dependent upon https://github.com/langchain-ai/langgraph/pull/6711
1. relax constraint in `factory.py` to allow for tools not
pre-registered in the `ModelRequest.tools` list
2. always add tool node if `wrap_tool_call` or `awrap_tool_call` is
implemented
3. add tests confirming you can register new tools at runtime in
`wrap_model_call` and execute them via `wrap_tool_call`
allows for the following pattern
```py
from langchain_core.messages import HumanMessage, ToolMessage
from langchain_core.tools import tool
from libs.langchain_v1.langchain.agents.factory import create_agent
from libs.langchain_v1.langchain.agents.middleware.types import (
AgentMiddleware,
ModelRequest,
ToolCallRequest,
)
@tool
def get_weather(location: str) -> str:
"""Get the current weather for a location."""
return f"The weather in {location} is sunny and 72°F."
@tool
def calculate_tip(bill_amount: float, tip_percentage: float = 20.0) -> str:
"""Calculate the tip amount for a bill."""
tip = bill_amount * (tip_percentage / 100)
return f"Tip: ${tip:.2f}, Total: ${bill_amount + tip:.2f}"
class DynamicToolMiddleware(AgentMiddleware):
"""Middleware that adds and handles a dynamic tool."""
def wrap_model_call(self, request: ModelRequest, handler):
updated = request.override(tools=[*request.tools, calculate_tip])
return handler(updated)
def wrap_tool_call(self, request: ToolCallRequest, handler):
if request.tool_call["name"] == "calculate_tip":
return handler(request.override(tool=calculate_tip))
return handler(request)
agent = create_agent(model="openai:gpt-4o-mini", tools=[get_weather], middleware=[DynamicToolMiddleware()])
result = agent.invoke({
"messages": [HumanMessage("What's the weather in NYC? Also calculate a 20% tip on a $85 bill")]
})
for msg in result["messages"]:
msg.pretty_print()
```
We need to set `{"metadata": {"lc_source": "summarization"}}` on the
invocation so that consumers (e.g. `deepagents-cli`) can see that a
summarization LLM call is being made, and therefore take any necessary
actions (such as updating the status line to say `'Currently
summarizing...'`
See https://github.com/langchain-ai/deepagents/pull/742 for more
Related to #34693 (but for outbound)
* Making `FakeToolCallingModel` generic on its `structured_response`
doesn't help anywhere in typing.
* There are more than 120 references of `FakeToolCallingModel` in the
code where you get ` error: Need type annotation for "model"
[var-annotated]` because mypy can't resolve the generic type (we don't
see them atm because they are in files temporarily excluded from mypy
checking). We would need to explicitly type them to
`FakeToolCallingModel[Any]`
Co-authored-by: Mason Daugherty <mason@langchain.dev>
description by @mdrxy
- Enable `test_responses_spec.py` integration tests that were previously
skipped at module level
- Widen `ToolStrategy.schema` type annotation from `type[SchemaT]` to
`type[SchemaT] | dict[str, Any]` to match actual supported usage (JSON
schema dicts were already handled at runtime)
- Fix type annotations and linting issues in test file (modernize to
`dict`/`list`, add return types, prefix unused `_request` param)
- Improve generic typing in `load_spec` utility with bounded `TypeVar`
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Fixes#34282
**Before:** When using agents with tools (like file reading, web search,
etc.), the conversation looks like this:
```
[User] "Read these 10 files and summarize them"
[AI] "I'll read all 10 files" + [tool_call: read_file x 10]
[Tool] "Contents of file1.txt..."
[Tool] "Contents of file2.txt..."
[Tool] "Contents of file3.txt..."
... (7 more tool responses)
```
When the conversation gets too long, `SummarizationMiddleware` kicks in
to compress older messages. The problem was:
If you asked to keep the last 6 messages, you'd get:
```
[Summary] "Here's what happened before..."
[Tool] "Contents of file5.txt..."
[Tool] "Contents of file6.txt..."
[Tool] "Contents of file7.txt..."
[Tool] "Contents of file8.txt..."
[Tool] "Contents of file9.txt..."
[Tool] "Contents of file10.txt..."
```
The AI's original request to read the files (`[AI]` message with
`tool_calls`) was summarized away, but the tool responses remained. This
caused the error:
```
Error code: 400 - "No tool call found for function call output with call_id..."
```
Many APIs require that every tool response has a matching tool request.
Without the AI message, the tool responses are "orphaned."
## The fix
Now when the cutoff lands on tool messages, we **move backward** to
include the AI message that requested those tools:
Same scenario, keeping last 6 messages:
```
[Summary] "Here's what happened before..."
[AI] "I'll read all 10 files" + [tool_call: read_file x 10]
[Tool] "Contents of file1.txt..."
[Tool] "Contents of file2.txt..."
... (all 10 tool responses)
```
The AI message is preserved along with its tool responses, keeping them
paired together.
## Practical examples
### Example 1: Parallel tool calls
**Scenario:** Agent reads 10 files in parallel, summarization triggers
(see above)
### Example 2: Mixed conversation
**Scenario:** User asks question, AI uses tools, user says thanks
```
[User] "What's the weather?"
[AI] "Let me check" + [tool_call: get_weather]
[Tool] "72F and sunny"
[AI] "It's 72F and sunny!"
[User] "Thanks!"
```
Keeping last 2 messages:
| Before (Bug) | After (Fix) |
|--------------|-------------|
| Only `[User] "Thanks!"` kept | `[AI] + [Tool] + [AI] + [User]` all
kept |
| Lost the weather info | Tool pair preserved with response |
### Example 3: Multiple tool sequences
```
[User] "Search for X"
[AI] [tool_call: search]
[Tool] "Results for X"
[User] "Now search for Y"
[AI] [tool_call: search]
[Tool] "Results for Y"
[User] "Great!"
```
**Keeping last 3 messages:** If cutoff lands on `[Tool] "Results for
Y"`, we now include `[AI] [tool_call: search]` to keep the pair
together.
Add unit coverage for chat model provider inference across common model
name prefixes. This improves regression protection without touching
runtime
---------
Co-authored-by: Mason Daugherty <github@mdrxy.com>
The agent should only make a single call to update the todo list at a
time. A parallel call doesn't make sense, but also cannot work as
there's no obvious reducer to use.
On parallel calls of the todo tool, we return ToolMessage containing to
guide the LLM to not call the tool in parallel.
---------
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Fixes#34517
Supersedes #34557, #34570
Fixes token inflation in `SummarizationMiddleware` that caused context
window overflow during summarization.
**Root cause:** When formatting messages for the summary prompt,
`str(messages)` was implicitly called, which includes all Pydantic
metadata fields (`usage_metadata`, `response_metadata`,
`additional_kwargs`, etc.). This caused the stringified representation
to use ~2.5x more tokens than `count_tokens_approximately` estimates.
**Problem:**
- Summarization triggers at 85% of context window based on
`count_tokens_approximately`
- But `str(messages)` in the prompt uses 2.5x more tokens
- Results in `ContextLengthExceeded`
**Fix:** Use `get_buffer_string()` to format messages, which produces
compact output:
```
Human: What's the weather?
AI: Let me check...[tool_calls]
Tool: 72°F and sunny
```
Instead of verbose Pydantic repr:
```python
[HumanMessage(content='What's the weather?', additional_kwargs={}, response_metadata={}), ...]
```
