Closes https://github.com/langchain-ai/langchain/issues/33983
* Adds `ModelRetryMiddleware` modeled after `ToolRetryMiddleware`
* Uses `on_failure` modes of `error` and `continue` to match the
`exit_behavior` modes of model + tool call limit middleware
* In a backwards compatible manner, aligns the API of
`ToolRetryMiddleware`'s `on_failure` with the above
* Centralize common "retry" utils across these middlewares
Moving all `ToolNode` related improvements back to LangGraph and
importing them in LC!
pairing w/ https://github.com/langchain-ai/langgraph/pull/6321
this fixes a couple of things:
1. `InjectedState`, store etc will continue to work as expected no
matter where the import is from
2. `ToolRuntime` is now usable w/in langgraph, woohoo!
The old `before_model_jump_to` classvar approach was quite clunky, this
is nicer imo and easier to document. Also moving from `jump_to` to
`can_jump_to` which is more idiomatic.
Before:
```py
class MyMiddleware(AgentMiddleware):
before_model_jump_to: ClassVar[list[JumpTo]] = ["end"]
def before_model(state, runtime) -> dict[str, Any]:
return {"jump_to": "end"}
```
After
```py
class MyMiddleware(AgentMiddleware):
@hook_config(can_jump_to=["end"])
def before_model(state, runtime) -> dict[str, Any]:
return {"jump_to": "end"}
```
Multiple improvements to HITL flow:
* On a `response` type resume, we should still append the tool call to
the last AIMessage (otherwise we have a ToolResult without a
corresponding ToolCall)
* When all interrupts have `response` types (so there's no pending tool
calls), we should jump back to the first node (instead of end) as we
enforced in the previous `post_model_hook_router`
* Added comments to `model_to_tools` router so clarify all of the
potential exit conditions
Additionally:
* Lockfile update to use latest LG alpha release
* Added test for `jump_to` behaving ephemerally, this was fixed in LG
but surfaced as a bug w/ `jump_to`.
* Bump version to v1.0.0a10 to prep for alpha release
---------
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
- Removes Codespell from deps, docs, and `Makefile`s
- Python version requirements in all `pyproject.toml` files now use the
`~=` (compatible release) specifier
- All dependency groups and main dependencies now use explicit lower and
upper bounds, reducing potential for breaking changes
## Overview
Adding new `AgentMiddleware` primitive that supports `before_model`,
`after_model`, and `prepare_model_request` hooks.
This is very exciting! It makes our `create_agent` prebuilt much more
extensible + capable. Still in alpha and subject to change.
This is different than the initial
[implementation](https://github.com/langchain-ai/langgraph/tree/nc/25aug/agent)
in that it:
* Fills in gaps w/ missing features, for ex -- new structured output,
optionality of tools + system prompt, sync and async model requests,
provider builtin tools
* Exposes private state extensions for middleware, enabling things like
model call tracking, etc
* Middleware can register tools
* Uses a `TypedDict` for `AgentState` -- dataclass subclassing is tricky
w/ required values + required decorators
* Addition of `model_settings` to `ModelRequest` so that we can pass
through things to bind (like cache kwargs for anthropic middleware)
## TODOs
### top prio
- [x] add middleware support to existing agent
- [x] top prio middlewares
- [x] summarization node
- [x] HITL
- [x] prompt caching
other ones
- [x] model call limits
- [x] tool calling limits
- [ ] usage (requires output state)
### secondary prio
- [x] improve typing for state updates from middleware (not working
right now w/ simple `AgentUpdate` and `AgentJump`, at least in Python)
- [ ] add support for public state (input / output modifications via
pregel channel mods) -- to be tackled in another PR
- [x] testing!
### docs
See https://github.com/langchain-ai/docs/pull/390
- [x] high level docs about middleware
- [x] summarization node
- [x] HITL
- [x] prompt caching
## open questions
Lots of open questions right now, many of them inlined as comments for
the short term, will catalog some more significant ones here.
---------
Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Adding `create_react_agent` and introducing `langchain.agents`!
## Enhanced Structured Output
`create_react_agent` supports coercion of outputs to structured data
types like `pydantic` models, dataclasses, typed dicts, or JSON schemas
specifications.
### Structural Changes
In langgraph < 1.0, `create_react_agent` implemented support for
structured output via an additional LLM call to the model after the
standard model / tool calling loop finished. This introduced extra
expense and was unnecessary.
This new version implements structured output support in the main loop,
allowing a model to choose between calling tools or generating
structured output (or both).
The same basic pattern for structured output generation works:
```py
from langchain.agents import create_react_agent
from langchain_core.messages import HumanMessage
from pydantic import BaseModel
class Weather(BaseModel):
temperature: float
condition: str
def weather_tool(city: str) -> str:
"""Get the weather for a city."""
return f"it's sunny and 70 degrees in {city}"
agent = create_react_agent("openai:gpt-4o-mini", tools=[weather_tool], response_format=Weather)
print(repr(result["structured_response"]))
#> Weather(temperature=70.0, condition='sunny')
```
### Advanced Configuration
The new API exposes two ways to configure how structured output is
generated. Under the hood, LangChain will attempt to pick the best
approach if not explicitly specified. That is, if provider native
support is available for a given model, that takes priority over
artificial tool calling.
1. Artificial tool calling (the default for most models)
LangChain generates a tool (or tools) under the hood that match the
schema of your response format. When the model calls those tools,
LangChain coerces the args to the desired format. Note, LangChain does
not validate outputs adhering to JSON schema specifications.
<details>
<summary>Extended example</summary>
```py
from langchain.agents import create_react_agent
from langchain_core.messages import HumanMessage
from langchain.agents.structured_output import ToolStrategy
from pydantic import BaseModel
class Weather(BaseModel):
temperature: float
condition: str
def weather_tool(city: str) -> str:
"""Get the weather for a city."""
return f"it's sunny and 70 degrees in {city}"
agent = create_react_agent(
"openai:gpt-4o-mini",
tools=[weather_tool],
response_format=ToolStrategy(
schema=Weather, tool_message_content="Final Weather result generated"
),
)
result = agent.invoke({"messages": [HumanMessage("What's the weather in Tokyo?")]})
for message in result["messages"]:
message.pretty_print()
"""
================================ Human Message =================================
What's the weather in Tokyo?
================================== Ai Message ==================================
Tool Calls:
weather_tool (call_Gg933BMHMwck50Q39dtBjXm7)
Call ID: call_Gg933BMHMwck50Q39dtBjXm7
Args:
city: Tokyo
================================= Tool Message =================================
Name: weather_tool
it's sunny and 70 degrees in Tokyo
================================== Ai Message ==================================
Tool Calls:
Weather (call_9xOkYUM7PuEXl9DQq9sWGv5l)
Call ID: call_9xOkYUM7PuEXl9DQq9sWGv5l
Args:
temperature: 70
condition: sunny
================================= Tool Message =================================
Name: Weather
Final Weather result generated
"""
print(repr(result["structured_response"]))
#> Weather(temperature=70.0, condition='sunny')
```
</details>
2. Provider implementations (limited to OpenAI, Groq)
Some providers support structured output generating directly. For those
cases, we offer the `ProviderStrategy` hint:
<details>
<summary>Extended example</summary>
```py
from langchain.agents import create_react_agent
from langchain_core.messages import HumanMessage
from langchain.agents.structured_output import ProviderStrategy
from pydantic import BaseModel
class Weather(BaseModel):
temperature: float
condition: str
def weather_tool(city: str) -> str:
"""Get the weather for a city."""
return f"it's sunny and 70 degrees in {city}"
agent = create_react_agent(
"openai:gpt-4o-mini",
tools=[weather_tool],
response_format=ProviderStrategy(Weather),
)
result = agent.invoke({"messages": [HumanMessage("What's the weather in Tokyo?")]})
for message in result["messages"]:
message.pretty_print()
"""
================================ Human Message =================================
What's the weather in Tokyo?
================================== Ai Message ==================================
Tool Calls:
weather_tool (call_OFJq1FngIXS6cvjWv5nfSFZp)
Call ID: call_OFJq1FngIXS6cvjWv5nfSFZp
Args:
city: Tokyo
================================= Tool Message =================================
Name: weather_tool
it's sunny and 70 degrees in Tokyo
================================== Ai Message ==================================
{"temperature":70,"condition":"sunny"}
Weather(temperature=70.0, condition='sunny')
"""
print(repr(result["structured_response"]))
#> Weather(temperature=70.0, condition='sunny')
```
Note! The final tool message has the custom content provided by the dev.
</details>
Prompted output was previously supported and is no longer supported via
the `response_format` argument to `create_react_agent`. If there's
significant demand for this, we'd be happy to engineer a solution.
## Error Handling
`create_react_agent` now exposes an API for managing errors associated
with structured output generation. There are two common problems with
structured output generation (w/ artificial tool calling):
1. **Parsing error** -- the model generates data that doesn't match the
desired structure for the output
2. **Multiple tool calls error** -- the model generates 2 or more tool
calls associated with structured output schemas
A developer can control the desired behavior for this via the
`handle_errors` arg to `ToolStrategy`.
<details>
<summary>Extended example</summary>
```py
from langchain_core.messages import HumanMessage
from pydantic import BaseModel
from langchain.agents import create_react_agent
from langchain.agents.structured_output import StructuredOutputValidationError, ToolStrategy
class Weather(BaseModel):
temperature: float
condition: str
def weather_tool(city: str) -> str:
"""Get the weather for a city."""
return f"it's sunny and 70 degrees in {city}"
def handle_validation_error(error: Exception) -> str:
if isinstance(error, StructuredOutputValidationError):
return (
f"Please call the {error.tool_name} call again with the correct arguments. "
f"Your mistake was: {error.source}"
)
raise error
agent = create_react_agent(
"openai:gpt-5",
tools=[weather_tool],
response_format=ToolStrategy(
schema=Weather,
handle_errors=handle_validation_error,
),
)
```
</details>
## Error Handling for Tool Calling
Tools fail for two main reasons:
1. **Invocation failure** -- the args generated by the model for the
tool are incorrect (missing, incompatible data types, etc)
2. **Execution failure** -- the tool execution itself fails due to a
developer error, network error, or some other exception.
By default, when tool **invocation** fails, the react agent will return
an artificial `ToolMessage` to the model asking it to correct its
mistakes and retry.
Now, when tool **execution** fails, the react agent raises the
`ToolException` by default instead of asking the model to retry. This
helps to avoid looping that should be avoided due to the aforementioned
issues.
Developers can configure their desired behavior for retries / error
handling via the `handle_tool_errors` arg to `ToolNode`.
## Pre-Bound Models
`create_react_agent` no longer supports inputs to `model` that have been
pre-bound w/ tools or other configuration. To properly support
structured output generation, the agent itself needs the power to bind
tools + structured output kwargs.
This also makes the devx cleaner - it's always expected that `model` is
an instance of `BaseChatModel` (or `str` that we coerce into a chat
model instance).
Dynamic model functions can return a pre-bound model **IF** structured
output is not also used. Dynamic model functions can then bind tools /
structured output logic.
## Import Changes
Users should now use `create_react_agent` from `langchain.agents`
instead of `langgraph.prebuilts`.
Other imports have a similar migration path, `ToolNode` and `AgentState`
for example.
* `chat_agent_executor.py` -> `react_agent.py`
Some notes:
1. Disabled blockbuster + some linting in `langchain/agents` -- beyond
ideal, but necessary to get this across the line for the alpha. We
should re-enable before official release.
