Normalizes raw Pydantic schemas to AutoStrategy when middleware
overrides response_format.
When middleware calls
`request.override(response_format=SomePydanticClass)` with a raw schema,
the code now wraps it in `AutoStrategy` before processing. This prevents
an `IndexError` crash that occurred because raw schemas were falling
through to the "no structured output" branch.
Fixes#35008
Co-authored-by: Guofang Tang <tinggofun@gmail.com>
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Alternative to https://github.com/langchain-ai/langchain/pull/35024.
Paving the way for summarization in `wrap_model_call` (which requires
state updates).
---
Add `ExtendedModelResponse` dataclass that allows `wrap_model_call`
middleware to return a `Command` alongside the model response for
additional state updates.
```py
@dataclass
class ExtendedModelResponse(Generic[ResponseT]):
model_response: ModelResponse[ResponseT]
command: Command
```
## Motivation
Previously, `wrap_model_call` middleware could only return a
`ModelResponse` or `AIMessage` — there was no way to inject additional
state updates (e.g. custom state fields) from the model call middleware
layer. `ExtendedModelResponse` fills this gap by accepting an optional
`Command`.
This feature is needed by the summarization middleware, which needs to
track summarization trigger points calculated during `wrap_model_call`.
## Why `Command` instead of a plain `state_update` dict?
We chose `Command` rather than the raw `state_update: dict` approach
from the earlier iteration because `Command` is the established
LangGraph primitive for state updates from nodes. Using `Command` means:
- State updates flow through the graph's reducers (e.g. `add_messages`)
rather than being merged as raw dicts. This makes messages updates
additive alongside the model response instead of replacing them.
- Consistency with `wrap_tool_call`, which already returns `Command`.
- Future-proof: as `Command` gains new capabilities (e.g. `goto`,
`send`), middleware can leverage them without API changes.
## Why keep `model_response` separate instead of using `Command`
directly?
The model node needs to distinguish the model's actual response
(messages + structured output) from supplementary middleware state
updates. If middleware returned only a `Command`, there would be no
clean way to extract the `ModelResponse` for structured output handling,
response validation, and the core model-to-tools routing logic. Keeping
`model_response` explicit preserves a clear boundary between "what the
model said" and "what middleware wants to update."
Also, in order to avoid breaking, the `handler` passed to
`wrap_tool_call` needs to always return a `ModelResponse`. There's no
easy way to preserve this if we pump it into a `Command`.
One nice thing about having this `ExtendedModelResponse` structure is
that it's extensible if we want to add more metadata in the future.
## Composition
When multiple middleware layers return `ExtendedModelResponse`, their
commands compose naturally:
- **Inner commands propagate outward:** At composition boundaries,
`ExtendedModelResponse` is unwrapped to its underlying `ModelResponse`
so outer middleware always sees a plain `ModelResponse` from
`handler()`. The inner command is captured and accumulated.
- **Commands are applied through reducers:** Each `Command` becomes a
separate state update applied through the graph's reducers. For
messages, this means they're additive (via `add_messages`), not
replacing.
- **Outer wins on conflicts:** For non-reducer state fields, commands
are applied inner-first then outer, so the outermost middleware's value
takes precedence on conflicting keys.
- **Retry-safe:** When outer middleware retries by calling `handler()`
again, accumulated inner commands are cleared and re-collected from the
fresh call.
```python
class Outer(AgentMiddleware):
def wrap_model_call(self, request, handler):
response = handler(request) # sees ModelResponse, not ExtendedModelResponse
return ExtendedModelResponse(
model_response=response,
command=Command(update={"outer_key": "val"}),
)
class Inner(AgentMiddleware):
def wrap_model_call(self, request, handler):
response = handler(request)
return ExtendedModelResponse(
model_response=response,
command=Command(update={"inner_key": "val"}),
)
# Final state merges both commands: {"inner_key": "val", "outer_key": "val"}
```
## Backwards compatibility
Fully backwards compatible. The `ModelCallResult` type alias is widened
from `ModelResponse | AIMessage` to `ModelResponse | AIMessage |
ExtendedModelResponse`, but existing middleware returning
`ModelResponse` or `AIMessage` continues to work identically.
## Internals
- `model_node` / `amodel_node` now return `list[Command]` instead of
`dict[str, Any]`
- `_build_commands` converts the model response + accumulated middleware
commands into a list of `Command` objects for LangGraph
- `_ComposedExtendedModelResponse` is the internal type that accumulates
commands across layers during composition
Closes https://github.com/langchain-ai/langchain/issues/33956
* Making `ModelRequest` generic on `ContextT` and `ResponseT` so that we
can thread type information through to `wrap_model_call`
* Making builtin middlewares generic on `ContextT` and `ResponseT` so
their context and response types can be inferred from the `create_agent`
signature
See new tests that verify backwards compatibility (for cases where folks
use custom middleware that wasn't parametrized).
This fixes:
1. Lack of access to context and response types in `wrap_model_call`
2. Lack of cohesion between middleware context + response types with
those specified in `create_agent`
See examples below:
### Type-safe context and response access
```python
class MyMiddleware(AgentMiddleware[AgentState[AnalysisResult], UserContext, AnalysisResult]):
def wrap_model_call(
self,
request: ModelRequest[UserContext],
handler: Callable[[ModelRequest[UserContext]], ModelResponse[AnalysisResult]],
) -> ModelResponse[AnalysisResult]:
# ✅ Now type-safe: IDE knows user_id exists and is str
user_id: str = request.runtime.context["user_id"]
# ❌ mypy error: "session_id" doesn't exist on UserContext
request.runtime.context["session_id"]
response = handler(request)
if response.structured_response is not None:
# ✅ Now type-safe: IDE knows sentiment exists and is str
sentiment: str = response.structured_response.sentiment
# ❌ mypy error: "summary" doesn't exist on AnalysisResult
response.structured_response.summary
return response
```
### Mismatched middleware/schema caught at `create_agent`
```python
class SessionMiddleware(AgentMiddleware[AgentState[Any], SessionContext, Any]):
...
# ❌ mypy error: SessionMiddleware expects SessionContext, not UserContext
create_agent(
model=model,
middleware=[SessionMiddleware()],
context_schema=UserContext, # mismatch!
)
class AnalysisMiddleware(AgentMiddleware[AgentState[AnalysisResult], ContextT, AnalysisResult]):
...
# ❌ mypy error: AnalysisMiddleware expects AnalysisResult, not SummaryResult
create_agent(
model=model,
middleware=[AnalysisMiddleware()],
response_format=SummaryResult, # mismatch!
)
```
