Closes https://github.com/langchain-ai/langchain/issues/33956
* Making `ModelRequest` generic on `ContextT` and `ResponseT` so that we
can thread type information through to `wrap_model_call`
* Making builtin middlewares generic on `ContextT` and `ResponseT` so
their context and response types can be inferred from the `create_agent`
signature
See new tests that verify backwards compatibility (for cases where folks
use custom middleware that wasn't parametrized).
This fixes:
1. Lack of access to context and response types in `wrap_model_call`
2. Lack of cohesion between middleware context + response types with
those specified in `create_agent`
See examples below:
### Type-safe context and response access
```python
class MyMiddleware(AgentMiddleware[AgentState[AnalysisResult], UserContext, AnalysisResult]):
def wrap_model_call(
self,
request: ModelRequest[UserContext],
handler: Callable[[ModelRequest[UserContext]], ModelResponse[AnalysisResult]],
) -> ModelResponse[AnalysisResult]:
# ✅ Now type-safe: IDE knows user_id exists and is str
user_id: str = request.runtime.context["user_id"]
# ❌ mypy error: "session_id" doesn't exist on UserContext
request.runtime.context["session_id"]
response = handler(request)
if response.structured_response is not None:
# ✅ Now type-safe: IDE knows sentiment exists and is str
sentiment: str = response.structured_response.sentiment
# ❌ mypy error: "summary" doesn't exist on AnalysisResult
response.structured_response.summary
return response
```
### Mismatched middleware/schema caught at `create_agent`
```python
class SessionMiddleware(AgentMiddleware[AgentState[Any], SessionContext, Any]):
...
# ❌ mypy error: SessionMiddleware expects SessionContext, not UserContext
create_agent(
model=model,
middleware=[SessionMiddleware()],
context_schema=UserContext, # mismatch!
)
class AnalysisMiddleware(AgentMiddleware[AgentState[AnalysisResult], ContextT, AnalysisResult]):
...
# ❌ mypy error: AnalysisMiddleware expects AnalysisResult, not SummaryResult
create_agent(
model=model,
middleware=[AnalysisMiddleware()],
response_format=SummaryResult, # mismatch!
)
```
dependent upon https://github.com/langchain-ai/langgraph/pull/6711
1. relax constraint in `factory.py` to allow for tools not
pre-registered in the `ModelRequest.tools` list
2. always add tool node if `wrap_tool_call` or `awrap_tool_call` is
implemented
3. add tests confirming you can register new tools at runtime in
`wrap_model_call` and execute them via `wrap_tool_call`
allows for the following pattern
```py
from langchain_core.messages import HumanMessage, ToolMessage
from langchain_core.tools import tool
from libs.langchain_v1.langchain.agents.factory import create_agent
from libs.langchain_v1.langchain.agents.middleware.types import (
AgentMiddleware,
ModelRequest,
ToolCallRequest,
)
@tool
def get_weather(location: str) -> str:
"""Get the current weather for a location."""
return f"The weather in {location} is sunny and 72°F."
@tool
def calculate_tip(bill_amount: float, tip_percentage: float = 20.0) -> str:
"""Calculate the tip amount for a bill."""
tip = bill_amount * (tip_percentage / 100)
return f"Tip: ${tip:.2f}, Total: ${bill_amount + tip:.2f}"
class DynamicToolMiddleware(AgentMiddleware):
"""Middleware that adds and handles a dynamic tool."""
def wrap_model_call(self, request: ModelRequest, handler):
updated = request.override(tools=[*request.tools, calculate_tip])
return handler(updated)
def wrap_tool_call(self, request: ToolCallRequest, handler):
if request.tool_call["name"] == "calculate_tip":
return handler(request.override(tool=calculate_tip))
return handler(request)
agent = create_agent(model="openai:gpt-4o-mini", tools=[get_weather], middleware=[DynamicToolMiddleware()])
result = agent.invoke({
"messages": [HumanMessage("What's the weather in NYC? Also calculate a 20% tip on a $85 bill")]
})
for msg in result["messages"]:
msg.pretty_print()
```
description by @mdrxy
- Enable `test_responses_spec.py` integration tests that were previously
skipped at module level
- Widen `ToolStrategy.schema` type annotation from `type[SchemaT]` to
`type[SchemaT] | dict[str, Any]` to match actual supported usage (JSON
schema dicts were already handled at runtime)
- Fix type annotations and linting issues in test file (modernize to
`dict`/`list`, add return types, prefix unused `_request` param)
- Improve generic typing in `load_spec` utility with bounded `TypeVar`
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Moving all `ToolNode` related improvements back to LangGraph and
importing them in LC!
pairing w/ https://github.com/langchain-ai/langgraph/pull/6321
this fixes a couple of things:
1. `InjectedState`, store etc will continue to work as expected no
matter where the import is from
2. `ToolRuntime` is now usable w/in langgraph, woohoo!
