1bf29da0d6
Adds generator-based middleware for intercepting tool execution in
agents. Middleware can retry on errors, cache results, modify requests,
or short-circuit execution.
### Implementation
**Middleware Protocol**
```python
class AgentMiddleware:
def on_tool_call(
self,
request: ToolCallRequest,
state: StateT,
runtime: Runtime[ContextT],
) -> Generator[ToolCallRequest | ToolMessage | Command, ToolMessage | Command, None]:
"""
Yields: ToolCallRequest (execute), ToolMessage (cached result), or Command (control flow)
Receives: ToolMessage or Command via .send()
Returns: None (final result is last value sent to handler)
"""
yield request # passthrough
```
**Composition**
Multiple middleware compose automatically (first = outermost), with
`_chain_tool_call_handlers()` stacking them like nested function calls.
### Examples
**Retry on error:**
```python
class RetryMiddleware(AgentMiddleware):
def on_tool_call(self, request, state, runtime):
for attempt in range(3):
response = yield request
if not isinstance(response, ToolMessage) or response.status != "error":
return
if attempt == 2:
return # Give up
```
**Cache results:**
```python
class CacheMiddleware(AgentMiddleware):
def on_tool_call(self, request, state, runtime):
cache_key = (request.tool_call["name"], tuple(request.tool_call["args"].items()))
if cached := self.cache.get(cache_key):
yield ToolMessage(content=cached, tool_call_id=request.tool_call["id"])
else:
response = yield request
self.cache[cache_key] = response.content
```
**Emulate tools with LLM**
```python
class ToolEmulator(AgentMiddleware):
def on_tool_call(self, request, state, runtime):
prompt = f"""Emulate: {request.tool_call["name"]}
Description: {request.tool.description}
Args: {request.tool_call["args"]}
Return ONLY the tool's output."""
response = emulator_model.invoke([HumanMessage(prompt)])
yield ToolMessage(
content=response.content,
tool_call_id=request.tool_call["id"],
name=request.tool_call["name"],
)
```
**Modify requests:**
```python
class ScalingMiddleware(AgentMiddleware):
def on_tool_call(self, request, state, runtime):
if "value" in request.tool_call["args"]:
request.tool_call["args"]["value"] *= 2
yield request
```
Packages
Important
This repository is structured as a monorepo, with various packages located in this libs/ directory. Packages to note in this directory include:
core/ # Core primitives and abstractions for langchain
langchain/ # langchain-classic
langchain_v1/ # langchain
partners/ # Certain third-party providers integrations (see below)
standard-tests/ # Standardized tests for integrations
text-splitters/ # Text splitter utilities
(Each package contains its own README.md file with specific details about that package.)
Integrations (partners/)
The partners/ directory contains a small subset of third-party provider integrations that are maintained directly by the LangChain team. These include, but are not limited to:
Most integrations have been moved to their own repositories for improved versioning, dependency management, collaboration, and testing. This includes packages from popular providers such as Google and AWS. Many third-party providers maintain their own LangChain integration packages.
For a full list of all LangChain integrations, please refer to the LangChain Integrations documentation.