mirror of
https://github.com/hwchase17/langchain.git
synced 2026-02-21 22:56:05 +00:00
2c3fec014f
Introduces a generator-based `on_model_call` hook that allows middleware
to intercept model calls with support for retry logic, error handling,
response transformation, and request modification.
## Overview
Middleware can now implement `on_model_call()` using a generator
protocol that:
- **Yields** `ModelRequest` to execute the model
- **Receives** `AIMessage` via `.send()` on success, or exception via
`.throw()` on error
- **Yields again** to retry or transform responses
- Uses **implicit last-yield semantics** (no return values from
generators)
## Usage Examples
### Basic Retry on Error
```python
from langchain.agents.middleware.types import AgentMiddleware
class RetryMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
for attempt in range(3):
try:
yield request # Execute model
break # Success
except Exception:
if attempt == 2:
raise # Max retries exceeded
```
### Response Transformation
```python
class UppercaseMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
result = yield request
modified = AIMessage(content=result.content.upper())
yield modified # Return transformed response
```
### Error Recovery
```python
class FallbackMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
try:
yield request
except Exception:
fallback = AIMessage(content="Service unavailable")
yield fallback # Convert error to fallback response
```
### Caching / Short-Circuit
```python
class CacheMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
if cached := get_cache(request):
yield cached # Skip model execution
else:
result = yield request
save_cache(request, result)
```
### Request Modification
```python
class SystemPromptMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
modified_request = ModelRequest(
model=request.model,
system_prompt="You are a helpful assistant.",
messages=request.messages,
tools=request.tools,
)
yield modified_request
```
### Function Decorator
```python
from langchain.agents.middleware.types import on_model_call
@on_model_call
def retry_three_times(request, state, runtime):
for attempt in range(3):
try:
yield request
break
except Exception:
if attempt == 2:
raise
agent = create_agent(model="openai:gpt-4o", middleware=[retry_three_times])
```
## Middleware Composition
Middleware compose with first in list as outermost layer:
```python
agent = create_agent(
model="openai:gpt-4o",
middleware=[
RetryMiddleware(), # Outer - wraps others
LoggingMiddleware(), # Middle
UppercaseMiddleware(), # Inner - closest to model
]
)
```
Packages
Important
This repository is structured as a monorepo, with various packages located in this libs/ directory. Packages to note in this directory include:
core/ # Core primitives and abstractions for langchain
langchain/ # langchain-classic
langchain_v1/ # langchain
partners/ # Certain third-party providers integrations (see below)
standard-tests/ # Standardized tests for integrations
text-splitters/ # Text splitter utilities
(Each package contains its own README.md file with specific details about that package.)
Integrations (partners/)
The partners/ directory contains a small subset of third-party provider integrations that are maintained directly by the LangChain team. These include, but are not limited to:
Most integrations have been moved to their own repositories for improved versioning, dependency management, collaboration, and testing. This includes packages from popular providers such as Google and AWS. Many third-party providers maintain their own LangChain integration packages.
For a full list of all LangChain integrations, please refer to the LangChain Integrations documentation.