mirror of
https://github.com/hwchase17/langchain.git
synced 2026-07-15 15:40:34 +00:00
2c3fec014f
Introduces a generator-based `on_model_call` hook that allows middleware
to intercept model calls with support for retry logic, error handling,
response transformation, and request modification.
## Overview
Middleware can now implement `on_model_call()` using a generator
protocol that:
- **Yields** `ModelRequest` to execute the model
- **Receives** `AIMessage` via `.send()` on success, or exception via
`.throw()` on error
- **Yields again** to retry or transform responses
- Uses **implicit last-yield semantics** (no return values from
generators)
## Usage Examples
### Basic Retry on Error
```python
from langchain.agents.middleware.types import AgentMiddleware
class RetryMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
for attempt in range(3):
try:
yield request # Execute model
break # Success
except Exception:
if attempt == 2:
raise # Max retries exceeded
```
### Response Transformation
```python
class UppercaseMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
result = yield request
modified = AIMessage(content=result.content.upper())
yield modified # Return transformed response
```
### Error Recovery
```python
class FallbackMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
try:
yield request
except Exception:
fallback = AIMessage(content="Service unavailable")
yield fallback # Convert error to fallback response
```
### Caching / Short-Circuit
```python
class CacheMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
if cached := get_cache(request):
yield cached # Skip model execution
else:
result = yield request
save_cache(request, result)
```
### Request Modification
```python
class SystemPromptMiddleware(AgentMiddleware):
def on_model_call(self, request, state, runtime):
modified_request = ModelRequest(
model=request.model,
system_prompt="You are a helpful assistant.",
messages=request.messages,
tools=request.tools,
)
yield modified_request
```
### Function Decorator
```python
from langchain.agents.middleware.types import on_model_call
@on_model_call
def retry_three_times(request, state, runtime):
for attempt in range(3):
try:
yield request
break
except Exception:
if attempt == 2:
raise
agent = create_agent(model="openai:gpt-4o", middleware=[retry_three_times])
```
## Middleware Composition
Middleware compose with first in list as outermost layer:
```python
agent = create_agent(
model="openai:gpt-4o",
middleware=[
RetryMiddleware(), # Outer - wraps others
LoggingMiddleware(), # Middle
UppercaseMiddleware(), # Inner - closest to model
]
)
```
The platform for reliable agents.
LangChain is a framework for building LLM-powered applications. It helps you chain together interoperable components and third-party integrations to simplify AI application development — all while future-proofing decisions as the underlying technology evolves.
pip install -U langchain
Documentation: To learn more about LangChain, check out the docs.
If you're looking for more advanced customization or agent orchestration, check out LangGraph, our framework for building controllable agent workflows.
Note
Looking for the JS/TS library? Check out LangChain.js.
Why use LangChain?
LangChain helps developers build applications powered by LLMs through a standard interface for models, embeddings, vector stores, and more.
Use LangChain for:
- Real-time data augmentation. Easily connect LLMs to diverse data sources and external/internal systems, drawing from LangChain’s vast library of integrations with model providers, tools, vector stores, retrievers, and more.
- Model interoperability. Swap models in and out as your engineering team experiments to find the best choice for your application’s needs. As the industry frontier evolves, adapt quickly — LangChain’s abstractions keep you moving without losing momentum.
LangChain’s ecosystem
While the LangChain framework can be used standalone, it also integrates seamlessly with any LangChain product, giving developers a full suite of tools when building LLM applications.
To improve your LLM application development, pair LangChain with:
- LangSmith - Helpful for agent evals and observability. Debug poor-performing LLM app runs, evaluate agent trajectories, gain visibility in production, and improve performance over time.
- LangGraph - Build agents that can reliably handle complex tasks with LangGraph, our low-level agent orchestration framework. LangGraph offers customizable architecture, long-term memory, and human-in-the-loop workflows — and is trusted in production by companies like LinkedIn, Uber, Klarna, and GitLab.
- LangGraph Platform - Deploy and scale agents effortlessly with a purpose-built deployment platform for long-running, stateful workflows. Discover, reuse, configure, and share agents across teams — and iterate quickly with visual prototyping in LangGraph Studio.
Additional resources
- Conceptual Guides: Explanations of key concepts behind the LangChain framework.
- Tutorials: Simple walkthroughs with guided examples on getting started with LangChain.
- API Reference: Detailed reference on navigating base packages and integrations for LangChain.
- LangChain Forum: Connect with the community and share all of your technical questions, ideas, and feedback.
- Chat LangChain: Ask questions & chat with our documentation.