mirror of
https://github.com/hwchase17/langchain.git
synced 2026-04-05 03:48:48 +00:00
df2ecd9448
* Add llm based tool selection middleware.
* Note that we might want some form of caching for when the agent is
inside an active tool calling loop as the tool selection isn't expected
to change during that time.
API:
```python
class LLMToolSelectorMiddleware(AgentMiddleware):
"""Uses an LLM to select relevant tools before calling the main model.
When an agent has many tools available, this middleware filters them down
to only the most relevant ones for the user's query. This reduces token usage
and helps the main model focus on the right tools.
Examples:
Limit to 3 tools:
```python
from langchain.agents.middleware import LLMToolSelectorMiddleware
middleware = LLMToolSelectorMiddleware(max_tools=3)
agent = create_agent(
model="openai:gpt-4o",
tools=[tool1, tool2, tool3, tool4, tool5],
middleware=[middleware],
)
```
Use a smaller model for selection:
```python
middleware = LLMToolSelectorMiddleware(model="openai:gpt-4o-mini", max_tools=2)
```
"""
def __init__(
self,
*,
model: str | BaseChatModel | None = None,
system_prompt: str = DEFAULT_SYSTEM_PROMPT,
max_tools: int | None = None,
always_include: list[str] | None = None,
) -> None:
"""Initialize the tool selector.
Args:
model: Model to use for selection. If not provided, uses the agent's main model.
Can be a model identifier string or BaseChatModel instance.
system_prompt: Instructions for the selection model.
max_tools: Maximum number of tools to select. If the model selects more,
only the first max_tools will be used. No limit if not specified.
always_include: Tool names to always include regardless of selection.
These do not count against the max_tools limit.
"""
```
```python
"""Test script for LLM tool selection middleware."""
from langchain.agents import create_agent
from langchain.agents.middleware import LLMToolSelectorMiddleware
from langchain_core.tools import tool
@tool
def get_weather(location: str) -> str:
"""Get current weather for a location."""
return f"Weather in {location}: 72°F, sunny"
@tool
def search_web(query: str) -> str:
"""Search the web for information."""
return f"Search results for: {query}"
@tool
def calculate(expression: str) -> str:
"""Perform mathematical calculations."""
return f"Result of {expression}: 42"
@tool
def send_email(to: str, subject: str) -> str:
"""Send an email to someone."""
return f"Email sent to {to} with subject: {subject}"
@tool
def get_stock_price(symbol: str) -> str:
"""Get current stock price for a symbol."""
return f"Stock price for {symbol}: $150.25"
@tool
def translate_text(text: str, target_language: str) -> str:
"""Translate text to another language."""
return f"Translated '{text}' to {target_language}"
@tool
def set_reminder(task: str, time: str) -> str:
"""Set a reminder for a task."""
return f"Reminder set: {task} at {time}"
@tool
def get_news(topic: str) -> str:
"""Get latest news about a topic."""
return f"Latest news about {topic}"
@tool
def book_flight(destination: str, date: str) -> str:
"""Book a flight to a destination."""
return f"Flight booked to {destination} on {date}"
@tool
def get_restaurant_recommendations(city: str, cuisine: str) -> str:
"""Get restaurant recommendations."""
return f"Top {cuisine} restaurants in {city}"
# Create agent with tool selection middleware
middleware = LLMToolSelectorMiddleware(
model="openai:gpt-4o-mini",
max_tools=3,
)
agent = create_agent(
model="openai:gpt-4o",
tools=[
get_weather,
search_web,
calculate,
send_email,
get_stock_price,
translate_text,
set_reminder,
get_news,
book_flight,
get_restaurant_recommendations,
],
middleware=[middleware],
)
# Test with a query that should select specific tools
response = agent.invoke(
{"messages": [{"role": "user", "content": "I need to find restaurants"}]}
)
print(response)
```
The platform for reliable agents.
LangChain is a framework for building LLM-powered applications. It helps you chain together interoperable components and third-party integrations to simplify AI application development — all while future-proofing decisions as the underlying technology evolves.
pip install -U langchain
Documentation: To learn more about LangChain, check out the docs.
If you're looking for more advanced customization or agent orchestration, check out LangGraph, our framework for building controllable agent workflows.
Note
Looking for the JS/TS library? Check out LangChain.js.
Why use LangChain?
LangChain helps developers build applications powered by LLMs through a standard interface for models, embeddings, vector stores, and more.
Use LangChain for:
- Real-time data augmentation. Easily connect LLMs to diverse data sources and external/internal systems, drawing from LangChain’s vast library of integrations with model providers, tools, vector stores, retrievers, and more.
- Model interoperability. Swap models in and out as your engineering team experiments to find the best choice for your application’s needs. As the industry frontier evolves, adapt quickly — LangChain’s abstractions keep you moving without losing momentum.
LangChain’s ecosystem
While the LangChain framework can be used standalone, it also integrates seamlessly with any LangChain product, giving developers a full suite of tools when building LLM applications.
To improve your LLM application development, pair LangChain with:
- LangSmith - Helpful for agent evals and observability. Debug poor-performing LLM app runs, evaluate agent trajectories, gain visibility in production, and improve performance over time.
- LangGraph - Build agents that can reliably handle complex tasks with LangGraph, our low-level agent orchestration framework. LangGraph offers customizable architecture, long-term memory, and human-in-the-loop workflows — and is trusted in production by companies like LinkedIn, Uber, Klarna, and GitLab.
- LangGraph Platform - Deploy and scale agents effortlessly with a purpose-built deployment platform for long-running, stateful workflows. Discover, reuse, configure, and share agents across teams — and iterate quickly with visual prototyping in LangGraph Studio.
Additional resources
- Conceptual Guides: Explanations of key concepts behind the LangChain framework.
- Tutorials: Simple walkthroughs with guided examples on getting started with LangChain.
- API Reference: Detailed reference on navigating base packages and integrations for LangChain.
- LangChain Forum: Connect with the community and share all of your technical questions, ideas, and feedback.
- Chat LangChain: Ask questions & chat with our documentation.