1. Main fix: when we don't have a response format or middleware, don't
draw a conditional edge back to the loop entrypoint (self loop on model)
2. Supplementary fix: when we jump to `end` and there is an
`after_agent` hook, jump there instead of `__end__`
Other improvements -- I can remove these if they're more harmful than
helpful
1. Use keyword only arguments for edge generator functions for clarity
2. Rename args to `model_destination` and `end_destination` for clarity
Using the `api_doc_build.yml` workflow will now only pull from the
`v0.3` branch for each `langchain-ai` repo used during the build
process. This ensures that upcoming updates to the `master`/`main`
branch for each repo won't affect the v0.3 reference docs if/when they
are re-built or updated.
To accommodate breaking changes (e.g., removal of deprecated params like
`callback_manager`).
Will revert once we have updated releases of anthropic and openai.
This reduces confusion w/ types like `AgentState`, different arg names,
etc.
Second attempt, following
https://github.com/langchain-ai/langchain/pull/33249
* Ability to pass through `cache` and name in `create_agent` as
compilation args for the agent
* Right now, removing `test_react_agent.py` but we should add these
tests back as implemented w/ the new agent
* Add conditional edge when structured output tools are present to allow
for retries
* Rename `tracking` to `model_call_limit` to be consistent w/ tool call
limits
We need in the future (I'm happy to own):
* Significant test refactor
* Significant test overhaul where we emphasize and enforce coverage
- retry_model_request hook lets a middleware decide to retry a failed
model request, with full ability to modify as much or as little of the
request before doing so
- ModelFallbackMiddleware tries each fallback model in order, until one
is successful, or fallback list is exhausted
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
* Add llm based tool selection middleware.
* Note that we might want some form of caching for when the agent is
inside an active tool calling loop as the tool selection isn't expected
to change during that time.
API:
```python
class LLMToolSelectorMiddleware(AgentMiddleware):
"""Uses an LLM to select relevant tools before calling the main model.
When an agent has many tools available, this middleware filters them down
to only the most relevant ones for the user's query. This reduces token usage
and helps the main model focus on the right tools.
Examples:
Limit to 3 tools:
```python
from langchain.agents.middleware import LLMToolSelectorMiddleware
middleware = LLMToolSelectorMiddleware(max_tools=3)
agent = create_agent(
model="openai:gpt-4o",
tools=[tool1, tool2, tool3, tool4, tool5],
middleware=[middleware],
)
```
Use a smaller model for selection:
```python
middleware = LLMToolSelectorMiddleware(model="openai:gpt-4o-mini", max_tools=2)
```
"""
def __init__(
self,
*,
model: str | BaseChatModel | None = None,
system_prompt: str = DEFAULT_SYSTEM_PROMPT,
max_tools: int | None = None,
always_include: list[str] | None = None,
) -> None:
"""Initialize the tool selector.
Args:
model: Model to use for selection. If not provided, uses the agent's main model.
Can be a model identifier string or BaseChatModel instance.
system_prompt: Instructions for the selection model.
max_tools: Maximum number of tools to select. If the model selects more,
only the first max_tools will be used. No limit if not specified.
always_include: Tool names to always include regardless of selection.
These do not count against the max_tools limit.
"""
```
```python
"""Test script for LLM tool selection middleware."""
from langchain.agents import create_agent
from langchain.agents.middleware import LLMToolSelectorMiddleware
from langchain_core.tools import tool
@tool
def get_weather(location: str) -> str:
"""Get current weather for a location."""
return f"Weather in {location}: 72°F, sunny"
@tool
def search_web(query: str) -> str:
"""Search the web for information."""
return f"Search results for: {query}"
@tool
def calculate(expression: str) -> str:
"""Perform mathematical calculations."""
return f"Result of {expression}: 42"
@tool
def send_email(to: str, subject: str) -> str:
"""Send an email to someone."""
return f"Email sent to {to} with subject: {subject}"
@tool
def get_stock_price(symbol: str) -> str:
"""Get current stock price for a symbol."""
return f"Stock price for {symbol}: $150.25"
@tool
def translate_text(text: str, target_language: str) -> str:
"""Translate text to another language."""
return f"Translated '{text}' to {target_language}"
@tool
def set_reminder(task: str, time: str) -> str:
"""Set a reminder for a task."""
return f"Reminder set: {task} at {time}"
@tool
def get_news(topic: str) -> str:
"""Get latest news about a topic."""
return f"Latest news about {topic}"
@tool
def book_flight(destination: str, date: str) -> str:
"""Book a flight to a destination."""
return f"Flight booked to {destination} on {date}"
@tool
def get_restaurant_recommendations(city: str, cuisine: str) -> str:
"""Get restaurant recommendations."""
return f"Top {cuisine} restaurants in {city}"
# Create agent with tool selection middleware
middleware = LLMToolSelectorMiddleware(
model="openai:gpt-4o-mini",
max_tools=3,
)
agent = create_agent(
model="openai:gpt-4o",
tools=[
get_weather,
search_web,
calculate,
send_email,
get_stock_price,
translate_text,
set_reminder,
get_news,
book_flight,
get_restaurant_recommendations,
],
middleware=[middleware],
)
# Test with a query that should select specific tools
response = agent.invoke(
{"messages": [{"role": "user", "content": "I need to find restaurants"}]}
)
print(response)
```
* Add server side tools to modifyModelRequest (represented as dicts)
* Update some of the logic in terms of which tools are bound to ToolNode
* We still have a constraint on changing the response format dynamically
when using tool strategy. structured_output_tools are being using in
some of the edges. The code is now raising an exception to explain that
it's a limitation of the implementation. (We can add support later.)
- supports 6 well-known PII types (email, credit_card, ip, mac_address,
url)
- 4 handling strategies (block, redact, mask, hash)
- supports custom PII types with detector functions or regex
- the built-in types were chosen because they are common, and detection
can be reliably implemented with stdlib
* Preserve Auto type for the response format. cc @sydney-runkle Creating
an extra type was the nicest devx I could find for this (makes it easy
to do isinstance(thingy, AutoStrategy)
Remaining issue to address:
* Going to sort out why we're including tools in the tool node
