Moving all `ToolNode` related improvements back to LangGraph and
importing them in LC!
pairing w/ https://github.com/langchain-ai/langgraph/pull/6321
this fixes a couple of things:
1. `InjectedState`, store etc will continue to work as expected no
matter where the import is from
2. `ToolRuntime` is now usable w/in langgraph, woohoo!
* attach the latest `AIMessage` to all `StructuredOutputError`s so that
relevant middleware can use as desired
* raise `StructuredOutputError` from `ProviderStrategy` logic in case of
failed parsing (so that we can retry from middleware)
* added a test suite w/ example custom middleware that retries for tool
+ provider strategy
Long term, we could add our own opinionated structured output retry
middleware, but this at least unblocks folks who want to use custom
retry logic in the short term :)
```py
class StructuredOutputRetryMiddleware(AgentMiddleware):
"""Retries model calls when structured output parsing fails."""
def __init__(self, max_retries: int) -> None:
self.max_retries = max_retries
def wrap_model_call(
self, request: ModelRequest, handler: Callable[[ModelRequest], ModelResponse]
) -> ModelResponse:
for attempt in range(self.max_retries + 1):
try:
return handler(request)
except StructuredOutputError as exc:
if attempt == self.max_retries:
raise
ai_content = exc.ai_message.content
error_message = (
f"Your previous response was:\n{ai_content}\n\n"
f"Error: {exc}. Please try again with a valid response."
)
request.messages.append(HumanMessage(content=error_message))
```
The LLM shouldn't be seeing parameters it cannot control in the
ToolMessage error it gets when it invokes a tool with incorrect args.
This fixes the behavior within langchain to address immediate issue.
We may want to change the behavior in langchain_core as well to prevent
validation of injected arguments. But this would be done in a separate
change
added some noqas, this is a quick patch to support a bug uncovered in
the quickstart, will resolve fully depending on where we centralize
ToolNode stuff.
the fact that this was broken showcases that we need significantly
better test coverage, this is literally the most minimalistic usage of
this middleware there could be 😿
will document these two gotchas better for custom middleware
```py
from langchain.agents.middleware.shell_tool import ShellToolMiddleware
from langchain.agents import create_agent
agent = create_agent(model="openai:gpt-4",middleware = [ShellToolMiddleware()])
agent.invoke({"messages":[{"role": "user", "content": "hi"}]})
```
mostly #33520
also tacking on change to make sure we're only looking at client side
calls for the jump to end
---------
Co-authored-by: Nuno Campos <nuno@boringbits.io>
- Both middleware share the same implementation, the only difference is
one uses Claude's server-side tool definition, whereas the other one
uses a generic tool definition compatible with all models
- Implemented 3 execution policies (responsible for actually running the
shell process)
- HostExecutionPolicy runs the shell as subprocess, appropriate for
already sandboxed environments, eg when run inside a dedicated docker
container
- CodexSandboxExecutionPolicy runs the shell using the sandbox command
from the Codex CLI which implements sandboxing techniques for Linux and
Mac OS.
- DockerExecutionPolicy runs the shell inside a dedicated Docker
container for isolation.
- Implements all behaviours described in
https://docs.claude.com/en/docs/agents-and-tools/tool-use/bash-tool#handle-large-outputs
including timeouts, truncation, output redaction, etc
---------
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Adds special private helper to allow direct injection of `ToolRuntime`
in tools, plus adding guards for generic annotations w/ `get_origin`.
Went w/ the private helper so that we didn't change behavior for other
injected types.
To make migration easier, things are more backwards compat
Very minimal footprint here
Will need to upgrade migration guide and other docs w/ this change
Adds `ToolRetryMiddleware` to automatically retry failed tool calls with
configurable exponential backoff, exception filtering, and error
handling.
## Example
```python
from langchain.agents import create_agent
from langchain.agents.middleware import ToolRetryMiddleware
from langchain_openai import ChatOpenAI
# Retry up to 3 times with exponential backoff
retry = ToolRetryMiddleware(
max_retries=3,
initial_delay=1.0,
backoff_factor=2.0,
)
agent = create_agent(
model=ChatOpenAI(model="gpt-4"),
tools=[search_tool, database_tool],
middleware=[retry],
)
# Tool failures are automatically retried
result = agent.invoke({"messages": [{"role": "user", "content": "Search for AI news"}]})
```
For advanced usage with specific exception handling:
```python
from requests.exceptions import Timeout, HTTPError
def should_retry(exc: Exception) -> bool:
# Only retry on 5xx errors or timeouts
if isinstance(exc, HTTPError):
return 500 <= exc.response.status_code < 600
return isinstance(exc, Timeout)
retry = ToolRetryMiddleware(
max_retries=4,
retry_on=should_retry,
tools=["search_database"], # Only apply to specific tools
)
```
Largely:
- Remove explicit `"Default is x"` since new refs show default inferred
from sig
- Inline code (useful for eventual parsing)
- Fix code block rendering (indentations)
The one risk point that I can see here is that model + tool call
counting now occurs in the `after_model` hook which introduces order
dependency (what if you have HITL execute before this hook and we jump
early to `model`, for example).
This is something users can work around at the moment and we can
document. We could also introduce a priority concept to middleware.
