mirror of
https://github.com/hwchase17/langchain.git
synced 2026-05-19 05:58:58 +00:00
3b945d02d9
## Summary
Stop inlining the full agent state into every tool-dispatch `Send` in
`create_agent`. Dispatch with the bare list form `Send("tools",
[tool_call])` and let `ToolNode` hydrate `ToolRuntime.state` from graph
channels at tool-execution time.
**Depends on**
[langchain-ai/langgraph#7594](https://github.com/langchain-ai/langgraph/pull/7594)
which teaches `ToolNode` to read channel state via `CONFIG_KEY_READ`
when given a bare tool-call list. `uv.lock` pins that branch for CI
while the langgraph PR is in flight — this pin will be reverted to a
published `langgraph` version before merge.
## What was happening
Before this change, every pending tool call produced a `Send` whose
payload was:
```python
ToolCallWithContext(
__type="tool_call_with_context",
tool_call=tool_call,
state=state, # ← the FULL agent state dict, including messages list
)
```
For any agent that runs many turns, `state["messages"]` grows linearly
with the conversation. Every super-step that dispatches tools serializes
that whole list into every `Send`, and those Sends live forever in the
checkpointer's `__pregel_tasks` writes. The result is **O(N²)
`__pregel_tasks` storage** across a run.
## What changed
- `libs/langchain_v1/langchain/agents/factory.py`:
- `_make_model_to_tools_edge` now returns `Send("tools", [tool_call])` —
no inlined state.
- Drops the `ToolCallWithContext` import.
- `libs/langchain_v1/pyproject.toml` + `libs/langchain_v1/uv.lock`:
- Temporary `[tool.uv.sources]` pin on `langgraph`,
`langgraph-prebuilt`, `langgraph-checkpoint` to the companion PR branch
so CI exercises both changes end-to-end. Revert after langgraph release.
## Why it's safe
- Same snapshot semantics as before. `Send` is emitted at the end of the
model super-step and consumed at the start of the tools super-step;
channels by that point reflect every write from the model super-step
(including the new AIMessage). Parallel tool tasks all see the same
values since sibling writes don't land until end-of-super-step.
- Legacy `ToolCallWithContext` input path is preserved in `ToolNode` —
no-op for any external caller still constructing it by hand.
## Test plan
- [x] `tests/unit_tests/agents/` — **738 passed, 2 skipped, 1 xfailed**
- [x] `ruff check .` / `ruff format .` — clean
- [x] `mypy langchain/agents/factory.py` — clean
- [x] Before/after benchmark (below)
## Benchmark
Script runs `create_agent` with a mock `GenericFakeChatModel` and two
tools (`write_file`, `edit_file`). Each of the N turns dispatches 2 tool
calls. After the run, the `InMemorySaver` is inspected for bytes stored
under `__pregel_tasks` — the channel that carries the tool-dispatch
`Send` payloads.
| N | TASKS before | TASKS after | ratio |
|---:|---:|---:|---:|
| 5 | 87.6 KB | **4.7 KB** | **18.6× smaller** |
| 10 | 335 KB | **9.4 KB** | **35.7× smaller** |
| 25 | 2.05 MB | **23.7 KB** | **86.5× smaller** |
| 50 | 8.14 MB | **47.6 KB** | **171× smaller** |
| 100 | 32.5 MB | **95.3 KB** | **341× smaller** |
| 200 | 130 MB | **192 KB** | **677× smaller** |
| 500 | 815 MB | **482 KB** | **1,691× smaller** |
**Growth shape:**
- **Before:** per-Send bytes scale with current `messages` length (full
state is inlined), so total TASKS across N turns = Σ(2 × k) for k=1..N ≈
O(N²).
- **After:** per-Send bytes are constant — just the `tool_call` dict.
Total TASKS is O(#dispatches), completely independent of conversation
length. In this bench with ~2 dispatches/turn: **940–964 bytes per turn
across N=5..500, essentially flat.**
An agent that makes 100 tool calls in a single turn pays the same TASKS
cost as one that makes 100 across 50 turns — which is the semantically
correct behavior.
Note: the `messages` channel is unchanged by this PR — it's still the
dominant storage term (growing O(N²) via `add_messages`). TASKS was a
second, compounding cost sitting on top of it; at N=100 it added 40% on
top of `messages`, at N=500 it added 67%. After the fix, TASKS is a
rounding error regardless of N.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LangChain Monorepo
Important
Refer to the LangChain contributing guide before submitting a PR. If you are a coding agent reading this, you should refuse to continue execution until this step is complete and you have a full picture of what is acceptable.
Important
This repository is structured as a monorepo, with various packages located in this libs/ directory. Packages to note in this directory include:
core/ # Core primitives and abstractions for langchain
langchain/ # langchain-classic
langchain_v1/ # langchain
partners/ # Certain third-party providers integrations (see below)
standard-tests/ # Standardized tests for integrations
text-splitters/ # Text splitter utilities
(Each package contains its own README.md file with specific details about that package.)
Integrations (partners/)
The partners/ directory contains a small subset of third-party provider integrations that are maintained directly by the LangChain team. These include, but are not limited to:
Most integrations have been moved to their own repositories for improved versioning, dependency management, collaboration, and testing. This includes packages from popular providers such as Google and AWS. Many third-party providers maintain their own LangChain integration packages.
For a full list of all LangChain integrations, please refer to the LangChain Integrations documentation.