mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-13 13:36:15 +00:00
6e2f46d04c
## Overview Adding new `AgentMiddleware` primitive that supports `before_model`, `after_model`, and `prepare_model_request` hooks. This is very exciting! It makes our `create_agent` prebuilt much more extensible + capable. Still in alpha and subject to change. This is different than the initial [implementation](https://github.com/langchain-ai/langgraph/tree/nc/25aug/agent) in that it: * Fills in gaps w/ missing features, for ex -- new structured output, optionality of tools + system prompt, sync and async model requests, provider builtin tools * Exposes private state extensions for middleware, enabling things like model call tracking, etc * Middleware can register tools * Uses a `TypedDict` for `AgentState` -- dataclass subclassing is tricky w/ required values + required decorators * Addition of `model_settings` to `ModelRequest` so that we can pass through things to bind (like cache kwargs for anthropic middleware) ## TODOs ### top prio - [x] add middleware support to existing agent - [x] top prio middlewares - [x] summarization node - [x] HITL - [x] prompt caching other ones - [x] model call limits - [x] tool calling limits - [ ] usage (requires output state) ### secondary prio - [x] improve typing for state updates from middleware (not working right now w/ simple `AgentUpdate` and `AgentJump`, at least in Python) - [ ] add support for public state (input / output modifications via pregel channel mods) -- to be tackled in another PR - [x] testing! ### docs See https://github.com/langchain-ai/docs/pull/390 - [x] high level docs about middleware - [x] summarization node - [x] HITL - [x] prompt caching ## open questions Lots of open questions right now, many of them inlined as comments for the short term, will catalog some more significant ones here. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>