mirror of
https://github.com/hwchase17/langchain.git
synced 2026-06-09 10:17:00 +00:00
docs(langchain): docstrings for summariziation middleware types (#34158)
improving devx :)
This commit is contained in:
@@ -58,10 +58,65 @@ _DEFAULT_FALLBACK_MESSAGE_COUNT = 15
|
|||||||
_SEARCH_RANGE_FOR_TOOL_PAIRS = 5
|
_SEARCH_RANGE_FOR_TOOL_PAIRS = 5
|
||||||
|
|
||||||
ContextFraction = tuple[Literal["fraction"], float]
|
ContextFraction = tuple[Literal["fraction"], float]
|
||||||
|
"""Fraction of model's maximum input tokens.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
To specify 50% of the model's max input tokens:
|
||||||
|
|
||||||
|
```python
|
||||||
|
("fraction", 0.5)
|
||||||
|
```
|
||||||
|
"""
|
||||||
|
|
||||||
ContextTokens = tuple[Literal["tokens"], int]
|
ContextTokens = tuple[Literal["tokens"], int]
|
||||||
|
"""Absolute number of tokens.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
To specify 3000 tokens:
|
||||||
|
|
||||||
|
```python
|
||||||
|
("tokens", 3000)
|
||||||
|
```
|
||||||
|
"""
|
||||||
|
|
||||||
ContextMessages = tuple[Literal["messages"], int]
|
ContextMessages = tuple[Literal["messages"], int]
|
||||||
|
"""Absolute number of messages.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
To specify 50 messages:
|
||||||
|
|
||||||
|
```python
|
||||||
|
("messages", 50)
|
||||||
|
```
|
||||||
|
"""
|
||||||
|
|
||||||
ContextSize = ContextFraction | ContextTokens | ContextMessages
|
ContextSize = ContextFraction | ContextTokens | ContextMessages
|
||||||
|
"""Union type for context size specifications.
|
||||||
|
|
||||||
|
Can be either:
|
||||||
|
|
||||||
|
- [`ContextFraction`][langchain.agents.middleware.summarization.ContextFraction]: A
|
||||||
|
fraction of the model's maximum input tokens.
|
||||||
|
- [`ContextTokens`][langchain.agents.middleware.summarization.ContextTokens]: An absolute
|
||||||
|
number of tokens.
|
||||||
|
- [`ContextMessages`][langchain.agents.middleware.summarization.ContextMessages]: An
|
||||||
|
absolute number of messages.
|
||||||
|
|
||||||
|
Depending on use with `trigger` or `keep` parameters, this type indicates either
|
||||||
|
when to trigger summarization or how much context to retain.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
```python
|
||||||
|
# ContextFraction
|
||||||
|
context_size: ContextSize = ("fraction", 0.5)
|
||||||
|
|
||||||
|
# ContextTokens
|
||||||
|
context_size: ContextSize = ("tokens", 3000)
|
||||||
|
|
||||||
|
# ContextMessages
|
||||||
|
context_size: ContextSize = ("messages", 50)
|
||||||
|
```
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
class SummarizationMiddleware(AgentMiddleware):
|
class SummarizationMiddleware(AgentMiddleware):
|
||||||
@@ -89,19 +144,48 @@ class SummarizationMiddleware(AgentMiddleware):
|
|||||||
model: The language model to use for generating summaries.
|
model: The language model to use for generating summaries.
|
||||||
trigger: One or more thresholds that trigger summarization.
|
trigger: One or more thresholds that trigger summarization.
|
||||||
|
|
||||||
Provide a single `ContextSize` tuple or a list of tuples, in which case
|
Provide a single
|
||||||
summarization runs when any threshold is breached.
|
[`ContextSize`][langchain.agents.middleware.summarization.ContextSize]
|
||||||
|
tuple or a list of tuples, in which case summarization runs when any
|
||||||
|
threshold is met.
|
||||||
|
|
||||||
Examples: `("messages", 50)`, `("tokens", 3000)`, `[("fraction", 0.8),
|
!!! example
|
||||||
("messages", 100)]`.
|
|
||||||
|
```python
|
||||||
|
# Trigger summarization when 50 messages is reached
|
||||||
|
("messages", 50)
|
||||||
|
|
||||||
|
# Trigger summarization when 3000 tokens is reached
|
||||||
|
("tokens", 3000)
|
||||||
|
|
||||||
|
# Trigger summarization either when 80% of model's max input tokens
|
||||||
|
# is reached or when 100 messages is reached (whichever comes first)
|
||||||
|
[("fraction", 0.8), ("messages", 100)]
|
||||||
|
```
|
||||||
|
|
||||||
|
See [`ContextSize`][langchain.agents.middleware.summarization.ContextSize]
|
||||||
|
for more details.
|
||||||
keep: Context retention policy applied after summarization.
|
keep: Context retention policy applied after summarization.
|
||||||
|
|
||||||
Provide a `ContextSize` tuple to specify how much history to preserve.
|
Provide a [`ContextSize`][langchain.agents.middleware.summarization.ContextSize]
|
||||||
|
tuple to specify how much history to preserve.
|
||||||
|
|
||||||
Defaults to keeping the most recent 20 messages.
|
Defaults to keeping the most recent `20` messages.
|
||||||
|
|
||||||
Examples: `("messages", 20)`, `("tokens", 3000)`, or
|
Does not support multiple values like `trigger`.
|
||||||
`("fraction", 0.3)`.
|
|
||||||
|
!!! example
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Keep the most recent 20 messages
|
||||||
|
("messages", 20)
|
||||||
|
|
||||||
|
# Keep the most recent 3000 tokens
|
||||||
|
("tokens", 3000)
|
||||||
|
|
||||||
|
# Keep the most recent 30% of the model's max input tokens
|
||||||
|
("fraction", 0.3)
|
||||||
|
```
|
||||||
token_counter: Function to count tokens in messages.
|
token_counter: Function to count tokens in messages.
|
||||||
summary_prompt: Prompt template for generating summaries.
|
summary_prompt: Prompt template for generating summaries.
|
||||||
trim_tokens_to_summarize: Maximum tokens to keep when preparing messages for
|
trim_tokens_to_summarize: Maximum tokens to keep when preparing messages for
|
||||||
|
|||||||
Reference in New Issue
Block a user