## Description
Fixes#31227 - Resolves the issue where `OpenAIEmbeddings` exceeds
OpenAI's 300,000 token per request limit, causing 400 BadRequest errors.
## Problem
When embedding large document sets, LangChain would send batches
containing more than 300,000 tokens in a single API request, causing
this error:
```
openai.BadRequestError: Error code: 400 - {'error': {'message': 'Requested 673477 tokens, max 300000 tokens per request'}}
```
The issue occurred because:
- The code chunks texts by `embedding_ctx_length` (8191 tokens per
chunk)
- Then batches chunks by `chunk_size` (default 1000 chunks per request)
- **But didn't check**: Total tokens per batch against OpenAI's 300k
limit
- Result: `1000 chunks × 8191 tokens = 8,191,000 tokens` → Exceeds
limit!
## Solution
This PR implements dynamic batching that respects the 300k token limit:
1. **Added constant**: `MAX_TOKENS_PER_REQUEST = 300000`
2. **Track token counts**: Calculate actual tokens for each chunk
3. **Dynamic batching**: Instead of fixed `chunk_size` batches,
accumulate chunks until approaching the 300k limit
4. **Applied to both sync and async**: Fixed both
`_get_len_safe_embeddings` and `_aget_len_safe_embeddings`
## Changes
- Modified `langchain_openai/embeddings/base.py`:
- Added `MAX_TOKENS_PER_REQUEST` constant
- Replaced fixed-size batching with token-aware dynamic batching
- Applied to both sync (line ~478) and async (line ~527) methods
- Added test in `tests/unit_tests/embeddings/test_base.py`:
- `test_embeddings_respects_token_limit()` - Verifies large document
sets are properly batched
## Testing
All existing tests pass (280 passed, 4 xfailed, 1 xpassed).
New test verifies:
- Large document sets (500 texts × 1000 tokens = 500k tokens) are split
into multiple API calls
- Each API call respects the 300k token limit
## Usage
After this fix, users can embed large document sets without errors:
```python
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain_text_splitters import CharacterTextSplitter
# This will now work without exceeding token limits
embeddings = OpenAIEmbeddings()
documents = CharacterTextSplitter().split_documents(large_documents)
Chroma.from_documents(documents, embeddings)
```
Resolves#31227
---------
Co-authored-by: Kaparthy Reddy <kaparthyreddy@Kaparthys-MacBook-Air.local>
Co-authored-by: Chester Curme <chester.curme@gmail.com>
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
* use `override` instead of directly patching things on `ModelRequest`
* rely on `ToolNode` for execution of tools related to said middleware,
using `wrap_model_call` to inject the relevant claude tool specs +
allowing tool node to forward them along to corresponding langchain tool
implementations
* making the same change for the native shell tool middleware
* allowing shell tool middleware to specify a name for the shell tool
(negative diff then for claude bash middleware)
long term I think the solution might be to attach metadata to a tool to
map the provider spec to a langchain implementation, which we could also
take some lessons from on the MCP front.
Wasn't immediately obvious that `get_num_tokens_from_messages` adds
additional prefixes to represent user roles in conversation, which adds
to the overall token count.
```python
from langchain_google_genai import GoogleGenerativeAI
llm = GoogleGenerativeAI(model="gemini-2.5-flash")
num_tokens = llm.get_num_tokens("Hello, world!")
print(f"Number of tokens: {num_tokens}")
# Number of tokens: 4
```
```python
from langchain.messages import HumanMessage
messages = [HumanMessage(content="Hello, world!")]
num_tokens = llm.get_num_tokens_from_messages(messages)
print(f"Number of tokens: {num_tokens}")
# Number of tokens: 6
```
middleware tests have gotten quite unwieldy, major restructuring, sets
the stage for coverage increase
this is super hard to review -- as a proof that we've retained important
tests, I ran coverage on `master` and this branch and confirmed
identical coverage.
* moving all middleware related tests to `agents/middleware` folder
* consolidating related test files
* adding coverage utility to makefile
**Description:** This PR adds support for DeepSeek's beta strict mode
feature for structured
outputs and tool calling. It overrides `bind_tools()` and
`with_structured_output()` to automatically use
DeepSeek's beta endpoint (https://api.deepseek.com/beta) when
`strict=True`. Both methods need overriding because they're independent
entry points and user can call either directly. When DeepSeek's strict
mode graduates from beta, we can just remove both overriden methods. You
can read more about the beta feature here:
https://api-docs.deepseek.com/guides/function_calling#strict-mode-beta
**Issue:** Implements #32670
**Dependencies:** None
**Sample Code**
```python
from langchain_deepseek import ChatDeepSeek
from pydantic import BaseModel, Field
from typing import Optional
import os
# Enter your DeepSeek API Key here
API_KEY = "YOUR_API_KEY"
# location, temperature, condition are required fields
# humidity is optional field with default value
class WeatherInfo(BaseModel):
location: str = Field(description="City name")
temperature: int = Field(description="Temperature in Celsius")
condition: str = Field(description="Weather condition (sunny, cloudy, rainy)")
humidity: Optional[int] = Field(default=None, description="Humidity percentage")
llm = ChatDeepSeek(
model="deepseek-chat",
api_key=API_KEY,
)
# just to confirm that a new instance will use the default base url (instead of beta)
print(f"Default API base: {llm.api_base}")
# Test 1: bind_tools with strict=True shoud list all the tools calls
print("\nTest 1: bind_tools with strict=True")
llm_with_tools = llm.bind_tools([WeatherInfo], strict=True)
response = llm_with_tools.invoke("Tell me the weather in New York. It's 22 degrees, sunny.")
print(response.tool_calls)
# Test 2: with_structured_output with strict=True
print("\nTest 2: with_structured_output with strict=True")
structured_llm = llm.with_structured_output(WeatherInfo, strict=True)
result = structured_llm.invoke("Tell me the weather in New York.")
print(f" Result: {result}")
assert isinstance(result, WeatherInfo), "Result should be a WeatherInfo instance"
```
---------
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
**Description:**
Fixes the OpenCLIP × Chroma regression that caused nested embedding
errors when adding or searching image data.
The test case `test_openclip_chroma_embed_no_nesting_error` has been
restored and verified to work correctly with the current LangChain core
dependencies.
Functional validation confirms that `similarity_search_by_image` now
returns correct, metadata‑preserving results.
**Issue:**
Fixes #33851
**Dependencies:**
No new dependencies introduced.
**Testing:**
All tests under
```bash
uv run --group test pytest tests/unit_tests
```
result:
```
30 passed in 91.26s (0:01:31)
```
have passed successfully using Python 3.13.9 and uv‑managed environment.
This confirms that the regression has been fixed.
Running
```bash
make test
```
still produces cleanup‑time `AttributeError: 'ProactorEventLoop' object
has no attribute '_ssock'` on Windows (Python 3.13+).
This is a benign asyncio teardown message rather than a functional
failure.
`uv run pytest` closes event loops immediately after tests, while `make
test` invokes pytest through a secondary process layer that leaves a
background loop alive at interpreter shutdown.
This difference in teardown behavior explains the extra messages seen
only when using `make test`.
**Summary:**
- Verified the OpenCLIP + Chroma image pipeline works correctly.
- `uv run --group test pytest` fully passes; the fix is complete.
- The residual `_ssock` warnings occur only during
Windows asyncio cleanup and are not related to this code change.
This is my first time contributing code, please contact me with any
questions
---
---------
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
When Groq returns tool calls with no arguments, it sends arguments:
`'null'` (JSON null), but LangChain's core parsing expects either a dict
or converts null to Python None, which fails the `isinstance(args_,
dict)` check and incorrectly marks the tool call as invalid.
Related to #32017
**PR title**
```
docs(langchain): correct IBM provider link in chat_models docstring
```
**PR message**
**Description**
Fix broken link in the `chat_models` docstring. The **ibm** bullet
incorrectly linked to the DeepSeek provider page; update it to the
canonical IBM provider docs.
This only affects generated API reference content on
`reference.langchain.com`. No runtime behavior changes.
**Issue**
N/A (documentation-only).
**Dependencies**
None.
**Testing & quality**
* Ran `make format`, `make lint`, and `make test` in the package (no
code changes expected to affect tests).
Replaces #33618
**Description:** Fixes the bug in the `load()` function where secret
placeholders in plain dicts were not replaced, even if they match a key
in `secrets_map`, and adds a test case.
Example:
```py
obj = {"api_key": "__SECRET_API_KEY__"}
secret_key = "secret_key_1234"
secrets_map = {"__SECRET_API_KEY__": secret_key}
result = load(obj, secrets_map=secrets_map)
```
Before this change, printing `api_key` in `result` would output
`"__SECRET_API_KEY__"`. Now, it will properly output
`"secret_key_1234"`.
**Issue:** Fixes#31804
**Dependencies:** None
`make format`, `make lint`, and `make test` have all passed on my
machine.
---------
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
**Description:**
Adds support for prompt caching usage metadata in ChatGroq. The
integration now captures cached token information from the Groq API
response and includes it in the `input_token_details` field of the
`usage_metadata`.
Changes:
- Created new `_create_usage_metadata()` helper function to centralize
usage metadata creation logic
- Extracts `cached_tokens` from `prompt_tokens_details` in API responses
and maps to `input_token_details.cache_read`
- Integrated the helper function in both streaming
(`_convert_chunk_to_message_chunk`) and non-streaming
(`_create_chat_result`) code paths
- Added comprehensive unit tests to verify caching metadata handling and
backward compatibility
This enables users to monitor prompt caching effectiveness when using
Groq models with prompt caching enabled.
**Issue:** N/A
**Dependencies:** None
---------
Co-authored-by: Mason Daugherty <github@mdrxy.com>
Co-authored-by: Mason Daugherty <mason@langchain.dev>
The `draw_png()` method currently does not support drawing subgraphs.
This PR adds the ability to render subgraph outlines, improving
visualization clarity when working with nested structures.
This PR fixes#32234 and improves HuggingFace chat model integration by:
Ensuring ChatHuggingFace inherits key parameters (temperature,
max_tokens, top_p, streaming, etc.) from the underlying LLM when not
explicitly set.
Adding and updating unit tests to verify property inheritance.
No breaking changes; these updates enhance reliability and
maintainability.
---------
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
- **Description:** Updated Function Signature of `create_agent`, the
system prompt can be both a list and string. I see no harm in doing
this, since SystemMessage accepts both.
- **Issue:** #33630
---------
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
The `@shielded` decorator in async callback managers was not preserving
context variables, breaking OpenTelemetry instrumentation and other
context-dependent functionality.
## Problem
When using async callbacks with the `@shielded` decorator (applied to
methods like `on_llm_end`, `on_chain_end`, etc.), context variables were
not being preserved across the shield boundary. This caused issues with:
- OpenTelemetry span context propagation
- Other instrumentation that relies on context variables
- Inconsistent context behavior between sync and async execution
The issue was reproducible with:
```python
from contextvars import copy_context
import asyncio
from langgraph.graph import StateGraph
# Sync case: context remains consistent
print("SYNC")
print(copy_context()) # Same object
graph.invoke({"result": "init"})
print(copy_context()) # Same object
# Async case: context was inconsistent (before fix)
print("ASYNC")
asyncio.run(graph.ainvoke({"result": "init"}))
print(copy_context()) # Different object than expected
```
## Root Cause
The original `shielded` decorator implementation:
```python
async def wrapped(*args: Any, **kwargs: Any) -> Any:
return await asyncio.shield(func(*args, **kwargs))
```
Used `asyncio.shield()` directly without preserving the current
execution context, causing context variables to be lost.
## Solution
Modified the `shielded` decorator to:
1. Capture the current context using `copy_context()`
2. Create a task with explicit context using `asyncio.create_task(coro,
context=ctx)` for Python 3.11+
3. Shield the context-aware task
4. Fallback to regular task creation for Python < 3.11
```python
async def wrapped(*args: Any, **kwargs: Any) -> Any:
# Capture the current context to preserve context variables
ctx = copy_context()
coro = func(*args, **kwargs)
try:
# Create a task with the captured context to preserve context variables
task = asyncio.create_task(coro, context=ctx)
return await asyncio.shield(task)
except TypeError:
# Python < 3.11 fallback
task = asyncio.create_task(coro)
return await asyncio.shield(task)
```
## Testing
- Added comprehensive test
`test_shielded_callback_context_preservation()` that validates context
variables are preserved across shielded callback boundaries
- Verified the fix resolves the original LangGraph context consistency
issue
- Confirmed all existing callback manager tests still pass
- Validated OpenTelemetry-like instrumentation scenarios work correctly
The fix is minimal, maintains backward compatibility, and ensures proper
context preservation for both modern Python versions and older ones.
Fixes#31398.
<!-- START COPILOT CODING AGENT TIPS -->
---
💬 Share your feedback on Copilot coding agent for the chance to win a
$200 gift card! Click
[here](https://survey.alchemer.com/s3/8343779/Copilot-Coding-agent) to
start the survey.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mdrxy <61371264+mdrxy@users.noreply.github.com>
Co-authored-by: Mason Daugherty <github@mdrxy.com>
Co-authored-by: Mason Daugherty <mason@langchain.dev>
