Fixes#37533
---
`langchain-core` defines `ContextOverflowError` so that application code
can catch an over-long prompt the same way regardless of which provider
raised it. The Anthropic, OpenAI, and Fireworks integrations already
promote their provider-specific context-length errors to a subclass of
it, but `langchain-groq` did not: a context overflow there surfaced as a
plain `groq.BadRequestError`, so anyone relying on the shared exception
had to special-case Groq.
This closes that gap for Groq. It adds a `GroqContextOverflowError` (a
subclass of both `groq.BadRequestError` and `ContextOverflowError`) and
a small promoter, `_handle_groq_invalid_request`, wired into the sync
and async `generate` and `stream` paths. Because Groq's SDK mirrors
OpenAI's, the implementation follows the same shape as the existing
partners, and the promoted error keeps the original `response` and
`body` so existing catchers that inspect `.response.status_code` keep
working. Anything that already catches `groq.BadRequestError` is
unaffected, since the new class is still a `BadRequestError`.
One detail worth a reviewer's eye: Groq returns the overflow as a 400
whose JSON body carries `"code": "context_length_exceeded"`, but the
SDK's `BadRequestError` does not expose that code as an attribute. The
SDK does fold the full JSON body into the error message, so detection
primarily matches `context_length_exceeded` against the stringified
error, with `reduce the length` from the message as a secondary signal
and an attribute check kept as defensive cover in case a future SDK adds
`.code`. The unit tests construct the error exactly as the SDK does for
a 4xx response and assert promotion across all four call paths, that an
unrelated `BadRequestError` is left untouched, and that
`response`/`body` are preserved.
I scoped this to Groq and left Mistral as a follow-up: Mistral surfaces
errors as raw `httpx.HTTPStatusError` rather than a typed SDK error, and
I could not verify its exact context-overflow signal (status code plus
body `code`/`message`) against an authoritative source well enough to
assert it in a unit test without live API access, so I would rather not
guess at the shape.
---------
Co-authored-by: Mason Daugherty <github@mdrxy.com>
Co-authored-by: Mason Daugherty <mason@langchain.dev>
Groq's `openai/gpt-oss-20b` can return reasoning content instead of
honoring forced tool choice in live integration tests. The forced
tool-choice coverage now uses Groq's recommended `qwen/qwen3.6-27b`
replacement with reasoning disabled, while leaving gpt-oss coverage in
place for reasoning-specific tests.
## Changes
- Add a dedicated tool-calling model constant for forced tool-choice
integration coverage.
- Update sync and async tool-choice tests to use `qwen/qwen3.6-27b` with
`reasoning_effort="none"`.
- Remove stale xfail markers from streaming tool-call tests and assert
the normalized aggregated stream output, including preserved tool-call
IDs.
Groq's standard integration suite already treats several tool-calling
checks as flaky because provider behavior is inconsistent. The forced
`tool_choice` check now hits the same provider-side `tool_use_failed`
400 on generic prompts, so the Groq-specific suite marks that case as
expected flaky instead of failing scheduled integration runs.
## Changes
- Add a Groq-specific `test_tool_choice` override that retries and
xfails the shared standard test.
- Keep the rest of the Groq tool-calling coverage unchanged, including
the existing xfail/retry behavior for related standard tests.
Groq's API now exposes a fourth service tier, `performance` — their
highest tier, providing reliable low latency for the most critical
production applications. `ChatGroq.service_tier` only accepted
`on_demand`, `flex`, and `auto`, so users who wanted to route requests
to the performance tier had no type-safe way to do so.
This widens the `service_tier` `Literal` to include `performance` and
documents it alongside the existing tiers. The value is passed straight
through to the Groq SDK as a constrained enum, so no validation or
mapping logic changes were needed.
Reference: [Groq service tiers
documentation](https://console.groq.com/docs/service-tiers).
An integration test case was added to `test_setting_service_tier_class`
mirroring the existing per-tier assertions; it exercises a live request
and so runs only with a Groq API key.
Package-version trace metadata now uses the LangChain-owned
`metadata["lc_versions"]` convention instead of the user-owned
`metadata["versions"]` key. Metadata merging is narrowed so only
`lc_versions` accumulates nested package-version entries, while generic
nested metadata keeps normal last-writer-wins behavior.
## Changes
- Renamed `BaseLanguageModel._add_version()` trace metadata from
`versions` to `lc_versions`, including docstrings and the non-dict
replacement warning.
- Scoped `_merge_metadata_dicts()` nested-map accumulation to only
`lc_versions`; duplicate package entries remain last-writer-wins and
`lc_versions` mappings are copied defensively.
- Preserved user-owned `metadata["versions"]` semantics by keeping it
out of package-version tracking and generic nested metadata merging.
- Updated runnable snapshots and partner package metadata assertions
across Anthropic, DeepSeek, Fireworks, Groq, Hugging Face, MistralAI,
Ollama, OpenAI, OpenRouter, Perplexity, and xAI to expect `lc_versions`.
## Testing
- Added/adjusted core tests for `lc_versions` accumulation, duplicate
package overwrite behavior, non-dict `lc_versions` replacement,
defensive copying, and `metadata["versions"]` last-writer-wins behavior.
- Ran focused core and partner metadata tests plus Ruff checks for
changed areas.
Following on the heels of #35293
TODO:
- Packages outside of this repo (e.g. LiteLLM, Nvidia, Google, AWS)
---
## Summary
Surface partner package versions in `metadata.versions` on LangSmith
traces. Mirrors the JS SDK's `_addVersion()` pattern
([langchainjs#10106](https://github.com/langchain-ai/langchainjs/pull/10106)).
Each model constructor records its package version via `_add_version()`
on `BaseLanguageModel`. The version dict accumulates through the class
hierarchy — `langchain-core` is added in
`BaseLanguageModel.model_post_init`, `langchain-openai` in
`BaseChatOpenAI._set_openai_chat_version`, and each leaf partner in its
uniquely-named `model_validator`. Traces end up with:
```json
{
"metadata": {
"versions": {
"langchain-core": "1.4.5",
"langchain-openai": "1.3.0",
"langchain-xai": "1.2.2"
}
}
}
```
### Changes
- `BaseLanguageModel._add_version(pkg, version)` — appends to
`self.metadata["versions"]`; accepts any `Mapping` type; emits a warning
if a non-mapping value is found and replaced
- `BaseLanguageModel.model_post_init` — adds `langchain-core` version;
calls `super()` for MRO safety
- `_merge_metadata_dicts` — one-level-deep (non-recursive) merge for
nested dict metadata keys
- `CallbackManager.add_metadata` — uses `_merge_metadata_dicts` instead
of flat `dict.update()` so nested metadata dicts (like `versions`)
coexist rather than clobber
- `merge_configs` — uses `_merge_metadata_dicts` for config merging
**Partners:**
- Each now calls `self._add_version("langchain-<pkg>", __version__)`
### Design decisions
- **Constructor-based, not `_get_ls_params`-based** — versions flow
through `self.metadata` (local metadata on traces), not through
`LangSmithParams`. This matches JS and makes child-class version
inheritance automatic (no merge/clobber issues).
- **`versions` is local (non-inheritable) metadata** — `self.metadata`
is passed to `CallbackManager.configure` as `local_metadata`
(`add_metadata(..., inherit=False)`), so `versions` is attached **once
per chat-model run** and is **not** propagated to child runs or
duplicated onto every streaming chunk. This is intentionally the
opposite of the inheritable-per-chunk metadata that #36588 was reducing
for performance — `versions` does not regress that path.
- **`add_metadata` deep-merge is a correctness fix, not just for
versions** — previously `add_metadata`/`merge_configs` did a flat
top-level `dict.update`/spread, so any nested metadata dict baked into a
config (e.g. via `.with_config({"metadata": {...}})`) would be wholly
replaced when a caller also passed `metadata`. `_merge_metadata_dicts`
merges one level deep so user-provided `config.metadata.versions` and
model-set `versions` coexist instead of clobbering. The merge runs once
per `configure` (not per chunk), so it is off the streaming hot path.
- **One level deep only** — `_merge_metadata_dicts` is deliberately
*not* a recursive deep merge; values nested more than one level are
last-writer-wins. This covers the `versions` case without the
ambiguity/cost of arbitrary-depth merging.
- **Warn on non-dict `metadata["versions"]`** — if a user sets
`metadata={"versions": "some-string"}`, `_add_version` emits a warning
and replaces the value with the version dict rather than silently
discarding user data or crashing. This is a soft breaking change for
anyone who previously stored non-dict values at this key.
### Follow-ups (tracked separately, out of scope here)
- JS `mergeConfigs` still flat-spreads nested metadata, so
`metadata.versions` can still clobber on the JS side until an equivalent
deep-merge lands.
---
Made by [Open SWE](https://openswe.vercel.app)
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Python's `or` operator treats `0` as falsy, so
`token_usage.get("total_tokens") or fallback` silently replaces a
provider-reported `total_tokens=0` with the computed sum of input +
output tokens. Providers can legitimately report zero tokens (e.g.,
cached responses, empty completions).
The same pattern exists in the dual-key lookups for
`input_tokens`/`output_tokens` in Groq and OpenRouter. While current
APIs don't return both key formats simultaneously (making the `or`-chain
functionally correct today), the semantics are still wrong; `0` should
not fall through to a fallback.
## Changes
- Replace `x.get(key) or fallback` with explicit `is not None` checks in
`_create_usage_metadata` across `langchain-openai`, `langchain-groq`,
and `langchain-openrouter` for `input_tokens`, `output_tokens`, and
`total_tokens`
- Fix a concrete bug in the `total_tokens` path: a provider-reported `0`
was silently replaced by the computed sum
- Harden dual-key lookups in Groq and OpenRouter to correctly preserve
zero values from the preferred key, should both key formats ever coexist
- Update OpenAI's single-key extraction for consistency — the old `or 0`
pattern happened to produce correct results (`0 or 0 == 0`) but was
semantically wrong
Add a `model` property to `ChatFireworks`, `ChatGroq`, and
`ChatOpenRouter` that returns `model_name`. These partners use
Pydantic's `Field(alias="model")` on `model_name`, which means
`instance.model` doesn't work as a read accessor after construction — it
raises an `AttributeError` or returns the field descriptor. `ChatOpenAI`
already has this property; this brings the remaining in-repo partners to
parity.
Use of the fixture `_base_vcr_config` is deprecated with alternative
function `base_vcr_config()`
This way:
* we don't need to import `_base_vcr_config` seen as unused (which leads
to ruff violations PLC0414 and F811)
* we don't need to make a copy since a new dict is created at each
function invocation
Co-authored-by: Mason Daugherty <mason@langchain.dev>
**Description:**
Adds support for prompt caching usage metadata in ChatGroq. The
integration now captures cached token information from the Groq API
response and includes it in the `input_token_details` field of the
`usage_metadata`.
Changes:
- Created new `_create_usage_metadata()` helper function to centralize
usage metadata creation logic
- Extracts `cached_tokens` from `prompt_tokens_details` in API responses
and maps to `input_token_details.cache_read`
- Integrated the helper function in both streaming
(`_convert_chunk_to_message_chunk`) and non-streaming
(`_create_chat_result`) code paths
- Added comprehensive unit tests to verify caching metadata handling and
backward compatibility
This enables users to monitor prompt caching effectiveness when using
Groq models with prompt caching enabled.
**Issue:** N/A
**Dependencies:** None
---------
Co-authored-by: Mason Daugherty <github@mdrxy.com>
Co-authored-by: Mason Daugherty <mason@langchain.dev>
- There was some ambiguous wording that has been updated to hopefully
clarify the functionality of `reasoning_format` in ChatGroq.
- Added support for `reasoning_effort`
- Added links to see models capable of `reasoning_format` and
`reasoning_effort`
- Other minor nits
Llama-3.1 started failing consistently with
> groq.BadRequestError: Error code: 400 - ***'error': ***'message':
"Failed to call a function. Please adjust your prompt. See
'failed_generation' for more details.", 'type': 'invalid_request_error',
'code': 'tool_use_failed', 'failed_generation':
'<function=brave_search>***"query": "Hello!"***</function>'***
- Test if models support forcing tool calls via `tool_choice`. If they
do, they should support
- `"any"` to specify any tool
- the tool name as a string to force calling a particular tool
- Add `tool_choice` to signature of `BaseChatModel.bind_tools` in core
- Deprecate `tool_choice_value` in standard tests in favor of a boolean
`has_tool_choice`
Will follow up with PRs in external repos (tested in AWS and Google
already).
The default model for `ChatGroq`, `"mixtral-8x7b-32768"`, is being
retired on March 20, 2025. Here we remove the default, such that model
names must be explicitly specified (being explicit is a good practice
here, and avoids the need for breaking changes down the line). This
change will be released in a minor version bump to 0.3.
This follows https://github.com/langchain-ai/langchain/pull/30161
(released in version 0.2.5), where we began generating warnings to this
effect.
Groq is retiring `mixtral-8x7b-32768`, which is currently the default
model for ChatGroq, on March 20. Here we emit a warning if the model is
not specified explicitly.
A version 0.3.0 will be released ahead of March 20 that removes the
default altogether.
