Python's `or` operator treats `0` as falsy, so
`token_usage.get("total_tokens") or fallback` silently replaces a
provider-reported `total_tokens=0` with the computed sum of input +
output tokens. Providers can legitimately report zero tokens (e.g.,
cached responses, empty completions).
The same pattern exists in the dual-key lookups for
`input_tokens`/`output_tokens` in Groq and OpenRouter. While current
APIs don't return both key formats simultaneously (making the `or`-chain
functionally correct today), the semantics are still wrong; `0` should
not fall through to a fallback.
## Changes
- Replace `x.get(key) or fallback` with explicit `is not None` checks in
`_create_usage_metadata` across `langchain-openai`, `langchain-groq`,
and `langchain-openrouter` for `input_tokens`, `output_tokens`, and
`total_tokens`
- Fix a concrete bug in the `total_tokens` path: a provider-reported `0`
was silently replaced by the computed sum
- Harden dual-key lookups in Groq and OpenRouter to correctly preserve
zero values from the preferred key, should both key formats ever coexist
- Update OpenAI's single-key extraction for consistency — the old `or 0`
pattern happened to produce correct results (`0 or 0 == 0`) but was
semantically wrong
Add a `model` property to `ChatFireworks`, `ChatGroq`, and
`ChatOpenRouter` that returns `model_name`. These partners use
Pydantic's `Field(alias="model")` on `model_name`, which means
`instance.model` doesn't work as a read accessor after construction — it
raises an `AttributeError` or returns the field descriptor. `ChatOpenAI`
already has this property; this brings the remaining in-repo partners to
parity.
Use of the fixture `_base_vcr_config` is deprecated with alternative
function `base_vcr_config()`
This way:
* we don't need to import `_base_vcr_config` seen as unused (which leads
to ruff violations PLC0414 and F811)
* we don't need to make a copy since a new dict is created at each
function invocation
Co-authored-by: Mason Daugherty <mason@langchain.dev>
**Description:**
Adds support for prompt caching usage metadata in ChatGroq. The
integration now captures cached token information from the Groq API
response and includes it in the `input_token_details` field of the
`usage_metadata`.
Changes:
- Created new `_create_usage_metadata()` helper function to centralize
usage metadata creation logic
- Extracts `cached_tokens` from `prompt_tokens_details` in API responses
and maps to `input_token_details.cache_read`
- Integrated the helper function in both streaming
(`_convert_chunk_to_message_chunk`) and non-streaming
(`_create_chat_result`) code paths
- Added comprehensive unit tests to verify caching metadata handling and
backward compatibility
This enables users to monitor prompt caching effectiveness when using
Groq models with prompt caching enabled.
**Issue:** N/A
**Dependencies:** None
---------
Co-authored-by: Mason Daugherty <github@mdrxy.com>
Co-authored-by: Mason Daugherty <mason@langchain.dev>
- There was some ambiguous wording that has been updated to hopefully
clarify the functionality of `reasoning_format` in ChatGroq.
- Added support for `reasoning_effort`
- Added links to see models capable of `reasoning_format` and
`reasoning_effort`
- Other minor nits
Llama-3.1 started failing consistently with
> groq.BadRequestError: Error code: 400 - ***'error': ***'message':
"Failed to call a function. Please adjust your prompt. See
'failed_generation' for more details.", 'type': 'invalid_request_error',
'code': 'tool_use_failed', 'failed_generation':
'<function=brave_search>***"query": "Hello!"***</function>'***
- Test if models support forcing tool calls via `tool_choice`. If they
do, they should support
- `"any"` to specify any tool
- the tool name as a string to force calling a particular tool
- Add `tool_choice` to signature of `BaseChatModel.bind_tools` in core
- Deprecate `tool_choice_value` in standard tests in favor of a boolean
`has_tool_choice`
Will follow up with PRs in external repos (tested in AWS and Google
already).
The default model for `ChatGroq`, `"mixtral-8x7b-32768"`, is being
retired on March 20, 2025. Here we remove the default, such that model
names must be explicitly specified (being explicit is a good practice
here, and avoids the need for breaking changes down the line). This
change will be released in a minor version bump to 0.3.
This follows https://github.com/langchain-ai/langchain/pull/30161
(released in version 0.2.5), where we began generating warnings to this
effect.
Groq is retiring `mixtral-8x7b-32768`, which is currently the default
model for ChatGroq, on March 20. Here we emit a warning if the model is
not specified explicitly.
A version 0.3.0 will be released ahead of March 20 that removes the
default altogether.
Here we allow standard tests to specify a value for `tool_choice` via a
`tool_choice_value` property, which defaults to None.
Chat models [available in
Together](https://docs.together.ai/docs/chat-models) have issues passing
standard tool calling tests:
- llama 3.1 models currently [appear to rely on user-side
parsing](https://docs.together.ai/docs/llama-3-function-calling) in
Together;
- Mixtral-8x7B and Mistral-7B (currently tested) consistently do not
call tools in some tests.
Specifying tool_choice also lets us remove an existing `xfail` and use a
smaller model in Groq tests.
- Mixtral with Groq has started consistently failing tool calling tests.
Here we restrict testing to llama 3.1.
- `.schema` is deprecated in pydantic proper in favor of
`.model_json_schema`.
