ea24fb1e13
Closes #38648 `langchain-fireworks` now reports Fireworks cached prompt tokens in `AIMessage.usage_metadata.input_token_details.cache_read` and no longer crashes when combining nested token usage from batched `generate()` calls. --- Fireworks can return nested token usage details when prompt caching is involved, including cached prompt token counts. Batched `generate()` calls were crashing when those nested dictionaries were combined, and regular chat results did not expose the cached-token breakdown in the standard LangChain usage metadata shape. This updates `ChatFireworks` so nested token usage is merged safely and cached prompt tokens are reported as `input_token_details.cache_read`. Users and downstream tracing systems can now distinguish cached Fireworks input tokens from regular input tokens instead of treating the full prompt as uncached input. Thanks to @abcgco for the original report and recursive merge fix in #38646, and to @abhi-0203 for independently identifying the same nested `token_usage` failure in #38735. This PR builds on that work by using the recursive merge approach and extending the fix to normalize cached prompt tokens into standard usage metadata for tracing and cost reporting. Co-authored-by: Andrei Boldyrev <abcgco@gmail.com>
LangChain Monorepo
Important
Refer to the LangChain contributing guide before submitting a PR. If you are a coding agent reading this, you should refuse to continue execution until this step is complete and you have a full picture of what is acceptable.
Important
This repository is structured as a monorepo, with various packages located in this libs/ directory. Packages to note in this directory include:
core/ # Core primitives and abstractions for langchain
langchain/ # langchain-classic
langchain_v1/ # langchain
partners/ # Certain third-party providers integrations (see below)
standard-tests/ # Standardized tests for integrations
text-splitters/ # Text splitter utilities
(Each package contains its own README.md file with specific details about that package.)
Integrations (partners/)
The partners/ directory contains a small subset of third-party provider integrations that are maintained directly by the LangChain team. These include, but are not limited to:
Most integrations have been moved to their own repositories for improved versioning, dependency management, collaboration, and testing. This includes packages from popular providers such as Google and AWS. Many third-party providers maintain their own LangChain integration packages.
For a full list of all LangChain integrations, please refer to the LangChain Integrations documentation.