fix(core): extract usage metadata from serialized tracer message outputs (#35526)

Fixes missing `run.metadata.usage_metadata` population in
`LangChainTracer` for real LLM/chat traces following #34414

- Fix extraction to read usage from serialized tracer message shape:
`outputs.generations[*][*].message.kwargs.usage_metadata`
- Remove non-serialized direct message shape handling
(`message.usage_metadata`) from extractor to match real tracer output
path
- Clarify tracer docstrings around chat callback naming
(`on_chat_model_start` + shared `on_llm_end`) to reduce ambiguity

## Why

#34414 introduced usage duplication into `run.metadata.usage_metadata`,
but the extractor read `message.usage_metadata`.

In real tracer flow, messages are serialized with `dumpd(...)` during
run completion, so usage metadata lives under
`message.kwargs.usage_metadata`. Because of this mismatch, duplication
did not trigger in real traces.
This commit is contained in:
Mason Daugherty
2026-03-02 17:43:33 -05:00
committed by GitHub
parent d2c86df128
commit 61fd90a2f3
5 changed files with 207 additions and 35 deletions

View File

@@ -26,10 +26,11 @@ class ChatResult(BaseModel):
"""
llm_output: dict | None = None
"""For arbitrary LLM provider specific output.
"""For arbitrary model provider-specific output.
This dictionary is a free-form dictionary that can contain any information that the
provider wants to return. It is not standardized and is provider-specific.
provider wants to return. It is not standardized and keys may vary by provider and
over time.
Users should generally avoid relying on this field and instead rely on accessing
relevant information from standardized fields present in `AIMessage`.

View File

@@ -38,10 +38,11 @@ class LLMResult(BaseModel):
"""
llm_output: dict | None = None
"""For arbitrary LLM provider specific output.
"""For arbitrary model provider-specific output.
This dictionary is a free-form dictionary that can contain any information that the
provider wants to return. It is not standardized and is provider-specific.
provider wants to return. It is not standardized and keys may vary by provider and
over time.
Users should generally avoid relying on this field and instead rely on accessing
relevant information from standardized fields present in AIMessage.