Following on the heels of #35293
TODO:
- Packages outside of this repo (e.g. LiteLLM, Nvidia, Google, AWS)
---
## Summary
Surface partner package versions in `metadata.versions` on LangSmith
traces. Mirrors the JS SDK's `_addVersion()` pattern
([langchainjs#10106](https://github.com/langchain-ai/langchainjs/pull/10106)).
Each model constructor records its package version via `_add_version()`
on `BaseLanguageModel`. The version dict accumulates through the class
hierarchy — `langchain-core` is added in
`BaseLanguageModel.model_post_init`, `langchain-openai` in
`BaseChatOpenAI._set_openai_chat_version`, and each leaf partner in its
uniquely-named `model_validator`. Traces end up with:
```json
{
"metadata": {
"versions": {
"langchain-core": "1.4.5",
"langchain-openai": "1.3.0",
"langchain-xai": "1.2.2"
}
}
}
```
### Changes
- `BaseLanguageModel._add_version(pkg, version)` — appends to
`self.metadata["versions"]`; accepts any `Mapping` type; emits a warning
if a non-mapping value is found and replaced
- `BaseLanguageModel.model_post_init` — adds `langchain-core` version;
calls `super()` for MRO safety
- `_merge_metadata_dicts` — one-level-deep (non-recursive) merge for
nested dict metadata keys
- `CallbackManager.add_metadata` — uses `_merge_metadata_dicts` instead
of flat `dict.update()` so nested metadata dicts (like `versions`)
coexist rather than clobber
- `merge_configs` — uses `_merge_metadata_dicts` for config merging
**Partners:**
- Each now calls `self._add_version("langchain-<pkg>", __version__)`
### Design decisions
- **Constructor-based, not `_get_ls_params`-based** — versions flow
through `self.metadata` (local metadata on traces), not through
`LangSmithParams`. This matches JS and makes child-class version
inheritance automatic (no merge/clobber issues).
- **`versions` is local (non-inheritable) metadata** — `self.metadata`
is passed to `CallbackManager.configure` as `local_metadata`
(`add_metadata(..., inherit=False)`), so `versions` is attached **once
per chat-model run** and is **not** propagated to child runs or
duplicated onto every streaming chunk. This is intentionally the
opposite of the inheritable-per-chunk metadata that #36588 was reducing
for performance — `versions` does not regress that path.
- **`add_metadata` deep-merge is a correctness fix, not just for
versions** — previously `add_metadata`/`merge_configs` did a flat
top-level `dict.update`/spread, so any nested metadata dict baked into a
config (e.g. via `.with_config({"metadata": {...}})`) would be wholly
replaced when a caller also passed `metadata`. `_merge_metadata_dicts`
merges one level deep so user-provided `config.metadata.versions` and
model-set `versions` coexist instead of clobbering. The merge runs once
per `configure` (not per chunk), so it is off the streaming hot path.
- **One level deep only** — `_merge_metadata_dicts` is deliberately
*not* a recursive deep merge; values nested more than one level are
last-writer-wins. This covers the `versions` case without the
ambiguity/cost of arbitrary-depth merging.
- **Warn on non-dict `metadata["versions"]`** — if a user sets
`metadata={"versions": "some-string"}`, `_add_version` emits a warning
and replaces the value with the version dict rather than silently
discarding user data or crashing. This is a soft breaking change for
anyone who previously stored non-dict values at this key.
### Follow-ups (tracked separately, out of scope here)
- JS `mergeConfigs` still flat-spreads nested metadata, so
`metadata.versions` can still clobber on the JS side until an equivalent
deep-merge lands.
---
Made by [Open SWE](https://openswe.vercel.app)
---------
Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
Test targets run with `-n auto`, which makes `pytest-benchmark` (present
via `langchain-tests`) auto-disable itself and emit a
`PytestBenchmarkWarning` once per xdist worker. Passing
`--benchmark-disable` turns the plugin off explicitly so the warning
never fires, matching what `core` and `langchain_v1` already do.
## Changes
- Add `--benchmark-disable` to the `-n auto` test targets across
`langchain` (unit) and 14 partner packages' integration targets:
`anthropic`, `chroma`, `deepseek`, `exa`, `fireworks`, `groq`,
`huggingface`, `mistralai`, `nomic`, `ollama`, `openai`, `openrouter`,
`qdrant`, `xai`.
- Deliberately excluded `text-splitters` and `model-profiles`: their
`test` group doesn't install `pytest-benchmark`, so the flag would fail
with `unrecognized arguments`. Verified by importing the plugin under
each package's actual dependency group before editing.
Standardize the `integration_tests` Makefile target across all 15
partner packages in `libs/partners/`, mirroring the deepagents
`libs/evals` pattern (`-v --tb=short`). Previously each partner had its
own ad-hoc flag stack (some missing `-n auto`, some with `-vvv`, others
with nothing), and every partner that used `-n auto` was emitting a
`PytestBenchmarkWarning` because `pytest-benchmark` is pulled in
transitively via `langchain-tests` even though no partner has benchmark
tests.
Python's `or` operator treats `0` as falsy, so
`token_usage.get("total_tokens") or fallback` silently replaces a
provider-reported `total_tokens=0` with the computed sum of input +
output tokens. Providers can legitimately report zero tokens (e.g.,
cached responses, empty completions).
The same pattern exists in the dual-key lookups for
`input_tokens`/`output_tokens` in Groq and OpenRouter. While current
APIs don't return both key formats simultaneously (making the `or`-chain
functionally correct today), the semantics are still wrong; `0` should
not fall through to a fallback.
## Changes
- Replace `x.get(key) or fallback` with explicit `is not None` checks in
`_create_usage_metadata` across `langchain-openai`, `langchain-groq`,
and `langchain-openrouter` for `input_tokens`, `output_tokens`, and
`total_tokens`
- Fix a concrete bug in the `total_tokens` path: a provider-reported `0`
was silently replaced by the computed sum
- Harden dual-key lookups in Groq and OpenRouter to correctly preserve
zero values from the preferred key, should both key formats ever coexist
- Update OpenAI's single-key extraction for consistency — the old `or 0`
pattern happened to produce correct results (`0 or 0 == 0`) but was
semantically wrong
Reduce CI log noise by suppressing pytest's per-test dot/verbose
streaming output. The `_test.yml` workflow now passes `PYTEST_EXTRA=-q`
to `make test`, which overrides the default verbosity with quiet mode —
failures still print in full, but the thousands of `.......` progress
lines are gone. Local `make test` is unaffected since `PYTEST_EXTRA`
defaults empty.
## Changes
- Add `PYTEST_EXTRA ?=` variable to all 21 package Makefiles and inject
it into each `test` target's pytest invocation
- Pass `PYTEST_EXTRA=-q` in `_test.yml` for both the main test step and
the min-version retest step
CI lint jobs use `uv run --all-groups` for all tools, but ruff doesn't
need dependency resolution — only mypy does. By splitting into
`UV_RUN_LINT` (ruff) and `UV_RUN_TYPE` (mypy), the CI-facing targets run
ruff with `--group lint` only, giving fast-fail feedback before mypy
triggers the full environment sync.
For packages where source code only conditionally imports heavy deps
(text-splitters, huggingface), `lint_package` also overrides
`UV_RUN_TYPE` to `--group lint --group typing`, skipping the ~3.5GB
`test_integration` download entirely. `lint_tests` keeps `--all-groups`
since test code legitimately imports those deps.
Additionally, `lint_imports.sh` was inconsistently wired — most packages
had the script but weren't calling it.
## Changes
**Makefile optimization**
- Introduce `UV_RUN_LINT` and `UV_RUN_TYPE` Make variables, both
defaulting to `uv run --all-groups`. For `lint_package` and
`lint_tests`, `UV_RUN_LINT` is overridden to `uv run --group lint` so
ruff runs instantly without syncing heavy deps
- For `text-splitters` and `huggingface`, override `UV_RUN_TYPE` on
`lint_package` to `uv run --group lint --group typing` — mypy runs
without downloading torch, CUDA, spacy, etc.
**mypy config for lean groups**
- Add `transformers` and `transformers.*` to `ignore_missing_imports` in
`text-splitters` pyproject.toml (conditional `try/except` import, same
treatment as existing `konlpy`/`nltk` entries)
- Add `torch`, `torch.*`, `langchain_community`, `langchain_community.*`
to `ignore_missing_imports` in `huggingface` pyproject.toml
- Add dual `# type: ignore[unreachable, unused-ignore]` in
`text-splitters/base.py` to handle the `PreTrainedTokenizerBase`
isinstance check that behaves differently depending on whether
transformers is installed
**lint_imports.sh consistency**
- Add `./scripts/lint_imports.sh` to the lint recipe in every package
that wasn't calling it (standard-tests, model-profiles, all 15
partners), and create the script for the two packages missing it
entirely (`model-profiles`, `openrouter`)
- Update all `lint_imports.sh` scripts to allow `from langchain.agents`
and `from langchain.tools` imports (legitimate v1 middleware
dependencies used by `langchain-anthropic` and `langchain-openai`)
