mirror of
https://github.com/hwchase17/langchain.git
synced 2026-07-12 02:55:21 +00:00
9636d899ea
Plumb an explicit resource-lifecycle contract through `BaseChatModel`, `RunnableWithFallbacks`, and the two largest partner integrations. Adds an opt-in `FallbackLatch` so `with_fallbacks(...)` can short-circuit the primary after a failure. Motivation: provider SDKs (`anthropic`, `openai`) back their clients with httpx connection pools that the SDKs only release best-effort from `__del__` — `asyncio.get_running_loop().create_task(self.aclose())` with a bare `except Exception: pass`. Long-lived workers that construct chat models per request (multi-tenant LangGraph deployments, agents-as-services) silently accumulate pools and leak memory + file descriptors. The fix today requires reaching into private attributes (`_async_client`, `root_async_client`, ...) on each provider. This PR makes teardown a first-class part of the chat-model API. ## langchain-core - `BaseChatModel.close()` / `aclose()` — default no-ops that subclasses override. `aclose()` dispatches to `close()` so async teardown works for sync-only subclasses. Adds `__enter__`/`__exit__`/`__aenter__`/ `__aexit__` so models can be used as context managers. - `RunnableWithFallbacks.close()` / `aclose()` — walks `runnable` and `fallbacks`, calling each one's lifecycle method. Per-runnable failures are suppressed so one bad close doesn't prevent the others from running. - `FallbackLatch` + `with_fallbacks(..., latch=...)` — opt-in circuit-breaker: once the primary raises a handled exception, latch trips and subsequent calls (on this wrapper, or any wrapper sharing the same latch instance) skip the primary. Useful when a primary failure is unlikely to recover within the wrapper's lifetime — wrong API key, sustained outage — so the default `try-primary-on-every-call` doesn't waste a round-trip on every retry. `latch.reset()` re-enables the primary. The latch propagates through `__getattr__` rebinds (e.g. `wrapper.bind_tools([...])`) so tool-bound and bare wrappers share one circuit. - Default-latch behaviour is unchanged: passing no `latch` retains the existing "retry primary on every call" semantics. ## langchain-anthropic - `ChatAnthropic.close()` / `aclose()` — closes `_client` (sync) and `_async_client` (async). Both are `cached_property` slots; guarded via `__dict__` so we don't materialize an uninstantiated cached client just to immediately close it. Idempotent. ## langchain-openai - `BaseChatOpenAI.close()` / `aclose()` — closes `root_client` and `root_async_client`, then clears the corresponding `client` / `async_client` attributes so the model can't be used after teardown. Idempotent. Tolerates the API-key-missing case where one client is `None`. Note: `BaseChatOpenAI`'s eager construction of both sync + async clients in its `model_validator` (even for async-only use) is a related inefficiency but not addressed here — it's a fixed per-instance cost rather than the per-request leak that `aclose()` solves. ## Tests - 7 new latch + propagation tests in `test_fallbacks.py` - 4 new lifecycle tests in `test_base.py` for `BaseChatModel` - 5 new tests in `test_chat_models.py` for `ChatAnthropic` - 5 new tests in `test_base.py` for `BaseChatOpenAI` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>