mirror of
https://github.com/hwchase17/langchain.git
synced 2026-06-09 10:17:00 +00:00
4000c22376
> [!IMPORTANT] > **Behavior change on upgrade — minor bump (`1.1.16` → `1.2.0`).** > > Streaming calls now raise `StreamChunkTimeoutError` (a `TimeoutError` subclass — existing `except TimeoutError:` / `except asyncio.TimeoutError:` handlers catch it) after 120s of content silence instead of hanging forever. Opt out with `stream_chunk_timeout=None` or `LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S=0`. > > Kernel-level TCP keepalive / `TCP_USER_TIMEOUT` are applied via a custom `httpx` transport. `httpx` disables its env-proxy auto-detection (`HTTP_PROXY` / `HTTPS_PROXY` / `ALL_PROXY` / `NO_PROXY` and macOS/Windows system proxy) whenever a transport is supplied, so to avoid silently breaking enterprise proxy users, `ChatOpenAI` now detects the "proxy-env-shadow" shape at construction and **skips the custom transport entirely** when **all** of these hold: > > - `http_socket_options` left at default (`None`) > - No `http_client` or `http_async_client` supplied > - No `openai_proxy` supplied > - A proxy env var / system proxy is visible to httpx > > On that shape the instance falls back to pre-PR behavior and env-proxy auto-detection still applies. A one-time `INFO` records the bypass. > > Users who explicitly set `http_socket_options=[...]` alongside an env proxy still get the shadowed behavior with a one-time `WARNING` log — they opted in. Full opt-outs below. --- Streaming chat completions can hang forever when the underlying TCP connection silently dies mid-stream (idle NAT/LB timeouts, sandboxed runtimes killing long-lived connections, peer gone without a FIN or RST). httpx's read timeout doesn't help here because it's reset by any bytes arriving on the socket, including OpenAI's SSE keepalive comments, so a stream that's quiet on content but still producing keepalives looks alive forever. This PR adds two knobs to `ChatOpenAI`, both on by default with opt-outs: - `stream_chunk_timeout` (default 120s): wraps the async streaming iterator in `asyncio.wait_for` per chunk. Measures the gap between *parsed* SSE chunks, so keepalives don't reset it. Fires on genuine content silence and raises `StreamChunkTimeoutError` — a `TimeoutError` subclass carrying `timeout_s`, `model_name`, and `chunks_received` as structured attributes (mirrored in the WARNING log's `extra=`) for alerting without message-regex. Override with the kwarg or `LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S`. - `http_socket_options`: applies `SO_KEEPALIVE` + `TCP_KEEPIDLE` / `TCP_KEEPINTVL` / `TCP_KEEPCNT` + `TCP_USER_TIMEOUT` on Linux (macOS equivalents where available). On platforms missing some options, they're dropped silently and the remaining set still does useful work. Pool limits are set explicitly on the custom transport to mirror the `openai` SDK — without that, passing `transport=` to `httpx.AsyncClient` silently shrinks the connection pool. ## Behavior change The default-shape proxy-env bypass (above) covers the common enterprise case. Beyond that: - Connections that would previously have hung forever will now error out via `StreamChunkTimeoutError`. - Users who explicitly opt into `http_socket_options` while also relying on env proxies will see a one-time `WARNING` and lose env-proxy auto-detection — the custom transport shadows it. This is the original shipped behavior, retained for anyone who *wants* socket tuning on top of an env-proxied setup. Full opt-outs: - `stream_chunk_timeout=None` or `LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S=0` - `http_socket_options=()` or `LANGCHAIN_OPENAI_TCP_KEEPALIVE=0` - Supply your own `http_client` **and** `http_async_client`. `http_socket_options` is applied per side: passing only one still leaves the other side's default builder getting socket options. Supply both (or combine with `http_socket_options=()`) to take full control. Unparseable or negative values for the `LANGCHAIN_OPENAI_*` env vars fall back to the default with a `WARNING` log rather than silently being accepted, so a misconfigured environment still boots but the fallback is discoverable. --------- Co-authored-by: Mason Daugherty <github@mdrxy.com> Co-authored-by: Mason Daugherty <mason@langchain.dev>
19 lines
560 B
Python
19 lines
560 B
Python
"""Module for OpenAI integrations."""
|
|
|
|
from langchain_openai.chat_models import AzureChatOpenAI, ChatOpenAI
|
|
from langchain_openai.chat_models._client_utils import StreamChunkTimeoutError
|
|
from langchain_openai.embeddings import AzureOpenAIEmbeddings, OpenAIEmbeddings
|
|
from langchain_openai.llms import AzureOpenAI, OpenAI
|
|
from langchain_openai.tools import custom_tool
|
|
|
|
__all__ = [
|
|
"AzureChatOpenAI",
|
|
"AzureOpenAI",
|
|
"AzureOpenAIEmbeddings",
|
|
"ChatOpenAI",
|
|
"OpenAI",
|
|
"OpenAIEmbeddings",
|
|
"StreamChunkTimeoutError",
|
|
"custom_tool",
|
|
]
|