ccurme 8119a7bc5c openai[patch]: support streaming token counts in AzureChatOpenAI (#30494) ... When OpenAI originally released `stream_options` to enable token usage during streaming, it was not supported in AzureOpenAI. It is now supported. Like the [OpenAI SDK](f66d2e6fdc/src/openai/resources/completions.py (L68)), ChatOpenAI does not return usage metadata during streaming by default (which adds an extra chunk to the stream). The OpenAI SDK requires users to pass `stream_options={"include_usage": True}`. ChatOpenAI implements a convenience argument `stream_usage: Optional[bool]`, and an attribute `stream_usage: bool = False`. Here we extend this to AzureChatOpenAI by moving the `stream_usage` attribute and `stream_usage` kwarg (on `_(a)stream`) from ChatOpenAI to BaseChatOpenAI. --- Additional consideration: we must be sensitive to the number of users using BaseChatOpenAI to interact with other APIs that do not support the `stream_options` parameter. Suppose OpenAI in the future updates the default behavior to stream token usage. Currently, BaseChatOpenAI only passes `stream_options` if `stream_usage` is True, so there would be no way to disable this new default behavior. To address this, we could update the `stream_usage` attribute to `Optional[bool] = None`, but this is technically a breaking change (as currently values of False are not passed to the client). IMO: if / when this change happens, we could accompany it with this update in a minor bump. --- Related previous PRs: - https://github.com/langchain-ai/langchain/pull/22628 - https://github.com/langchain-ai/langchain/pull/22854 - https://github.com/langchain-ai/langchain/pull/23552 --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>		2025-03-26 15:16:37 -04:00
..
chat_models	openai[patch]: support streaming token counts in AzureChatOpenAI (#30494)	2025-03-26 15:16:37 -04:00
embeddings	openai[patch]: add standard tests for embeddings (#28540)	2024-12-05 17:00:27 +00:00
llms	openai[patch]: fix Azure LLM test (#29302)	2025-01-19 17:25:42 +00:00
__init__.py
test_compile.py