docs: Update a note about how to track azure openai's token usage when streaming (#30409)

- **Description:** Update a note about how to track azure openai's token
usage when streaming
  - **Issue:** #30390 
  - **Dependencies:** None
  - **Twitter handle:** None

---------

Co-authored-by: Chester Curme <chester.curme@gmail.com>
This commit is contained in:
axiangcoding
2025-03-22 02:18:50 +08:00
committed by GitHub
parent 1de7fa8f3a
commit 428de88398

View File

@@ -29,6 +29,31 @@
"%pip install -qU langchain-anthropic langchain-openai"
]
},
{
"cell_type": "markdown",
"id": "c71b676a",
"metadata": {},
"source": [
":::note A note on streaming with OpenAI\n",
"\n",
"OpenAI's Chat Completions API does not stream token usage statistics by default (see API reference\n",
"[here](https://platform.openai.com/docs/api-reference/completions/create#completions-create-stream_options)).\n",
"To recover token counts when streaming with `ChatOpenAI`, set `stream_usage=True` as\n",
"demonstrated in this guide.\n",
"\n",
"For `AzureChatOpenAI`, set `stream_options={\"include_usage\": True}` when calling\n",
"`.(a)stream`, or initialize with:\n",
"\n",
"```python\n",
"AzureChatOpenAI(\n",
" ...,\n",
" model_kwargs={\"stream_options\": {\"include_usage\": True}},\n",
")\n",
"```\n",
"\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "598ae1e2-a52d-4459-81fd-cdc68b06742a",