mirror of
https://github.com/hwchase17/langchain.git
synced 2025-05-09 17:18:31 +00:00
a43df006de
**langchain_openai: Support of reasoning summary streaming** **Description:** OpenAI API now supports streaming reasoning summaries for reasoning models (o1, o3, o3-mini, o4-mini). More info about it: https://platform.openai.com/docs/guides/reasoning#reasoning-summaries It is supported only in Responses API (not Completion API), so you need to create LangChain Open AI model as follows to support reasoning summaries streaming: ``` llm = ChatOpenAI( model="o4-mini", # also o1, o3, o3-mini support reasoning streaming use_responses_api=True, # reasoning streaming works only with responses api, not completion api model_kwargs={ "reasoning": { "effort": "high", # also "low" and "medium" supported "summary": "auto" # some models support "concise" summary, some "detailed", but auto will always work } } ) ``` Now, if you stream events from llm: ``` async for event in llm.astream_events(prompt, version="v2"): print(event) ``` or ``` for chunk in llm.stream(prompt): print (chunk) ``` OpenAI API will send you new types of events: `response.reasoning_summary_text.added` `response.reasoning_summary_text.delta` `response.reasoning_summary_text.done` These events are new, so they were ignored. So I have added support of these events in function `_convert_responses_chunk_to_generation_chunk`, so reasoning chunks or full reasoning added to the chunk additional_kwargs. Example of how this reasoning summary may be printed: ``` async for event in llm.astream_events(prompt, version="v2"): if event["event"] == "on_chat_model_stream": chunk: AIMessageChunk = event["data"]["chunk"] if "reasoning_summary_chunk" in chunk.additional_kwargs: print(chunk.additional_kwargs["reasoning_summary_chunk"], end="") elif "reasoning_summary" in chunk.additional_kwargs: print("\n\nFull reasoning step summary:", chunk.additional_kwargs["reasoning_summary"]) elif chunk.content and chunk.content[0]["type"] == "text": print(chunk.content[0]["text"], end="") ``` or ``` for chunk in llm.stream(prompt): if "reasoning_summary_chunk" in chunk.additional_kwargs: print(chunk.additional_kwargs["reasoning_summary_chunk"], end="") elif "reasoning_summary" in chunk.additional_kwargs: print("\n\nFull reasoning step summary:", chunk.additional_kwargs["reasoning_summary"]) elif chunk.content and chunk.content[0]["type"] == "text": print(chunk.content[0]["text"], end="") ``` --------- Co-authored-by: Chester Curme <chester.curme@gmail.com> |
||
|---|---|---|
| .. | ||
| ai21 | ||
| anthropic | ||
| astradb | ||
| azure-dynamic-sessions | ||
| chroma | ||
| couchbase | ||
| deepseek | ||
| exa | ||
| fireworks | ||
| groq | ||
| huggingface | ||
| ibm | ||
| milvus | ||
| mistralai | ||
| mongodb | ||
| nomic | ||
| ollama | ||
| openai | ||
| perplexity | ||
| pinecone | ||
| prompty | ||
| qdrant | ||
| together | ||
| unstructured | ||
| voyageai | ||
| xai | ||