github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-21 14:18:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

langchain_openai[patch]: Invoke callback prior to yielding token (#16909)

Browse Source 
All models should be calling the callback for new token prior to
yielding the token.

Not doing this can cause callbacks for downstream steps to be called
prior to the callback for the new token; causing issues in
astream_events APIs and other things that depend in callback ordering
being correct.

We need to make this change for all chat models.
This commit is contained in:
Eugene Yurtsev 2024-02-01 16:43:10 -08:00 committed by GitHub
parent b1a847366c
commit a265878d71
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
libs/partners/openai/langchain_openai/chat_models/base.py
View File

@ -424,9 +424,9 @@ class ChatOpenAI(BaseChatModel):
chunk = ChatGenerationChunk( chunk = ChatGenerationChunk(
message=chunk, generation_info=generation_info or None message=chunk, generation_info=generation_info or None
) )
yield chunk
if run_manager: if run_manager:
run_manager.on_llm_new_token(chunk.text, chunk=chunk, logprobs=logprobs) run_manager.on_llm_new_token(chunk.text, chunk=chunk, logprobs=logprobs)
yield chunk
def _generate( def _generate(
self, self,
@ -516,11 +516,11 @@ class ChatOpenAI(BaseChatModel):
chunk = ChatGenerationChunk( chunk = ChatGenerationChunk(
message=chunk, generation_info=generation_info or None message=chunk, generation_info=generation_info or None
) )
yield chunk
if run_manager: if run_manager:
await run_manager.on_llm_new_token( await run_manager.on_llm_new_token(
token=chunk.text, chunk=chunk, logprobs=logprobs token=chunk.text, chunk=chunk, logprobs=logprobs
) )
yield chunk
async def _agenerate( async def _agenerate(
self, self,