github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-24 07:35:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

community[patch]: Invoke on_llm_new_token callback before yielding chunk (#24938)

Browse Source 
**Description**: Invoke on_llm_new_token callback before yielding chunk
in streaming mode
**Issue**:
[#16913](https://github.com/langchain-ai/langchain/issues/16913)
This commit is contained in:
Anneli Samuel 2024-08-01 19:39:04 +03:00 committed by GitHub
parent ff6274d32d
commit 2204d8cb7d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
libs/community/langchain_community/chat_models/mlx.py
View File

@ -186,9 +186,9 @@ class ChatMLX(BaseChatModel):
# yield text, if any # yield text, if any
if text: if text:
chunk = ChatGenerationChunk(message=AIMessageChunk(content=text)) chunk = ChatGenerationChunk(message=AIMessageChunk(content=text))
yield chunk
if run_manager: if run_manager:
run_manager.on_llm_new_token(text, chunk=chunk) run_manager.on_llm_new_token(text, chunk=chunk)
yield chunk
# break if stop sequence found # break if stop sequence found
if token == eos_token_id or (stop is not None and text in stop): if token == eos_token_id or (stop is not None and text in stop):