github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-07-20 11:31:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix output final text for HuggingFaceTextGenInference when streaming (#6211)

Browse Source 
The LLM integration
[HuggingFaceTextGenInference](https://github.com/hwchase17/langchain/blob/master/langchain/llms/huggingface_text_gen_inference.py)
already has streaming support.

However, when streaming is enabled, it always returns an empty string as
the final output text when the LLM is finished. This is because `text`
is instantiated with an empty string and never updated.

This PR fixes the collection of the final output text by concatenating
new tokens.
This commit is contained in:
Jan Pawellek 2023-06-19 02:01:15 +02:00 committed by GitHub
parent b3bccabc66
commit ea6a5b03e0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
langchain/llms/huggingface_text_gen_inference.py
View File

@ -169,4 +169,5 @@ class HuggingFaceTextGenInference(LLM):
if not token.special: if not token.special:
if text_callback: if text_callback:
text_callback(token.text) text_callback(token.text)
text += token.text
return text return text