github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-05 14:43:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

community[patch]: allows using text-generation-inference /generate route with HuggingFaceEndpoint (#20100)

Browse Source 
- **Description:** allows to use the /generate route of
`text-generation-inference` with the `HuggingFaceEndpoint`
This commit is contained in:
Alexander Dicke 2024-04-30 05:09:55 +02:00 committed by GitHub
parent ea43c669f2
commit d7e12750df
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 8 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
libs/community/langchain_community/llms/huggingface_endpoint.py
View File

@ -258,7 +258,10 @@ class HuggingFaceEndpoint(LLM):
stream=False, stream=False,
task=self.task, task=self.task,
) )
response_text = json.loads(response.decode())[0]["generated_text"] try:
response_text = json.loads(response.decode())[0]["generated_text"]
except KeyError:
response_text = json.loads(response.decode())["generated_text"]
# Maybe the generation has stopped at one of the stop sequences: # Maybe the generation has stopped at one of the stop sequences:
# then we remove this stop sequence from the end of the generated text # then we remove this stop sequence from the end of the generated text
@ -289,7 +292,10 @@ class HuggingFaceEndpoint(LLM):
stream=False, stream=False,
task=self.task, task=self.task,
) )
response_text = json.loads(response.decode())[0]["generated_text"] try:
response_text = json.loads(response.decode())[0]["generated_text"]
except KeyError:
response_text = json.loads(response.decode())["generated_text"]
# Maybe the generation has stopped at one of the stop sequences: # Maybe the generation has stopped at one of the stop sequences:
# then remove this stop sequence from the end of the generated text # then remove this stop sequence from the end of the generated text