mirror of
https://github.com/hwchase17/langchain.git
synced 2025-08-31 18:38:48 +00:00
community[patch]: huggingface hub character removal bug fix (#16233)
- **Description:** Some text-generation models on huggingface repeat the prompt in their generated response, but not all do! The tests use "gpt2" which DOES repeat the prompt and as such, the HuggingFaceHub class is hardcoded to remove the first few characters of the response (to match the len(prompt)). However, if you are using a model (such as the very popular "meta-llama/Llama-2-7b-chat-hf") that DOES NOT repeat the prompt in it's generated text, then the beginning of the generated text will be cut off. This code change fixes that bug by first checking whether the prompt is repeated in the generated response and removing it conditionally. - **Issue:** #16232 - **Dependencies:** N/A - **Twitter handle:** N/A
This commit is contained in:
@@ -112,8 +112,10 @@ class HuggingFaceHub(LLM):
|
||||
if "error" in response:
|
||||
raise ValueError(f"Error raised by inference API: {response['error']}")
|
||||
if self.client.task == "text-generation":
|
||||
# Text generation return includes the starter text.
|
||||
text = response[0]["generated_text"][len(prompt) :]
|
||||
# Text generation sometimes return includes the starter text.
|
||||
text = response[0]["generated_text"]
|
||||
if text.startswith(prompt):
|
||||
text = response[0]["generated_text"][len(prompt) :]
|
||||
elif self.client.task == "text2text-generation":
|
||||
text = response[0]["generated_text"]
|
||||
elif self.client.task == "summarization":
|
||||
|
Reference in New Issue
Block a user