github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-21 06:14:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

update(llms/TGI): Allow None as temperature value (#10212)

Browse Source 
Text Generation Inference's client permits the use of a None temperature
as seen
[here](033230ae66/clients/python/text_generation/client.py (L71C9-L71C20)).
While I haved dived into TGI's server code and don't know about the
implications of using None as a temperature setting, I think we should
grant users the option to pass None as a temperature parameter to TGI.
This commit is contained in:
Junlin Zhou 2023-09-05 16:07:57 +08:00 committed by GitHub
parent be152b6a56
commit 8b95dabfe3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
libs/langchain/langchain/llms/huggingface_text_gen_inference.py
View File

@ -65,7 +65,7 @@ class HuggingFaceTextGenInference(LLM):
typical_p: Optional[float] = 0.95 typical_p: Optional[float] = 0.95
"""Typical Decoding mass. See [Typical Decoding for Natural Language """Typical Decoding mass. See [Typical Decoding for Natural Language
Generation](https://arxiv.org/abs/2202.00666) for more information.""" Generation](https://arxiv.org/abs/2202.00666) for more information."""
temperature: float = 0.8 temperature: Optional[float] = 0.8
"""The value used to module the logits distribution.""" """The value used to module the logits distribution."""
repetition_penalty: Optional[float] = None repetition_penalty: Optional[float] = None
"""The parameter for repetition penalty. 1.0 means no penalty. """The parameter for repetition penalty. 1.0 means no penalty.