github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-06 23:24:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

community: aligning ChatLiteLLM default parameters with litellm (#30360)

Browse Source 
**Description:**
Since `ChatLiteLLM` is forwarding most parameters to
`litellm.completion(...)`, there is no reason to set other default
values than the ones defined by `litellm`.

In the case of parameter 'n', it also provokes an issue when trying to
call a serverless endpoint on Azure, as it is considered an extra
parameter. So we need to keep it optional.

We can debate about backward compatibility of this change: in my
opinion, there should not be big issues since from my experience,
calling `litellm.completion()` without these parameters works fine.

**Issue:** 
- #29679 

**Dependencies:** None
This commit is contained in:
Florian Chappaz 2025-03-19 14:07:28 +01:00 committed by GitHub
parent 57ffacadd0
commit 07cb41ea9e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
libs/community/langchain_community/chat_models/litellm.py
View File

@ -259,7 +259,7 @@ class ChatLiteLLM(BaseChatModel):
organization: Optional[str] = None organization: Optional[str] = None
custom_llm_provider: Optional[str] = None custom_llm_provider: Optional[str] = None
request_timeout: Optional[Union[float, Tuple[float, float]]] = None request_timeout: Optional[Union[float, Tuple[float, float]]] = None
temperature: Optional[float] = 1 temperature: Optional[float] = None
"""Run inference with this temperature. Must be in the closed """Run inference with this temperature. Must be in the closed
interval [0.0, 1.0].""" interval [0.0, 1.0]."""
model_kwargs: Dict[str, Any] = Field(default_factory=dict) model_kwargs: Dict[str, Any] = Field(default_factory=dict)
@ -270,12 +270,12 @@ class ChatLiteLLM(BaseChatModel):
top_k: Optional[int] = None top_k: Optional[int] = None
"""Decode using top-k sampling: consider the set of top_k most probable tokens. """Decode using top-k sampling: consider the set of top_k most probable tokens.
Must be positive.""" Must be positive."""
n: int = 1 n: Optional[int] = None
"""Number of chat completions to generate for each prompt. Note that the API may """Number of chat completions to generate for each prompt. Note that the API may
not return the full n completions if duplicates are generated.""" not return the full n completions if duplicates are generated."""
max_tokens: Optional[int] = None max_tokens: Optional[int] = None
max_retries: int = 6 max_retries: int = 1
@property @property
def _default_params(self) -> Dict[str, Any]: def _default_params(self) -> Dict[str, Any]: