community: aligning ChatLiteLLM default parameters with litellm (#30360)

**Description:**
Since `ChatLiteLLM` is forwarding most parameters to
`litellm.completion(...)`, there is no reason to set other default
values than the ones defined by `litellm`.

In the case of parameter 'n', it also provokes an issue when trying to
call a serverless endpoint on Azure, as it is considered an extra
parameter. So we need to keep it optional.

We can debate about backward compatibility of this change: in my
opinion, there should not be big issues since from my experience,
calling `litellm.completion()` without these parameters works fine.

**Issue:** 
- #29679 

**Dependencies:** None
This commit is contained in:
Florian Chappaz 2025-03-19 14:07:28 +01:00 committed by GitHub
parent 57ffacadd0
commit 07cb41ea9e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -259,7 +259,7 @@ class ChatLiteLLM(BaseChatModel):
organization: Optional[str] = None
custom_llm_provider: Optional[str] = None
request_timeout: Optional[Union[float, Tuple[float, float]]] = None
temperature: Optional[float] = 1
temperature: Optional[float] = None
"""Run inference with this temperature. Must be in the closed
interval [0.0, 1.0]."""
model_kwargs: Dict[str, Any] = Field(default_factory=dict)
@ -270,12 +270,12 @@ class ChatLiteLLM(BaseChatModel):
top_k: Optional[int] = None
"""Decode using top-k sampling: consider the set of top_k most probable tokens.
Must be positive."""
n: int = 1
n: Optional[int] = None
"""Number of chat completions to generate for each prompt. Note that the API may
not return the full n completions if duplicates are generated."""
max_tokens: Optional[int] = None
max_retries: int = 6
max_retries: int = 1
@property
def _default_params(self) -> Dict[str, Any]: