github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-07-08 22:15:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: VLLMOpenAI -- create() got an unexpected keyword argument 'api_key' (#13517)

Browse Source 
The issue was accuring because of `openai` update in Completions. its
not accepting `api_key` and 'api_base' args.

The fix is we check for the openai version and if ats v1 then remove
these keys from args before passing them to `Compilation.create(...)`
when sending from `VLLMOpenAI`

Fixed: #13507 

@eyu
@efriis 
@hwchase17

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
This commit is contained in:
umair mehmood 2023-11-20 06:49:55 +05:00 committed by GitHub
parent 6bc08266e0
commit 69d39e2173
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
libs/langchain/langchain/llms/vllm.py
View File

@ -5,6 +5,7 @@ from langchain.llms.base import BaseLLM
from langchain.llms.openai import BaseOpenAI
from langchain.pydantic_v1 import Field, root_validator
from langchain.schema.output import Generation, LLMResult
from langchain.utils.openai import is_openai_v1
class VLLM(BaseLLM):
@ -148,17 +149,21 @@ class VLLMOpenAI(BaseOpenAI):
@property
def _invocation_params(self) -> Dict[str, Any]:
"""Get the parameters used to invoke the model."""
openai_creds: Dict[str, Any] = {
"api_key": self.openai_api_key,
"api_base": self.openai_api_base,
}
return {
params: Dict[str, Any] = {
"model": self.model_name,
**openai_creds,
**self._default_params,
"logit_bias": None,
}
if not is_openai_v1():
params.update(
{
"api_key": self.openai_api_key,
"api_base": self.openai_api_base,
}
)
return params
@property
def _llm_type(self) -> str: