mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-12 00:11:17 +00:00
docs(openai): add comprehensive documentation and examples for extra_body
+ others (#32149)
This PR addresses the common issue where users struggle to pass custom parameters to OpenAI-compatible APIs like LM Studio, vLLM, and others. The problem occurs when users try to use `model_kwargs` for custom parameters, which causes API errors. ## Problem Users attempting to pass custom parameters (like LM Studio's `ttl` parameter) were getting errors: ```python # ❌ This approach fails llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", model_kwargs={"ttl": 5} # Causes TypeError: unexpected keyword argument 'ttl' ) ``` ## Solution The `extra_body` parameter is the correct way to pass custom parameters to OpenAI-compatible APIs: ```python # ✅ This approach works correctly llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 5} # Custom parameters go in extra_body ) ``` ## Changes Made 1. **Enhanced Documentation**: Updated the `extra_body` parameter docstring with comprehensive examples for LM Studio, vLLM, and other providers 2. **Added Documentation Section**: Created a new "OpenAI-compatible APIs" section in the main class docstring with practical examples 3. **Unit Tests**: Added tests to verify `extra_body` functionality works correctly: - `test_extra_body_parameter()`: Verifies custom parameters are included in request payload - `test_extra_body_with_model_kwargs()`: Ensures `extra_body` and `model_kwargs` work together 4. **Clear Guidance**: Documented when to use `extra_body` vs `model_kwargs` ## Examples Added **LM Studio with TTL (auto-eviction):** ```python ChatOpenAI( base_url="http://localhost:1234/v1", api_key="lm-studio", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 300} # Auto-evict after 5 minutes ) ``` **vLLM with custom sampling:** ```python ChatOpenAI( base_url="http://localhost:8000/v1", api_key="EMPTY", model="meta-llama/Llama-2-7b-chat-hf", extra_body={ "use_beam_search": True, "best_of": 4 } ) ``` ## Why This Works - `model_kwargs` parameters are passed directly to the OpenAI client's `create()` method, causing errors for non-standard parameters - `extra_body` parameters are included in the HTTP request body, which is exactly what OpenAI-compatible APIs expect for custom parameters Fixes #32115. <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click [here](https://survey.alchemer.com/s3/8343779/Copilot-Coding-agent) to start the survey. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: mdrxy <61371264+mdrxy@users.noreply.github.com> Co-authored-by: Mason Daugherty <github@mdrxy.com> Co-authored-by: Mason Daugherty <mason@langchain.dev>
This commit is contained in:
@@ -13,13 +13,13 @@ Not pinning your version will ensure you always have the latest tests, but it ma
|
||||
also break your CI if we introduce tests that your integration doesn't pass.
|
||||
|
||||
Pip:
|
||||
|
||||
|
||||
```bash
|
||||
pip install -U langchain-tests
|
||||
```
|
||||
|
||||
Poetry:
|
||||
|
||||
|
||||
```bash
|
||||
poetry add langchain-tests
|
||||
```
|
||||
@@ -32,44 +32,44 @@ To add standard tests to an integration package's e.g. ChatModel, you need to cr
|
||||
2. An integration test class that inherits from ChatModelIntegrationTests
|
||||
|
||||
`tests/unit_tests/test_standard.py`:
|
||||
|
||||
```python
|
||||
"""Standard LangChain interface tests"""
|
||||
|
||||
from typing import Type
|
||||
```python
|
||||
"""Standard LangChain interface tests"""
|
||||
|
||||
import pytest
|
||||
from langchain_core.language_models import BaseChatModel
|
||||
from langchain_tests.unit_tests import ChatModelUnitTests
|
||||
from typing import Type
|
||||
|
||||
from langchain_parrot_chain import ChatParrotChain
|
||||
import pytest
|
||||
from langchain_core.language_models import BaseChatModel
|
||||
from langchain_tests.unit_tests import ChatModelUnitTests
|
||||
|
||||
from langchain_parrot_chain import ChatParrotChain
|
||||
|
||||
|
||||
class TestParrotChainStandard(ChatModelUnitTests):
|
||||
@pytest.fixture
|
||||
def chat_model_class(self) -> Type[BaseChatModel]:
|
||||
return ChatParrotChain
|
||||
```
|
||||
class TestParrotChainStandard(ChatModelUnitTests):
|
||||
@pytest.fixture
|
||||
def chat_model_class(self) -> Type[BaseChatModel]:
|
||||
return ChatParrotChain
|
||||
```
|
||||
|
||||
`tests/integration_tests/test_standard.py`:
|
||||
|
||||
```python
|
||||
"""Standard LangChain interface tests"""
|
||||
|
||||
from typing import Type
|
||||
```python
|
||||
"""Standard LangChain interface tests"""
|
||||
|
||||
import pytest
|
||||
from langchain_core.language_models import BaseChatModel
|
||||
from langchain_tests.integration_tests import ChatModelIntegrationTests
|
||||
from typing import Type
|
||||
|
||||
from langchain_parrot_chain import ChatParrotChain
|
||||
import pytest
|
||||
from langchain_core.language_models import BaseChatModel
|
||||
from langchain_tests.integration_tests import ChatModelIntegrationTests
|
||||
|
||||
from langchain_parrot_chain import ChatParrotChain
|
||||
|
||||
|
||||
class TestParrotChainStandard(ChatModelIntegrationTests):
|
||||
@pytest.fixture
|
||||
def chat_model_class(self) -> Type[BaseChatModel]:
|
||||
return ChatParrotChain
|
||||
```
|
||||
class TestParrotChainStandard(ChatModelIntegrationTests):
|
||||
@pytest.fixture
|
||||
def chat_model_class(self) -> Type[BaseChatModel]:
|
||||
return ChatParrotChain
|
||||
```
|
||||
|
||||
## Reference
|
||||
|
||||
|
Reference in New Issue
Block a user