mirror of
https://github.com/hwchase17/langchain.git
synced 2026-06-09 10:17:00 +00:00
docs(openai): add comprehensive documentation and examples for extra_body + others (#32149)
This PR addresses the common issue where users struggle to pass custom parameters to OpenAI-compatible APIs like LM Studio, vLLM, and others. The problem occurs when users try to use `model_kwargs` for custom parameters, which causes API errors. ## Problem Users attempting to pass custom parameters (like LM Studio's `ttl` parameter) were getting errors: ```python # ❌ This approach fails llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", model_kwargs={"ttl": 5} # Causes TypeError: unexpected keyword argument 'ttl' ) ``` ## Solution The `extra_body` parameter is the correct way to pass custom parameters to OpenAI-compatible APIs: ```python # ✅ This approach works correctly llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 5} # Custom parameters go in extra_body ) ``` ## Changes Made 1. **Enhanced Documentation**: Updated the `extra_body` parameter docstring with comprehensive examples for LM Studio, vLLM, and other providers 2. **Added Documentation Section**: Created a new "OpenAI-compatible APIs" section in the main class docstring with practical examples 3. **Unit Tests**: Added tests to verify `extra_body` functionality works correctly: - `test_extra_body_parameter()`: Verifies custom parameters are included in request payload - `test_extra_body_with_model_kwargs()`: Ensures `extra_body` and `model_kwargs` work together 4. **Clear Guidance**: Documented when to use `extra_body` vs `model_kwargs` ## Examples Added **LM Studio with TTL (auto-eviction):** ```python ChatOpenAI( base_url="http://localhost:1234/v1", api_key="lm-studio", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 300} # Auto-evict after 5 minutes ) ``` **vLLM with custom sampling:** ```python ChatOpenAI( base_url="http://localhost:8000/v1", api_key="EMPTY", model="meta-llama/Llama-2-7b-chat-hf", extra_body={ "use_beam_search": True, "best_of": 4 } ) ``` ## Why This Works - `model_kwargs` parameters are passed directly to the OpenAI client's `create()` method, causing errors for non-standard parameters - `extra_body` parameters are included in the HTTP request body, which is exactly what OpenAI-compatible APIs expect for custom parameters Fixes #32115. <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click [here](https://survey.alchemer.com/s3/8343779/Copilot-Coding-agent) to start the survey. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: mdrxy <61371264+mdrxy@users.noreply.github.com> Co-authored-by: Mason Daugherty <github@mdrxy.com> Co-authored-by: Mason Daugherty <mason@langchain.dev>
This commit is contained in:
@@ -2,7 +2,6 @@
|
||||
|
||||
This is the partner package for tying Fireworks.ai and LangChain. Fireworks really strive to provide good support for LangChain use cases, so if you run into any issues please let us know. You can reach out to us [in our Discord channel](https://discord.com/channels/1137072072808472616/)
|
||||
|
||||
|
||||
## Installation
|
||||
|
||||
To use the `langchain-fireworks` package, follow these installation steps:
|
||||
@@ -11,8 +10,6 @@ To use the `langchain-fireworks` package, follow these installation steps:
|
||||
pip install langchain-fireworks
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Basic usage
|
||||
|
||||
### Setting up
|
||||
@@ -21,12 +18,15 @@ pip install langchain-fireworks
|
||||
|
||||
Once you've signed in and obtained an API key, follow these steps to set the `FIREWORKS_API_KEY` environment variable:
|
||||
- **Linux/macOS:** Open your terminal and execute the following command:
|
||||
|
||||
```bash
|
||||
export FIREWORKS_API_KEY='your_api_key'
|
||||
```
|
||||
|
||||
**Note:** To make this environment variable persistent across terminal sessions, add the above line to your `~/.bashrc`, `~/.bash_profile`, or `~/.zshrc` file.
|
||||
|
||||
- **Windows:** For Command Prompt, use:
|
||||
|
||||
```cmd
|
||||
set FIREWORKS_API_KEY=your_api_key
|
||||
```
|
||||
@@ -44,7 +44,6 @@ llm = Fireworks(
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
### Calling the Model Directly
|
||||
|
||||
You can call the model directly with string prompts to get completions.
|
||||
@@ -66,15 +65,14 @@ output = llm.generate(
|
||||
print(output.generations)
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Advanced usage
|
||||
|
||||
### Tool use: LangChain Agent + Fireworks function calling model
|
||||
Please checkout how to teach Fireworks function calling model to use a calculator [here](https://github.com/fw-ai/cookbook/blob/main/learn/function-calling/notebooks_langchain/fireworks_langchain_tool_usage.ipynb).
|
||||
|
||||
Please checkout how to teach Fireworks function calling model to use a calculator [in this notebook](https://github.com/fw-ai/cookbook/blob/main/learn/function-calling/notebooks_langchain/fireworks_langchain_tool_usage.ipynb).
|
||||
|
||||
Fireworks focus on delivering the best experience for fast model inference as well as tool use. You can check out [our blog](https://fireworks.ai/blog/firefunction-v1-gpt-4-level-function-calling) for more details on how it compares to GPT-4, the punchline is that it is on par with GPT-4 in terms of function calling use cases, but it is way faster and much cheaper.
|
||||
|
||||
### RAG: LangChain agent + Fireworks function calling model + MongoDB + Nomic AI embeddings
|
||||
|
||||
Please check out the [cookbook here](https://github.com/fw-ai/cookbook/blob/main/integrations/MongoDB/project_rag_with_mongodb/mongodb_agent.ipynb) for an end to end flow
|
||||
|
||||
Reference in New Issue
Block a user