mirror of
https://github.com/hwchase17/langchain.git
synced 2026-06-09 10:17:00 +00:00
54542b9385
This PR addresses the common issue where users struggle to pass custom parameters to OpenAI-compatible APIs like LM Studio, vLLM, and others. The problem occurs when users try to use `model_kwargs` for custom parameters, which causes API errors. ## Problem Users attempting to pass custom parameters (like LM Studio's `ttl` parameter) were getting errors: ```python # ❌ This approach fails llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", model_kwargs={"ttl": 5} # Causes TypeError: unexpected keyword argument 'ttl' ) ``` ## Solution The `extra_body` parameter is the correct way to pass custom parameters to OpenAI-compatible APIs: ```python # ✅ This approach works correctly llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 5} # Custom parameters go in extra_body ) ``` ## Changes Made 1. **Enhanced Documentation**: Updated the `extra_body` parameter docstring with comprehensive examples for LM Studio, vLLM, and other providers 2. **Added Documentation Section**: Created a new "OpenAI-compatible APIs" section in the main class docstring with practical examples 3. **Unit Tests**: Added tests to verify `extra_body` functionality works correctly: - `test_extra_body_parameter()`: Verifies custom parameters are included in request payload - `test_extra_body_with_model_kwargs()`: Ensures `extra_body` and `model_kwargs` work together 4. **Clear Guidance**: Documented when to use `extra_body` vs `model_kwargs` ## Examples Added **LM Studio with TTL (auto-eviction):** ```python ChatOpenAI( base_url="http://localhost:1234/v1", api_key="lm-studio", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 300} # Auto-evict after 5 minutes ) ``` **vLLM with custom sampling:** ```python ChatOpenAI( base_url="http://localhost:8000/v1", api_key="EMPTY", model="meta-llama/Llama-2-7b-chat-hf", extra_body={ "use_beam_search": True, "best_of": 4 } ) ``` ## Why This Works - `model_kwargs` parameters are passed directly to the OpenAI client's `create()` method, causing errors for non-standard parameters - `extra_body` parameters are included in the HTTP request body, which is exactly what OpenAI-compatible APIs expect for custom parameters Fixes #32115. <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click [here](https://survey.alchemer.com/s3/8343779/Copilot-Coding-agent) to start the survey. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: mdrxy <61371264+mdrxy@users.noreply.github.com> Co-authored-by: Mason Daugherty <github@mdrxy.com> Co-authored-by: Mason Daugherty <mason@langchain.dev>
106 lines
2.8 KiB
Markdown
106 lines
2.8 KiB
Markdown
# langchain-exa
|
|
|
|
This package contains the LangChain integrations for Exa Cloud generative models.
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
pip install -U langchain-exa
|
|
```
|
|
|
|
## Exa Search Retriever
|
|
|
|
You can retrieve search results as follows
|
|
|
|
```python
|
|
from langchain_exa import ExaSearchRetriever
|
|
|
|
exa_api_key = "YOUR API KEY"
|
|
|
|
# Create a new instance of the ExaSearchRetriever
|
|
exa = ExaSearchRetriever(exa_api_key=exa_api_key)
|
|
|
|
# Search for a query and save the results
|
|
results = exa.invoke("What is the capital of France?")
|
|
|
|
# Print the results
|
|
print(results)
|
|
```
|
|
|
|
### Advanced Features
|
|
|
|
You can use advanced features like text limits, summaries, and live crawling:
|
|
|
|
```python
|
|
from langchain_exa import ExaSearchRetriever, TextContentsOptions
|
|
|
|
# Create a new instance with advanced options
|
|
exa = ExaSearchRetriever(
|
|
exa_api_key="YOUR API KEY",
|
|
k=20, # Number of results (1-100)
|
|
type="auto", # Can be "neural", "keyword", or "auto"
|
|
livecrawl="always", # Can be "always", "fallback", or "never"
|
|
summary=True, # Get an AI-generated summary of each result
|
|
text_contents_options={"max_characters": 3000} # Limit text length
|
|
)
|
|
|
|
# Search for a query with custom summary prompt
|
|
exa_with_custom_summary = ExaSearchRetriever(
|
|
exa_api_key="YOUR API KEY",
|
|
summary={"query": "generate one line summary in simple words."} # Custom summary prompt
|
|
)
|
|
```
|
|
|
|
## Exa Search Results
|
|
|
|
You can run the ExaSearchResults module as follows
|
|
|
|
```python
|
|
from langchain_exa import ExaSearchResults
|
|
|
|
# Initialize the ExaSearchResults tool
|
|
search_tool = ExaSearchResults(exa_api_key="YOUR API KEY")
|
|
|
|
# Perform a search query
|
|
search_results = search_tool._run(
|
|
query="When was the last time the New York Knicks won the NBA Championship?",
|
|
num_results=5,
|
|
text_contents_options=True,
|
|
highlights=True
|
|
)
|
|
|
|
print("Search Results:", search_results)
|
|
```
|
|
|
|
## Exa Find Similar Results
|
|
|
|
You can run the ExaFindSimilarResults module as follows
|
|
|
|
```python
|
|
from langchain_exa import ExaFindSimilarResults
|
|
|
|
# Initialize the ExaFindSimilarResults tool
|
|
find_similar_tool = ExaFindSimilarResults(exa_api_key="YOUR API KEY")
|
|
|
|
# Find similar results based on a URL
|
|
similar_results = find_similar_tool._run(
|
|
url="http://espn.com",
|
|
num_results=5,
|
|
text_contents_options=True,
|
|
highlights=True
|
|
)
|
|
|
|
print("Similar Results:", similar_results)
|
|
```
|
|
|
|
## Configuration Options
|
|
|
|
All Exa tools support the following common parameters:
|
|
|
|
- `num_results` (1-100): Number of search results to return
|
|
- `type`: Search type - "neural", "keyword", or "auto"
|
|
- `livecrawl`: Live crawling mode - "always", "fallback", or "never"
|
|
- `summary`: Get AI-generated summaries (True/False or custom prompt dict)
|
|
- `text_contents_options`: Dict to limit text length (e.g. `{"max_characters": 2000}`)
|
|
- `highlights`: Include highlighted text snippets (True/False)
|