mirror of
https://github.com/hwchase17/langchain.git
synced 2025-08-07 12:06:43 +00:00
54542b9385
This PR addresses the common issue where users struggle to pass custom parameters to OpenAI-compatible APIs like LM Studio, vLLM, and others. The problem occurs when users try to use `model_kwargs` for custom parameters, which causes API errors. ## Problem Users attempting to pass custom parameters (like LM Studio's `ttl` parameter) were getting errors: ```python # ❌ This approach fails llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", model_kwargs={"ttl": 5} # Causes TypeError: unexpected keyword argument 'ttl' ) ``` ## Solution The `extra_body` parameter is the correct way to pass custom parameters to OpenAI-compatible APIs: ```python # ✅ This approach works correctly llm = ChatOpenAI( base_url="http://localhost:1234/v1", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 5} # Custom parameters go in extra_body ) ``` ## Changes Made 1. **Enhanced Documentation**: Updated the `extra_body` parameter docstring with comprehensive examples for LM Studio, vLLM, and other providers 2. **Added Documentation Section**: Created a new "OpenAI-compatible APIs" section in the main class docstring with practical examples 3. **Unit Tests**: Added tests to verify `extra_body` functionality works correctly: - `test_extra_body_parameter()`: Verifies custom parameters are included in request payload - `test_extra_body_with_model_kwargs()`: Ensures `extra_body` and `model_kwargs` work together 4. **Clear Guidance**: Documented when to use `extra_body` vs `model_kwargs` ## Examples Added **LM Studio with TTL (auto-eviction):** ```python ChatOpenAI( base_url="http://localhost:1234/v1", api_key="lm-studio", model="mlx-community/QwQ-32B-4bit", extra_body={"ttl": 300} # Auto-evict after 5 minutes ) ``` **vLLM with custom sampling:** ```python ChatOpenAI( base_url="http://localhost:8000/v1", api_key="EMPTY", model="meta-llama/Llama-2-7b-chat-hf", extra_body={ "use_beam_search": True, "best_of": 4 } ) ``` ## Why This Works - `model_kwargs` parameters are passed directly to the OpenAI client's `create()` method, causing errors for non-standard parameters - `extra_body` parameters are included in the HTTP request body, which is exactly what OpenAI-compatible APIs expect for custom parameters Fixes #32115. <!-- START COPILOT CODING AGENT TIPS --> --- 💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click [here](https://survey.alchemer.com/s3/8343779/Copilot-Coding-agent) to start the survey. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: mdrxy <61371264+mdrxy@users.noreply.github.com> Co-authored-by: Mason Daugherty <github@mdrxy.com> Co-authored-by: Mason Daugherty <mason@langchain.dev>
88 lines
5.2 KiB
Markdown
88 lines
5.2 KiB
Markdown
<picture>
|
||
<source media="(prefers-color-scheme: light)" srcset="docs/static/img/logo-dark.svg">
|
||
<source media="(prefers-color-scheme: dark)" srcset="docs/static/img/logo-light.svg">
|
||
<img alt="LangChain Logo" src="docs/static/img/logo-dark.svg" width="80%">
|
||
</picture>
|
||
|
||
<div>
|
||
<br>
|
||
</div>
|
||
|
||
[](https://github.com/langchain-ai/langchain/releases)
|
||
[](https://github.com/langchain-ai/langchain/actions/workflows/check_diffs.yml)
|
||
[](https://opensource.org/licenses/MIT)
|
||
[](https://pypistats.org/packages/langchain-core)
|
||
[](https://star-history.com/#langchain-ai/langchain)
|
||
[](https://github.com/langchain-ai/langchain/issues)
|
||
[](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/langchain-ai/langchain)
|
||
[<img src="https://github.com/codespaces/badge.svg" title="Open in Github Codespace" width="150" height="20">](https://codespaces.new/langchain-ai/langchain)
|
||
[](https://twitter.com/langchainai)
|
||
[](https://codspeed.io/langchain-ai/langchain)
|
||
|
||
> [!NOTE]
|
||
> Looking for the JS/TS library? Check out [LangChain.js](https://github.com/langchain-ai/langchainjs).
|
||
|
||
LangChain is a framework for building LLM-powered applications. It helps you chain
|
||
together interoperable components and third-party integrations to simplify AI
|
||
application development — all while future-proofing decisions as the underlying
|
||
technology evolves.
|
||
|
||
```bash
|
||
pip install -U langchain
|
||
```
|
||
|
||
To learn more about LangChain, check out
|
||
[the docs](https://python.langchain.com/docs/introduction/). If you’re looking for more
|
||
advanced customization or agent orchestration, check out
|
||
[LangGraph](https://langchain-ai.github.io/langgraph/), our framework for building
|
||
controllable agent workflows.
|
||
|
||
## Why use LangChain?
|
||
|
||
LangChain helps developers build applications powered by LLMs through a standard
|
||
interface for models, embeddings, vector stores, and more.
|
||
|
||
Use LangChain for:
|
||
|
||
- **Real-time data augmentation**. Easily connect LLMs to diverse data sources and
|
||
external / internal systems, drawing from LangChain’s vast library of integrations with
|
||
model providers, tools, vector stores, retrievers, and more.
|
||
- **Model interoperability**. Swap models in and out as your engineering team
|
||
experiments to find the best choice for your application’s needs. As the industry
|
||
frontier evolves, adapt quickly — LangChain’s abstractions keep you moving without
|
||
losing momentum.
|
||
|
||
## LangChain’s ecosystem
|
||
|
||
While the LangChain framework can be used standalone, it also integrates seamlessly
|
||
with any LangChain product, giving developers a full suite of tools when building LLM
|
||
applications.
|
||
|
||
To improve your LLM application development, pair LangChain with:
|
||
|
||
- [LangSmith](http://www.langchain.com/langsmith) - Helpful for agent evals and
|
||
observability. Debug poor-performing LLM app runs, evaluate agent trajectories, gain
|
||
visibility in production, and improve performance over time.
|
||
- [LangGraph](https://langchain-ai.github.io/langgraph/) - Build agents that can
|
||
reliably handle complex tasks with LangGraph, our low-level agent orchestration
|
||
framework. LangGraph offers customizable architecture, long-term memory, and
|
||
human-in-the-loop workflows — and is trusted in production by companies like LinkedIn,
|
||
Uber, Klarna, and GitLab.
|
||
- [LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/langgraph_platform/) - Deploy
|
||
and scale agents effortlessly with a purpose-built deployment platform for long
|
||
running, stateful workflows. Discover, reuse, configure, and share agents across
|
||
teams — and iterate quickly with visual prototyping in
|
||
[LangGraph Studio](https://langchain-ai.github.io/langgraph/concepts/langgraph_studio/).
|
||
|
||
## Additional resources
|
||
|
||
- [Tutorials](https://python.langchain.com/docs/tutorials/): Simple walkthroughs with
|
||
guided examples on getting started with LangChain.
|
||
- [How-to Guides](https://python.langchain.com/docs/how_to/): Quick, actionable code
|
||
snippets for topics such as tool calling, RAG use cases, and more.
|
||
- [Conceptual Guides](https://python.langchain.com/docs/concepts/): Explanations of key
|
||
concepts behind the LangChain framework.
|
||
- [LangChain Forum](https://forum.langchain.com/): Connect with the community and share all of your technical questions, ideas, and feedback.
|
||
- [API Reference](https://python.langchain.com/api_reference/): Detailed reference on
|
||
navigating base packages and integrations for LangChain.
|