github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-29 18:08:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

community: Rename deployments server to AI gateway (#26368)

Browse Source 
We recently renamed `MLflow Deployments Server` to `MLflow AI Gateway`
in mlflow. This PR updates the relevant notebooks to use `MLflow AI
gateway`

---

Thank you for contributing to LangChain!

- [x] **PR title**: "package: description"
- Where "package" is whichever of langchain, community, core,
experimental, etc. is being modified. Use "docs: ..." for purely docs
changes, "templates: ..." for template changes, "infra: ..." for CI
changes.
  - Example: "community: add foobar LLM"


- [x] **PR message**: ***Delete this entire checklist*** and replace
with
    - **Description:** a description of the change
    - **Issue:** the issue # it fixes, if applicable
    - **Dependencies:** any dependencies required for this change
- **Twitter handle:** if your PR gets announced, and you'd like a
mention, we'll gladly shout you out!


- [x] **Add tests and docs**: If you're adding a new integration, please
include
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.


- [x] **Lint and test**: Run `make format`, `make lint` and `make test`
from the root of the package(s) you've modified. See contribution
guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:
- Make sure optional dependencies are imported within a function.
- Please do not add dependencies to pyproject.toml files (even optional
ones) unless they are required for unit tests.
- Most PRs should not touch more than one package.
- Changes should be backwards compatible.
- If you are adding something to community, do not re-import it in
langchain.

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.

---------

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
This commit is contained in:
Harutaka Kawamura 2024-09-18 06:36:04 +09:00 committed by GitHub
parent 5ced41bf50
commit 6ed50e78c9
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 9 additions and 165 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
docs/docs/integrations/providers/mlflow.mdx
View File

@ -1,12 +1,12 @@
# MLflow Deployments for LLMs # MLflow AI Gateway for LLMs
>[The MLflow Deployments for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large >[The MLflow AI Gateway for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large
> language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface > language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface
> that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests. > that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests.
## Installation and Setup ## Installation and Setup
Install `mlflow` with MLflow Deployments dependencies: Install `mlflow` with MLflow GenAI dependencies:
```sh ```sh
pip install 'mlflow[genai]' pip install 'mlflow[genai]'
@ -39,10 +39,10 @@ endpoints:
openai_api_key: $OPENAI_API_KEY openai_api_key: $OPENAI_API_KEY
``` ```
Start the deployments server: Start the gateway server:
```sh ```sh
mlflow deployments start-server --config-path /path/to/config.yaml mlflow gateway start --config-path /path/to/config.yaml
``` ```
## Example provided by `MLflow` ## Example provided by `MLflow`

160
docs/docs/integrations/providers/mlflow_ai_gateway.mdx
View File

@ -1,160 +0,0 @@
# MLflow AI Gateway
:::warning
MLflow AI Gateway has been deprecated. Please use [MLflow Deployments for LLMs](/docs/integrations/providers/mlflow/) instead.
:::
>[The MLflow AI Gateway](https://www.mlflow.org/docs/latest/index.html) service is a powerful tool designed to streamline the usage and management of various large
> language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface
> that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests.
## Installation and Setup
Install `mlflow` with MLflow AI Gateway dependencies:
```sh
pip install 'mlflow[gateway]'
```
Set the OpenAI API key as an environment variable:
```sh
export OPENAI_API_KEY=...
```
Create a configuration file:
```yaml
routes:
- name: completions
route_type: llm/v1/completions
model:
provider: openai
name: text-davinci-003
config:
openai_api_key: $OPENAI_API_KEY
- name: embeddings
route_type: llm/v1/embeddings
model:
provider: openai
name: text-embedding-ada-002
config:
openai_api_key: $OPENAI_API_KEY
```
Start the Gateway server:
```sh
mlflow gateway start --config-path /path/to/config.yaml
```
## Example provided by `MLflow`
>The `mlflow.langchain` module provides an API for logging and loading `LangChain` models.
> This module exports multivariate LangChain models in the langchain flavor and univariate LangChain
> models in the pyfunc flavor.
See the [API documentation and examples](https://www.mlflow.org/docs/latest/python_api/mlflow.langchain.html?highlight=langchain#module-mlflow.langchain).
## Completions Example
```python
import mlflow
from langchain.chains import LLMChain, PromptTemplate
from langchain_community.llms import MlflowAIGateway
gateway = MlflowAIGateway(
gateway_uri="http://127.0.0.1:5000",
route="completions",
params={
"temperature": 0.0,
"top_p": 0.1,
},
)
llm_chain = LLMChain(
llm=gateway,
prompt=PromptTemplate(
input_variables=["adjective"],
template="Tell me a {adjective} joke",
),
)
result = llm_chain.run(adjective="funny")
print(result)
with mlflow.start_run():
model_info = mlflow.langchain.log_model(chain, "model")
model = mlflow.pyfunc.load_model(model_info.model_uri)
print(model.predict([{"adjective": "funny"}]))
```
## Embeddings Example
```python
from langchain_community.embeddings import MlflowAIGatewayEmbeddings
embeddings = MlflowAIGatewayEmbeddings(
gateway_uri="http://127.0.0.1:5000",
route="embeddings",
)
print(embeddings.embed_query("hello"))
print(embeddings.embed_documents(["hello"]))
```
## Chat Example
```python
from langchain_community.chat_models import ChatMLflowAIGateway
from langchain_core.messages import HumanMessage, SystemMessage
chat = ChatMLflowAIGateway(
gateway_uri="http://127.0.0.1:5000",
route="chat",
params={
"temperature": 0.1
}
)
messages = [
SystemMessage(
content="You are a helpful assistant that translates English to French."
),
HumanMessage(
content="Translate this sentence from English to French: I love programming."
),
]
print(chat(messages))
```
## Databricks MLflow AI Gateway
Databricks MLflow AI Gateway is in private preview.
Please contact a Databricks representative to enroll in the preview.
```python
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
from langchain_community.llms import MlflowAIGateway
gateway = MlflowAIGateway(
gateway_uri="databricks",
route="completions",
)
llm_chain = LLMChain(
llm=gateway,
prompt=PromptTemplate(
input_variables=["adjective"],
template="Tell me a {adjective} joke",
),
)
result = llm_chain.run(adjective="funny")
print(result)
```

4
docs/vercel.json
View File

@ -73,6 +73,10 @@
{ {
"source": "/v0.2/docs/templates/:path(.*/?)*", "source": "/v0.2/docs/templates/:path(.*/?)*",
"destination": "https://github.com/langchain-ai/langchain/tree/master/templates/:path*" "destination": "https://github.com/langchain-ai/langchain/tree/master/templates/:path*"
},
{
"source": "/docs/integrations/providers/mlflow_ai_gateway(/?)",
"destination": "/docs/integrations/providers/mlflow/"
} }
] ]
} }