From 6ed50e78c96d5328215806d9c09fa3486033b263 Mon Sep 17 00:00:00 2001 From: Harutaka Kawamura Date: Wed, 18 Sep 2024 06:36:04 +0900 Subject: [PATCH] community: Rename deployments server to AI gateway (#26368) We recently renamed `MLflow Deployments Server` to `MLflow AI Gateway` in mlflow. This PR updates the relevant notebooks to use `MLflow AI gateway` --- Thank you for contributing to LangChain! - [x] **PR title**: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] **PR message**: ***Delete this entire checklist*** and replace with - **Description:** a description of the change - **Issue:** the issue # it fixes, if applicable - **Dependencies:** any dependencies required for this change - **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [x] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. --------- Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> Co-authored-by: Erick Friis --- docs/docs/integrations/providers/mlflow.mdx | 10 +- .../providers/mlflow_ai_gateway.mdx | 160 ------------------ docs/vercel.json | 4 + 3 files changed, 9 insertions(+), 165 deletions(-) delete mode 100644 docs/docs/integrations/providers/mlflow_ai_gateway.mdx diff --git a/docs/docs/integrations/providers/mlflow.mdx b/docs/docs/integrations/providers/mlflow.mdx index cb4d5aba840..861154a0b8a 100644 --- a/docs/docs/integrations/providers/mlflow.mdx +++ b/docs/docs/integrations/providers/mlflow.mdx @@ -1,12 +1,12 @@ -# MLflow Deployments for LLMs +# MLflow AI Gateway for LLMs ->[The MLflow Deployments for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large +>[The MLflow AI Gateway for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large > language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface > that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests. ## Installation and Setup -Install `mlflow` with MLflow Deployments dependencies: +Install `mlflow` with MLflow GenAI dependencies: ```sh pip install 'mlflow[genai]' @@ -39,10 +39,10 @@ endpoints: openai_api_key: $OPENAI_API_KEY ``` -Start the deployments server: +Start the gateway server: ```sh -mlflow deployments start-server --config-path /path/to/config.yaml +mlflow gateway start --config-path /path/to/config.yaml ``` ## Example provided by `MLflow` diff --git a/docs/docs/integrations/providers/mlflow_ai_gateway.mdx b/docs/docs/integrations/providers/mlflow_ai_gateway.mdx deleted file mode 100644 index 912ea449eba..00000000000 --- a/docs/docs/integrations/providers/mlflow_ai_gateway.mdx +++ /dev/null @@ -1,160 +0,0 @@ -# MLflow AI Gateway - -:::warning - -MLflow AI Gateway has been deprecated. Please use [MLflow Deployments for LLMs](/docs/integrations/providers/mlflow/) instead. - -::: - ->[The MLflow AI Gateway](https://www.mlflow.org/docs/latest/index.html) service is a powerful tool designed to streamline the usage and management of various large -> language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface -> that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests. - -## Installation and Setup - -Install `mlflow` with MLflow AI Gateway dependencies: - -```sh -pip install 'mlflow[gateway]' -``` - -Set the OpenAI API key as an environment variable: - -```sh -export OPENAI_API_KEY=... -``` - -Create a configuration file: - -```yaml -routes: - - name: completions - route_type: llm/v1/completions - model: - provider: openai - name: text-davinci-003 - config: - openai_api_key: $OPENAI_API_KEY - - - name: embeddings - route_type: llm/v1/embeddings - model: - provider: openai - name: text-embedding-ada-002 - config: - openai_api_key: $OPENAI_API_KEY -``` - -Start the Gateway server: - -```sh -mlflow gateway start --config-path /path/to/config.yaml -``` - -## Example provided by `MLflow` - ->The `mlflow.langchain` module provides an API for logging and loading `LangChain` models. -> This module exports multivariate LangChain models in the langchain flavor and univariate LangChain -> models in the pyfunc flavor. - -See the [API documentation and examples](https://www.mlflow.org/docs/latest/python_api/mlflow.langchain.html?highlight=langchain#module-mlflow.langchain). - - - -## Completions Example - -```python -import mlflow -from langchain.chains import LLMChain, PromptTemplate -from langchain_community.llms import MlflowAIGateway - -gateway = MlflowAIGateway( - gateway_uri="http://127.0.0.1:5000", - route="completions", - params={ - "temperature": 0.0, - "top_p": 0.1, - }, -) - -llm_chain = LLMChain( - llm=gateway, - prompt=PromptTemplate( - input_variables=["adjective"], - template="Tell me a {adjective} joke", - ), -) -result = llm_chain.run(adjective="funny") -print(result) - -with mlflow.start_run(): - model_info = mlflow.langchain.log_model(chain, "model") - -model = mlflow.pyfunc.load_model(model_info.model_uri) -print(model.predict([{"adjective": "funny"}])) -``` - -## Embeddings Example - -```python -from langchain_community.embeddings import MlflowAIGatewayEmbeddings - -embeddings = MlflowAIGatewayEmbeddings( - gateway_uri="http://127.0.0.1:5000", - route="embeddings", -) - -print(embeddings.embed_query("hello")) -print(embeddings.embed_documents(["hello"])) -``` - -## Chat Example - -```python -from langchain_community.chat_models import ChatMLflowAIGateway -from langchain_core.messages import HumanMessage, SystemMessage - -chat = ChatMLflowAIGateway( - gateway_uri="http://127.0.0.1:5000", - route="chat", - params={ - "temperature": 0.1 - } -) - -messages = [ - SystemMessage( - content="You are a helpful assistant that translates English to French." - ), - HumanMessage( - content="Translate this sentence from English to French: I love programming." - ), -] -print(chat(messages)) -``` - -## Databricks MLflow AI Gateway - -Databricks MLflow AI Gateway is in private preview. -Please contact a Databricks representative to enroll in the preview. - -```python -from langchain.chains import LLMChain -from langchain_core.prompts import PromptTemplate -from langchain_community.llms import MlflowAIGateway - -gateway = MlflowAIGateway( - gateway_uri="databricks", - route="completions", -) - -llm_chain = LLMChain( - llm=gateway, - prompt=PromptTemplate( - input_variables=["adjective"], - template="Tell me a {adjective} joke", - ), -) -result = llm_chain.run(adjective="funny") -print(result) -``` diff --git a/docs/vercel.json b/docs/vercel.json index 5f88f694cdc..ed15a82a067 100644 --- a/docs/vercel.json +++ b/docs/vercel.json @@ -73,6 +73,10 @@ { "source": "/v0.2/docs/templates/:path(.*/?)*", "destination": "https://github.com/langchain-ai/langchain/tree/master/templates/:path*" + }, + { + "source": "/docs/integrations/providers/mlflow_ai_gateway(/?)", + "destination": "/docs/integrations/providers/mlflow/" } ] }