diff --git a/docs/docs/integrations/providers/mlflow.mdx b/docs/docs/integrations/providers/mlflow.mdx index cb4d5aba840..861154a0b8a 100644 --- a/docs/docs/integrations/providers/mlflow.mdx +++ b/docs/docs/integrations/providers/mlflow.mdx @@ -1,12 +1,12 @@ -# MLflow Deployments for LLMs +# MLflow AI Gateway for LLMs ->[The MLflow Deployments for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large +>[The MLflow AI Gateway for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large > language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface > that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests. ## Installation and Setup -Install `mlflow` with MLflow Deployments dependencies: +Install `mlflow` with MLflow GenAI dependencies: ```sh pip install 'mlflow[genai]' @@ -39,10 +39,10 @@ endpoints: openai_api_key: $OPENAI_API_KEY ``` -Start the deployments server: +Start the gateway server: ```sh -mlflow deployments start-server --config-path /path/to/config.yaml +mlflow gateway start --config-path /path/to/config.yaml ``` ## Example provided by `MLflow` diff --git a/docs/docs/integrations/providers/mlflow_ai_gateway.mdx b/docs/docs/integrations/providers/mlflow_ai_gateway.mdx deleted file mode 100644 index 912ea449eba..00000000000 --- a/docs/docs/integrations/providers/mlflow_ai_gateway.mdx +++ /dev/null @@ -1,160 +0,0 @@ -# MLflow AI Gateway - -:::warning - -MLflow AI Gateway has been deprecated. Please use [MLflow Deployments for LLMs](/docs/integrations/providers/mlflow/) instead. - -::: - ->[The MLflow AI Gateway](https://www.mlflow.org/docs/latest/index.html) service is a powerful tool designed to streamline the usage and management of various large -> language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface -> that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests. - -## Installation and Setup - -Install `mlflow` with MLflow AI Gateway dependencies: - -```sh -pip install 'mlflow[gateway]' -``` - -Set the OpenAI API key as an environment variable: - -```sh -export OPENAI_API_KEY=... -``` - -Create a configuration file: - -```yaml -routes: - - name: completions - route_type: llm/v1/completions - model: - provider: openai - name: text-davinci-003 - config: - openai_api_key: $OPENAI_API_KEY - - - name: embeddings - route_type: llm/v1/embeddings - model: - provider: openai - name: text-embedding-ada-002 - config: - openai_api_key: $OPENAI_API_KEY -``` - -Start the Gateway server: - -```sh -mlflow gateway start --config-path /path/to/config.yaml -``` - -## Example provided by `MLflow` - ->The `mlflow.langchain` module provides an API for logging and loading `LangChain` models. -> This module exports multivariate LangChain models in the langchain flavor and univariate LangChain -> models in the pyfunc flavor. - -See the [API documentation and examples](https://www.mlflow.org/docs/latest/python_api/mlflow.langchain.html?highlight=langchain#module-mlflow.langchain). - - - -## Completions Example - -```python -import mlflow -from langchain.chains import LLMChain, PromptTemplate -from langchain_community.llms import MlflowAIGateway - -gateway = MlflowAIGateway( - gateway_uri="http://127.0.0.1:5000", - route="completions", - params={ - "temperature": 0.0, - "top_p": 0.1, - }, -) - -llm_chain = LLMChain( - llm=gateway, - prompt=PromptTemplate( - input_variables=["adjective"], - template="Tell me a {adjective} joke", - ), -) -result = llm_chain.run(adjective="funny") -print(result) - -with mlflow.start_run(): - model_info = mlflow.langchain.log_model(chain, "model") - -model = mlflow.pyfunc.load_model(model_info.model_uri) -print(model.predict([{"adjective": "funny"}])) -``` - -## Embeddings Example - -```python -from langchain_community.embeddings import MlflowAIGatewayEmbeddings - -embeddings = MlflowAIGatewayEmbeddings( - gateway_uri="http://127.0.0.1:5000", - route="embeddings", -) - -print(embeddings.embed_query("hello")) -print(embeddings.embed_documents(["hello"])) -``` - -## Chat Example - -```python -from langchain_community.chat_models import ChatMLflowAIGateway -from langchain_core.messages import HumanMessage, SystemMessage - -chat = ChatMLflowAIGateway( - gateway_uri="http://127.0.0.1:5000", - route="chat", - params={ - "temperature": 0.1 - } -) - -messages = [ - SystemMessage( - content="You are a helpful assistant that translates English to French." - ), - HumanMessage( - content="Translate this sentence from English to French: I love programming." - ), -] -print(chat(messages)) -``` - -## Databricks MLflow AI Gateway - -Databricks MLflow AI Gateway is in private preview. -Please contact a Databricks representative to enroll in the preview. - -```python -from langchain.chains import LLMChain -from langchain_core.prompts import PromptTemplate -from langchain_community.llms import MlflowAIGateway - -gateway = MlflowAIGateway( - gateway_uri="databricks", - route="completions", -) - -llm_chain = LLMChain( - llm=gateway, - prompt=PromptTemplate( - input_variables=["adjective"], - template="Tell me a {adjective} joke", - ), -) -result = llm_chain.run(adjective="funny") -print(result) -``` diff --git a/docs/vercel.json b/docs/vercel.json index 5f88f694cdc..ed15a82a067 100644 --- a/docs/vercel.json +++ b/docs/vercel.json @@ -73,6 +73,10 @@ { "source": "/v0.2/docs/templates/:path(.*/?)*", "destination": "https://github.com/langchain-ai/langchain/tree/master/templates/:path*" + }, + { + "source": "/docs/integrations/providers/mlflow_ai_gateway(/?)", + "destination": "/docs/integrations/providers/mlflow/" } ] }