mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-08 14:31:55 +00:00
Add Javelin integration (#10275)
We are introducing the py integration to Javelin AI Gateway www.getjavelin.io. Javelin is an enterprise-scale fast llm router & gateway. Could you please review and let us know if there is anything missing. Javelin AI Gateway wraps Embedding, Chat and Completion LLMs. Uses javelin_sdk under the covers (pip install javelin_sdk). Author: Sharath Rajasekar, Twitter: @sharathr, @javelinai Thanks!!
This commit is contained in:
Sharath Rajasekar
committed by
GitHub
GitHub
parent
957956ba6d
commit
96023f94d9
@@ -465,6 +465,7 @@
|
||||
"PromptLayer": "https://python.langchain.com/docs/integrations/callbacks/promptlayer",
|
||||
"Log10": "https://python.langchain.com/docs/integrations/providers/log10",
|
||||
"MLflow AI Gateway": "https://python.langchain.com/docs/integrations/providers/mlflow_ai_gateway",
|
||||
"Javelin AI Gateway": "https://python.langchain.com/docs/integrations/providers/javelin_ai_gateway",
|
||||
"Flyte": "https://python.langchain.com/docs/integrations/providers/flyte",
|
||||
"Arthur": "https://python.langchain.com/docs/integrations/providers/arthur_tracking",
|
||||
"Chatbots": "https://python.langchain.com/docs/use_cases/chatbots",
|
||||
@@ -1245,6 +1246,7 @@
|
||||
"Context": "https://python.langchain.com/docs/integrations/callbacks/context",
|
||||
"Label Studio": "https://python.langchain.com/docs/integrations/callbacks/labelstudio",
|
||||
"MLflow AI Gateway": "https://python.langchain.com/docs/integrations/providers/mlflow_ai_gateway",
|
||||
"Javelin AI Gateway": "https://python.langchain.com/docs/integrations/providers/javelin_ai_gateway",
|
||||
"Chatbots": "https://python.langchain.com/docs/use_cases/chatbots",
|
||||
"Conversational Retrieval Agent": "https://python.langchain.com/docs/use_cases/question_answering/how_to/conversational_retrieval_agents",
|
||||
"Structure answers with OpenAI functions": "https://python.langchain.com/docs/use_cases/question_answering/integrations/openai_functions_retrieval_qa",
|
||||
@@ -1879,6 +1881,15 @@
|
||||
"ChatMLflowAIGateway": {
|
||||
"MLflow AI Gateway": "https://python.langchain.com/docs/integrations/providers/mlflow_ai_gateway"
|
||||
},
|
||||
"JavelinAIGateway": {
|
||||
"Javelin AI Gateway": "https://python.langchain.com/docs/integrations/providers/javelin_ai_gateway"
|
||||
},
|
||||
"JavelinAIGatewayEmbeddings": {
|
||||
"Javelin AI Gateway": "https://python.langchain.com/docs/integrations/providers/javelin_ai_gateway"
|
||||
},
|
||||
"ChatJavelinAIGateway": {
|
||||
"Javelin AI Gateway": "https://python.langchain.com/docs/integrations/providers/javelin_ai_gateway"
|
||||
},
|
||||
"SingleStoreDB": {
|
||||
"SingleStoreDB": "https://python.langchain.com/docs/integrations/vectorstores/singlestoredb"
|
||||
},
|
||||
|
||||
242
docs/extras/integrations/llms/javelin.ipynb
Normal file
242
docs/extras/integrations/llms/javelin.ipynb
Normal file
@@ -0,0 +1,242 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "62bacc68-1976-44eb-9316-d5baf54bf595",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Javelin AI Gateway Tutorial\n",
|
||||
"\n",
|
||||
"This Jupyter Notebook will explore how to interact with the Javelin AI Gateway using the Python SDK. \n",
|
||||
"The Javelin AI Gateway facilitates the utilization of large language models (LLMs) like OpenAI, Cohere, Anthropic, and others by \n",
|
||||
"providing a secure and unified endpoint. The gateway itself provides a centralized mechanism to roll out models systematically, \n",
|
||||
"provide access security, policy & cost guardrails for enterprises, etc., \n",
|
||||
"\n",
|
||||
"For a complete listing of all the features & benefits of Javelin, please visit www.getjavelin.io\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e52185f8-132b-4585-b73d-6fee928ac199",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Step 1: Introduction\n",
|
||||
"[The Javelin AI Gateway](https://www.getjavelin.io) is an enterprise-grade API Gateway for AI applications. It integrates robust access security, ensuring secure interactions with large language models. Learn more in the [official documentation](https://docs.getjavelin.io).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2e2acdb3-e3b8-422b-b077-7a0d63d18349",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Step 2: Installation\n",
|
||||
"Before we begin, we must install the `javelin_sdk` and set up the Javelin API key as an environment variable. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "e91518a4-43ce-443e-b4c0-dbc652eb749f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Requirement already satisfied: javelin_sdk in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (0.1.8)\n",
|
||||
"Requirement already satisfied: httpx<0.25.0,>=0.24.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from javelin_sdk) (0.24.1)\n",
|
||||
"Requirement already satisfied: pydantic<2.0.0,>=1.10.7 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from javelin_sdk) (1.10.12)\n",
|
||||
"Requirement already satisfied: certifi in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (2023.5.7)\n",
|
||||
"Requirement already satisfied: httpcore<0.18.0,>=0.15.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (0.17.3)\n",
|
||||
"Requirement already satisfied: idna in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (3.4)\n",
|
||||
"Requirement already satisfied: sniffio in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpx<0.25.0,>=0.24.0->javelin_sdk) (1.3.0)\n",
|
||||
"Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from pydantic<2.0.0,>=1.10.7->javelin_sdk) (4.7.1)\n",
|
||||
"Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<0.25.0,>=0.24.0->javelin_sdk) (0.14.0)\n",
|
||||
"Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<0.25.0,>=0.24.0->javelin_sdk) (3.7.1)\n",
|
||||
"Note: you may need to restart the kernel to use updated packages.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"pip install 'javelin_sdk'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "53b546dc-9ca3-4602-9a7b-d733d99e8e2f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Step 3: Completions Example\n",
|
||||
"This section will demonstrate how to interact with the Javelin AI Gateway to get completions from a large language model. Here is a Python script that demonstrates this:\n",
|
||||
"(note) assumes that you have setup a route in the gateway called 'eng_dept03'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "d36949f0-5354-44ca-9a31-70c769344319",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "ImportError",
|
||||
"evalue": "cannot import name 'JavelinAIGateway' from 'langchain.llms' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/llms/__init__.py)",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
||||
"\u001b[0;31mImportError\u001b[0m Traceback (most recent call last)",
|
||||
"Cell \u001b[0;32mIn[6], line 2\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchains\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m LLMChain\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mllms\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m JavelinAIGateway\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mprompts\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m PromptTemplate\n\u001b[1;32m 5\u001b[0m route_completions \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124meng_dept03\u001b[39m\u001b[38;5;124m\"\u001b[39m\n",
|
||||
"\u001b[0;31mImportError\u001b[0m: cannot import name 'JavelinAIGateway' from 'langchain.llms' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/llms/__init__.py)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from langchain.chains import LLMChain\n",
|
||||
"from langchain.llms import JavelinAIGateway\n",
|
||||
"from langchain.prompts import PromptTemplate\n",
|
||||
"\n",
|
||||
"route_completions = \"eng_dept03\"\n",
|
||||
"\n",
|
||||
"gateway = JavelinAIGateway(\n",
|
||||
" gateway_uri=\"http://localhost:8000\", # replace with service URL or host/port of Javelin\n",
|
||||
" route=route_completions,\n",
|
||||
" model_name=\"text-davinci-003\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"prompt = PromptTemplate(\"Translate the following English text to French: {text}\")\n",
|
||||
"\n",
|
||||
"llmchain = LLMChain(llm=gateway, prompt=prompt)\n",
|
||||
"result = llmchain.run(\"podcast player\")\n",
|
||||
"\n",
|
||||
"print(result)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6b63fe93-2e77-4ea9-b8e7-dec2b96b8e95",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Step 4: Embeddings Example\n",
|
||||
"This section demonstrates how to use the Javelin AI Gateway to obtain embeddings for text queries and documents. Here is a Python script that illustrates this:\n",
|
||||
"(note) assumes that you have setup a route in the gateway called 'embeddings'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"id": "878e6c1d-be7f-49de-825c-43c266c8714e",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "ImportError",
|
||||
"evalue": "cannot import name 'JavelinAIGatewayEmbeddings' from 'langchain.embeddings' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/embeddings/__init__.py)",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
||||
"\u001b[0;31mImportError\u001b[0m Traceback (most recent call last)",
|
||||
"Cell \u001b[0;32mIn[9], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01membeddings\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m JavelinAIGatewayEmbeddings\n\u001b[1;32m 2\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01membeddings\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mopenai\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m OpenAIEmbeddings\n\u001b[1;32m 4\u001b[0m embeddings \u001b[38;5;241m=\u001b[39m JavelinAIGatewayEmbeddings(\n\u001b[1;32m 5\u001b[0m gateway_uri\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mhttp://localhost:8000\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# replace with service URL or host/port of Javelin\u001b[39;00m\n\u001b[1;32m 6\u001b[0m route\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124membeddings\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 7\u001b[0m )\n",
|
||||
"\u001b[0;31mImportError\u001b[0m: cannot import name 'JavelinAIGatewayEmbeddings' from 'langchain.embeddings' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/embeddings/__init__.py)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from langchain.embeddings import JavelinAIGatewayEmbeddings\n",
|
||||
"from langchain.embeddings.openai import OpenAIEmbeddings\n",
|
||||
"\n",
|
||||
"embeddings = JavelinAIGatewayEmbeddings(\n",
|
||||
" gateway_uri=\"http://localhost:8000\", # replace with service URL or host/port of Javelin\n",
|
||||
" route=\"embeddings\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(embeddings.embed_query(\"hello\"))\n",
|
||||
"print(embeddings.embed_documents([\"hello\"]))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "07c6691b-d333-4598-b2b7-c0933ed75937",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Step 5: Chat Example\n",
|
||||
"This section illustrates how to interact with the Javelin AI Gateway to facilitate a chat with a large language model. Here is a Python script that demonstrates this:\n",
|
||||
"(note) assumes that you have setup a route in the gateway called 'mychatbot_route'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"id": "653ef88c-36cd-4730-9c12-43c246b551f1",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "ImportError",
|
||||
"evalue": "cannot import name 'ChatJavelinAIGateway' from 'langchain.chat_models' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/chat_models/__init__.py)",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
||||
"\u001b[0;31mImportError\u001b[0m Traceback (most recent call last)",
|
||||
"Cell \u001b[0;32mIn[8], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchat_models\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m ChatJavelinAIGateway\n\u001b[1;32m 2\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mlangchain\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mschema\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m HumanMessage, SystemMessage\n\u001b[1;32m 4\u001b[0m messages \u001b[38;5;241m=\u001b[39m [\n\u001b[1;32m 5\u001b[0m SystemMessage(\n\u001b[1;32m 6\u001b[0m content\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mYou are a helpful assistant that translates English to French.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 10\u001b[0m ),\n\u001b[1;32m 11\u001b[0m ]\n",
|
||||
"\u001b[0;31mImportError\u001b[0m: cannot import name 'ChatJavelinAIGateway' from 'langchain.chat_models' (/usr/local/Caskroom/miniconda/base/lib/python3.11/site-packages/langchain/chat_models/__init__.py)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from langchain.chat_models import ChatJavelinAIGateway\n",
|
||||
"from langchain.schema import HumanMessage, SystemMessage\n",
|
||||
"\n",
|
||||
"messages = [\n",
|
||||
" SystemMessage(\n",
|
||||
" content=\"You are a helpful assistant that translates English to French.\"\n",
|
||||
" ),\n",
|
||||
" HumanMessage(\n",
|
||||
" content=\"Artificial Intelligence has the power to transform humanity and make the world a better place\"\n",
|
||||
" ),\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"chat = ChatJavelinAIGateway(\n",
|
||||
" gateway_uri=\"http://localhost:8000\", # replace with service URL or host/port of Javelin\n",
|
||||
" route=\"mychatbot_route\",\n",
|
||||
" model_name=\"gpt-3.5-turbo\",\n",
|
||||
" params={\n",
|
||||
" \"temperature\": 0.1\n",
|
||||
" }\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(chat(messages))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6eb9cf33-6505-4e05-808b-645856463a8e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Step 6: Conclusion\n",
|
||||
"This tutorial introduced the Javelin AI Gateway and demonstrated how to interact with it using the Python SDK. \n",
|
||||
"Remember to check the Javelin [Python SDK](https://www.github.com/getjavelin.io/javelin-python) for more examples and to explore the official documentation for additional details.\n",
|
||||
"\n",
|
||||
"Happy coding!"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.4"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
92
docs/extras/integrations/providers/javelin_ai_gateway.mdx
Normal file
92
docs/extras/integrations/providers/javelin_ai_gateway.mdx
Normal file
@@ -0,0 +1,92 @@
|
||||
# Javelin AI Gateway
|
||||
|
||||
[The Javelin AI Gateway](https://www.getjavelin.io) service is a high-performance, enterprise grade API Gateway for AI applications.
|
||||
It is designed to streamline the usage and access of various large language model (LLM) providers,
|
||||
such as OpenAI, Cohere, Anthropic and custom large language models within an organization by incorporating
|
||||
robust access security for all interactions with LLMs.
|
||||
|
||||
Javelin offers a high-level interface that simplifies the interaction with LLMs by providing a unified endpoint
|
||||
to handle specific LLM related requests.
|
||||
|
||||
See the Javelin AI Gateway [documentation](https://docs.getjavelin.io) for more details.
|
||||
[Javelin Python SDK](https://www.github.com/getjavelin/javelin-python) is an easy to use client library meant to be embedded into AI Applications
|
||||
|
||||
## Installation and Setup
|
||||
|
||||
Install `javelin_sdk` to interact with Javelin AI Gateway:
|
||||
|
||||
```sh
|
||||
pip install 'javelin_sdk'
|
||||
```
|
||||
|
||||
Set the Javelin's API key as an environment variable:
|
||||
|
||||
```sh
|
||||
export JAVELIN_API_KEY=...
|
||||
```
|
||||
|
||||
## Completions Example
|
||||
|
||||
```python
|
||||
|
||||
from langchain.chains import LLMChain
|
||||
from langchain.llms import JavelinAIGateway
|
||||
from langchain.prompts import PromptTemplate
|
||||
|
||||
route_completions = "eng_dept03"
|
||||
|
||||
gateway = JavelinAIGateway(
|
||||
gateway_uri="http://localhost:8000",
|
||||
route=route_completions,
|
||||
model_name="text-davinci-003",
|
||||
)
|
||||
|
||||
llmchain = LLMChain(llm=gateway, prompt=prompt)
|
||||
result = llmchain.run("podcast player")
|
||||
|
||||
print(result)
|
||||
|
||||
```
|
||||
|
||||
## Embeddings Example
|
||||
|
||||
```python
|
||||
from langchain.embeddings import JavelinAIGatewayEmbeddings
|
||||
from langchain.embeddings.openai import OpenAIEmbeddings
|
||||
|
||||
embeddings = JavelinAIGatewayEmbeddings(
|
||||
gateway_uri="http://localhost:8000",
|
||||
route="embeddings",
|
||||
)
|
||||
|
||||
print(embeddings.embed_query("hello"))
|
||||
print(embeddings.embed_documents(["hello"]))
|
||||
```
|
||||
|
||||
## Chat Example
|
||||
```python
|
||||
from langchain.chat_models import ChatJavelinAIGateway
|
||||
from langchain.schema import HumanMessage, SystemMessage
|
||||
|
||||
messages = [
|
||||
SystemMessage(
|
||||
content="You are a helpful assistant that translates English to French."
|
||||
),
|
||||
HumanMessage(
|
||||
content="Artificial Intelligence has the power to transform humanity and make the world a better place"
|
||||
),
|
||||
]
|
||||
|
||||
chat = ChatJavelinAIGateway(
|
||||
gateway_uri="http://localhost:8000",
|
||||
route="mychatbot_route",
|
||||
model_name="gpt-3.5-turbo"
|
||||
params={
|
||||
"temperature": 0.1
|
||||
}
|
||||
)
|
||||
|
||||
print(chat(messages))
|
||||
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user