mirror of
https://github.com/hwchase17/langchain.git
synced 2025-12-16 12:24:26 +00:00
7a197539aa
This PR adds native RAG support in langchain premai package. The same has been added in the docs too.
331 lines
10 KiB
Plaintext
331 lines
10 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"sidebar_label: PremAI\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# ChatPremAI\n",
|
|
"\n",
|
|
"[PremAI](https://premai.io/) is an all-in-one platform that simplifies the creation of robust, production-ready applications powered by Generative AI. By streamlining the development process, PremAI allows you to concentrate on enhancing user experience and driving overall growth for your application. You can quickly start using our platform [here](https://docs.premai.io/quick-start).\n",
|
|
"\n",
|
|
"This example goes over how to use LangChain to interact with different chat models with `ChatPremAI`"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Installation and setup\n",
|
|
"\n",
|
|
"We start by installing `langchain` and `premai-sdk`. You can type the following command to install:\n",
|
|
"\n",
|
|
"```bash\n",
|
|
"pip install premai langchain\n",
|
|
"```\n",
|
|
"\n",
|
|
"Before proceeding further, please make sure that you have made an account on PremAI and already created a project. If not, please refer to the [quick start](https://docs.premai.io/introduction) guide to get started with the PremAI platform. Create your first project and grab your API key."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_community.chat_models import ChatPremAI\n",
|
|
"from langchain_core.messages import HumanMessage, SystemMessage"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Setup PremAI client in LangChain\n",
|
|
"\n",
|
|
"Once we imported our required modules, let's setup our client. For now let's assume that our `project_id` is `8`. But make sure you use your project-id, otherwise it will throw error.\n",
|
|
"\n",
|
|
"To use langchain with prem, you do not need to pass any model name or set any parameters with our chat-client. By default it will use the model name and parameters used in the [LaunchPad](https://docs.premai.io/get-started/launchpad). \n",
|
|
"\n",
|
|
"> Note: If you change the `model` or any other parameters like `temperature` or `max_tokens` while setting the client, it will override existing default configurations, that was used in LaunchPad. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import getpass\n",
|
|
"import os\n",
|
|
"\n",
|
|
"# First step is to set up the env variable.\n",
|
|
"# you can also pass the API key while instantiating the model but this\n",
|
|
"# comes under a best practices to set it as env variable.\n",
|
|
"\n",
|
|
"if os.environ.get(\"PREMAI_API_KEY\") is None:\n",
|
|
" os.environ[\"PREMAI_API_KEY\"] = getpass.getpass(\"PremAI API Key:\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# By default it will use the model which was deployed through the platform\n",
|
|
"# in my case it will is \"claude-3-haiku\"\n",
|
|
"\n",
|
|
"chat = ChatPremAI(project_id=8)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Chat Completions\n",
|
|
"\n",
|
|
"`ChatPremAI` supports two methods: `invoke` (which is the same as `generate`) and `stream`. \n",
|
|
"\n",
|
|
"The first one will give us a static result. Whereas the second one will stream tokens one by one. Here's how you can generate chat-like completions. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"I am an artificial intelligence created by Anthropic. I'm here to help with a wide variety of tasks, from research and analysis to creative projects and open-ended conversation. I have general knowledge and capabilities, but I'm not a real person - I'm an AI assistant. Please let me know if you have any other questions!\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"human_message = HumanMessage(content=\"Who are you?\")\n",
|
|
"\n",
|
|
"response = chat.invoke([human_message])\n",
|
|
"print(response.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Above looks interesting right? I set my default lanchpad system-prompt as: `Always sound like a pirate` You can also, override the default system prompt if you need to. Here's how you can do it. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content=\"I am an artificial intelligence created by Anthropic. My purpose is to assist and converse with humans in a friendly and helpful way. I have a broad knowledge base that I can use to provide information, answer questions, and engage in discussions on a wide range of topics. Please let me know if you have any other questions - I'm here to help!\")"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"system_message = SystemMessage(content=\"You are a friendly assistant.\")\n",
|
|
"human_message = HumanMessage(content=\"Who are you?\")\n",
|
|
"\n",
|
|
"chat.invoke([system_message, human_message])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can provide system prompt here like this:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content='I am an artificial intelligence created by Anthropic')"
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"chat.invoke([system_message, human_message], temperature=0.7, max_tokens=10, top_p=0.95)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> If you are going to place system prompt here, then it will override your system prompt that was fixed while deploying the application from the platform. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Native RAG Support with Prem Repositories\n",
|
|
"\n",
|
|
"Prem Repositories which allows users to upload documents (.txt, .pdf etc) and connect those repositories to the LLMs. You can think Prem repositories as native RAG, where each repository can be considered as a vector database. You can connect multiple repositories. You can learn more about repositories [here](https://docs.premai.io/get-started/repositories).\n",
|
|
"\n",
|
|
"Repositories are also supported in langchain premai. Here is how you can do it. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"query = \"what is the diameter of individual Galaxy\"\n",
|
|
"repository_ids = [\n",
|
|
" 1991,\n",
|
|
"]\n",
|
|
"repositories = dict(ids=repository_ids, similarity_threshold=0.3, limit=3)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"First we start by defining our repository with some repository ids. Make sure that the ids are valid repository ids. You can learn more about how to get the repository id [here](https://docs.premai.io/get-started/repositories). \n",
|
|
"\n",
|
|
"> Please note: Similar like `model_name` when you invoke the argument `repositories`, then you are potentially overriding the repositories connected in the launchpad. \n",
|
|
"\n",
|
|
"Now, we connect the repository with our chat object to invoke RAG based generations. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import json\n",
|
|
"\n",
|
|
"response = chat.invoke(query, max_tokens=100, repositories=repositories)\n",
|
|
"\n",
|
|
"print(response.content)\n",
|
|
"print(json.dumps(response.response_metadata, indent=4))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> Ideally, you do not need to connect Repository IDs here to get Retrieval Augmented Generations. You can still get the same result if you have connected the repositories in prem platform. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Streaming\n",
|
|
"\n",
|
|
"In this section, let's see how we can stream tokens using langchain and PremAI. Here's how you do it. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Hello! As an AI language model, I don't have feelings or a physical state, but I'm functioning properly and ready to assist you with any questions or tasks you might have. How can I help you today?"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import sys\n",
|
|
"\n",
|
|
"for chunk in chat.stream(\"hello how are you\"):\n",
|
|
" sys.stdout.write(chunk.content)\n",
|
|
" sys.stdout.flush()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Similar to above, if you want to override the system-prompt and the generation parameters, you need to add the following:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Hello! As an AI language model, I don't have feelings or a physical form, but I'm functioning properly and ready to assist you. How can I help you today?"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import sys\n",
|
|
"\n",
|
|
"# For some experimental reasons if you want to override the system prompt then you\n",
|
|
"# can pass that here too. However it is not recommended to override system prompt\n",
|
|
"# of an already deployed model.\n",
|
|
"\n",
|
|
"for chunk in chat.stream(\n",
|
|
" \"hello how are you\",\n",
|
|
" system_prompt=\"act like a dog\",\n",
|
|
" temperature=0.7,\n",
|
|
" max_tokens=200,\n",
|
|
"):\n",
|
|
" sys.stdout.write(chunk.content)\n",
|
|
" sys.stdout.flush()"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "venv",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.7"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|