github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-09 06:53:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Bagatur/clarifai update (#7324)

Browse Source 
This PR improves upon the Clarifai LangChain integration with improved docs, errors, args and the addition of embedding model support in LancChain for Clarifai's embedding models and an overview of the various ways you can integrate with Clarifai added to the docs.

---------

Co-authored-by: Matthew Zeiler <zeiler@clarifai.com>
This commit is contained in:
Bagatur
2023-07-07 02:23:20 -04:00
committed by GitHub
parent 9954eff8fd
commit a9c5b4bcea
10 changed files with 542 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

208
docs/extras/modules/data_connection/text_embedding/integrations/clarifai.ipynb Normal file
View File

@@ -0,0 +1,208 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "9597802c",
"metadata": {},
"source": [
"# Clarifai\n",
"\n",
">[Clarifai](https://www.clarifai.com/) is an AI Platform that provides the full AI lifecycle ranging from data exploration, data labeling, model training, evaluation, and inference.\n",
"\n",
"This example goes over how to use LangChain to interact with `Clarifai` [models](https://clarifai.com/explore/models). Text embedding models in particular can be found [here](https://clarifai.com/explore/models?page=1&perPage=24&filterData=%5B%7B%22field%22%3A%22model_type_id%22%2C%22value%22%3A%5B%22text-embedder%22%5D%7D%5D).\n",
"\n",
"To use Clarifai, you must have an account and a Personal Access Token (PAT) key. \n",
"[Check here](https://clarifai.com/settings/security) to get or create a PAT."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2a773d8d",
"metadata": {},
"source": [
"# Dependencies"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "91ea14ce-831d-409a-a88f-30353acdabd1",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Install required dependencies\n",
"!pip install clarifai"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "426f1156",
"metadata": {},
"source": [
"# Imports\n",
"Here we will be setting the personal access token. You can find your PAT under [settings/security](https://clarifai.com/settings/security) in your Clarifai account."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "3f5dc9d7-65e3-4b5b-9086-3327d016cfe0",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdin",
"output_type": "stream",
"text": [
" ········\n"
]
}
],
"source": [
"# Please login and get your API key from https://clarifai.com/settings/security \n",
"from getpass import getpass\n",
"\n",
"CLARIFAI_PAT = getpass()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "6fb585dd",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Import the required modules\n",
"from langchain.embeddings import ClarifaiEmbeddings\n",
"from langchain import PromptTemplate, LLMChain"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "16521ed2",
"metadata": {},
"source": [
"# Input\n",
"Create a prompt template to be used with the LLM Chain:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "035dea0f",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"template = \"\"\"Question: {question}\n",
"\n",
"Answer: Let's think step by step.\"\"\"\n",
"\n",
"prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c8905eac",
"metadata": {},
"source": [
"# Setup\n",
"Set the user id and app id to the application in which the model resides. You can find a list of public models on https://clarifai.com/explore/models\n",
"\n",
"You will have to also initialize the model id and if needed, the model version id. Some models have many versions, you can choose the one appropriate for your task."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "1fe9bf15",
"metadata": {},
"outputs": [],
"source": [
"USER_ID = 'openai'\n",
"APP_ID = 'embed'\n",
"MODEL_ID = 'text-embedding-ada'\n",
"\n",
"# You can provide a specific model version as the model_version_id arg.\n",
"# MODEL_VERSION_ID = \"MODEL_VERSION_ID\""
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "3f3458d9",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Initialize a Clarifai embedding model\n",
"embeddings = ClarifaiEmbeddings(pat=CLARIFAI_PAT, user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "a641dbd9",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"text = \"This is a test document.\""
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "32b4d5f4-2b8e-4681-856f-19a3dd141ae4",
"metadata": {},
"outputs": [],
"source": [
"query_result = embeddings.embed_query(text)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "47076457-1880-48ac-970f-872ead6f0d94",
"metadata": {},
"outputs": [],
"source": [
"doc_result = embeddings.embed_documents([text])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

12
docs/extras/modules/data_connection/vectorstores/integrations/clarifai.ipynb
View File

@@ -8,12 +8,12 @@
"source": [
"# Clarifai\n",
"\n",
">[Clarifai](https://www.clarifai.com/) is a AI Platform that provides the full AI lifecycle ranging from data exploration, data labeling, model building and inference. A Clarifai application can be used as a vector database after uploading inputs. \n",
">[Clarifai](https://www.clarifai.com/) is an AI Platform that provides the full AI lifecycle ranging from data exploration, data labeling, model training, evaluation, and inference. A Clarifai application can be used as a vector database after uploading inputs. \n",
"\n",
"This notebook shows how to use functionality related to the `Clarifai` vector database.\n",
"\n",
"To use Clarifai, you must have an account and a Personal Access Token key. \n",
"Here are the [installation instructions](https://clarifai.com/settings/security )."
"To use Clarifai, you must have an account and a Personal Access Token (PAT) key. \n",
"[Check here](https://clarifai.com/settings/security) to get or create a PAT."
]
},
{
@@ -58,7 +58,7 @@
"# Please login and get your API key from https://clarifai.com/settings/security \n",
"from getpass import getpass\n",
"\n",
"CLARIFAI_PAT_KEY = getpass()"
"CLARIFAI_PAT = getpass()"
]
},
{
@@ -92,7 +92,7 @@
"metadata": {},
"source": [
"# Setup\n",
"Setup the user id and app id where the text data will be uploaded. \n",
"Setup the user id and app id where the text data will be uploaded. Note: when creating that application please select an appropriate base workflow for indexing your text documents such as the Language-Understanding workflow.\n",
"\n",
"You will have to first create an account on [Clarifai](https://clarifai.com/login) and then create an application."
]
@@ -139,7 +139,7 @@
"metadata": {},
"outputs": [],
"source": [
"clarifai_vector_db = Clarifai.from_texts(user_id=USER_ID, app_id=APP_ID, texts=texts, pat=CLARIFAI_PAT_KEY, number_of_docs=NUMBER_OF_DOCS, metadatas = metadatas)"
"clarifai_vector_db = Clarifai.from_texts(user_id=USER_ID, app_id=APP_ID, texts=texts, pat=CLARIFAI_PAT, number_of_docs=NUMBER_OF_DOCS, metadatas = metadatas)"
]
},
{

37
docs/extras/modules/model_io/models/llms/integrations/clarifai.ipynb
View File

@@ -8,9 +8,12 @@
"source": [
"# Clarifai\n",
"\n",
">[Clarifai](https://www.clarifai.com/) is a AI Platform that provides the full AI lifecycle ranging from data exploration, data labeling, model building and inference.\n",
">[Clarifai](https://www.clarifai.com/) is an AI Platform that provides the full AI lifecycle ranging from data exploration, data labeling, model training, evaluation, and inference.\n",
"\n",
"This example goes over how to use LangChain to interact with `Clarifai` [models](https://clarifai.com/explore/models)."
"This example goes over how to use LangChain to interact with `Clarifai` [models](https://clarifai.com/explore/models). \n",
"\n",
"To use Clarifai, you must have an account and a Personal Access Token (PAT) key. \n",
"[Check here](https://clarifai.com/settings/security) to get or create a PAT."
]
},
{
@@ -42,7 +45,7 @@
"metadata": {},
"source": [
"# Imports\n",
"Here we will be setting the personal access token. You can find your PAT under settings/security on the platform."
"Here we will be setting the personal access token. You can find your PAT under [settings/security](https://clarifai.com/settings/security) in your Clarifai account."
]
},
{
@@ -52,17 +55,25 @@
"metadata": {
"tags": []
},
"outputs": [],
"outputs": [
{
"name": "stdin",
"output_type": "stream",
"text": [
" ········\n"
]
}
],
"source": [
"# Please login and get your API key from https://clarifai.com/settings/security \n",
"from getpass import getpass\n",
"\n",
"CLARIFAI_PAT_KEY = getpass()"
"CLARIFAI_PAT = getpass()"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"id": "6fb585dd",
"metadata": {
"tags": []
@@ -81,12 +92,12 @@
"metadata": {},
"source": [
"# Input\n",
"Create a prompt template to be used with the LLM Chain"
"Create a prompt template to be used with the LLM Chain:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"id": "035dea0f",
"metadata": {
"tags": []
@@ -121,10 +132,10 @@
"source": [
"USER_ID = 'openai'\n",
"APP_ID = 'chat-completion'\n",
"MODEL_ID = 'chatgpt-3_5-turbo'\n",
"MODEL_ID = 'GPT-3_5-turbo'\n",
"\n",
"# You can provide a specific model version\n",
"# model_version_id = \"MODEL_VERSION_ID\""
"# You can provide a specific model version as the model_version_id arg.\n",
"# MODEL_VERSION_ID = \"MODEL_VERSION_ID\""
]
},
{
@@ -137,7 +148,7 @@
"outputs": [],
"source": [
"# Initialize a Clarifai LLM\n",
"clarifai_llm = Clarifai(clarifai_pat_key=CLARIFAI_PAT_KEY, user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID)"
"clarifai_llm = Clarifai(pat=CLARIFAI_PAT, user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID)"
]
},
{
@@ -171,7 +182,7 @@
{
"data": {
"text/plain": [
"'Justin Bieber was born on March 1, 1994. So, we need to look at the Super Bowl that was played in the year 1994. \\n\\nThe Super Bowl in 1994 was Super Bowl XXVIII (28). It was played on January 30, 1994, between the Dallas Cowboys and the Buffalo Bills. \\n\\nThe Dallas Cowboys won the Super Bowl in 1994, defeating the Buffalo Bills by a score of 30-13. \\n\\nTherefore, the Dallas Cowboys are the NFL team that won the Super Bowl in the year Justin Bieber was born.'"
"'Justin Bieber was born on March 1, 1994. So, we need to figure out the Super Bowl winner for the 1994 season. The NFL season spans two calendar years, so the Super Bowl for the 1994 season would have taken place in early 1995. \\n\\nThe Super Bowl in question is Super Bowl XXIX, which was played on January 29, 1995. The game was won by the San Francisco 49ers, who defeated the San Diego Chargers by a score of 49-26. Therefore, the San Francisco 49ers won the Super Bowl in the year Justin Bieber was born.'"
]
},
"execution_count": 7,