mirror of
https://github.com/hwchase17/langchain.git
synced 2025-10-26 13:21:40 +00:00
e2bc8f19c0
## **Description:** This PR updates the internal documentation link for the RAG tutorials to reflect the updated path. Previously, the link pointed to the root `/docs/tutorials/`, which was generic. It now correctly routes to the RAG-specific tutorial page for the following vector store docs. 1. AstraDBVectorStore 2. Clickhouse 3. CouchbaseSearchVectorStore 4. DatabricksVectorSearch 5. ElasticsearchStore 6. FAISS 7. Milvus 8. MongoDBAtlasVectorSearch 9. openGauss 10. PGVector 11. PGVectorStore 12. PineconeVectorStore 13. QdrantVectorStore 14. Redis 15. SQLServer ## **Issue:** N/A ## **Dependencies:** None ## **Twitter handle:** N/A
383 lines
11 KiB
Plaintext
383 lines
11 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "683953b3",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Pinecone\n",
|
|
"\n",
|
|
">[Pinecone](https://docs.pinecone.io/docs/overview) is a vector database with broad functionality.\n",
|
|
"\n",
|
|
"This notebook shows how to use functionality related to the `Pinecone` vector database.\n",
|
|
"\n",
|
|
"## Setup\n",
|
|
"\n",
|
|
"To use the `PineconeVectorStore` you first need to install the partner package, as well as the other packages used throughout this notebook."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b4c41cad-08ef-4f72-a545-2151e4598efe",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"pip install -qU langchain langchain-pinecone langchain-openai"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1917d123",
|
|
"metadata": {},
|
|
"source": [
|
|
"Migration note: if you are migrating from the `langchain_community.vectorstores` implementation of Pinecone, you may need to remove your `pinecone-client` v2 dependency before installing `langchain-pinecone`, which relies on `pinecone-client` v6."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ef6dc4de",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Credentials\n",
|
|
"\n",
|
|
"Create a new Pinecone account, or sign into your existing one, and create an API key to use in this notebook."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "eb554814",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import getpass\n",
|
|
"import os\n",
|
|
"\n",
|
|
"from pinecone import Pinecone\n",
|
|
"\n",
|
|
"if not os.getenv(\"PINECONE_API_KEY\"):\n",
|
|
" os.environ[\"PINECONE_API_KEY\"] = getpass.getpass(\"Enter your Pinecone API key: \")\n",
|
|
"\n",
|
|
"pinecone_api_key = os.environ.get(\"PINECONE_API_KEY\")\n",
|
|
"\n",
|
|
"pc = Pinecone(api_key=pinecone_api_key)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6ef1d828",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "23b5ac5e",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n",
|
|
"# os.environ[\"LANGSMITH_TRACING\"] = \"true\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "658706a3",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Initialization\n",
|
|
"\n",
|
|
"Before initializing our vector store, let's connect to a Pinecone index. If one named `index_name` doesn't exist, it will be created."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "276a06dd",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from pinecone import ServerlessSpec\n",
|
|
"\n",
|
|
"index_name = \"langchain-test-index\" # change if desired\n",
|
|
"\n",
|
|
"if not pc.has_index(index_name):\n",
|
|
" pc.create_index(\n",
|
|
" name=index_name,\n",
|
|
" dimension=1536,\n",
|
|
" metric=\"cosine\",\n",
|
|
" spec=ServerlessSpec(cloud=\"aws\", region=\"us-east-1\"),\n",
|
|
" )\n",
|
|
"\n",
|
|
"index = pc.Index(index_name)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "1485db56",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_openai import OpenAIEmbeddings\n",
|
|
"\n",
|
|
"embeddings = OpenAIEmbeddings(model=\"text-embedding-3-small\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "6e104aee",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_pinecone import PineconeVectorStore\n",
|
|
"\n",
|
|
"vector_store = PineconeVectorStore(index=index, embedding=embeddings)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "48721e29",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Manage vector store\n",
|
|
"\n",
|
|
"Once you have created your vector store, we can interact with it by adding and deleting different items.\n",
|
|
"\n",
|
|
"### Add items to vector store\n",
|
|
"\n",
|
|
"We can add items to our vector store by using the `add_documents` function."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "70e688f4",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from uuid import uuid4\n",
|
|
"\n",
|
|
"from langchain_core.documents import Document\n",
|
|
"\n",
|
|
"document_1 = Document(\n",
|
|
" page_content=\"I had chocolate chip pancakes and scrambled eggs for breakfast this morning.\",\n",
|
|
" metadata={\"source\": \"tweet\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_2 = Document(\n",
|
|
" page_content=\"The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees.\",\n",
|
|
" metadata={\"source\": \"news\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_3 = Document(\n",
|
|
" page_content=\"Building an exciting new project with LangChain - come check it out!\",\n",
|
|
" metadata={\"source\": \"tweet\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_4 = Document(\n",
|
|
" page_content=\"Robbers broke into the city bank and stole $1 million in cash.\",\n",
|
|
" metadata={\"source\": \"news\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_5 = Document(\n",
|
|
" page_content=\"Wow! That was an amazing movie. I can't wait to see it again.\",\n",
|
|
" metadata={\"source\": \"tweet\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_6 = Document(\n",
|
|
" page_content=\"Is the new iPhone worth the price? Read this review to find out.\",\n",
|
|
" metadata={\"source\": \"website\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_7 = Document(\n",
|
|
" page_content=\"The top 10 soccer players in the world right now.\",\n",
|
|
" metadata={\"source\": \"website\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_8 = Document(\n",
|
|
" page_content=\"LangGraph is the best framework for building stateful, agentic applications!\",\n",
|
|
" metadata={\"source\": \"tweet\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_9 = Document(\n",
|
|
" page_content=\"The stock market is down 500 points today due to fears of a recession.\",\n",
|
|
" metadata={\"source\": \"news\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"document_10 = Document(\n",
|
|
" page_content=\"I have a bad feeling I am going to get deleted :(\",\n",
|
|
" metadata={\"source\": \"tweet\"},\n",
|
|
")\n",
|
|
"\n",
|
|
"documents = [\n",
|
|
" document_1,\n",
|
|
" document_2,\n",
|
|
" document_3,\n",
|
|
" document_4,\n",
|
|
" document_5,\n",
|
|
" document_6,\n",
|
|
" document_7,\n",
|
|
" document_8,\n",
|
|
" document_9,\n",
|
|
" document_10,\n",
|
|
"]\n",
|
|
"uuids = [str(uuid4()) for _ in range(len(documents))]\n",
|
|
"vector_store.add_documents(documents=documents, ids=uuids)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "120922b3",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Delete items from vector store"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "5b8437cd",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"vector_store.delete(ids=[uuids[-1]])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5ee21c89",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Query vector store\n",
|
|
"\n",
|
|
"Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. \n",
|
|
"\n",
|
|
"### Query directly\n",
|
|
"\n",
|
|
"Performing a simple similarity search can be done as follows:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "ffbcb3fb",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"results = vector_store.similarity_search(\n",
|
|
" \"LangChain provides abstractions to make working with LLMs easy\",\n",
|
|
" k=2,\n",
|
|
" filter={\"source\": \"tweet\"},\n",
|
|
")\n",
|
|
"for res in results:\n",
|
|
" print(f\"* {res.page_content} [{res.metadata}]\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "79f3494d",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Similarity search with score\n",
|
|
"\n",
|
|
"You can also search with score:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "5fb24583",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"results = vector_store.similarity_search_with_score(\n",
|
|
" \"Will it be hot tomorrow?\", k=1, filter={\"source\": \"news\"}\n",
|
|
")\n",
|
|
"for res, score in results:\n",
|
|
" print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1855941b",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Other search methods\n",
|
|
"\n",
|
|
"There are more search methods (such as MMR) not listed in this notebook, to find all of them be sure to read the [API reference](https://python.langchain.com/api_reference/pinecone/vectorstores/langchain_pinecone.vectorstores.PineconeVectorStore.html).\n",
|
|
"\n",
|
|
"### Query by turning into retriever\n",
|
|
"\n",
|
|
"You can also transform the vector store into a retriever for easier usage in your chains."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "78140e87",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"retriever = vector_store.as_retriever(\n",
|
|
" search_type=\"similarity_score_threshold\",\n",
|
|
" search_kwargs={\"k\": 1, \"score_threshold\": 0.4},\n",
|
|
")\n",
|
|
"retriever.invoke(\"Stealing from the bank is a crime\", filter={\"source\": \"news\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "72990cb5",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Usage for retrieval-augmented generation\n",
|
|
"\n",
|
|
"For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n",
|
|
"\n",
|
|
"- [Tutorials](/docs/tutorials/rag)\n",
|
|
"- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n",
|
|
"- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/retrieval)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0d5722bc",
|
|
"metadata": {},
|
|
"source": [
|
|
"## API reference\n",
|
|
"\n",
|
|
"For detailed documentation of all features and configurations head to the API reference: https://python.langchain.com/api_reference/pinecone/vectorstores/langchain_pinecone.vectorstores.PineconeVectorStore.html"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": ".venv",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.15"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|