github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-07-16 17:53:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

IBM DB2 vector store documentation addition (#31008)

Browse Source
This commit is contained in:
Yiwei 2025-06-27 11:34:32 -07:00 committed by GitHub
parent 9f17fabc43
commit 375f53adac
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 432 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
docs/docs/integrations/providers/ibm.mdx
View File

@ -1,7 +1,9 @@
# IBM
The `LangChain` integrations related to [IBM watsonx.ai](https://www.ibm.com/products/watsonx-ai) platform.
LangChain integrations related to IBM technologies, including the
[IBM watsonx.ai](https://www.ibm.com/products/watsonx-ai) platform and DB2 database.
## Watsonx AI
IBM® watsonx.ai™ AI studio is part of the IBM [watsonx](https://www.ibm.com/watsonx)™ AI and data platform, bringing together new generative
AI capabilities powered by [foundation models](https://www.ibm.com/products/watsonx-ai/foundation-models) and traditional machine learning (ML)
into a powerful studio spanning the AI lifecycle. Tune and guide models with your enterprise data to meet your needs with easy-to-use tools for
@ -14,7 +16,7 @@ Watsonx.ai offers:
- **Hybrid, multi-cloud deployments:** IBM provides the flexibility to integrate and deploy your AI workloads into your hybrid-cloud stack of choice.
## Installation and Setup
### Installation and Setup
Install the integration package with
```bash
@ -28,9 +30,9 @@ import os
os.environ["WATSONX_APIKEY"] = "your IBM watsonx.ai api key"
```
## Chat Model
### Chat Model
### ChatWatsonx
#### ChatWatsonx
See a [usage example](/docs/integrations/chat/ibm_watsonx).
@ -38,9 +40,9 @@ See a [usage example](/docs/integrations/chat/ibm_watsonx).
from langchain_ibm import ChatWatsonx
```
## LLMs
### LLMs
### WatsonxLLM
#### WatsonxLLM
See a [usage example](/docs/integrations/llms/ibm_watsonx).
@ -48,9 +50,9 @@ See a [usage example](/docs/integrations/llms/ibm_watsonx).
from langchain_ibm import WatsonxLLM
```
## Embedding Models
### Embedding Models
### WatsonxEmbeddings
#### WatsonxEmbeddings
See a [usage example](/docs/integrations/text_embedding/ibm_watsonx).
@ -58,9 +60,9 @@ See a [usage example](/docs/integrations/text_embedding/ibm_watsonx).
from langchain_ibm import WatsonxEmbeddings
```
## Reranker
### Reranker
### WatsonxRerank
#### WatsonxRerank
See a [usage example](/docs/integrations/retrievers/ibm_watsonx_ranker).
@ -68,12 +70,35 @@ See a [usage example](/docs/integrations/retrievers/ibm_watsonx_ranker).
from langchain_ibm import WatsonxRerank
```
## Toolkit
### Toolkit
### WatsonxToolkit
#### WatsonxToolkit
See a [usage example](/docs/integrations/tools/ibm_watsonx).
```python
from langchain_ibm import WatsonxToolkit
```
## DB2
### Vector stores
#### IBM DB2 Vector Store and Vector Search
The IBM DB2 relational database v12.1.2 and above offers the abilities of vector store
and vector search. Installation of `langchain-db2` package will give Langchain users
the support of DB2 vector store and vector search.
See detailed usage examples in the guide [here](/docs/integrations/vectorstores/db2).
Installation: This is a seperate package for vector store feature only and can be run
without the `langchain-ibm` package.
```bash
pip install -U langchain-db2
```
Usage:
```python
from langchain_db2 import db2vs
from langchain_db2.db2vs import DB2VS
```

391
docs/docs/integrations/vectorstores/db2.ipynb Normal file
View File

@ -0,0 +1,391 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "dd33e9d5-9dba-4aac-9f7f-4cf9e6686593",
"metadata": {},
"source": [
"# IBM Db2 Vector Store and Vector Search\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "67520bec",
"metadata": {},
"source": [
"LangChain's Db2 integration (langchain-db2) provides vector store and vector search capabilities for working with IBM relational database Db2 version v12.1.2 and above, distributed under the MIT license. Users can use the provided implementations as-is or customize them for specific needs.\n",
" Key features include:\n",
"\n",
" * Vector storage with metadata\n",
" * Vector similarity search and max marginal relevance search, with metadata filtering options\n",
" * Support for dot production, cosine, and euclidean distance metrics\n",
" * Performance optimization by index creation and Approximate nearest neighbors search. (Will be added shortly)"
]
},
{
"cell_type": "markdown",
"id": "bc94b35a",
"metadata": {},
"source": [
"## Setup"
]
},
{
"cell_type": "markdown",
"id": "7bd80054-c803-47e1-a259-c40ed073c37d",
"metadata": {},
"source": [
"### Prerequisites for using Langchain with Db2 Vector Store and Search\n",
"\n",
"Install package `langchain-db2` which is the integration package for the db2 LangChain Vector Store and Search.\n",
"\n",
"The installation of the package should also install its dependencies like `langchain-core` and `ibm_db`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2bbb989d-c6fb-4ab9-bafd-a95fd48538d0",
"metadata": {},
"outputs": [],
"source": [
"# pip install -U langchain-db2"
]
},
{
"cell_type": "markdown",
"id": "0fceaa5a-95da-4ebd-8b8d-5e73bb653172",
"metadata": {},
"source": [
"### Connect to Db2 Vector Store\n",
"\n",
"The following sample code will show how to connect to Db2 Database. Besides the dependencies above, you will need a Db2 database instance (with version v12.1.2+, which has the vector datatype support) running."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4421e4b7-2c7e-4bcd-82b3-9576595edd0f",
"metadata": {},
"outputs": [],
"source": [
"import ibm_db\n",
"import ibm_db_dbi\n",
"\n",
"database = \"\"\n",
"username = \"\"\n",
"password = \"\"\n",
"\n",
"try:\n",
" connection = ibm_db_dbi.connect(database, username, password)\n",
" print(\"Connection successful!\")\n",
"except Exception as e:\n",
" print(\"Connection failed!\")"
]
},
{
"cell_type": "markdown",
"id": "b11cf362-01b0-485d-8527-31b0fbb5028e",
"metadata": {},
"source": [
"### Import the required dependencies"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "43ea59e3-2910-45a6-b195-5f06094bb7c9",
"metadata": {},
"outputs": [],
"source": [
"from langchain_community.embeddings import HuggingFaceEmbeddings\n",
"from langchain_community.vectorstores.utils import DistanceStrategy\n",
"from langchain_core.documents import Document\n",
"from langchain_db2 import db2vs\n",
"from langchain_db2.db2vs import DB2VS"
]
},
{
"cell_type": "markdown",
"id": "66d56383",
"metadata": {},
"source": [
"## Initialization"
]
},
{
"cell_type": "markdown",
"id": "0aac10dc-a9cc-4fdb-901c-1b7a4bbbe5a7",
"metadata": {},
"source": [
"### Create Documents"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "70ac6982-b13a-4e8c-9c47-57c6d136ac60",
"metadata": {},
"outputs": [],
"source": [
"# Define a list of documents\n",
"documents_json_list = [\n",
" {\n",
" \"id\": \"doc_1_2_P4\",\n",
" \"text\": \"Db2 handles LOB data differently than other kinds of data. As a result, you sometimes need to take additional actions when you define LOB columns and insert the LOB data.\",\n",
" \"link\": \"https://www.ibm.com/docs/en/db2-for-zos/12?topic=programs-storing-lob-data-in-tables\",\n",
" },\n",
" {\n",
" \"id\": \"doc_11.1.0_P1\",\n",
" \"text\": \"Db2® column-organized tables add columnar capabilities to Db2 databases, which include data that is stored with column organization and vector processing of column data. Using this table format with star schema data marts provides significant improvements to storage, query performance, and ease of use through simplified design and tuning.\",\n",
" \"link\": \"https://www.ibm.com/docs/en/db2/11.1.0?topic=organization-column-organized-tables\",\n",
" },\n",
" {\n",
" \"id\": \"id_22.3.4.3.1_P2\",\n",
" \"text\": \"Data structures are elements that are required to use Db2®. You can access and use these elements to organize your data. Examples of data structures include tables, table spaces, indexes, index spaces, keys, views, and databases.\",\n",
" \"link\": \"https://www.ibm.com/docs/en/zos-basic-skills?topic=concepts-db2-data-structures\",\n",
" },\n",
" {\n",
" \"id\": \"id_3.4.3.1_P3\",\n",
" \"text\": \"Db2® maintains a set of tables that contain information about the data that Db2 controls. These tables are collectively known as the catalog. The catalog tables contain information about Db2 objects such as tables, views, and indexes. When you create, alter, or drop an object, Db2 inserts, updates, or deletes rows of the catalog that describe the object.\",\n",
" \"link\": \"https://www.ibm.com/docs/en/zos-basic-skills?topic=objects-db2-catalog\",\n",
" },\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eaa942d6-5954-4898-8c32-3627b923a3a5",
"metadata": {},
"outputs": [],
"source": [
"# Create Langchain Documents\n",
"\n",
"documents_langchain = []\n",
"\n",
"for doc in documents_json_list:\n",
" metadata = {\"id\": doc[\"id\"], \"link\": doc[\"link\"]}\n",
" doc_langchain = Document(page_content=doc[\"text\"], metadata=metadata)\n",
" documents_langchain.append(doc_langchain)"
]
},
{
"cell_type": "markdown",
"id": "6823f5e6-997c-4f15-927b-bd44c61f105f",
"metadata": {},
"source": [
"### Create Vector Stores with different distance metrics\n",
"\n",
"First we will create three vector stores each with different distance strategies. \n",
"\n",
"(You can manually connect to the Db2 Database and will see three tables : \n",
"Documents_DOT, Documents_COSINE and Documents_EUCLIDEAN. )\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ed1b253e-5f5c-4a81-983c-74645213a170",
"metadata": {},
"outputs": [],
"source": [
"# Create Db2 Vector Stores using different distance strategies\n",
"\n",
"# When using our API calls, start by initializing your vector store with a subset of your documents\n",
"# through from_documents(), then incrementally add more documents using add_texts().\n",
"# This approach prevents system overload and ensures efficient document processing.\n",
"\n",
"model = HuggingFaceEmbeddings(model_name=\"sentence-transformers/all-mpnet-base-v2\")\n",
"\n",
"vector_store_dot = DB2VS.from_documents(\n",
" documents_langchain,\n",
" model,\n",
" client=connection,\n",
" table_name=\"Documents_DOT\",\n",
" distance_strategy=DistanceStrategy.DOT_PRODUCT,\n",
")\n",
"vector_store_max = DB2VS.from_documents(\n",
" documents_langchain,\n",
" model,\n",
" client=connection,\n",
" table_name=\"Documents_COSINE\",\n",
" distance_strategy=DistanceStrategy.COSINE,\n",
")\n",
"vector_store_euclidean = DB2VS.from_documents(\n",
" documents_langchain,\n",
" model,\n",
" client=connection,\n",
" table_name=\"Documents_EUCLIDEAN\",\n",
" distance_strategy=DistanceStrategy.EUCLIDEAN_DISTANCE,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "804b9142",
"metadata": {},
"source": [
"## Manage vector store"
]
},
{
"cell_type": "markdown",
"id": "77c29505-8688-4b87-9a99-e648fbb2d425",
"metadata": {},
"source": [
"### Demonstrating add and delete operations for texts, along with basic similarity search\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "306563ae-577b-4bc7-8a92-3dd6a59310f5",
"metadata": {},
"outputs": [],
"source": [
"def manage_texts(vector_stores):\n",
" \"\"\"\n",
" Adds texts to each vector store, demonstrates error handling for duplicate additions,\n",
" and performs deletion of texts. Showcases similarity searches and index creation for each vector store.\n",
"\n",
" Args:\n",
" - vector_stores (list): A list of DB2VS instances.\n",
" \"\"\"\n",
" texts = [\"Rohan\", \"Shailendra\"]\n",
" metadata = [\n",
" {\"id\": \"100\", \"link\": \"Document Example Test 1\"},\n",
" {\"id\": \"101\", \"link\": \"Document Example Test 2\"},\n",
" ]\n",
"\n",
" for i, vs in enumerate(vector_stores, start=1):\n",
" # Adding texts\n",
" try:\n",
" vs.add_texts(texts, metadata)\n",
" print(f\"\\n\\n\\nAdd texts complete for vector store {i}\\n\\n\\n\")\n",
" except Exception as ex:\n",
" print(f\"\\n\\n\\nExpected error on duplicate add for vector store {i}\\n\\n\\n\")\n",
"\n",
" # Deleting texts using the value of 'id'\n",
" vs.delete([metadata[0][\"id\"], metadata[1][\"id\"]])\n",
" print(f\"\\n\\n\\nDelete texts complete for vector store {i}\\n\\n\\n\")\n",
"\n",
" # Similarity search\n",
" results = vs.similarity_search(\"How are LOBS stored in Db2 Database\", 2)\n",
" print(f\"\\n\\n\\nSimilarity search results for vector store {i}: {results}\\n\\n\\n\")\n",
"\n",
"\n",
"vector_store_list = [\n",
" vector_store_dot,\n",
" vector_store_max,\n",
" vector_store_euclidean,\n",
"]\n",
"manage_texts(vector_store_list)"
]
},
{
"cell_type": "markdown",
"id": "37ebcb44",
"metadata": {},
"source": [
"## Query vector store"
]
},
{
"cell_type": "markdown",
"id": "7223d048-5c0b-4e91-a91b-a7daa9f86758",
"metadata": {},
"source": [
"### Demonstrate advanced searches on vector stores, with and without attribute filtering \n",
"\n",
"With filtering, we only select the document id 101 and nothing else"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "37ca2e7d-9803-4260-95e7-62776d4fb820",
"metadata": {},
"outputs": [],
"source": [
"# Conduct advanced searches\n",
"def conduct_advanced_searches(vector_stores):\n",
" query = \"How are LOBS stored in Db2 Database\"\n",
" # Constructing a filter for direct comparison against document metadata\n",
" # This filter aims to include documents whose metadata 'id' is exactly '101'\n",
" filter_criteria = {\"id\": [\"101\"]} # Direct comparison filter\n",
"\n",
" for i, vs in enumerate(vector_stores, start=1):\n",
" print(f\"\\n--- Vector Store {i} Advanced Searches ---\")\n",
" # Similarity search without a filter\n",
" print(\"\\nSimilarity search results without filter:\")\n",
" print(vs.similarity_search(query, 2))\n",
"\n",
" # Similarity search with a filter\n",
" print(\"\\nSimilarity search results with filter:\")\n",
" print(vs.similarity_search(query, 2, filter=filter_criteria))\n",
"\n",
" # Similarity search with relevance score\n",
" print(\"\\nSimilarity search with relevance score:\")\n",
" print(vs.similarity_search_with_score(query, 2))\n",
"\n",
" # Similarity search with relevance score with filter\n",
" print(\"\\nSimilarity search with relevance score with filter:\")\n",
" print(vs.similarity_search_with_score(query, 2, filter=filter_criteria))\n",
"\n",
" # Max marginal relevance search\n",
" print(\"\\nMax marginal relevance search results:\")\n",
" print(vs.max_marginal_relevance_search(query, 2, fetch_k=20, lambda_mult=0.5))\n",
"\n",
" # Max marginal relevance search with filter\n",
" print(\"\\nMax marginal relevance search results with filter:\")\n",
" print(\n",
" vs.max_marginal_relevance_search(\n",
" query, 2, fetch_k=20, lambda_mult=0.5, filter=filter_criteria\n",
" )\n",
" )\n",
"\n",
"\n",
"conduct_advanced_searches(vector_store_list)"
]
},
{
"cell_type": "markdown",
"id": "d7db8d14",
"metadata": {},
"source": [
"## Usage for retrieval-augmented generation"
]
},
{
"cell_type": "markdown",
"id": "eb5e7288",
"metadata": {},
"source": [
"## API reference"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "pythonEnv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.21"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

4
libs/packages.yml
View File

@ -683,3 +683,7 @@ packages:
repo: surrealdb/langchain-surrealdb
downloads: 291
downloads_updated_at: '2025-06-25T20:06:46.206625+00:00'
- name: langchain-db2
path: libs/langchain-db2
repo: langchain-ai/langchain-ibm
provider_page: ibm