mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-26 16:43:35 +00:00
docs: add Gel integration (#31186)
Continued from https://github.com/langchain-ai/langchain/pull/31050 --------- Co-authored-by: deepbuzin <contactbuzin@gmail.com>
This commit is contained in:
ccurme
GitHub
parent
65fbbb0249
commit
ff9183fd3c
56
docs/docs/integrations/providers/gel.mdx
Normal file
56
docs/docs/integrations/providers/gel.mdx
Normal file
@ -0,0 +1,56 @@
|
||||
# Gel
|
||||
|
||||
[Gel](https://www.geldata.com/) is a powerful data platform built on top of PostgreSQL.
|
||||
|
||||
- Think in objects and graphs instead of tables and JOINs.
|
||||
- Use the advanced Python SDK, integrated GUI, migrations engine, Auth and AI layers, and much more.
|
||||
- Run locally, remotely, or in a [fully managed cloud](https://www.geldata.com/cloud).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install langchain-gel
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
1. Run `gel project init`
|
||||
2. Edit the schema. You need the following types to use the LangChain vectorstore:
|
||||
|
||||
```gel
|
||||
using extension pgvector;
|
||||
|
||||
module default {
|
||||
scalar type EmbeddingVector extending ext::pgvector::vector<1536>;
|
||||
|
||||
type Record {
|
||||
required collection: str;
|
||||
text: str;
|
||||
embedding: EmbeddingVector;
|
||||
external_id: str {
|
||||
constraint exclusive;
|
||||
};
|
||||
metadata: json;
|
||||
|
||||
index ext::pgvector::hnsw_cosine(m := 16, ef_construction := 128)
|
||||
on (.embedding)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> Note: this is the minimal setup. Feel free to add as many types, properties and links as you want!
|
||||
> Learn more about taking advantage of Gel's schema by reading the [docs](https://docs.geldata.com/learn/schema).
|
||||
|
||||
3. Run the migration: `gel migration create && gel migrate`.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from langchain_gel import GelVectorStore
|
||||
|
||||
vector_store = GelVectorStore(
|
||||
embeddings=embeddings,
|
||||
)
|
||||
```
|
||||
|
||||
See the full usage example [here](/docs/integrations/vectorstores/gel).
|
||||
450
docs/docs/integrations/vectorstores/gel.ipynb
Normal file
450
docs/docs/integrations/vectorstores/gel.ipynb
Normal file
@ -0,0 +1,450 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7679dd7b-7ed4-4755-a499-824deadba708",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Gel \n",
|
||||
"\n",
|
||||
"> An implementation of LangChain vectorstore abstraction using `gel` as the backend.\n",
|
||||
"\n",
|
||||
"[Gel](https://www.geldata.com/) is an open-source PostgreSQL data layer optimized for fast development to production cycle. It comes with a high-level strictly typed graph-like data model, composable hierarchical query language, full SQL support, migrations, Auth and AI modules.\n",
|
||||
"\n",
|
||||
"The code lives in an integration package called [langchain-gel](https://github.com/geldata/langchain-gel).\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"First install relevant packages:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "92df32f0",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! pip install -qU gel langchain-gel "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "68ef6ebb",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Initialization\n",
|
||||
"\n",
|
||||
"In order to use Gel as a backend for your `VectorStore`, you're going to need a working Gel instance.\n",
|
||||
"Fortunately, it doesn't have to involve Docker containers or anything complicated, unless you want to!\n",
|
||||
"\n",
|
||||
"To set up a local instance, run:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "b79938d3",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! gel project init --non-interactive"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "08e79230",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you are using [Gel Cloud](https://cloud.geldata.com/) (and you should!), add one more argument to that command:\n",
|
||||
"\n",
|
||||
"```bash\n",
|
||||
"gel project init --server-instance <org-name>/<instance-name>\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"For a comprehensive list of ways to run Gel, take a look at [Running Gel](https://docs.geldata.com/reference/running) section of the reference docs.\n",
|
||||
"\n",
|
||||
"### Set up the schema\n",
|
||||
"\n",
|
||||
"[Gel schema](https://docs.geldata.com/reference/datamodel) is an explicit high-level description of your application's data model. \n",
|
||||
"Aside from enabling you to define exactly how your data is going to be laid out, it drives Gel's many powerful features such as links, access policies, functions, triggers, constraints, indexes, and more.\n",
|
||||
"\n",
|
||||
"The LangChain's `VectorStore` expects the following layout for the schema:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"id": "9a7edd58",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"schema_content = \"\"\"\n",
|
||||
"using extension pgvector;\n",
|
||||
" \n",
|
||||
"module default {\n",
|
||||
" scalar type EmbeddingVector extending ext::pgvector::vector<1536>;\n",
|
||||
"\n",
|
||||
" type Record {\n",
|
||||
" required collection: str;\n",
|
||||
" text: str;\n",
|
||||
" embedding: EmbeddingVector; \n",
|
||||
" external_id: str {\n",
|
||||
" constraint exclusive;\n",
|
||||
" };\n",
|
||||
" metadata: json;\n",
|
||||
"\n",
|
||||
" index ext::pgvector::hnsw_cosine(m := 16, ef_construction := 128)\n",
|
||||
" on (.embedding)\n",
|
||||
" } \n",
|
||||
"}\n",
|
||||
"\"\"\".strip()\n",
|
||||
"\n",
|
||||
"with open(\"dbschema/default.gel\", \"w\") as f:\n",
|
||||
" f.write(schema_content)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "90320ef1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In order to apply schema changes to the database, run a migration using Gel's [migration mechanism](https://docs.geldata.com/reference/datamodel/migrations):"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "cdff483e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! gel migration create --non-interactive\n",
|
||||
"! gel migrate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b2290ef2",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"From this point onward, `GelVectorStore` can be used as a drop-in replacement for any other vectorstore available in LangChain."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ec44dfcc",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Instantiation\n",
|
||||
"\n",
|
||||
"import EmbeddingTabs from \"@theme/EmbeddingTabs\";\n",
|
||||
"\n",
|
||||
"<EmbeddingTabs/>\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "94f5c129",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# | output: false\n",
|
||||
"# | echo: false\n",
|
||||
"from langchain_openai import OpenAIEmbeddings\n",
|
||||
"\n",
|
||||
"embeddings = OpenAIEmbeddings(model=\"text-embedding-3-small\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "979a65bd-742f-4b0d-be1e-c0baae245ec6",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain_gel import GelVectorStore\n",
|
||||
"\n",
|
||||
"vector_store = GelVectorStore(\n",
|
||||
" embeddings=embeddings,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "61a224a1-d70b-4daf-86ba-ab6e43c08b50",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Manage vector store\n",
|
||||
"\n",
|
||||
"### Add items to vector store\n",
|
||||
"\n",
|
||||
"Note that adding documents by ID will over-write any existing documents that match that ID."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "88a288cc-ffd4-4800-b011-750c72b9fd10",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain_core.documents import Document\n",
|
||||
"\n",
|
||||
"docs = [\n",
|
||||
" Document(\n",
|
||||
" page_content=\"there are cats in the pond\",\n",
|
||||
" metadata={\"id\": \"1\", \"location\": \"pond\", \"topic\": \"animals\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"ducks are also found in the pond\",\n",
|
||||
" metadata={\"id\": \"2\", \"location\": \"pond\", \"topic\": \"animals\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"fresh apples are available at the market\",\n",
|
||||
" metadata={\"id\": \"3\", \"location\": \"market\", \"topic\": \"food\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"the market also sells fresh oranges\",\n",
|
||||
" metadata={\"id\": \"4\", \"location\": \"market\", \"topic\": \"food\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"the new art exhibit is fascinating\",\n",
|
||||
" metadata={\"id\": \"5\", \"location\": \"museum\", \"topic\": \"art\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"a sculpture exhibit is also at the museum\",\n",
|
||||
" metadata={\"id\": \"6\", \"location\": \"museum\", \"topic\": \"art\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"a new coffee shop opened on Main Street\",\n",
|
||||
" metadata={\"id\": \"7\", \"location\": \"Main Street\", \"topic\": \"food\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"the book club meets at the library\",\n",
|
||||
" metadata={\"id\": \"8\", \"location\": \"library\", \"topic\": \"reading\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"the library hosts a weekly story time for kids\",\n",
|
||||
" metadata={\"id\": \"9\", \"location\": \"library\", \"topic\": \"reading\"},\n",
|
||||
" ),\n",
|
||||
" Document(\n",
|
||||
" page_content=\"a cooking class for beginners is offered at the community center\",\n",
|
||||
" metadata={\"id\": \"10\", \"location\": \"community center\", \"topic\": \"classes\"},\n",
|
||||
" ),\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"vector_store.add_documents(docs, ids=[doc.metadata[\"id\"] for doc in docs])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0c712fa3",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Delete items from vector store"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "a5b2b71f-49eb-407d-b03a-dea4c0a517d6",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"vector_store.delete(ids=[\"3\"])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "59f82250-7903-4279-8300-062542c83416",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Query vector store\n",
|
||||
"\n",
|
||||
"Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. \n",
|
||||
"\n",
|
||||
"### Filtering Support\n",
|
||||
"\n",
|
||||
"The vectorstore supports a set of filters that can be applied against the metadata fields of the documents.\n",
|
||||
"\n",
|
||||
"| Operator | Meaning/Category |\n",
|
||||
"|----------|-------------------------|\n",
|
||||
"| \\$eq | Equality (==) |\n",
|
||||
"| \\$ne | Inequality (!=) |\n",
|
||||
"| \\$lt | Less than (<) |\n",
|
||||
"| \\$lte | Less than or equal (<=) |\n",
|
||||
"| \\$gt | Greater than (>) |\n",
|
||||
"| \\$gte | Greater than or equal (>=) |\n",
|
||||
"| \\$in | Special Cased (in) |\n",
|
||||
"| \\$nin | Special Cased (not in) |\n",
|
||||
"| \\$between | Special Cased (between) |\n",
|
||||
"| \\$like | Text (like) |\n",
|
||||
"| \\$ilike | Text (case-insensitive like) |\n",
|
||||
"| \\$and | Logical (and) |\n",
|
||||
"| \\$or | Logical (or) |\n",
|
||||
"\n",
|
||||
"### Query directly\n",
|
||||
"\n",
|
||||
"Performing a simple similarity search can be done as follows:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "f15a2359-6dc3-4099-8214-785f167a9ca4",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"results = vector_store.similarity_search(\n",
|
||||
" \"kitty\", k=10, filter={\"id\": {\"$in\": [\"1\", \"5\", \"2\", \"9\"]}}\n",
|
||||
")\n",
|
||||
"for doc in results:\n",
|
||||
" print(f\"* {doc.page_content} [{doc.metadata}]\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d92ea049-1b1f-4ae9-9525-35750fe2e52e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you provide a dict with multiple fields, but no operators, the top level will be interpreted as a logical **AND** filter"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "88f919e4-e4b0-4b5f-99b3-24c675c26d33",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"vector_store.similarity_search(\n",
|
||||
" \"ducks\",\n",
|
||||
" k=10,\n",
|
||||
" filter={\n",
|
||||
" \"id\": {\"$in\": [\"1\", \"5\", \"2\", \"9\"]},\n",
|
||||
" \"location\": {\"$in\": [\"pond\", \"market\"]},\n",
|
||||
" },\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "88f423a4-6575-4fb8-9be2-a3da01106591",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"vector_store.similarity_search(\n",
|
||||
" \"ducks\",\n",
|
||||
" k=10,\n",
|
||||
" filter={\n",
|
||||
" \"$and\": [\n",
|
||||
" {\"id\": {\"$in\": [\"1\", \"5\", \"2\", \"9\"]}},\n",
|
||||
" {\"location\": {\"$in\": [\"pond\", \"market\"]}},\n",
|
||||
" ]\n",
|
||||
" },\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2e65adc1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you want to execute a similarity search and receive the corresponding scores you can run:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7d92e7b3",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"results = vector_store.similarity_search_with_score(query=\"cats\", k=1)\n",
|
||||
"for doc, score in results:\n",
|
||||
" print(f\"* [SIM={score:3f}] {doc.page_content} [{doc.metadata}]\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8d40db8c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Query by turning into retriever\n",
|
||||
"\n",
|
||||
"You can also transform the vector store into a retriever for easier usage in your chains. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7cd1fb75",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"retriever = vector_store.as_retriever(search_kwargs={\"k\": 1})\n",
|
||||
"retriever.invoke(\"kitty\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7ecd77a0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Usage for retrieval-augmented generation\n",
|
||||
"\n",
|
||||
"For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n",
|
||||
"\n",
|
||||
"- [Tutorials](/docs/tutorials/)\n",
|
||||
"- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n",
|
||||
"- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/retrieval)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "33a5f0e6",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## API reference\n",
|
||||
"\n",
|
||||
"For detailed documentation of all GelVectorStore features and configurations head to the API reference: https://python.langchain.com/api_reference/"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.13.2"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@ -652,6 +652,10 @@ packages:
|
||||
provider_page: sap
|
||||
downloads: 350
|
||||
downloads_updated_at: '2025-05-08T20:27:18.592135+00:00'
|
||||
- name: langchain-gel
|
||||
path: .
|
||||
repo: geldata/langchain-gel
|
||||
provider_page: gel
|
||||
- name: langchain-aerospike
|
||||
path: .
|
||||
repo: aerospike/langchain-aerospike
|
||||
|
||||
Loading…
Reference in New Issue
Block a user