mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-26 16:43:35 +00:00
docs: ElasticsearchCache in cache integrations documentation (#20790)
The package for LangChain integrations with Elasticsearch https://github.com/langchain-ai/langchain-elastic is going to contain a LLM cache integration in the next release (see https://github.com/langchain-ai/langchain-elastic/pull/14). This is the documentation contribution on the page dedicated to cache integrations
This commit is contained in:
Giacomo Berardi
GitHub
parent
8c085fc697
commit
61f14f00d7
@ -1428,6 +1428,15 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 82,
|
||||
"id": "14ca942820e8140c",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-03-12T00:12:57.462226Z",
|
||||
"start_time": "2024-03-12T00:12:55.166201Z"
|
||||
},
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -1439,7 +1448,9 @@
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'"
|
||||
"text/plain": [
|
||||
"'\\n\\nWhy was the math book sad? Because it had too many problems.'"
|
||||
]
|
||||
},
|
||||
"execution_count": 82,
|
||||
"metadata": {},
|
||||
@ -1450,16 +1461,7 @@
|
||||
"%%time\n",
|
||||
"# The first time, it is not yet in cache, so it should take longer\n",
|
||||
"llm(\"Tell me a joke\")"
|
||||
],
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-03-12T00:12:57.462226Z",
|
||||
"start_time": "2024-03-12T00:12:55.166201Z"
|
||||
}
|
||||
},
|
||||
"id": "14ca942820e8140c",
|
||||
"execution_count": 82
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@ -1482,7 +1484,9 @@
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'"
|
||||
"text/plain": [
|
||||
"'\\n\\nWhy was the math book sad? Because it had too many problems.'"
|
||||
]
|
||||
},
|
||||
"execution_count": 83,
|
||||
"metadata": {},
|
||||
@ -1495,6 +1499,141 @@
|
||||
"llm(\"Tell me a joke\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "306ff47b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## `Elasticsearch` Cache\n",
|
||||
"A caching layer for LLMs that uses Elasticsearch.\n",
|
||||
"\n",
|
||||
"First install the LangChain integration with Elasticsearch."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9ee5cd3e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install -U langchain-elasticsearch"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9e70b0a0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Use the class `ElasticsearchCache`.\n",
|
||||
"\n",
|
||||
"Simple example:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "1762c9c1",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from elasticsearch import Elasticsearch\n",
|
||||
"from langchain.globals import set_llm_cache\n",
|
||||
"from langchain_elasticsearch import ElasticsearchCache\n",
|
||||
"\n",
|
||||
"es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n",
|
||||
"set_llm_cache(\n",
|
||||
" ElasticsearchCache(\n",
|
||||
" es_connection=es_client,\n",
|
||||
" index_name=\"llm-chat-cache\",\n",
|
||||
" metadata={\"project\": \"my_chatgpt_project\"},\n",
|
||||
" )\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d4fac5d6",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The `index_name` parameter can also accept aliases. This allows to use the \n",
|
||||
"[ILM: Manage the index lifecycle](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html)\n",
|
||||
"that we suggest to consider for managing retention and controlling cache growth.\n",
|
||||
"\n",
|
||||
"Look at the class docstring for all parameters."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "eaf9dfd7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Index the generated text\n",
|
||||
"\n",
|
||||
"The cached data won't be searchable by default.\n",
|
||||
"The developer can customize the building of the Elasticsearch document in order to add indexed text fields,\n",
|
||||
"where to put, for example, the text generated by the LLM.\n",
|
||||
"\n",
|
||||
"This can be done by subclassing end overriding methods.\n",
|
||||
"The new cache class can be applied also to a pre-existing cache index:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "5104c2c0",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import json\n",
|
||||
"from typing import Any, Dict, List\n",
|
||||
"\n",
|
||||
"from elasticsearch import Elasticsearch\n",
|
||||
"from langchain.globals import set_llm_cache\n",
|
||||
"from langchain_core.caches import RETURN_VAL_TYPE\n",
|
||||
"from langchain_elasticsearch import ElasticsearchCache\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class SearchableElasticsearchCache(ElasticsearchCache):\n",
|
||||
" @property\n",
|
||||
" def mapping(self) -> Dict[str, Any]:\n",
|
||||
" mapping = super().mapping\n",
|
||||
" mapping[\"mappings\"][\"properties\"][\"parsed_llm_output\"] = {\n",
|
||||
" \"type\": \"text\",\n",
|
||||
" \"analyzer\": \"english\",\n",
|
||||
" }\n",
|
||||
" return mapping\n",
|
||||
"\n",
|
||||
" def build_document(\n",
|
||||
" self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE\n",
|
||||
" ) -> Dict[str, Any]:\n",
|
||||
" body = super().build_document(prompt, llm_string, return_val)\n",
|
||||
" body[\"parsed_llm_output\"] = self._parse_output(body[\"llm_output\"])\n",
|
||||
" return body\n",
|
||||
"\n",
|
||||
" @staticmethod\n",
|
||||
" def _parse_output(data: List[str]) -> List[str]:\n",
|
||||
" return [\n",
|
||||
" json.loads(output)[\"kwargs\"][\"message\"][\"kwargs\"][\"content\"]\n",
|
||||
" for output in data\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n",
|
||||
"set_llm_cache(\n",
|
||||
" SearchableElasticsearchCache(es_connection=es_client, index_name=\"llm-chat-cache\")\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "db0dea73",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"When overriding the mapping and the document building, \n",
|
||||
"please only make additive modifications, keeping the base mapping intact."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0c69d84d",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user