From 9b453741184b45798ac9faf84692f5aabc38ae97 Mon Sep 17 00:00:00 2001
From: nareshnagpal06 <33876096+nareshnagpal06@users.noreply.github.com>
Date: Tue, 4 Jun 2024 18:40:29 +0100
Subject: [PATCH] =?UTF-8?q?docs:=20Added=20Semantic=20Cache=20Example=20wi?=
 =?UTF-8?q?th=20BedrockChat=20using=20Bedrock=20Embedding=E2=80=A6=20(#221?=
 =?UTF-8?q?90)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

…s and Opensearch Semantic Cache

Thank you for contributing to LangChain!

- [ ] **PR title**: "package: description"
- Where "package" is whichever of langchain, community, core,
experimental, etc. is being modified. Use "docs: ..." for purely docs
changes, "templates: ..." for template changes, "infra: ..." for CI
changes.
  - Example: "community: add foobar LLM"


- [ ] **PR message**: ***Delete this entire checklist*** and replace
with
    - **Description:** a description of the change
    - **Issue:** the issue # it fixes, if applicable
    - **Dependencies:** any dependencies required for this change
- **Twitter handle:** if your PR gets announced, and you'd like a
mention, we'll gladly shout you out!


- [ ] **Add tests and docs**: If you're adding a new integration, please
include
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.


- [ ] **Lint and test**: Run `make format`, `make lint` and `make test`
from the root of the package(s) you've modified. See contribution
guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:
- Make sure optional dependencies are imported within a function.
- Please do not add dependencies to pyproject.toml files (even optional
ones) unless they are required for unit tests.
- Most PRs should not touch more than one package.
- Changes should be backwards compatible.
- If you are adding something to community, do not re-import it in
langchain.

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
---
 docs/docs/integrations/chat/bedrock.ipynb | 71 +++++++++++++++++++++++
 1 file changed, 71 insertions(+)

diff --git a/docs/docs/integrations/chat/bedrock.ipynb b/docs/docs/integrations/chat/bedrock.ipynb
index 1a74fb273ee..3d7b92e7c98 100644
--- a/docs/docs/integrations/chat/bedrock.ipynb
+++ b/docs/docs/integrations/chat/bedrock.ipynb
@@ -137,6 +137,77 @@
     "for chunk in chat.stream(messages):\n",
     "    print(chunk.content, end=\"\", flush=True)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c36575b3",
+   "metadata": {},
+   "source": [
+    "### LLM Caching with OpenSearch Semantic Cache\n",
+    "\n",
+    "Use OpenSearch as a semantic cache to cache prompts and responses and evaluate hits based on semantic similarity.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "375d4e56",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.globals import set_llm_cache\n",
+    "from langchain_aws import BedrockEmbeddings, ChatBedrock\n",
+    "from langchain_community.cache import OpenSearchSemanticCache\n",
+    "from langchain_core.messages import HumanMessage\n",
+    "\n",
+    "bedrock_embeddings = BedrockEmbeddings(\n",
+    "    model_id=\"amazon.titan-embed-text-v1\", region_name=\"us-east-1\"\n",
+    ")\n",
+    "\n",
+    "chat = ChatBedrock(\n",
+    "    model_id=\"anthropic.claude-3-haiku-20240307-v1:0\", model_kwargs={\"temperature\": 0.5}\n",
+    ")\n",
+    "\n",
+    "# Enable LLM cache. Make sure OpenSearch is set up and running. Update URL accordingly.\n",
+    "set_llm_cache(\n",
+    "    OpenSearchSemanticCache(\n",
+    "        opensearch_url=\"http://localhost:9200\", embedding=bedrock_embeddings\n",
+    "    )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bb5d25bb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%time\n",
+    "# The first time, it is not yet in cache, so it should take longer\n",
+    "messages = [HumanMessage(content=\"tell me about Amazon Bedrock\")]\n",
+    "response_text = chat.invoke(messages)\n",
+    "\n",
+    "print(response_text)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6cfb3086",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%time\n",
+    "# The second time, while not a direct hit, the question is semantically similar to the original question,\n",
+    "# so it uses the cached result!\n",
+    "\n",
+    "messages = [HumanMessage(content=\"what is amazon bedrock\")]\n",
+    "response_text = chat.invoke(messages)\n",
+    "\n",
+    "print(response_text)"
+   ]
   }
  ],
  "metadata": {