cr

Adds support for [llmonitor](https://llmonitor.com) callbacks.

It enables:
- Requests tracking / logging / analytics
- Error debugging
- Cost analytics
- User tracking

Let me know if anythings neds to be changed for merge.

Thank you!

.github/CONTRIBUTING.md

@@ -44,7 +44,7 @@ If you are adding an issue, please try to keep it focused on a single, modular b
 If two issues are related, or blocking, please link them rather than combining them.
 
 We will try to keep these issues as up to date as possible, though
-with the rapid rate of develop in this field some may get out of date.
+with the rapid rate of development in this field some may get out of date.
 If you notice this happening, please let us know.
 
 ### 🙋Getting Help
@@ -87,7 +87,7 @@ This will install all requirements for running the package, examples, linting, f
 
 ❗Note: If during installation you receive a `WheelFileValidationError` for `debugpy`, please make sure you are running Poetry v1.5.1. This bug was present in older versions of Poetry (e.g. 1.4.1) and has been resolved in newer releases. If you are still seeing this bug on v1.5.1, you may also try disabling "modern installation" (`poetry config installer.modern-installation false`) and re-installing requirements. See [this `debugpy` issue](https://github.com/microsoft/debugpy/issues/1246) for more details.
 
-Now, you should be able to run the common tasks in the following section. To double check, run `make test`, all tests should pass. If they don't you may need to pip install additional dependencies, such as `numexpr` and `openapi_schema_pydantic`.
+Now assuming `make` and `pytest` are installed, you should be able to run the common tasks in the following section. To double check, run `make test` under `libs/langchain`, all tests should pass. If they don't, you may need to pip install additional dependencies, such as `numexpr` and `openapi_schema_pydantic`.
 
 ## ✅ Common Tasks
 
@@ -134,7 +134,7 @@ We recognize linting can be annoying - if you do not want to do it, please conta
 ### Spellcheck
 
 Spellchecking for this project is done via [codespell](https://github.com/codespell-project/codespell).
-Note that `codespell` finds common typos, so could have false-positive (correctly spelled but rarely used) and false-negatives (not finding misspelled) words.
+Note that `codespell` finds common typos, so it could have false-positive (correctly spelled but rarely used) and false-negatives (not finding misspelled) words.
 
 To check spelling for this project:
 

.github/workflows/_release.yml

@@ -31,13 +31,15 @@ jobs:
         working-directory: ${{ inputs.working-directory }}
     steps:
       - uses: actions/checkout@v3
-      - name: Install poetry
-        run: pipx install "poetry==$POETRY_VERSION"
-      - name: Set up Python 3.10
-        uses: actions/setup-python@v4
+
+      - name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
         with:
           python-version: "3.10"
-          cache: "poetry"
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: ${{ inputs.working-directory }}
+          cache-key: release
+
       - name: Build project for distribution
         run: poetry build
       - name: Check Version

.github/workflows/langchain_experimental_ci.yml

@@ -81,3 +81,35 @@ jobs:
 
       - name: Run tests
         run: make test
+  extended-tests:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ${{ env.WORKDIR }}
+    strategy:
+      matrix:
+        python-version:
+          - "3.8"
+          - "3.9"
+          - "3.10"
+          - "3.11"
+    name: Python ${{ matrix.python-version }} extended tests
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: libs/experimental
+          cache-key: extended
+
+      - name: Install dependencies
+        shell: bash
+        run: |
+          echo "Running extended tests, installing dependencies with poetry..."
+          poetry install -E extended_testing
+
+      - name: Run extended tests
+        run: make extended_tests

docs/api_reference/create_api_rst.py

@@ -228,7 +228,7 @@ Classes
     :toctree: {module}
 """
 
-            for class_ in classes:
+            for class_ in sorted(classes, key=lambda c: c["qualified_name"]):
                 if not class_["is_public"]:
                     continue
 

docs/api_reference/guide_imports.json

@@ -341,7 +341,7 @@
         "HugeGraph QA Chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_hugegraph_qa",
         "GraphSparqlQAChain": "https://python.langchain.com/docs/use_cases/more/graph/graph_sparql_qa",
         "ArangoDB QA chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_arangodb_qa",
-        "Graph DB QA chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_cypher_qa",
+        "Neo4j DB QA chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_cypher_qa",
         "How to use a SmartLLMChain": "https://python.langchain.com/docs/use_cases/more/self_check/smart_llm",
         "Multi-Agent Simulated Environment: Petting Zoo": "https://python.langchain.com/docs/use_cases/agent_simulations/petting_zoo",
         "Multi-agent decentralized speaker selection": "https://python.langchain.com/docs/use_cases/agent_simulations/multiagent_bidding",
@@ -3202,10 +3202,10 @@
         "Graph QA": "https://python.langchain.com/docs/use_cases/more/graph/graph_qa"
     },
     "GraphCypherQAChain": {
-        "Graph DB QA chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_cypher_qa"
+        "Neo4j DB QA chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_cypher_qa"
     },
     "Neo4jGraph": {
-        "Graph DB QA chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_cypher_qa"
+        "Neo4j DB QA chain": "https://python.langchain.com/docs/use_cases/more/graph/graph_cypher_qa"
     },
     "LLMBashChain": {
         "Bash chain": "https://python.langchain.com/docs/use_cases/more/code_writing/llm_bash"

docs/docs_skeleton/docs/get_started/introduction.mdx

@@ -42,9 +42,9 @@ Log and stream intermediate steps of any chain
 ## Examples, ecosystem, and resources
 ### [Use cases](/docs/use_cases/)
 Walkthroughs and best-practices for common end-to-end use cases, like:
-- [Chatbots](/docs/use_cases/chatbots/)
+- [Chatbots](/docs/use_cases/chatbots)
 - [Answering questions using sources](/docs/use_cases/question_answering/)
-- [Analyzing structured data](/docs/use_cases/tabular.html)
+- [Analyzing structured data](/docs/use_cases/sql)
 - and much more...
 
 ### [Guides](/docs/guides/)
@@ -56,9 +56,8 @@ LangChain is part of a rich ecosystem of tools that integrate with our framework
 ### [Additional resources](/docs/additional_resources/)
 Our community is full of prolific developers, creative builders, and fantastic teachers. Check out [YouTube tutorials](/docs/additional_resources/youtube.html) for great tutorials from folks in the community, and [Gallery](https://github.com/kyrolabs/awesome-langchain) for a list of awesome LangChain projects, compiled by the folks at [KyroLabs](https://kyrolabs.com).
 
-<h3><span style={{color:"#2e8555"}}> Support </span></h3>
-
-Join us on [GitHub](https://github.com/hwchase17/langchain) or [Discord](https://discord.gg/6adMQxSpJS) to ask questions, share feedback, meet other developers building with LangChain, and dream about the future of LLM’s.
+### [Community](/docs/community)
+Head to the [Community navigator](/docs/community) to find places to ask questions, share feedback, meet other developers, and dream about the future of LLM’s.
 
 ## API reference
 

docs/docs_skeleton/docs/guides/safety/index.mdx

@@ -2,5 +2,5 @@
 
 One of the key concerns with using LLMs is that they may generate harmful or unethical text. This is an area of active research in the field. Here we present some built-in chains inspired by this research, which are intended to make the outputs of LLMs safer.
 
-- [Moderation chain](/docs/use_cases/safety/moderation): Explicitly check if any output text is harmful and flag it.
-- [Constitutional chain](/docs/use_cases/safety/constitutional_chain): Prompt the model with a set of principles which should guide it's behavior.
+- [Moderation chain](/docs/guides/safety/moderation): Explicitly check if any output text is harmful and flag it.
+- [Constitutional chain](/docs/guides/safety/constitutional_chain): Prompt the model with a set of principles which should guide it's behavior.

docs/docs_skeleton/docs/modules/memory/types/buffer.mdx

@@ -1,4 +1,4 @@
-# Conversation buffer memory
+# Conversation Buffer
 
 This notebook shows how to use `ConversationBufferMemory`. This memory allows for storing of messages and then extracts the messages in a variable.
 

docs/docs_skeleton/docs/modules/memory/types/buffer_window.mdx

@@ -1,4 +1,4 @@
-# Conversation buffer window memory
+# Conversation Buffer Window
 
 `ConversationBufferWindowMemory` keeps a list of the interactions of the conversation over time. It only uses the last K interactions. This can be useful for keeping a sliding window of the most recent interactions, so the buffer does not get too large
 

docs/docs_skeleton/docs/modules/memory/types/entity_summary_memory.mdx

@@ -1,4 +1,4 @@
-# Entity memory
+# Entity
 
 Entity Memory remembers given facts about specific entities in a conversation. It extracts information on entities (using an LLM) and builds up its knowledge about that entity over time (also using an LLM).
 

docs/docs_skeleton/docs/modules/memory/types/index.mdx

@@ -4,5 +4,5 @@ sidebar_position: 2
 # Memory Types
 
 There are many different types of memory.
-Each have their own parameters, their own return types, and are useful in different scenarios.
+Each has their own parameters, their own return types, and is useful in different scenarios.
 Please see their individual page for more detail on each one.

docs/docs_skeleton/docs/modules/memory/types/summary.mdx

@@ -1,4 +1,4 @@
-# Conversation summary memory
+# Conversation Summary
 Now let's take a look at using a slightly more complex type of memory - `ConversationSummaryMemory`. This type of memory creates a summary of the conversation over time. This can be useful for condensing information from the conversation over time.
 Conversation summary memory summarizes the conversation as it happens and stores the current summary in memory. This memory can then be used to inject the summary of the conversation so far into a prompt/chain. This memory is most useful for longer conversations, where keeping the past message history in the prompt verbatim would take up too many tokens.
 

docs/docs_skeleton/docs/modules/memory/types/vectorstore_retriever_memory.mdx

@@ -1,4 +1,4 @@
-# Vector store-backed memory
+# Backed by a Vector Store
 
 `VectorStoreRetrieverMemory` stores memories in a VectorDB and queries the top-K most "salient" docs every time it is called.
 

docs/extras/ecosystem/dependents.mdx

@@ -51,7 +51,7 @@ Dependents stats for `langchain-ai/langchain`
 |[e2b-dev/e2b](https://github.com/e2b-dev/e2b) | 5365 |
 |[mage-ai/mage-ai](https://github.com/mage-ai/mage-ai) | 5352 |
 |[wenda-LLM/wenda](https://github.com/wenda-LLM/wenda) | 5192 |
-|[LangChain-Chinese-Getting-Started-Guide](https://github.com/liaokongVFX/LangChain-Chinese-Getting-Started-Guide) | 5129 |
+|[liaokongVFX/LangChain-Chinese-Getting-Started-Guide](https://github.com/liaokongVFX/LangChain-Chinese-Getting-Started-Guide) | 5129 |
 |[zilliztech/GPTCache](https://github.com/zilliztech/GPTCache) | 4993 |
 |[GreyDGL/PentestGPT](https://github.com/GreyDGL/PentestGPT) | 4831 |
 |[zauberzeug/nicegui](https://github.com/zauberzeug/nicegui) | 4824 |

docs/extras/guides/debugging.md

@@ -8,7 +8,7 @@ Here's a few different tools and functionalities to aid in debugging.
 
 ## Tracing
 
-Platforms with tracing capabilities like [LangSmith](/docs/guides/langsmith/) and [WandB](/docs/ecosystem/integrations/agent_with_wandb_tracing) are the most comprehensive solutions for debugging. These platforms make it easy to not only log and visualize LLM apps, but also to actively debug, test and refine them.
+Platforms with tracing capabilities like [LangSmith](/docs/guides/langsmith/) and [WandB](/docs/integrations/providers/wandb_tracing) are the most comprehensive solutions for debugging. These platforms make it easy to not only log and visualize LLM apps, but also to actively debug, test and refine them.
 
 For anyone building production-grade LLM applications, we highly recommend using a platform like this.
 

docs/extras/integrations/callbacks/infino.ipynb

@@ -1,86 +1,73 @@
 {
  "cells": [
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "8d10861f-a550-4443-bc63-4ce2ae13b841",
    "metadata": {},
    "source": [
-    "# Infino - LangChain LLM Monitoring Example\n",
+    "# Infino\n",
     "\n",
-    "This example shows how one can track the following while calling OpenAI models via LangChain and [Infino](https://github.com/infinohq/infino):\n",
+    "This example shows how one can track the following while calling OpenAI models via `LangChain` and [Infino](https://github.com/infinohq/infino):\n",
     "\n",
     "* prompt input,\n",
-    "* response from chatgpt or any other LangChain model,\n",
+    "* response from `ChatGPT` or any other `LangChain` model,\n",
     "* latency,\n",
     "* errors,\n",
     "* number of tokens consumed"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "3a5a0976-9953-41d8-880c-eb3f2992e936",
+   "cell_type": "markdown",
+   "id": "64d14c88-b71c-4524-ab1b-4250a7dbb62b",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Requirement already satisfied: matplotlib in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (3.7.1)\n",
-      "Requirement already satisfied: contourpy>=1.0.1 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (1.0.7)\n",
-      "Requirement already satisfied: cycler>=0.10 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (0.11.0)\n",
-      "Requirement already satisfied: fonttools>=4.22.0 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (4.39.4)\n",
-      "Requirement already satisfied: kiwisolver>=1.0.1 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (1.4.4)\n",
-      "Requirement already satisfied: numpy>=1.20 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (1.24.3)\n",
-      "Requirement already satisfied: packaging>=20.0 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (23.1)\n",
-      "Requirement already satisfied: pillow>=6.2.0 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (9.5.0)\n",
-      "Requirement already satisfied: pyparsing>=2.3.1 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (3.0.9)\n",
-      "Requirement already satisfied: python-dateutil>=2.7 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from matplotlib) (2.8.2)\n",
-      "Requirement already satisfied: six>=1.5 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n",
-      "Requirement already satisfied: infinopy in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (0.0.1)\n",
-      "Requirement already satisfied: docker in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from infinopy) (6.1.3)\n",
-      "Requirement already satisfied: requests in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from infinopy) (2.31.0)\n",
-      "Requirement already satisfied: packaging>=14.0 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from docker->infinopy) (23.1)\n",
-      "Requirement already satisfied: urllib3>=1.26.0 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from docker->infinopy) (2.0.2)\n",
-      "Requirement already satisfied: websocket-client>=0.32.0 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from docker->infinopy) (1.5.2)\n",
-      "Requirement already satisfied: charset-normalizer<4,>=2 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from requests->infinopy) (3.1.0)\n",
-      "Requirement already satisfied: idna<4,>=2.5 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from requests->infinopy) (3.4)\n",
-      "Requirement already satisfied: certifi>=2017.4.17 in /Users/vinaykakade/.pyenv/versions/3.10.11/lib/python3.10/site-packages (from requests->infinopy) (2023.5.7)\n"
-     ]
-    }
-   ],
+   "source": [
+    "## Initializing"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ed46c894-caa6-49b2-85d1-f275374fa308",
+   "metadata": {},
+   "outputs": [],
    "source": [
     "# Install necessary dependencies.\n",
     "!pip install infinopy\n",
-    "!pip install matplotlib\n",
-    "\n",
+    "!pip install matplotlib"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3a5a0976-9953-41d8-880c-eb3f2992e936",
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "# Remove the (1) import sys and sys.path.append(..) and (2) uncomment `!pip install langchain` after merging the PR for Infino/LangChain integration.\n",
     "import sys\n",
     "\n",
     "sys.path.append(\"../../../../../langchain\")\n",
-    "#!pip install langchain\n",
     "\n",
     "\n",
     "import datetime as dt\n",
-    "from infinopy import InfinoClient\n",
     "import json\n",
     "from langchain.llms import OpenAI\n",
-    "from langchain.callbacks import InfinoCallbackHandler\n",
     "import matplotlib.pyplot as plt\n",
     "import matplotlib.dates as md\n",
     "import os\n",
     "import time\n",
-    "import sys"
+    "import sys\n",
+    "\n",
+    "from infinopy import InfinoClient\n",
+    "from langchain.callbacks import InfinoCallbackHandler"
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "9f90210d-c805-4a0c-81e4-d5298942afc4",
    "metadata": {},
    "source": [
-    "## Start Infino server, initialize the Infino client\n"
+    "## Start Infino server, initialize the Infino client"
    ]
   },
   {
@@ -106,7 +93,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "b6b81cda-b841-43ee-8c5e-b1576555765f",
    "metadata": {},
@@ -148,7 +134,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "dce1b820-3f1a-4b94-b848-4c6032cadc18",
    "metadata": {},
@@ -214,7 +199,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "b68ec697-c922-4fd9-aad1-f49c6ac24e8a",
    "metadata": {},
@@ -326,7 +310,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "c3d61822-1781-4bc6-97a2-2abc5c2b2e75",
    "metadata": {},
@@ -364,12 +347,11 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "4b171074-c775-48e0-a4b3-f550e2c8eccb",
    "metadata": {},
    "source": [
-    "## Step 5: Stop infino server"
+    "## Stop infino server"
    ]
   },
   {
@@ -415,7 +397,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.4"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/integrations/callbacks/llmonitor.md

@@ -0,0 +1,63 @@
+# LLMonitor
+
+[LLMonitor](https://llmonitor.com) is an open-source observability platform that provides cost tracking, user tracking and powerful agent tracing.
+
+<video controls width='100%' >
+  <source src='https://llmonitor.com/videos/demo-annotated.mp4'/>
+</video>
+
+## Setup
+Create an account on [llmonitor.com](https://llmonitor.com), create an `App`, and then copy the associated `tracking id`.
+Once you have it, set it as an environment variable by running:
+```bash
+export LLMONITOR_APP_ID="..."
+```
+
+If you'd prefer not to set an environment variable, you can pass the key directly when initializing the callback handler:
+```python
+from langchain.callbacks import LLMonitorCallbackHandler
+
+handler = LLMonitorCallbackHandler(app_id="...")
+```
+
+## Usage with LLM/Chat models
+```python
+from langchain.llms import OpenAI
+from langchain.chat_models import ChatOpenAI
+from langchain.callbacks import LLMonitorCallbackHandler
+
+handler = LLMonitorCallbackHandler(app_id="...")
+
+llm = OpenAI(
+    callbacks=[handler],
+)
+
+chat = ChatOpenAI(
+    callbacks=[handler],
+    metadata={"userId": "123"},  # you can assign user ids to models in the metadata
+)
+```
+
+
+## Usage with agents
+```python
+from langchain.agents import load_tools, initialize_agent, AgentType
+from langchain.llms import OpenAI
+from langchain.callbacks import LLMonitorCallbackHandler
+
+handler = LLMonitorCallbackHandler(app_id="...")
+
+llm = OpenAI(temperature=0)
+tools = load_tools(["serpapi", "llm-math"], llm=llm)
+agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)
+agent.run(
+    "Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?",
+    callbacks=[handler],
+    metadata={
+        "agentName": "Leo DiCaprio's girlfriend",  # you can assign a custom agent in the metadata
+    },
+)
+```
+
+## Support
+For any question or issue with integration you can reach out to the LLMonitor team on [Discord](http://discord.com/invite/8PafSG58kK) or via [email](mailto:vince@llmonitor.com).

docs/extras/integrations/chat_loaders/discord.ipynb

@@ -0,0 +1,325 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "c4ff9336-1cf3-459e-bd70-d1314c1da6a0",
+   "metadata": {},
+   "source": [
+    "# Discord\n",
+    "\n",
+    "This notebook shows how to create your own chat loader that works on copy-pasted messages (from dms) to a list of LangChain messages.\n",
+    "\n",
+    "The process has four steps:\n",
+    "1. Create the chat .txt file by copying chats from the Discord app and pasting them in a file on your local computer\n",
+    "2. Copy the chat loader definition from below to a local file.\n",
+    "3. Initialize the `DiscordChatLoader` with the file path pointed to the text file.\n",
+    "4. Call `loader.load()` (or `loader.lazy_load()`) to perform the conversion.\n",
+    "\n",
+    "## 1. Creat message dump\n",
+    "\n",
+    "Currently (2023/08/23) this loader only supports .txt files in the format generated by copying messages in the app to your clipboard and pasting in a file. Below is an example."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "e4ccfdfa-6869-4d67-90a0-ab99f01b7553",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Overwriting discord_chats.txt\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%writefile discord_chats.txt\n",
+    "talkingtower — 08/15/2023 11:10 AM\n",
+    "Love music! Do you like jazz?\n",
+    "reporterbob — 08/15/2023 9:27 PM\n",
+    "Yes! Jazz is fantastic. Ever heard this one?\n",
+    "Website\n",
+    "Listen to classic jazz track...\n",
+    "\n",
+    "talkingtower — Yesterday at 5:03 AM\n",
+    "Indeed! Great choice. 🎷\n",
+    "reporterbob — Yesterday at 5:23 AM\n",
+    "Thanks! How about some virtual sightseeing?\n",
+    "Website\n",
+    "Virtual tour of famous landmarks...\n",
+    "\n",
+    "talkingtower — Today at 2:38 PM\n",
+    "Sounds fun! Let's explore.\n",
+    "reporterbob — Today at 2:56 PM\n",
+    "Enjoy the tour! See you around.\n",
+    "talkingtower — Today at 3:00 PM\n",
+    "Thank you! Goodbye! 👋\n",
+    "reporterbob — Today at 3:02 PM\n",
+    "Farewell! Happy exploring."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "359565a7-dad3-403c-a73c-6414b1295127",
+   "metadata": {},
+   "source": [
+    "## 2. Define chat loader\n",
+    "\n",
+    "LangChain currently does not support "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "a429e0c4-4d7d-45f8-bbbb-c7fc5229f6af",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import logging\n",
+    "import re\n",
+    "from typing import Iterator, List\n",
+    "\n",
+    "from langchain import schema\n",
+    "from langchain.chat_loaders import base as chat_loaders\n",
+    "\n",
+    "logger = logging.getLogger()\n",
+    "\n",
+    "\n",
+    "class DiscordChatLoader(chat_loaders.BaseChatLoader):\n",
+    "    \n",
+    "    def __init__(self, path: str):\n",
+    "        \"\"\"\n",
+    "        Initialize the Discord chat loader.\n",
+    "\n",
+    "        Args:\n",
+    "            path: Path to the exported Discord chat text file.\n",
+    "        \"\"\"\n",
+    "        self.path = path\n",
+    "        self._message_line_regex = re.compile(\n",
+    "            r\"(.+?) — (\\w{3,9} \\d{1,2}(?:st|nd|rd|th)?(?:, \\d{4})? \\d{1,2}:\\d{2} (?:AM|PM)|Today at \\d{1,2}:\\d{2} (?:AM|PM)|Yesterday at \\d{1,2}:\\d{2} (?:AM|PM))\",  # noqa\n",
+    "            flags=re.DOTALL,\n",
+    "        )\n",
+    "\n",
+    "    def _load_single_chat_session_from_txt(\n",
+    "        self, file_path: str\n",
+    "    ) -> chat_loaders.ChatSession:\n",
+    "        \"\"\"\n",
+    "        Load a single chat session from a text file.\n",
+    "\n",
+    "        Args:\n",
+    "            file_path: Path to the text file containing the chat messages.\n",
+    "\n",
+    "        Returns:\n",
+    "            A `ChatSession` object containing the loaded chat messages.\n",
+    "        \"\"\"\n",
+    "        with open(file_path, \"r\", encoding=\"utf-8\") as file:\n",
+    "            lines = file.readlines()\n",
+    "\n",
+    "        results: List[schema.BaseMessage] = []\n",
+    "        current_sender = None\n",
+    "        current_timestamp = None\n",
+    "        current_content = []\n",
+    "        for line in lines:\n",
+    "            if re.match(\n",
+    "                r\".+? — (\\d{2}/\\d{2}/\\d{4} \\d{1,2}:\\d{2} (?:AM|PM)|Today at \\d{1,2}:\\d{2} (?:AM|PM)|Yesterday at \\d{1,2}:\\d{2} (?:AM|PM))\",  # noqa\n",
+    "                line,\n",
+    "            ):\n",
+    "                if current_sender and current_content:\n",
+    "                    results.append(\n",
+    "                        schema.HumanMessage(\n",
+    "                            content=\"\".join(current_content).strip(),\n",
+    "                            additional_kwargs={\n",
+    "                                \"sender\": current_sender,\n",
+    "                                \"events\": [{\"message_time\": current_timestamp}],\n",
+    "                            },\n",
+    "                        )\n",
+    "                    )\n",
+    "                current_sender, current_timestamp = line.split(\" — \")[:2]\n",
+    "                current_content = [\n",
+    "                    line[len(current_sender) + len(current_timestamp) + 4 :].strip()\n",
+    "                ]\n",
+    "            elif re.match(r\"\\[\\d{1,2}:\\d{2} (?:AM|PM)\\]\", line.strip()):\n",
+    "                results.append(\n",
+    "                    schema.HumanMessage(\n",
+    "                        content=\"\".join(current_content).strip(),\n",
+    "                        additional_kwargs={\n",
+    "                            \"sender\": current_sender,\n",
+    "                            \"events\": [{\"message_time\": current_timestamp}],\n",
+    "                        },\n",
+    "                    )\n",
+    "                )\n",
+    "                current_timestamp = line.strip()[1:-1]\n",
+    "                current_content = []\n",
+    "            else:\n",
+    "                current_content.append(\"\\n\" + line.strip())\n",
+    "\n",
+    "        if current_sender and current_content:\n",
+    "            results.append(\n",
+    "                schema.HumanMessage(\n",
+    "                    content=\"\".join(current_content).strip(),\n",
+    "                    additional_kwargs={\n",
+    "                        \"sender\": current_sender,\n",
+    "                        \"events\": [{\"message_time\": current_timestamp}],\n",
+    "                    },\n",
+    "                )\n",
+    "            )\n",
+    "\n",
+    "        return chat_loaders.ChatSession(messages=results)\n",
+    "\n",
+    "    def lazy_load(self) -> Iterator[chat_loaders.ChatSession]:\n",
+    "        \"\"\"\n",
+    "        Lazy load the messages from the chat file and yield them in the required format.\n",
+    "\n",
+    "        Yields:\n",
+    "            A `ChatSession` object containing the loaded chat messages.\n",
+    "        \"\"\"\n",
+    "        yield self._load_single_chat_session_from_txt(self.path)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c8240393-48be-44d2-b0d6-52c215cd8ac2",
+   "metadata": {},
+   "source": [
+    "## 2. Create loader\n",
+    "\n",
+    "We will point to the file we just wrote to disk."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "1268de40-b0e5-445d-9cd8-54856cd0293a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = DiscordChatLoader(\n",
+    "    path=\"./discord_chats.txt\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4928df4b-ae31-48a7-bd76-be3ecee1f3e0",
+   "metadata": {},
+   "source": [
+    "## 3. Load Messages\n",
+    "\n",
+    "Assuming the format is correct, the loader will convert the chats to langchain messages."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "c8a0836d-4a22-4790-bfe9-97f2145bb0d6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List\n",
+    "from langchain.chat_loaders.base import ChatSession\n",
+    "from langchain.chat_loaders.utils import (\n",
+    "    map_ai_messages,\n",
+    "    merge_chat_runs,\n",
+    ")\n",
+    "\n",
+    "raw_messages = loader.lazy_load()\n",
+    "# Merge consecutive messages from the same sender into a single message\n",
+    "merged_messages = merge_chat_runs(raw_messages)\n",
+    "# Convert messages from \"talkingtower\" to AI messages\n",
+    "messages: List[ChatSession] = list(map_ai_messages(merged_messages, sender=\"talkingtower\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "1913963b-c44e-4f7a-aba7-0423c9b8bd59",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'messages': [AIMessage(content='Love music! Do you like jazz?', additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': '08/15/2023 11:10 AM\\n'}]}, example=False),\n",
+       "   HumanMessage(content='Yes! Jazz is fantastic. Ever heard this one?\\nWebsite\\nListen to classic jazz track...', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': '08/15/2023 9:27 PM\\n'}]}, example=False),\n",
+       "   AIMessage(content='Indeed! Great choice. 🎷', additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': 'Yesterday at 5:03 AM\\n'}]}, example=False),\n",
+       "   HumanMessage(content='Thanks! How about some virtual sightseeing?\\nWebsite\\nVirtual tour of famous landmarks...', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': 'Yesterday at 5:23 AM\\n'}]}, example=False),\n",
+       "   AIMessage(content=\"Sounds fun! Let's explore.\", additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': 'Today at 2:38 PM\\n'}]}, example=False),\n",
+       "   HumanMessage(content='Enjoy the tour! See you around.', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': 'Today at 2:56 PM\\n'}]}, example=False),\n",
+       "   AIMessage(content='Thank you! Goodbye! 👋', additional_kwargs={'sender': 'talkingtower', 'events': [{'message_time': 'Today at 3:00 PM\\n'}]}, example=False),\n",
+       "   HumanMessage(content='Farewell! Happy exploring.', additional_kwargs={'sender': 'reporterbob', 'events': [{'message_time': 'Today at 3:02 PM\\n'}]}, example=False)]}]"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "messages"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8595a518-5c89-44aa-94a7-ca51e7e2a5fa",
+   "metadata": {},
+   "source": [
+    "### Next Steps\n",
+    "\n",
+    "You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message  "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "08ff0a1e-fca0-4da3-aacd-d7401f99d946",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Thank you! Have a wonderful day! 🌟"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "llm = ChatOpenAI()\n",
+    "\n",
+    "for chunk in llm.stream(messages[0]['messages']):\n",
+    "    print(chunk.content, end=\"\", flush=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "50a5251f-074a-4a3c-a2b0-b1de85e0ac6a",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/chat_loaders/facebook.ipynb

@@ -0,0 +1,579 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "e4bd269b",
+   "metadata": {},
+   "source": [
+    "# Facebook Messenger\n",
+    "\n",
+    "This notebook shows how to load data from Facebook in a format you can finetune on. The overall steps are:\n",
+    "\n",
+    "1. Download your messenger data to disk.\n",
+    "2. Create the Chat Loader and call `loader.load()` (or `loader.lazy_load()`) to perform the conversion.\n",
+    "3. Optionally use `merge_chat_runs` to combine message from the same sender in sequence, and/or `map_ai_messages` to convert messages from the specified sender to the \"AIMessage\" class. Once you've done this, call `convert_messages_for_finetuning` to prepare your data for fine-tuning.\n",
+    "\n",
+    "\n",
+    "Once this has been done, you can fine-tune your model. To do so you would complete the following steps:\n",
+    "\n",
+    "4. Upload your messages to OpenAI and run a fine-tuning job.\n",
+    "6. Use the resulting model in your LangChain app!\n",
+    "\n",
+    "\n",
+    "Let's begin.\n",
+    "\n",
+    "\n",
+    "## 1. Download Data\n",
+    "\n",
+    "To download your own messenger data, following instructions [here](https://www.zapptales.com/en/download-facebook-messenger-chat-history-how-to/). IMPORTANT - make sure to download them in JSON format (not HTML).\n",
+    "\n",
+    "We are hosting an example dump at [this google drive link](https://drive.google.com/file/d/1rh1s1o2i7B-Sk1v9o8KNgivLVGwJ-osV/view?usp=sharing) that we will use in this walkthrough."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "647f2158-a42e-4634-b283-b8492caf542a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "File file.zip downloaded.\n",
+      "File file.zip has been unzipped.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# This uses some example data\n",
+    "import requests\n",
+    "import zipfile\n",
+    "\n",
+    "def download_and_unzip(url: str, output_path: str = 'file.zip') -> None:\n",
+    "    file_id = url.split('/')[-2]\n",
+    "    download_url = f'https://drive.google.com/uc?export=download&id={file_id}'\n",
+    "\n",
+    "    response = requests.get(download_url)\n",
+    "    if response.status_code != 200:\n",
+    "        print('Failed to download the file.')\n",
+    "        return\n",
+    "\n",
+    "    with open(output_path, 'wb') as file:\n",
+    "        file.write(response.content)\n",
+    "        print(f'File {output_path} downloaded.')\n",
+    "\n",
+    "    with zipfile.ZipFile(output_path, 'r') as zip_ref:\n",
+    "        zip_ref.extractall()\n",
+    "        print(f'File {output_path} has been unzipped.')\n",
+    "\n",
+    "# URL of the file to download\n",
+    "url = 'https://drive.google.com/file/d/1rh1s1o2i7B-Sk1v9o8KNgivLVGwJ-osV/view?usp=sharing'\n",
+    "\n",
+    "# Download and unzip\n",
+    "download_and_unzip(url)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "48ef8bb1-fc28-453c-835a-94a552f05a91",
+   "metadata": {},
+   "source": [
+    "## 2. Create Chat Loader\n",
+    "\n",
+    "We have 2 different `FacebookMessengerChatLoader` classes, one for an entire directory of chats, and one to load individual files. We"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "a0869bc6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "directory_path = \"./hogwarts\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "0460bf25",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.facebook_messenger import (\n",
+    "    SingleFileFacebookMessengerChatLoader,\n",
+    "    FolderFacebookMessengerChatLoader,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "f61ee277",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = SingleFileFacebookMessengerChatLoader(\n",
+    "    path=\"./hogwarts/inbox/HermioneGranger/messages_Hermione_Granger.json\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "ec466ad7",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[HumanMessage(content=\"Hi Hermione! How's your summer going so far?\", additional_kwargs={'sender': 'Harry Potter'}, example=False),\n",
+       " HumanMessage(content=\"Harry! Lovely to hear from you. My summer is going well, though I do miss everyone. I'm spending most of my time going through my books and researching fascinating new topics. How about you?\", additional_kwargs={'sender': 'Hermione Granger'}, example=False),\n",
+       " HumanMessage(content=\"I miss you all too. The Dursleys are being their usual unpleasant selves but I'm getting by. At least I can practice some spells in my room without them knowing. Let me know if you find anything good in your researching!\", additional_kwargs={'sender': 'Harry Potter'}, example=False)]"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chat_session = loader.load()[0]\n",
+    "chat_session[\"messages\"][:3]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "8a3ee473",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = FolderFacebookMessengerChatLoader(\n",
+    "    path=\"./hogwarts\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "9f41e122",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "9"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chat_sessions = loader.load()\n",
+    "len(chat_sessions)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d4aa3580-adc1-4b48-9bba-0e8e8d9f44ce",
+   "metadata": {},
+   "source": [
+    "## 3. Prepare for fine-tuning\n",
+    "\n",
+    "Calling `load()` returns all the chat messages we could extract as human messages. When conversing with chat bots, conversations typically follow a more strict alternating dialogue pattern relative to real conversations. \n",
+    "\n",
+    "You can choose to merge message \"runs\" (consecutive messages from the same sender) and select a sender to represent the \"AI\". The fine-tuned LLM will learn to generate these AI messages."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "5a78030d-b757-4bbe-8a6c-841056f46df7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.utils import (\n",
+    "    merge_chat_runs,\n",
+    "    map_ai_messages,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "ff35b028-78bf-4c5b-9ec6-939fe67de7f7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "merged_sessions = merge_chat_runs(chat_sessions)\n",
+    "alternating_sessions = list(map_ai_messages(merged_sessions, \"Harry Potter\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "4b11906e-a496-4d01-9f0d-1938c14147bf",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[AIMessage(content=\"Professor Snape, I was hoping I could speak with you for a moment about something that's been concerning me lately.\", additional_kwargs={'sender': 'Harry Potter'}, example=False),\n",
+       " HumanMessage(content=\"What is it, Potter? I'm quite busy at the moment.\", additional_kwargs={'sender': 'Severus Snape'}, example=False),\n",
+       " AIMessage(content=\"I apologize for the interruption, sir. I'll be brief. I've noticed some strange activity around the school grounds at night. I saw a cloaked figure lurking near the Forbidden Forest last night. I'm worried someone may be plotting something sinister.\", additional_kwargs={'sender': 'Harry Potter'}, example=False)]"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Now all of Harry Potter's messages will take the AI message class\n",
+    "# which maps to the 'assistant' role in OpenAI's training format\n",
+    "alternating_sessions[0]['messages'][:3]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d985478d-062e-47b9-ae9a-102f59be07c0",
+   "metadata": {},
+   "source": [
+    "#### Now we can convert to OpenAI format dictionaries"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "21372331",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.adapters.openai import convert_messages_for_finetuning"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "id": "92c5ae7a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Prepared 9 dialogues for training\n"
+     ]
+    }
+   ],
+   "source": [
+    "training_data = convert_messages_for_finetuning(alternating_sessions)\n",
+    "print(f\"Prepared {len(training_data)} dialogues for training\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "dfcbd181",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'role': 'assistant',\n",
+       "  'content': \"Professor Snape, I was hoping I could speak with you for a moment about something that's been concerning me lately.\"},\n",
+       " {'role': 'user',\n",
+       "  'content': \"What is it, Potter? I'm quite busy at the moment.\"},\n",
+       " {'role': 'assistant',\n",
+       "  'content': \"I apologize for the interruption, sir. I'll be brief. I've noticed some strange activity around the school grounds at night. I saw a cloaked figure lurking near the Forbidden Forest last night. I'm worried someone may be plotting something sinister.\"}]"
+      ]
+     },
+     "execution_count": 33,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "training_data[0][:3]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f1a9fd64-4f9f-42d3-b5dc-2a340e51e9e7",
+   "metadata": {},
+   "source": [
+    "OpenAI currently requires at least 10 training examples for a fine-tuning job, though they recommend between 50-100 for most tasks. Since we only have 9 chat sessions, we can subdivide them (optionally with some overlap) so that each training example is comprised of a portion of a whole conversation.\n",
+    "\n",
+    "Facebook chat sessions (1 per person) often span multiple days and conversations,\n",
+    "so the long-range dependencies may not be that important to model anyhow."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 42,
+   "id": "13cd290a-b1e9-4686-bb5e-d99de8b8612b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "100"
+      ]
+     },
+     "execution_count": 42,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Our chat is alternating, we will make each datapoint a group of 8 messages,\n",
+    "# with 2 messages overlapping\n",
+    "chunk_size = 8\n",
+    "overlap = 2\n",
+    "\n",
+    "training_examples = [\n",
+    "    conversation_messages[i: i + chunk_size] \n",
+    "    for conversation_messages in training_data\n",
+    "    for i in range(\n",
+    "        0, len(conversation_messages) - chunk_size + 1, \n",
+    "        chunk_size - overlap)\n",
+    "]\n",
+    "\n",
+    "len(training_examples)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cc8baf41-ff07-4492-96bd-b2472ee7cef9",
+   "metadata": {},
+   "source": [
+    "## 4. Fine-tune the model\n",
+    "\n",
+    "It's time to fine-tune the model. Make sure you have `openai` installed\n",
+    "and have set your `OPENAI_API_KEY` appropriately"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 43,
+   "id": "95ce3f63-3c80-44b2-9060-534ad74e16fa",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# %pip install -U openai --quiet"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 58,
+   "id": "ab9e28eb",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "File file-zCyNBeg4snpbBL7VkvsuhCz8 ready afer 30.55 seconds.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import json\n",
+    "from io import BytesIO\n",
+    "import time\n",
+    "\n",
+    "import openai\n",
+    "\n",
+    "# We will write the jsonl file in memory\n",
+    "my_file = BytesIO()\n",
+    "for m in training_examples:\n",
+    "    my_file.write((json.dumps({\"messages\": m}) + \"\\n\").encode('utf-8'))\n",
+    "\n",
+    "my_file.seek(0)\n",
+    "training_file = openai.File.create(\n",
+    "  file=my_file,\n",
+    "  purpose='fine-tune'\n",
+    ")\n",
+    "\n",
+    "# OpenAI audits each training file for compliance reasons.\n",
+    "# This make take a few minutes\n",
+    "status = openai.File.retrieve(training_file.id).status\n",
+    "start_time = time.time()\n",
+    "while status != \"processed\":\n",
+    "    print(f\"Status=[{status}]... {time.time() - start_time:.2f}s\", end=\"\\r\", flush=True)\n",
+    "    time.sleep(5)\n",
+    "    status = openai.File.retrieve(training_file.id).status\n",
+    "print(f\"File {training_file.id} ready after {time.time() - start_time:.2f} seconds.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "759a7f51-fde9-4b75-aaa9-e600e6537bd1",
+   "metadata": {},
+   "source": [
+    "With the file ready, it's time to kick off a training job."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 59,
+   "id": "3f451425",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "job = openai.FineTuningJob.create(\n",
+    "    training_file=training_file.id,\n",
+    "    model=\"gpt-3.5-turbo\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "489b23ef-5e14-42a9-bafb-44220ec6960b",
+   "metadata": {},
+   "source": [
+    "Grab a cup of tea while your model is being prepared. This may take some time!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 60,
+   "id": "bac1637a-c087-4523-ade1-c47f9bf4c6f4",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Status=[running]... 908.87s\r"
+     ]
+    }
+   ],
+   "source": [
+    "status = openai.FineTuningJob.retrieve(job.id).status\n",
+    "start_time = time.time()\n",
+    "while status != \"succeeded\":\n",
+    "    print(f\"Status=[{status}]... {time.time() - start_time:.2f}s\", end=\"\\r\", flush=True)\n",
+    "    time.sleep(5)\n",
+    "    job = openai.FineTuningJob.retrieve(job.id)\n",
+    "    status = job.status"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 66,
+   "id": "535895e1-bc69-40e5-82ed-e24ed2baeeee",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "ft:gpt-3.5-turbo-0613:personal::7rDwkaOq\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(job.fine_tuned_model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "502ff73b-f9e9-49ce-ba45-401811e57946",
+   "metadata": {},
+   "source": [
+    "## 5. Use in LangChain\n",
+    "\n",
+    "You can use the resulting model ID directly the `ChatOpenAI` model class."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 67,
+   "id": "3925d60d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "model = ChatOpenAI(\n",
+    "    model=job.fine_tuned_model,\n",
+    "    temperature=1,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 69,
+   "id": "7190cf2e-ab34-4ceb-bdad-45f24f069c29",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
+    "\n",
+    "prompt = ChatPromptTemplate.from_messages(\n",
+    "    [\n",
+    "        (\"human\", \"{input}\"),\n",
+    "    ]\n",
+    ")\n",
+    "\n",
+    "chain = prompt | model | StrOutputParser()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 72,
+   "id": "f02057e9-f914-40b1-9c9d-9432ff594b98",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The usual - Potions, Transfiguration, Defense Against the Dark Arts. What about you?"
+     ]
+    }
+   ],
+   "source": [
+    "for tok in chain.stream({\"input\": \"What classes are you taking?\"}):\n",
+    "    print(tok, end=\"\", flush=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "35331503-3cc6-4d64-955e-64afe6b5fef3",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/chat_loaders/gmail.ipynb

@@ -0,0 +1,179 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b3d1705d",
+   "metadata": {},
+   "source": [
+    "# GMail\n",
+    "\n",
+    "This loader goes over how to load data from GMail. There are many ways you could want to load data from GMail. This loader is currently fairly opionated in how to do so. The way it does it is it first looks for all messages that you have sent. It then looks for messages where you are responding to a previous email. It then fetches that previous email, and creates a training example of that email, followed by your email.\n",
+    "\n",
+    "Note that there are clear limitations here. For example, all examples created are only looking at the previous email for context.\n",
+    "\n",
+    "To use:\n",
+    "\n",
+    "- Set up a Google Developer Account: Go to the Google Developer Console, create a project, and enable the Gmail API for that project. This will give you a credentials.json file that you'll need later.\n",
+    "\n",
+    "- Install the Google Client Library: Run the following command to install the Google Client Library:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "84578039",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install --upgrade google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "be18f796",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os.path\n",
+    "import base64\n",
+    "import json\n",
+    "import re\n",
+    "import time\n",
+    "from google.auth.transport.requests import Request\n",
+    "from google.oauth2.credentials import Credentials\n",
+    "from google_auth_oauthlib.flow import InstalledAppFlow\n",
+    "from googleapiclient.discovery import build\n",
+    "import logging\n",
+    "import requests\n",
+    "\n",
+    "SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']\n",
+    "\n",
+    "\n",
+    "creds = None\n",
+    "# The file token.json stores the user's access and refresh tokens, and is\n",
+    "# created automatically when the authorization flow completes for the first\n",
+    "# time.\n",
+    "if os.path.exists('email_token.json'):\n",
+    "    creds = Credentials.from_authorized_user_file('email_token.json', SCOPES)\n",
+    "# If there are no (valid) credentials available, let the user log in.\n",
+    "if not creds or not creds.valid:\n",
+    "    if creds and creds.expired and creds.refresh_token:\n",
+    "        creds.refresh(Request())\n",
+    "    else:\n",
+    "        flow = InstalledAppFlow.from_client_secrets_file(               \n",
+    "            # your creds file here. Please create json file as here https://cloud.google.com/docs/authentication/getting-started\n",
+    "            'creds.json', SCOPES)\n",
+    "        creds = flow.run_local_server(port=0)\n",
+    "    # Save the credentials for the next run\n",
+    "    with open('email_token.json', 'w') as token:\n",
+    "        token.write(creds.to_json())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "a2793ba0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.gmail import GMailLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "2154597f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = GMailLoader(creds=creds, n=3)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "0b7d11bd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "data = loader.load()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "74764bc7",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "2"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Sometimes there can be errors which we silently ignore\n",
+    "len(data)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "d9360a85",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.utils import (\n",
+    "    map_ai_messages,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "a9646f7a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This makes messages sent by hchase@langchain.com the AI Messages\n",
+    "# This means you will train an LLM to predict as if it's responding as hchase\n",
+    "training_data = list(map_ai_messages(data, sender=\"Harrison Chase <hchase@langchain.com>\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d1a182f0",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/chat_loaders/imessage.ipynb

@@ -0,0 +1,420 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "01fcfa2f-33a9-48f3-835a-b1956c394d6b",
+   "metadata": {},
+   "source": [
+    "# iMessage\n",
+    "\n",
+    "This notebook shows how to use the iMessage chat loader. This class helps convert iMessage conversations to LangChain chat messages.\n",
+    "\n",
+    "On MacOS, iMessage stores conversations in a sqlite database at `~/Library/Messages/chat.db` (at least for macOS Ventura 13.4). \n",
+    "The `IMessageChatLoader` loads from this database file. \n",
+    "\n",
+    "1. Create the `IMessageChatLoader` with the file path pointed to `chat.db` database you'd like to process.\n",
+    "2. Call `loader.load()` (or `loader.lazy_load()`) to perform the conversion. Optionally use `merge_chat_runs` to combine message from the same sender in sequence, and/or `map_ai_messages` to convert messages from the specified sender to the \"AIMessage\" class.\n",
+    "\n",
+    "## 1. Access Chat DB\n",
+    "\n",
+    "It's likely that your terminal is denied access to `~/Library/Messages`. To use this class, you can copy the DB to an accessible directory (e.g., Documents) and load from there. Alternatively (and not recommended), you can grant full disk access for your terminal emulator in System Settings > Securityand Privacy > Full Disk Access.\n",
+    "\n",
+    "We have created an example database you can use at [this linked drive file](https://drive.google.com/file/d/1NebNKqTA2NXApCmeH6mu0unJD2tANZzo/view?usp=sharing)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "036ce7e0-a38f-4cbe-89a6-a205ae7c23be",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "File chat.db downloaded.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# This uses some example data\n",
+    "import requests\n",
+    "\n",
+    "def download_drive_file(url: str, output_path: str = 'chat.db') -> None:\n",
+    "    file_id = url.split('/')[-2]\n",
+    "    download_url = f'https://drive.google.com/uc?export=download&id={file_id}'\n",
+    "\n",
+    "    response = requests.get(download_url)\n",
+    "    if response.status_code != 200:\n",
+    "        print('Failed to download the file.')\n",
+    "        return\n",
+    "\n",
+    "    with open(output_path, 'wb') as file:\n",
+    "        file.write(response.content)\n",
+    "        print(f'File {output_path} downloaded.')\n",
+    "\n",
+    "url = 'https://drive.google.com/file/d/1NebNKqTA2NXApCmeH6mu0unJD2tANZzo/view?usp=sharing'\n",
+    "\n",
+    "# Download file to chat.db\n",
+    "download_drive_file(url)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cf60f703-76f1-4602-a723-02c59535c1af",
+   "metadata": {},
+   "source": [
+    "## 2. Create the Chat Loader\n",
+    "\n",
+    "Provide the loader with the file path to the zip directory. You can optionally specify the user id that maps to an ai message as well an configure whether to merge message runs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "4b8b432a-d2bc-49e1-b35f-761730a8fd6d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.imessage import IMessageChatLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "8ec6661b-0aca-48ae-9e2b-6412856c287b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = IMessageChatLoader(\n",
+    "    path=\"./chat.db\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8805a7c5-84b4-49f5-8989-0022f2054ace",
+   "metadata": {},
+   "source": [
+    "## 3. Load messages\n",
+    "\n",
+    "The `load()` (or `lazy_load`) methods return a list of \"ChatSessions\" that currently just contain a list of messages per loaded conversation. All messages are mapped to \"HumanMessage\" objects to start. \n",
+    "\n",
+    "You can optionally choose to merge message \"runs\" (consecutive messages from the same sender) and select a sender to represent the \"AI\". The fine-tuned LLM will learn to generate these AI messages."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "fcd69b3e-020d-4a15-8a0d-61c2d34e1ee1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List\n",
+    "from langchain.chat_loaders.base import ChatSession\n",
+    "from langchain.chat_loaders.utils import (\n",
+    "    map_ai_messages,\n",
+    "    merge_chat_runs,\n",
+    ")\n",
+    "\n",
+    "raw_messages = loader.lazy_load()\n",
+    "# Merge consecutive messages from the same sender into a single message\n",
+    "merged_messages = merge_chat_runs(raw_messages)\n",
+    "# Convert messages from \"Tortoise\" to AI messages. Do you have a guess who these conversations are between?\n",
+    "chat_sessions: List[ChatSession] = list(map_ai_messages(merged_messages, sender=\"Tortoise\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "370b8c26-c7a8-434c-a225-45c20ff14a03",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[AIMessage(content=\"Slow and steady, that's my motto.\", additional_kwargs={'message_time': 1693182723, 'sender': 'Tortoise'}, example=False),\n",
+       " HumanMessage(content='Speed is key!', additional_kwargs={'message_time': 1693182753, 'sender': 'Hare'}, example=False),\n",
+       " AIMessage(content='A balanced approach is more reliable.', additional_kwargs={'message_time': 1693182783, 'sender': 'Tortoise'}, example=False)]"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Now all of the Tortoise's messages will take the AI message class\n",
+    "# which maps to the 'assistant' role in OpenAI's training format\n",
+    "alternating_sessions[0]['messages'][:3]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "05208f9d-3193-4a8d-86a5-13df2c8197e5",
+   "metadata": {},
+   "source": [
+    "## 3. Prepare for fine-tuning\n",
+    "\n",
+    "Now it's time to convert our chat  messages to OpenAI dictionaries. We can use the `convert_messages_for_finetuning` utility to do so."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "8834861f-f37f-4c08-96c6-917269bf09b8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.adapters.openai import convert_messages_for_finetuning"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "ce7ab0f9-6e6a-4a1c-8b86-c635251d437e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Prepared 10 dialogues for training\n"
+     ]
+    }
+   ],
+   "source": [
+    "training_data = convert_messages_for_finetuning(alternating_sessions)\n",
+    "print(f\"Prepared {len(training_data)} dialogues for training\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b494d64c-8056-42ae-b4c1-a9cfabc002ea",
+   "metadata": {},
+   "source": [
+    "## 4. Fine-tune the model\n",
+    "\n",
+    "It's time to fine-tune the model. Make sure you have `openai` installed\n",
+    "and have set your `OPENAI_API_KEY` appropriately"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "b4b60daa-b899-4291-a09a-412ce9c218fc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# %pip install -U openai --quiet"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "2cca6c95-c0d6-4826-b4fa-1c403f217f93",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "File file-zHIgf4r8LltZG3RFpkGd4Sjf ready after 10.19 seconds.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import json\n",
+    "from io import BytesIO\n",
+    "import time\n",
+    "\n",
+    "import openai\n",
+    "\n",
+    "# We will write the jsonl file in memory\n",
+    "my_file = BytesIO()\n",
+    "for m in training_data:\n",
+    "    my_file.write((json.dumps({\"messages\": m}) + \"\\n\").encode('utf-8'))\n",
+    "\n",
+    "my_file.seek(0)\n",
+    "training_file = openai.File.create(\n",
+    "  file=my_file,\n",
+    "  purpose='fine-tune'\n",
+    ")\n",
+    "\n",
+    "# OpenAI audits each training file for compliance reasons.\n",
+    "# This make take a few minutes\n",
+    "status = openai.File.retrieve(training_file.id).status\n",
+    "start_time = time.time()\n",
+    "while status != \"processed\":\n",
+    "    print(f\"Status=[{status}]... {time.time() - start_time:.2f}s\", end=\"\\r\", flush=True)\n",
+    "    time.sleep(5)\n",
+    "    status = openai.File.retrieve(training_file.id).status\n",
+    "print(f\"File {training_file.id} ready after {time.time() - start_time:.2f} seconds.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "60ee0476-3113-4dc8-a886-bce878c60b07",
+   "metadata": {},
+   "source": [
+    "With the file ready, it's time to kick off a training job."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "c376ddca-5b4f-4e5a-bf4e-6beeb467eacc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "job = openai.FineTuningJob.create(\n",
+    "    training_file=training_file.id,\n",
+    "    model=\"gpt-3.5-turbo\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "09344c60-0bee-4989-b8d1-4a8821553cc3",
+   "metadata": {},
+   "source": [
+    "Grab a cup of tea while your model is being prepared. This may take some time!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "22eae900-04ca-456b-ba51-1dfff1f8e0e1",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Status=[running]... 524.95s\r"
+     ]
+    }
+   ],
+   "source": [
+    "status = openai.FineTuningJob.retrieve(job.id).status\n",
+    "start_time = time.time()\n",
+    "while status != \"succeeded\":\n",
+    "    print(f\"Status=[{status}]... {time.time() - start_time:.2f}s\", end=\"\\r\", flush=True)\n",
+    "    time.sleep(5)\n",
+    "    job = openai.FineTuningJob.retrieve(job.id)\n",
+    "    status = job.status"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "39e72616-a7d9-44b8-a4eb-506611d119f4",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "ft:gpt-3.5-turbo-0613:personal::7sKoRdlz\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(job.fine_tuned_model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0d717749-b1b6-451f-b3c5-3286b82d45b9",
+   "metadata": {},
+   "source": [
+    "## 5. Use in LangChain\n",
+    "\n",
+    "You can use the resulting model ID directly the `ChatOpenAI` model class."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "1579dfca-95c6-47b7-8549-1195b9dce5b0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "model = ChatOpenAI(\n",
+    "    model=job.fine_tuned_model,\n",
+    "    temperature=1,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "id": "6f53d1b1-dcbf-4976-a61a-17f74c6f1b0a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
+    "\n",
+    "prompt = ChatPromptTemplate.from_messages(\n",
+    "    [\n",
+    "        (\"system\", \"You are speaking to hare.\"),\n",
+    "        (\"human\", \"{input}\"),\n",
+    "    ]\n",
+    ")\n",
+    "\n",
+    "chain = prompt | model | StrOutputParser()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 41,
+   "id": "6619c9bc-54ea-4136-bd9a-44557f7da724",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "A symbol of interconnectedness."
+     ]
+    }
+   ],
+   "source": [
+    "for tok in chain.stream({\"input\": \"What's the golden thread?\"}):\n",
+    "    print(tok, end=\"\", flush=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "88e0d1a1-48a9-4d9d-9f4e-010cbbb65af8",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/chat_loaders/index.mdx

@@ -0,0 +1,188 @@
+---
+sidebar_position: 0
+---
+
+# Chat loaders
+
+Like document loaders, chat loaders are utilities designed to help load conversations from popular communication platforms such as Facebook, Slack, Discord, etc. These are loaded into memory as LangChain chat message objects. Such utilities facilitate tasks such as fine-tuning a language model to match your personal style or voice. 
+
+This brief guide will illustrate the process using [OpenAI's fine-tuning API](https://platform.openai.com/docs/guides/fine-tuning) comprised of six steps:
+
+1. Export your Facebook Messenger chat data in a compatible format for your intended chat loader.
+2. Load the chat data into memory as LangChain chat message objects. (_this is what is covered in each integration notebook in this section of the documentation_).
+    - Assign a person to the "AI" role and optionally filter, group, and merge messages.
+3. Export these acquired messages in a format expected by the fine-tuning API.
+4. Upload this data to OpenAI.
+5. Fine-tune your model.
+6. Implement the fine-tuned model in LangChain.
+
+This guide is not wholly comprehensive but is designed to take you through the fundamentals of going from raw data to fine-tuned model.
+
+We will demonstrate the procedure through an example of fine-tuning a `gpt-3.5-turbo` model on Facebook Messenger data. 
+
+### 1. Export your chat data
+
+To export your Facebook messenger data, you can follow the [instructions here](https://www.zapptales.com/en/download-facebook-messenger-chat-history-how-to/). 
+
+:::important JSON format
+You must select "JSON format" (instead of HTML) when exporting your data to be compatible with the current loader.
+:::
+
+OpenAI requires at least 10 examples to fine-tune your model, but they recommend between 50-100 for more optimal results.
+You can use the example data stored at [this google drive link](https://drive.google.com/file/d/1rh1s1o2i7B-Sk1v9o8KNgivLVGwJ-osV/view?usp=sharing) to test the process.
+
+### 2. Load the chat
+
+Once you've obtained your chat data, you can load it into memory as LangChain chat message objects. Here’s an example of loading data using the Python code:
+
+```python
+from langchain.chat_loaders.facebook_messenger import FolderFacebookMessengerChatLoader
+
+loader = FolderFacebookMessengerChatLoader(
+    path="./facebook_messenger_chats",
+)
+
+chat_sessions = loader.load()
+```
+
+In this snippet, we point the loader to a directory of Facebook chat dumps which are then loaded as multiple "sessions" of messages, one session per conversation file.
+
+Once you've loaded the messages, you should decide which person you want to fine-tune the model to (usually yourself). You can also decide to merge consecutive messages from the same sender into a single chat message.
+For both of these tasks, you can use the chat_loaders utilities to do so:
+
+```
+from langchain.chat_loaders.utils import (
+    merge_chat_runs,
+    map_ai_messages,
+)
+
+merged_sessions = merge_chat_runs(chat_sessions)
+alternating_sessions = list(map_ai_messages(merged_sessions, "My Name"))
+```
+
+### 3. Export messages to OpenAI format
+
+Convert the chat messages to dictionaries using the `convert_messages_for_finetuning` function. Then, group the data into chunks for better context modeling and overlap management.
+
+```python
+from langchain.adapters.openai import convert_messages_for_finetuning
+
+openai_messages = convert_messages_for_finetuning(chat_sessions)
+```
+
+At this point, the data is ready for upload to OpenAI. You can choose to split up conversations into smaller chunks for training if you
+do not have enough conversations to train on. Feel free to play around with different chunk sizes or with adding system messages to the fine-tuning data.
+
+```python
+chunk_size = 8
+overlap = 2
+
+message_groups = [
+    conversation_messages[i: i + chunk_size] 
+    for conversation_messages in openai_messages
+    for i in range(
+        0, len(conversation_messages) - chunk_size + 1, 
+        chunk_size - overlap)
+]
+
+len(message_groups)
+# 9
+```
+
+### 4. Upload the data to OpenAI
+
+Ensure you have set your OpenAI API key by following these [instructions](https://platform.openai.com/account/api-keys), then upload the training file.
+An audit is performed to ensure data compliance, so you may have to wait a few minutes for the dataset to become ready for use.
+
+```python
+import time
+import json
+import io
+
+import openai
+
+my_file = io.BytesIO()
+for group in message_groups:
+    my_file.write((json.dumps({"messages": group}) + "\n").encode('utf-8'))
+
+my_file.seek(0)
+training_file = openai.File.create(
+  file=my_file,
+  purpose='fine-tune'
+)
+
+# Wait while the file is processed
+status = openai.File.retrieve(training_file.id).status
+start_time = time.time()
+while status != "processed":
+    print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
+    time.sleep(5)
+    status = openai.File.retrieve(training_file.id).status
+print(f"File {training_file.id} ready after {time.time() - start_time:.2f} seconds.")
+```
+
+Once this is done, you can proceed to the model training!
+
+### 5. Fine-tune the model
+
+Start the fine-tuning job with your chosen base model.
+
+```python
+job = openai.FineTuningJob.create(
+    training_file=training_file.id,
+    model="gpt-3.5-turbo",
+)
+```
+
+This might take a while. Check the status with `openai.FineTuningJob.retrieve(job.id).status` and wait for it to report `succeeded`.
+
+```python
+# It may take 10-20+ minutes to complete training.
+status = openai.FineTuningJob.retrieve(job.id).status
+start_time = time.time()
+while status != "succeeded":
+    print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)
+    time.sleep(5)
+    job = openai.FineTuningJob.retrieve(job.id)
+    status = job.status
+```
+
+### 6. Use the model in LangChain
+
+You're almost there! Use the fine-tuned model in LangChain.
+
+```python
+from langchain import chat_models
+
+model_name = job.fine_tuned_model
+# Example: ft:gpt-3.5-turbo-0613:personal::5mty86jblapsed
+model = chat_models.ChatOpenAI(model=model_name)
+```
+
+```python
+from langchain.prompts import ChatPromptTemplate
+from langchain.schema.output_parser import StrOutputParser 
+
+prompt = ChatPromptTemplate.from_messages(
+    [
+        ("human", "{input}"),
+    ]
+)
+
+chain = prompt | model | StrOutputParser()
+
+for tok in chain.stream({"input": "What classes are you taking?"}):
+    print(tok, end="", flush=True)
+
+# The usual - Potions, Transfiguration, Defense Against the Dark Arts. What about you?
+```
+
+And that's it! You've successfully fine-tuned a model and used it in LangChain.
+
+## Supported Chat Loaders
+
+LangChain currently supports the following chat loaders. Feel free to contribute more!
+
+import DocCardList from "@theme/DocCardList";
+
+<DocCardList />

docs/extras/integrations/chat_loaders/slack.ipynb

@@ -0,0 +1,163 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "01fcfa2f-33a9-48f3-835a-b1956c394d6b",
+   "metadata": {},
+   "source": [
+    "# Slack\n",
+    "\n",
+    "This notebook shows how to use the Slack chat loader. This class helps map exported slack conversations to LangChain chat messages.\n",
+    "\n",
+    "The process has three steps:\n",
+    "1. Export the desired conversation thread by following the [instructions here](https://slack.com/help/articles/1500001548241-Request-to-export-all-conversations).\n",
+    "2. Create the `SlackChatLoader` with the file path pointed to the json file or directory of JSON files\n",
+    "3. Call `loader.load()` (or `loader.lazy_load()`) to perform the conversion. Optionally use `merge_chat_runs` to combine message from the same sender in sequence, and/or `map_ai_messages` to convert messages from the specified sender to the \"AIMessage\" class.\n",
+    "\n",
+    "## 1. Creat message dump\n",
+    "\n",
+    "Currently (2023/08/23) this loader best supports a zip directory of files in the format generated by exporting your a direct message converstion from Slack. Follow up-to-date instructions from slack on how to do so.\n",
+    "\n",
+    "We have an example in the LangChain repo."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "a79d35bf-5f21-4063-84bf-a60845c1c51f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import requests\n",
+    "\n",
+    "permalink = \"https://raw.githubusercontent.com/langchain-ai/langchain/342087bdfa3ac31d622385d0f2d09cf5e06c8db3/libs/langchain/tests/integration_tests/examples/slack_export.zip\"\n",
+    "response = requests.get(permalink)\n",
+    "with open(\"slack_dump.zip\", \"wb\") as f:\n",
+    "    f.write(response.content)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cf60f703-76f1-4602-a723-02c59535c1af",
+   "metadata": {},
+   "source": [
+    "## 2. Create the Chat Loader\n",
+    "\n",
+    "Provide the loader with the file path to the zip directory. You can optionally specify the user id that maps to an ai message as well an configure whether to merge message runs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "4b8b432a-d2bc-49e1-b35f-761730a8fd6d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.slack import SlackChatLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "8ec6661b-0aca-48ae-9e2b-6412856c287b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = SlackChatLoader(\n",
+    "    path=\"slack_dump.zip\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8805a7c5-84b4-49f5-8989-0022f2054ace",
+   "metadata": {},
+   "source": [
+    "## 3. Load messages\n",
+    "\n",
+    "The `load()` (or `lazy_load`) methods return a list of \"ChatSessions\" that currently just contain a list of messages per loaded conversation."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "fcd69b3e-020d-4a15-8a0d-61c2d34e1ee1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List\n",
+    "from langchain.chat_loaders.base import ChatSession\n",
+    "from langchain.chat_loaders.utils import (\n",
+    "    map_ai_messages,\n",
+    "    merge_chat_runs,\n",
+    ")\n",
+    "\n",
+    "raw_messages = loader.lazy_load()\n",
+    "# Merge consecutive messages from the same sender into a single message\n",
+    "merged_messages = merge_chat_runs(raw_messages)\n",
+    "# Convert messages from \"U0500003428\" to AI messages\n",
+    "messages: List[ChatSession] = list(map_ai_messages(merged_messages, sender=\"U0500003428\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7d033f87-cd0c-4f44-a753-41b871c1e919",
+   "metadata": {},
+   "source": [
+    "### Next Steps\n",
+    "\n",
+    "You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "7d8a1629-5d9e-49b3-b978-3add57027d59",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hi, \n",
+      "\n",
+      "I hope you're doing well. I wanted to reach out and ask if you'd be available to meet up for coffee sometime next week. I'd love to catch up and hear about what's been going on in your life. Let me know if you're interested and we can find a time that works for both of us. \n",
+      "\n",
+      "Looking forward to hearing from you!\n",
+      "\n",
+      "Best, [Your Name]"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "llm = ChatOpenAI()\n",
+    "\n",
+    "for chunk in llm.stream(messages[1]['messages']):\n",
+    "    print(chunk.content, end=\"\", flush=True)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/chat_loaders/telegram.ipynb

@@ -0,0 +1,206 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "735455a6-f82e-4252-b545-27385ef883f4",
+   "metadata": {},
+   "source": [
+    "# Telegram\n",
+    "\n",
+    "This notebook shows how to use the Telegram chat loader. This class helps map exported Telegram conversations to LangChain chat messages.\n",
+    "\n",
+    "The process has three steps:\n",
+    "1. Export  the chat .txt file by copying chats from the Discord app and pasting them in a file on your local computer\n",
+    "2. Create the `TelegramChatLoader` with the file path pointed to the json file or directory of JSON files\n",
+    "3. Call `loader.load()` (or `loader.lazy_load()`) to perform the conversion. Optionally use `merge_chat_runs` to combine message from the same sender in sequence, and/or `map_ai_messages` to convert messages from the specified sender to the \"AIMessage\" class.\n",
+    "\n",
+    "## 1. Creat message dump\n",
+    "\n",
+    "Currently (2023/08/23) this loader best supports json files in the format generated by exporting your chat history from the [Telegram Desktop App](https://desktop.telegram.org/).\n",
+    "\n",
+    "**Important:** There are 'lite' versions of telegram such as \"Telegram for MacOS\" that lack the export functionality. Please make sure you use the correct app to export the file.\n",
+    "\n",
+    "To make the export:\n",
+    "1. Download and open telegram desktop\n",
+    "2. Select a conversation\n",
+    "3. Navigate to the conversation settings (currently the three dots in the top right corner)\n",
+    "4. Click \"Export Chat History\"\n",
+    "5. Unselect photos and other media. Select \"Machine-readable JSON\" format to export.\n",
+    "\n",
+    "An example is below: "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "285f2044-0f58-4b92-addb-9f8569076734",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Overwriting telegram_conversation.json\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%writefile telegram_conversation.json\n",
+    "{\n",
+    " \"name\": \"Jiminy\",\n",
+    " \"type\": \"personal_chat\",\n",
+    " \"id\": 5965280513,\n",
+    " \"messages\": [\n",
+    "  {\n",
+    "   \"id\": 1,\n",
+    "   \"type\": \"message\",\n",
+    "   \"date\": \"2023-08-23T13:11:23\",\n",
+    "   \"date_unixtime\": \"1692821483\",\n",
+    "   \"from\": \"Jiminy Cricket\",\n",
+    "   \"from_id\": \"user123450513\",\n",
+    "   \"text\": \"You better trust your conscience\",\n",
+    "   \"text_entities\": [\n",
+    "    {\n",
+    "     \"type\": \"plain\",\n",
+    "     \"text\": \"You better trust your conscience\"\n",
+    "    }\n",
+    "   ]\n",
+    "  },\n",
+    "  {\n",
+    "   \"id\": 2,\n",
+    "   \"type\": \"message\",\n",
+    "   \"date\": \"2023-08-23T13:13:20\",\n",
+    "   \"date_unixtime\": \"1692821600\",\n",
+    "   \"from\": \"Batman & Robin\",\n",
+    "   \"from_id\": \"user6565661032\",\n",
+    "   \"text\": \"What did you just say?\",\n",
+    "   \"text_entities\": [\n",
+    "    {\n",
+    "     \"type\": \"plain\",\n",
+    "     \"text\": \"What did you just say?\"\n",
+    "    }\n",
+    "   ]\n",
+    "  }\n",
+    " ]\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7cc109f4-4c92-4cd3-8143-c322776c3f03",
+   "metadata": {},
+   "source": [
+    "## 2. Create the Chat Loader\n",
+    "\n",
+    "All that's required is the file path. You can optionally specify the user name that maps to an ai message as well an configure whether to merge message runs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "111f7767-573c-42d4-86f0-bd766bbaa071",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.telegram import TelegramChatLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "a4226efa-2640-4990-a20c-6861d1887329",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = TelegramChatLoader(\n",
+    "    path=\"./telegram_conversation.json\", \n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "71699fb7-7815-4c89-8d96-30e8fada6923",
+   "metadata": {},
+   "source": [
+    "## 3. Load messages\n",
+    "\n",
+    "The `load()` (or `lazy_load`) methods return a list of \"ChatSessions\" that currently just contain a list of messages per loaded conversation."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "81121efb-c875-4a77-ad1e-fe26b3d7e812",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List\n",
+    "from langchain.chat_loaders.base import ChatSession\n",
+    "from langchain.chat_loaders.utils import (\n",
+    "    map_ai_messages,\n",
+    "    merge_chat_runs,\n",
+    ")\n",
+    "\n",
+    "raw_messages = loader.lazy_load()\n",
+    "# Merge consecutive messages from the same sender into a single message\n",
+    "merged_messages = merge_chat_runs(raw_messages)\n",
+    "# Convert messages from \"Jiminy Cricket\" to AI messages\n",
+    "messages: List[ChatSession] = list(map_ai_messages(merged_messages, sender=\"Jiminy Cricket\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b9089c05-7375-41ca-a2f9-672a845314e4",
+   "metadata": {},
+   "source": [
+    "### Next Steps\n",
+    "\n",
+    "You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message  "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "637a6f5d-6944-4722-9361-a76ef5e9dd2a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "I said, \"You better trust your conscience.\""
+     ]
+    }
+   ],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "llm = ChatOpenAI()\n",
+    "\n",
+    "for chunk in llm.stream(messages[0]['messages']):\n",
+    "    print(chunk.content, end=\"\", flush=True)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/chat_loaders/twitter.ipynb

@@ -0,0 +1,77 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "d86853d2",
+   "metadata": {},
+   "source": [
+    "# Twitter (via Apify)\n",
+    "\n",
+    "This notebook shows how to load chat messages from Twitter to finetune on. We do this by utilizing Apify. \n",
+    "\n",
+    "First, use Apify to export tweets. An example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "e5034b4e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "from langchain.schema import AIMessage\n",
+    "from langchain.adapters.openai import convert_message_to_dict"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "8bf0fb93",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with open('example_data/dataset_twitter-scraper_2023-08-23_22-13-19-740.json') as f:\n",
+    "    data = json.load(f)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "468124fa",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Filter out tweets that reference other tweets, because it's a bit weird\n",
+    "tweets = [d[\"full_text\"] for d in data if \"t.co\" not in d['full_text']]\n",
+    "# Create them as AI messages\n",
+    "messages = [AIMessage(content=t) for t in tweets]\n",
+    "# Add in a system message at the start\n",
+    "# TODO: we could try to extract the subject from the tweets, and put that in the system message.\n",
+    "system_message = {\"role\": \"system\", \"content\": \"write a tweet\"}\n",
+    "data = [[system_message, convert_message_to_dict(m)] for m in messages]"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/chat_loaders/whatsapp.ipynb

@@ -0,0 +1,204 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "735455a6-f82e-4252-b545-27385ef883f4",
+   "metadata": {},
+   "source": [
+    "# WhatsApp\n",
+    "\n",
+    "This notebook shows how to use the WhatsApp chat loader. This class helps map exported Telegram conversations to LangChain chat messages.\n",
+    "\n",
+    "The process has three steps:\n",
+    "1. Export the chat conversations to computer\n",
+    "2. Create the `WhatsAppChatLoader` with the file path pointed to the json file or directory of JSON files\n",
+    "3. Call `loader.load()` (or `loader.lazy_load()`) to perform the conversion.\n",
+    "\n",
+    "## 1. Creat message dump\n",
+    "\n",
+    "To make the export of your WhatsApp conversation(s), complete the following steps:\n",
+    "\n",
+    "1. Open the target conversation\n",
+    "2. Click the three dots in the top right corner and select \"More\".\n",
+    "3. Then select \"Export chat\" and choose \"Without media\".\n",
+    "\n",
+    "An example of the data format for each converation is below: "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "285f2044-0f58-4b92-addb-9f8569076734",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Writing whatsapp_chat.txt\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%writefile whatsapp_chat.txt\n",
+    "[8/15/23, 9:12:33 AM] Dr. Feather: ‎Messages and calls are end-to-end encrypted. No one outside of this chat, not even WhatsApp, can read or listen to them.\n",
+    "[8/15/23, 9:12:43 AM] Dr. Feather: I spotted a rare Hyacinth Macaw yesterday in the Amazon Rainforest. Such a magnificent creature!\n",
+    "‎[8/15/23, 9:12:48 AM] Dr. Feather: ‎image omitted\n",
+    "[8/15/23, 9:13:15 AM] Jungle Jane: That's stunning! Were you able to observe its behavior?\n",
+    "‎[8/15/23, 9:13:23 AM] Dr. Feather: ‎image omitted\n",
+    "[8/15/23, 9:14:02 AM] Dr. Feather: Yes, it seemed quite social with other macaws. They're known for their playful nature.\n",
+    "[8/15/23, 9:14:15 AM] Jungle Jane: How's the research going on parrot communication?\n",
+    "‎[8/15/23, 9:14:30 AM] Dr. Feather: ‎image omitted\n",
+    "[8/15/23, 9:14:50 AM] Dr. Feather: It's progressing well. We're learning so much about how they use sound and color to communicate.\n",
+    "[8/15/23, 9:15:10 AM] Jungle Jane: That's fascinating! Can't wait to read your paper on it.\n",
+    "[8/15/23, 9:15:20 AM] Dr. Feather: Thank you! I'll send you a draft soon.\n",
+    "[8/15/23, 9:25:16 PM] Jungle Jane: Looking forward to it! Keep up the great work."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7cc109f4-4c92-4cd3-8143-c322776c3f03",
+   "metadata": {},
+   "source": [
+    "## 2. Create the Chat Loader\n",
+    "\n",
+    "The WhatsAppChatLoader accepts the resulting zip file, unzipped directory, or the path to any of the chat `.txt` files therein.\n",
+    "\n",
+    "Provide that as well as the user name you want to take on the role of \"AI\" when finetuning."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "111f7767-573c-42d4-86f0-bd766bbaa071",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_loaders.whatsapp import WhatsAppChatLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "a4226efa-2640-4990-a20c-6861d1887329",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = WhatsAppChatLoader(\n",
+    "    path=\"./whatsapp_chat.txt\", \n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "71699fb7-7815-4c89-8d96-30e8fada6923",
+   "metadata": {},
+   "source": [
+    "## 3. Load messages\n",
+    "\n",
+    "The `load()` (or `lazy_load`) methods return a list of \"ChatSessions\" that currently store the list of messages per loaded conversation."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "81121efb-c875-4a77-ad1e-fe26b3d7e812",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'messages': [AIMessage(content='I spotted a rare Hyacinth Macaw yesterday in the Amazon Rainforest. Such a magnificent creature!', additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:12:43 AM'}]}, example=False),\n",
+       "   HumanMessage(content=\"That's stunning! Were you able to observe its behavior?\", additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:13:15 AM'}]}, example=False),\n",
+       "   AIMessage(content=\"Yes, it seemed quite social with other macaws. They're known for their playful nature.\", additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:14:02 AM'}]}, example=False),\n",
+       "   HumanMessage(content=\"How's the research going on parrot communication?\", additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:14:15 AM'}]}, example=False),\n",
+       "   AIMessage(content=\"It's progressing well. We're learning so much about how they use sound and color to communicate.\", additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:14:50 AM'}]}, example=False),\n",
+       "   HumanMessage(content=\"That's fascinating! Can't wait to read your paper on it.\", additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:15:10 AM'}]}, example=False),\n",
+       "   AIMessage(content=\"Thank you! I'll send you a draft soon.\", additional_kwargs={'sender': 'Dr. Feather', 'events': [{'message_time': '8/15/23, 9:15:20 AM'}]}, example=False),\n",
+       "   HumanMessage(content='Looking forward to it! Keep up the great work.', additional_kwargs={'sender': 'Jungle Jane', 'events': [{'message_time': '8/15/23, 9:25:16 PM'}]}, example=False)]}]"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from typing import List\n",
+    "from langchain.chat_loaders.base import ChatSession\n",
+    "from langchain.chat_loaders.utils import (\n",
+    "    map_ai_messages,\n",
+    "    merge_chat_runs,\n",
+    ")\n",
+    "\n",
+    "raw_messages = loader.lazy_load()\n",
+    "# Merge consecutive messages from the same sender into a single message\n",
+    "merged_messages = merge_chat_runs(raw_messages)\n",
+    "# Convert messages from \"Dr. Feather\" to AI messages\n",
+    "messages: List[ChatSession] = list(map_ai_messages(merged_messages, sender=\"Dr. Feather\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b9089c05-7375-41ca-a2f9-672a845314e4",
+   "metadata": {},
+   "source": [
+    "### Next Steps\n",
+    "\n",
+    "You can then use these messages how you see fit, such as finetuning a model, few-shot example selection, or directly make predictions for the next message."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "637a6f5d-6944-4722-9361-a76ef5e9dd2a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Thank you for the encouragement! I'll do my best to continue studying and sharing fascinating insights about parrot communication."
+     ]
+    }
+   ],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "llm = ChatOpenAI()\n",
+    "\n",
+    "for chunk in llm.stream(messages[0]['messages']):\n",
+    "    print(chunk.content, end=\"\", flush=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "16156643-cfbd-444f-b4ae-198eb44f0267",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/document_loaders/Etherscan.ipynb

@@ -8,9 +8,9 @@
     "# Etherscan Loader\n",
     "## Overview\n",
     "\n",
-    "The Etherscan loader use etherscan api to load transacactions histories under specific account on Ethereum Mainnet.\n",
+    "The Etherscan loader use etherscan api to load transaction histories under specific account on Ethereum Mainnet.\n",
     "\n",
-    "You will need a Etherscan api key to proceed. The free api key has 5 calls per seconds quota.\n",
+    "You will need a Etherscan api key to proceed. The free api key has 5 calls per second quota.\n",
     "\n",
     "The loader supports the following six functinalities:\n",
     "* Retrieve normal transactions under specific account on Ethereum Mainet\n",

docs/extras/integrations/document_loaders/cube_semantic.ipynb

@@ -106,15 +106,39 @@
     "  - `column_data_type`\n",
     "  - `column_title`\n",
     "  - `column_description`\n",
-    "  - `column_values`"
+    "  - `column_values`\n",
+    "  - `cube_data_obj_type`"
    ]
   },
   {
-   "attachments": {},
-   "cell_type": "markdown",
+   "cell_type": "code",
+   "execution_count": null,
    "metadata": {},
+   "outputs": [],
    "source": [
-    "> page_content='Users View City, None' metadata={'table_name': 'users_view', 'column_name': 'users_view.city', 'column_data_type': 'string', 'column_title': 'Users View City', 'column_description': 'None', 'column_member_type': 'dimension', 'column_values': ['Austin', 'Chicago', 'Los Angeles', 'Mountain View', 'New York', 'Palo Alto', 'San Francisco', 'Seattle']}"
+    "# Given string containing page content\n",
+    "page_content = 'Users View City, None'\n",
+    "\n",
+    "# Given dictionary containing metadata\n",
+    "metadata = {\n",
+    "    'table_name': 'users_view',\n",
+    "    'column_name': 'users_view.city',\n",
+    "    'column_data_type': 'string',\n",
+    "    'column_title': 'Users View City',\n",
+    "    'column_description': 'None',\n",
+    "    'column_member_type': 'dimension',\n",
+    "    'column_values': [\n",
+    "        'Austin',\n",
+    "        'Chicago',\n",
+    "        'Los Angeles',\n",
+    "        'Mountain View',\n",
+    "        'New York',\n",
+    "        'Palo Alto',\n",
+    "        'San Francisco',\n",
+    "        'Seattle'\n",
+    "    ],\n",
+    "    'cube_data_obj_type': 'view'\n",
+    "}"
    ]
   }
  ],

docs/extras/integrations/document_loaders/google_drive.ipynb

@@ -38,7 +38,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "878928a6-a5ae-4f74-b351-64e3b01733fe",
    "metadata": {
     "tags": []
@@ -50,7 +50,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "id": "2216c83f-68e4-4d2f-8ea2-5878fb18bbe7",
    "metadata": {
     "tags": []
@@ -66,7 +66,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "id": "8f3b6aa0-b45d-4e37-8c50-5bebe70fdb9d",
    "metadata": {
     "tags": []
@@ -93,7 +93,7 @@
    "source": [
     "loader = GoogleDriveLoader(\n",
     "    folder_id=\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\",\n",
-    "    file_types=[\"document\", \"sheet\"]\n",
+    "    file_types=[\"document\", \"sheet\"],\n",
     "    recursive=False\n",
     ")"
    ]
@@ -110,7 +110,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "94207e39",
    "metadata": {},
    "outputs": [],
@@ -121,7 +121,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "id": "a15fbee0",
    "metadata": {},
    "outputs": [],
@@ -136,7 +136,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "id": "98410bda",
    "metadata": {},
    "outputs": [],
@@ -146,21 +146,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "id": "e3e72221",
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "Document(page_content='\\n  \\n    \\n      Team\\n      Location\\n      Stanley Cups\\n    \\n    \\n      Blues\\n      STL\\n      1\\n    \\n    \\n      Flyers\\n      PHI\\n      2\\n    \\n    \\n      Maple Leafs\\n      TOR\\n      13\\n    \\n  \\n', metadata={'filetype': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'page_number': 1, 'page_name': 'Stanley Cups', 'text_as_html': '<table border=\"1\" class=\"dataframe\">\\n  <tbody>\\n    <tr>\\n      <td>Team</td>\\n      <td>Location</td>\\n      <td>Stanley Cups</td>\\n    </tr>\\n    <tr>\\n      <td>Blues</td>\\n      <td>STL</td>\\n      <td>1</td>\\n    </tr>\\n    <tr>\\n      <td>Flyers</td>\\n      <td>PHI</td>\\n      <td>2</td>\\n    </tr>\\n    <tr>\\n      <td>Maple Leafs</td>\\n      <td>TOR</td>\\n      <td>13</td>\\n    </tr>\\n  </tbody>\\n</table>', 'category': 'Table', 'source': 'https://drive.google.com/file/d/1aA6L2AR3g0CR-PW03HEZZo4NaVlKpaP7/view'})"
-      ]
-     },
-     "execution_count": 4,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "docs[0]"
    ]
@@ -175,7 +164,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "id": "0e2d093f",
    "metadata": {},
    "outputs": [],
@@ -190,7 +179,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": null,
    "id": "b35ddcc6",
    "metadata": {},
    "outputs": [],
@@ -200,21 +189,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
    "id": "3cc141e0",
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "Document(page_content='\\n  \\n    \\n      Team\\n      Location\\n      Stanley Cups\\n    \\n    \\n      Blues\\n      STL\\n      1\\n    \\n    \\n      Flyers\\n      PHI\\n      2\\n    \\n    \\n      Maple Leafs\\n      TOR\\n      13\\n    \\n  \\n', metadata={'filetype': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'page_number': 1, 'page_name': 'Stanley Cups', 'text_as_html': '<table border=\"1\" class=\"dataframe\">\\n  <tbody>\\n    <tr>\\n      <td>Team</td>\\n      <td>Location</td>\\n      <td>Stanley Cups</td>\\n    </tr>\\n    <tr>\\n      <td>Blues</td>\\n      <td>STL</td>\\n      <td>1</td>\\n    </tr>\\n    <tr>\\n      <td>Flyers</td>\\n      <td>PHI</td>\\n      <td>2</td>\\n    </tr>\\n    <tr>\\n      <td>Maple Leafs</td>\\n      <td>TOR</td>\\n      <td>13</td>\\n    </tr>\\n  </tbody>\\n</table>', 'category': 'Table', 'source': 'https://drive.google.com/file/d/1aA6L2AR3g0CR-PW03HEZZo4NaVlKpaP7/view'})"
-      ]
-     },
-     "execution_count": 7,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "docs[0]"
    ]
@@ -226,6 +204,309 @@
    "metadata": {},
    "outputs": [],
    "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "83ac576b-48c9-4aad-a35e-e978ea32f746",
+   "metadata": {},
+   "source": [
+    "# Extended usage\n",
+    "An external component can manage the complexity of Google Drive : `langchain-googledrive`\n",
+    "It's compatible with the ̀`langchain.document_loaders.GoogleDriveLoader` and can be used\n",
+    "in its place.\n",
+    "\n",
+    "To be compatible with containers, the authentication uses an environment variable ̀GOOGLE_ACCOUNT_FILE` to credential file (for user or service)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b94f7119-bc1e-4ca3-907f-9d81e837ac59",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install langchain-googledrive"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c4c7474e-49cb-48a1-b3a0-77fba8e2dd70",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "folder_id='root'\n",
+    "#folder_id='1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5'"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8357f7f1-e2b1-41ef-8e38-48fcc3897dba",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Use the advanced version.\n",
+    "from langchain_googledrive.document_loaders import GoogleDriveLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "16ab9d3d-1782-4cb9-ab56-d87edbb25a18",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = GoogleDriveLoader(\n",
+    "    folder_id=folder_id,\n",
+    "    recursive=False,\n",
+    "    num_results=2,  # Maximum number of file to load\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ebac43aa-dd64-4964-802a-a90172415fd1",
+   "metadata": {},
+   "source": [
+    "By default, all files with these mime-type can be converted to `Document`.\n",
+    "- text/text\n",
+    "- text/plain\n",
+    "- text/html\n",
+    "- text/csv\n",
+    "- text/markdown\n",
+    "- image/png\n",
+    "- image/jpeg\n",
+    "- application/epub+zip\n",
+    "- application/pdf\n",
+    "- application/rtf\n",
+    "- application/vnd.google-apps.document (GDoc)\n",
+    "- application/vnd.google-apps.presentation (GSlide)\n",
+    "- application/vnd.google-apps.spreadsheet (GSheet)\n",
+    "- application/vnd.google.colaboratory (Notebook colab)\n",
+    "- application/vnd.openxmlformats-officedocument.presentationml.presentation (PPTX)\n",
+    "- application/vnd.openxmlformats-officedocument.wordprocessingml.document (DOCX)\n",
+    "\n",
+    "It's possible to update or customize this. See the documentation of `GDriveLoader`.\n",
+    "\n",
+    "But, the corresponding packages must be installed."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b4560f35-a37d-44e2-be0b-adaa245b3b3d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install unstructured"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6cb08da3-27df-46de-b60e-583bb7e31af4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for doc in loader.load():\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cd13d7d1-db7a-498d-ac98-76ccd9ad9019",
+   "metadata": {},
+   "source": [
+    "## Customize the search pattern\n",
+    "\n",
+    "All parameter compatible with Google [`list()`](https://developers.google.com/drive/api/v3/reference/files/list)\n",
+    "API can be set.\n",
+    "\n",
+    "To specify the new pattern of the Google request, you can use a `PromptTemplate()`.\n",
+    "The variables for the prompt can be set with `kwargs` in the constructor.\n",
+    "Some pre-formated request are proposed (use `{query}`, `{folder_id}` and/or `{mime_type}`):\n",
+    "\n",
+    "You can customize the criteria to select the files. A set of predefined filter are proposed:\n",
+    "| template                               | description                                                           |\n",
+    "| -------------------------------------- | --------------------------------------------------------------------- |\n",
+    "| gdrive-all-in-folder                   | Return all compatible files from a `folder_id`                        |\n",
+    "| gdrive-query                           | Search `query` in all drives                                          |\n",
+    "| gdrive-by-name                         | Search file with name `query`                                        |\n",
+    "| gdrive-query-in-folder                 | Search `query` in `folder_id` (and sub-folders if `recursive=true`)  |\n",
+    "| gdrive-mime-type                       | Search a specific `mime_type`                                         |\n",
+    "| gdrive-mime-type-in-folder             | Search a specific `mime_type` in `folder_id`                          |\n",
+    "| gdrive-query-with-mime-type            | Search `query` with a specific `mime_type`                            |\n",
+    "| gdrive-query-with-mime-type-and-folder | Search `query` with a specific `mime_type` and in `folder_id`         |\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "81348d59-8fd6-45d4-9de3-5df5cff5c7e2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = GoogleDriveLoader(\n",
+    "    folder_id=folder_id,\n",
+    "    recursive=False,\n",
+    "    template=\"gdrive-query\",  # Default template to use\n",
+    "    query=\"machine learning\",\n",
+    "    num_results=2,            # Maximum number of file to load\n",
+    "    supportsAllDrives=False,  # GDrive `list()` parameter\n",
+    ")\n",
+    "for doc in loader.load():\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "46c6ba5b-d4b1-4f0f-9801-5c1314021605",
+   "metadata": {},
+   "source": [
+    "You can customize your pattern."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5a5a323b-8d96-46b7-b46a-fd69bd2c8e04",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts.prompt import PromptTemplate\n",
+    "loader = GoogleDriveLoader(\n",
+    "    folder_id=folder_id,\n",
+    "    recursive=False,\n",
+    "    template=PromptTemplate(\n",
+    "        input_variables=[\"query\", \"query_name\"],\n",
+    "        template=\"fullText contains '{query}' and name contains '{query_name}' and trashed=false\",\n",
+    "        ),  # Default template to use\n",
+    "    query=\"machine learning\",\n",
+    "    query_name=\"ML\",    \n",
+    "    num_results=2,  # Maximum number of file to load\n",
+    ")\n",
+    "for doc in loader.load():\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "375bb465-8f69-407b-94bd-ffa3718ef500",
+   "metadata": {},
+   "source": [
+    "### Modes for GSlide and GSheet\n",
+    "The parameter mode accepts different values:\n",
+    "\n",
+    "- \"document\": return the body of each document\n",
+    "- \"snippets\": return the description of each file (set in metadata of Google Drive files).\n",
+    "\n",
+    "\n",
+    "The conversion can manage in Markdown format:\n",
+    "- bullet\n",
+    "- link\n",
+    "- table\n",
+    "- titles\n",
+    "\n",
+    "The parameter `gslide_mode` accepts different values:\n",
+    "\n",
+    "- \"single\" : one document with &lt;PAGE BREAK&gt;\n",
+    "- \"slide\" : one document by slide\n",
+    "- \"elements\" : one document for each elements.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7493d7b0-0600-49af-8107-7f4597c92de7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = GoogleDriveLoader(\n",
+    "    template=\"gdrive-mime-type\",\n",
+    "    mime_type=\"application/vnd.google-apps.presentation\", # Only GSlide files\n",
+    "    gslide_mode=\"slide\",\n",
+    "    num_results=2,  # Maximum number of file to load\n",
+    ")\n",
+    "for doc in loader.load():\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9bf338fb-02d7-452f-8679-c50419b13464",
+   "metadata": {},
+   "source": [
+    "The parameter `gsheet_mode` accepts different values:\n",
+    "- `\"single\"`: Generate one document by line\n",
+    "- `\"elements\"` : one document with markdown array and &lt;PAGE BREAK&gt; tags."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "469f5af0-67db-4f15-8aee-88cde480729b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = GoogleDriveLoader(\n",
+    "    template=\"gdrive-mime-type\",\n",
+    "    mime_type=\"application/vnd.google-apps.spreadsheet\", # Only GSheet files\n",
+    "    gsheet_mode=\"elements\",\n",
+    "    num_results=2,  # Maximum number of file to load\n",
+    ")\n",
+    "for doc in loader.load():\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "09acb864-e919-4add-9e06-deba6f7f0cd8",
+   "metadata": {},
+   "source": [
+    "## Advanced usage\n",
+    "All Google File have a 'description' in the metadata. This field can be used to memorize a summary of the document or others indexed tags (See method `lazy_update_description_with_summary()`).\n",
+    "\n",
+    "If you use the `mode=\"snippet\"`, only the description will be used for the body. Else, the `metadata['summary']` has the field.\n",
+    "\n",
+    "Sometime, a specific filter can be used to extract some information from the filename, to select some files with specific criteria. You can use a filter.\n",
+    "\n",
+    "Sometimes, many documents are returned. It's not necessary to have all documents in memory at the same time. You can use the lazy versions of methods, to get one document at a time. It's better to use a complex query in place of a recursive search. For each folder, a query must be applied if you activate `recursive=True`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a5e9c8eb-a266-4ae6-a760-d7826a0aa7c5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "loader = GoogleDriveLoader(\n",
+    "                gdrive_api_file=os.environ[\"GOOGLE_ACCOUNT_FILE\"],\n",
+    "                num_results=2,\n",
+    "                template=\"gdrive-query\",\n",
+    "                filter=lambda search, file: \"#test\" not in file.get('description',''),\n",
+    "                query='machine learning',\n",
+    "                supportsAllDrives=False,\n",
+    "                )\n",
+    "for doc in loader.load():\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "51efa73a-4e2d-4f9c-abaf-6c9bde2ff69d",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -244,7 +525,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.13"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,

docs/extras/integrations/document_transformers/docai.ipynb

@@ -0,0 +1,283 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "48438efb-9f0d-473b-a91c-9f1e29c2539d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders.blob_loaders import Blob\n",
+    "from langchain.document_loaders.parsers import DocAIParser"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f95ac25b-f025-40c3-95b8-77919fc4da7f",
+   "metadata": {},
+   "source": [
+    "DocAI is a Google Cloud platform to transform unstructured data from documents into structured data, making it easier to understand, analyze, and consume. You can read more about it: https://cloud.google.com/document-ai/docs/overview "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "51946817-798c-4d11-abd6-db2ae53a0270",
+   "metadata": {},
+   "source": [
+    "First, you need to set up a GCS bucket and create your own OCR processor as described here: https://cloud.google.com/document-ai/docs/create-processor\n",
+    "The GCS_OUTPUT_PATH should be a path to a folder on GCS (starting with `gs://`) and a processor name should look like `projects/PROJECT_NUMBER/locations/LOCATION/processors/PROCESSOR_ID`. You can get it either programmatically or copy from the `Prediction endpoint` section of the `Processor details` tab in the Google Cloud Console."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "ac85f7f3-3ef6-41d5-920a-b55f2939c202",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "PROJECT = \"PUT_SOMETHING_HERE\"\n",
+    "GCS_OUTPUT_PATH = \"PUT_SOMETHING_HERE\"\n",
+    "PROCESSOR_NAME = \"PUT_SOMETHING_HERE\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fad2bcca-1c0e-4888-b82d-15823ba57e60",
+   "metadata": {},
+   "source": [
+    "Now, let's create a parser:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "dcc0c65a-86c5-448d-8b21-2e564b1903b7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "parser = DocAIParser(location=\"us\", processor_name=PROCESSOR_NAME, gcs_output_path=GCS_OUTPUT_PATH)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b8b5a3ff-650a-4ad3-a73a-395f86e4c9e1",
+   "metadata": {},
+   "source": [
+    "Let's go and parse an Alphabet's take from here: https://abc.xyz/assets/a7/5b/9e5ae0364b12b4c883f3cf748226/goog-exhibit-99-1-q1-2023-19.pdf. Copy it to your GCS bucket first, and adjust the path below."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "373cc18e-a311-4c8d-8180-47e4ade1d2ad",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "blob = Blob(path=\"gs://vertex-pgt/examples/goog-exhibit-99-1-q1-2023-19.pdf\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "6ef84fad-2981-456d-a6b4-3a6a1a46d511",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "docs = list(parser.lazy_parse(blob))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3f8e4ee1-e07d-4c29-a120-4d56aae91859",
+   "metadata": {},
+   "source": [
+    "We'll get one document per page, 11 in total:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "343919f5-35d2-47fb-9790-de464649ebdf",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "11\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(len(docs))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b104ae56-011b-4abe-ac07-e999c69494c5",
+   "metadata": {},
+   "source": [
+    "You can run end-to-end parsing of a blob one-by-one. If you have many documents, it might be a better approach to batch them together and maybe even detach parsing from handling the results of parsing."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "9ecc1b99-5cef-47b0-a125-dbb2c41d2224",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "['projects/543079149601/locations/us/operations/16447136779727347991']\n"
+     ]
+    }
+   ],
+   "source": [
+    "operations = parser.docai_parse([blob])\n",
+    "print([op.operation.name for op in operations])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a2d24d63-c2c7-454c-9df3-2a9cf51309a6",
+   "metadata": {},
+   "source": [
+    "You can check whether operations are finished:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "ab11efb0-e514-4f44-9ba5-3d638a59c9e6",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "parser.is_running(operations)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "602ca0bc-080a-4a4e-a413-0e705aeab189",
+   "metadata": {},
+   "source": [
+    "And when they're finished, you can parse the results:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "ec1e6041-bc10-47d4-ba64-d09055c14f27",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "False"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "parser.is_running(operations)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "95d89da4-1c8a-413d-8473-ddd4a39375a5",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "DocAIParsingResults(source_path='gs://vertex-pgt/examples/goog-exhibit-99-1-q1-2023-19.pdf', parsed_path='gs://vertex-pgt/test/run1/16447136779727347991/0')\n"
+     ]
+    }
+   ],
+   "source": [
+    "results = parser.get_results(operations)\n",
+    "print(results[0])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "87e5b606-1679-46c7-9577-4cf9bc93a752",
+   "metadata": {},
+   "source": [
+    "And now we can finally generate Documents from parsed results:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "08e8878d-889b-41ad-9500-2f772d38782f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "docs = list(parser.parse_from_results(results))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "c59525fb-448d-444b-8f12-c4aea791e19b",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "11\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(len(docs))"
+   ]
+  }
+ ],
+ "metadata": {
+  "environment": {
+   "kernel": "python3",
+   "name": "common-cpu.m109",
+   "type": "gcloud",
+   "uri": "gcr.io/deeplearning-platform-release/base-cpu:m109"
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/llms/azure_openai_example.ipynb

@@ -30,7 +30,45 @@
     "```python\n",
     "import os\n",
     "os.environ[\"OPENAI_API_TYPE\"] = \"azure\"\n",
-    "...\n",
+    "```\n",
+    "\n",
+    "## Azure Active Directory Authentication\n",
+    "There are two ways you can authenticate to Azure OpenAI:\n",
+    "- API Key\n",
+    "- Azure Active Directory (AAD)\n",
+    "\n",
+    "Using the API key is the easiest way to get started. You can find your API key in the Azure portal under your Azure OpenAI resource.\n",
+    "\n",
+    "However, if you have complex security requirements - you may want to use Azure Active Directory. You can find more information on how to use AAD with Azure OpenAI [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/managed-identity).\n",
+    "\n",
+    "If you are developing locally, you will need to have the Azure CLI installed and be logged in. You can install the Azure CLI [here](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). Then, run `az login` to log in.\n",
+    "\n",
+    "Add a role an Azure role assignment `Cognitive Services OpenAI User` scoped to your Azure OpenAI resource. This will allow you to get a token from AAD to use with Azure OpenAI. You can grant this role assignment to a user, group, service principal, or managed identity. For more information about Azure OpenAI RBAC roles see [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/role-based-access-control).\n",
+    "\n",
+    "To use AAD in Python with LangChain, install the `azure-identity` package. Then, set `OPENAI_API_TYPE` to `azure_ad`. Next, use the `DefaultAzureCredential` class to get a token from AAD by calling `get_token` as shown below. Finally, set the `OPENAI_API_KEY` environment variable to the token value.\n",
+    "\n",
+    "```python\n",
+    "import os\n",
+    "from azure.identity import DefaultAzureCredential\n",
+    "\n",
+    "# Get the Azure Credential\n",
+    "credential = DefaultAzureCredential()\n",
+    "\n",
+    "# Set the API type to `azure_ad`\n",
+    "os.environ[\"OPENAI_API_TYPE\"] = \"azure_ad\"\n",
+    "# Set the API_KEY to the token from the Azure credential\n",
+    "os.environ[\"OPENAI_API_KEY\"] = credential.get_token(\"https://cognitiveservices.azure.com/.default\").token\n",
+    "```\n",
+    "\n",
+    "The `DefaultAzureCredential` class is an easy way to get started with AAD authentication. You can also customize the credential chain if necessary. In the example shown below, we first try Managed Identity, then fall back to the Azure CLI. This is useful if you are running your code in Azure, but want to develop locally.\n",
+    "\n",
+    "```python\n",
+    "from azure.identity import ChainedTokenCredential, ManagedIdentityCredential, AzureCliCredential\n",
+    "\n",
+    "credential = ChainedTokenCredential(\n",
+    "    ManagedIdentityCredential(),\n",
+    "    AzureCliCredential()\n",
+    ")\n",
     "```\n",
     "\n",
     "## Deployments\n",
@@ -144,7 +182,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "\u001b[1mAzureOpenAI\u001b[0m\n",
+      "\u001B[1mAzureOpenAI\u001B[0m\n",
       "Params: {'deployment_name': 'text-davinci-002', 'model_name': 'text-davinci-002', 'temperature': 0.7, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}\n"
      ]
     }

docs/extras/integrations/llms/google_vertex_ai_palm.ipynb

@@ -1,17 +1,28 @@
 {
  "cells": [
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Google Cloud Platform Vertex AI PaLM \n",
+    "# Google Vertex AI PaLM \n",
     "\n",
-    "Note: This is seperate from the Google PaLM integration, it exposes [Vertex AI PaLM API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on Google Cloud. \n",
+    "**Note:** This is seperate from the `Google PaLM` integration, it exposes [Vertex AI PaLM API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`. \n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting up"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud's AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
     "\n",
-    "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) Customer Data to train its foundation models as part of Google Cloud`s AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
-    "\n",
-    "To use Vertex AI PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
+    "To use `Vertex AI PaLM` you must have the `google-cloud-aiplatform` Python package installed and either:\n",
     "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",
     "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n",
     "\n",
@@ -19,8 +30,7 @@
     "\n",
     "For more information, see: \n",
     "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n",
-    "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n",
-    "\n"
+    "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth"
    ]
   },
   {
@@ -40,7 +50,22 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.llms import VertexAI\n",
+    "from langchain.llms import VertexAI"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Question-answering example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "from langchain import PromptTemplate, LLMChain"
    ]
   },
@@ -98,13 +123,21 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "You can now leverage the Codey API for code generation within Vertex AI. The model names are:\n",
-    "- code-bison: for code suggestion\n",
-    "- code-gecko: for code completion"
+    "## Code generation example"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can now leverage the `Codey API` for code generation within `Vertex AI`. \n",
+    "\n",
+    "The model names are:\n",
+    "- `code-bison`: for code suggestion\n",
+    "- `code-gecko`: for code completion"
    ]
   },
   {
@@ -191,7 +224,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   },
   "vscode": {
    "interpreter": {

docs/extras/integrations/llms/llamacpp.ipynb

@@ -7,9 +7,20 @@
     "# Llama.cpp\n",
     "\n",
     "[llama-cpp-python](https://github.com/abetlen/llama-cpp-python) is a Python binding for [llama.cpp](https://github.com/ggerganov/llama.cpp). \n",
-    "It supports [several LLMs](https://github.com/ggerganov/llama.cpp).\n",
     "\n",
-    "This notebook goes over how to run `llama-cpp-python` within LangChain."
+    "It supports inference for [many LLMs](https://github.com/ggerganov/llama.cpp), which can be accessed on [HuggingFace](https://huggingface.co/TheBloke).\n",
+    "\n",
+    "This notebook goes over how to run `llama-cpp-python` within LangChain.\n",
+    "\n",
+    "**Note: new versions of `llama-cpp-python` use GGUF model files (see [here](https://github.com/abetlen/llama-cpp-python/pull/633)).**\n",
+    "\n",
+    "This is a breaking change.\n",
+    " \n",
+    "To convert existing GGML models to GGUF you can run the following in [llama.cpp](https://github.com/ggerganov/llama.cpp):\n",
+    "\n",
+    "```\n",
+    "python ./convert-llama-ggmlv3-to-gguf.py --eps 1e-5 --input models/openorca-platypus2-13b.ggmlv3.q4_0.bin --output models/openorca-platypus2-13b.gguf.q4_0.bin\n",
+    "```"
    ]
   },
   {
@@ -19,7 +30,7 @@
     "## Installation\n",
     "\n",
     "There are different options on how to install the llama-cpp package: \n",
-    "- only CPU usage\n",
+    "- CPU usage\n",
     "- CPU + GPU (using one of many BLAS backends)\n",
     "- Metal GPU (MacOS with Apple Silicon Chip) \n",
     "\n",
@@ -171,7 +182,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 3,
    "metadata": {
     "tags": []
    },
@@ -192,7 +203,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 2,
    "metadata": {
     "tags": []
    },
@@ -207,15 +218,14 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 4,
    "metadata": {
     "tags": []
    },
    "outputs": [],
    "source": [
     "# Callbacks support token-wise streaming\n",
-    "callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])\n",
-    "# Verbose is required to pass to the callback manager"
+    "callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])"
    ]
   },
   {
@@ -240,12 +250,12 @@
    "source": [
     "# Make sure the model path is correct for your system!\n",
     "llm = LlamaCpp(\n",
-    "    model_path=\"/Users/rlm/Desktop/Code/llama/llama-2-7b-ggml/llama-2-7b-chat.ggmlv3.q4_0.bin\",\n",
+    "    model_path=\"/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin\",\n",
     "    temperature=0.75,\n",
     "    max_tokens=2000,\n",
     "    top_p=1,\n",
-    "    callback_manager=callback_manager,\n",
-    "    verbose=True,\n",
+    "    callback_manager=callback_manager, \n",
+    "    verbose=True, # Verbose is required to pass to the callback manager\n",
     ")"
    ]
   },
@@ -375,7 +385,6 @@
    ],
    "source": [
     "question = \"What NFL team won the Super Bowl in the year Justin Bieber was born?\"\n",
-    "\n",
     "llm_chain.run(question)"
    ]
   },
@@ -397,7 +406,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -406,46 +415,32 @@
     "\n",
     "# Make sure the model path is correct for your system!\n",
     "llm = LlamaCpp(\n",
-    "    model_path=\"./ggml-model-q4_0.bin\",\n",
+    "    model_path=\"/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin\",\n",
     "    n_gpu_layers=n_gpu_layers,\n",
     "    n_batch=n_batch,\n",
     "    callback_manager=callback_manager,\n",
-    "    verbose=True,\n",
+    "    verbose=True, # Verbose is required to pass to the callback manager\n",
     ")"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm_chain = LLMChain(prompt=prompt, llm=llm)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      " We are looking for an NFL team that won the Super Bowl when Justin Bieber (born March 1, 1994) was born. \n",
       "\n",
-      "First, let's look up which year is closest to when Justin Bieber was born:\n",
       "\n",
-      "* The year before he was born: 1993\n",
-      "* The year of his birth: 1994\n",
-      "* The year after he was born: 1995\n",
+      "1. Identify Justin Bieber's birth date: Justin Bieber was born on March 1, 1994.\n",
       "\n",
-      "We want to know what NFL team won the Super Bowl in the year that is closest to when Justin Bieber was born. Therefore, we should look up the NFL team that won the Super Bowl in either 1993 or 1994.\n",
+      "2. Find the Super Bowl winner of that year: The NFL season of 1993 with the Super Bowl being played in January or of 1994.\n",
       "\n",
-      "Now let's find out which NFL team did win the Super Bowl in either of those years:\n",
+      "3. Determine which team won the game: The Dallas Cowboys faced the Buffalo Bills in Super Bowl XXVII on January 31, 1993 (as the year is mis-labelled due to a error). The Dallas Cowboys won this matchup.\n",
       "\n",
-      "* In 1993, the San Francisco 49ers won the Super Bowl against the Dallas Cowboys by a score of 20-16.\n",
-      "* In 1994, the San Francisco 49ers won the Super Bowl again, this time against the San Diego Chargers by a score of 49-26.\n"
+      "So, Justin Bieber was born when the Dallas Cowboys were the reigning NFL Super Bowl."
      ]
     },
     {
@@ -453,27 +448,27 @@
      "output_type": "stream",
      "text": [
       "\n",
-      "llama_print_timings:        load time =   238.10 ms\n",
-      "llama_print_timings:      sample time =    84.23 ms /   256 runs   (    0.33 ms per token)\n",
-      "llama_print_timings: prompt eval time =   238.04 ms /    49 tokens (    4.86 ms per token)\n",
-      "llama_print_timings:        eval time = 10391.96 ms /   255 runs   (   40.75 ms per token)\n",
-      "llama_print_timings:       total time = 15664.80 ms\n"
+      "llama_print_timings:        load time =   427.63 ms\n",
+      "llama_print_timings:      sample time =   115.85 ms /   164 runs   (    0.71 ms per token,  1415.67 tokens per second)\n",
+      "llama_print_timings: prompt eval time =   427.53 ms /    45 tokens (    9.50 ms per token,   105.26 tokens per second)\n",
+      "llama_print_timings:        eval time =  4526.53 ms /   163 runs   (   27.77 ms per token,    36.01 tokens per second)\n",
+      "llama_print_timings:       total time =  5293.77 ms\n"
      ]
     },
     {
      "data": {
       "text/plain": [
-       "\" We are looking for an NFL team that won the Super Bowl when Justin Bieber (born March 1, 1994) was born. \\n\\nFirst, let's look up which year is closest to when Justin Bieber was born:\\n\\n* The year before he was born: 1993\\n* The year of his birth: 1994\\n* The year after he was born: 1995\\n\\nWe want to know what NFL team won the Super Bowl in the year that is closest to when Justin Bieber was born. Therefore, we should look up the NFL team that won the Super Bowl in either 1993 or 1994.\\n\\nNow let's find out which NFL team did win the Super Bowl in either of those years:\\n\\n* In 1993, the San Francisco 49ers won the Super Bowl against the Dallas Cowboys by a score of 20-16.\\n* In 1994, the San Francisco 49ers won the Super Bowl again, this time against the San Diego Chargers by a score of 49-26.\\n\""
+       "\"\\n\\n1. Identify Justin Bieber's birth date: Justin Bieber was born on March 1, 1994.\\n\\n2. Find the Super Bowl winner of that year: The NFL season of 1993 with the Super Bowl being played in January or of 1994.\\n\\n3. Determine which team won the game: The Dallas Cowboys faced the Buffalo Bills in Super Bowl XXVII on January 31, 1993 (as the year is mis-labelled due to a error). The Dallas Cowboys won this matchup.\\n\\nSo, Justin Bieber was born when the Dallas Cowboys were the reigning NFL Super Bowl.\""
       ]
      },
-     "execution_count": 8,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
     "question = \"What NFL team won the Super Bowl in the year Justin Bieber was born?\"\n",
-    "\n",
     "llm_chain.run(question)"
    ]
   },
@@ -503,15 +498,14 @@
    "source": [
     "n_gpu_layers = 1  # Metal set to 1 is enough.\n",
     "n_batch = 512  # Should be between 1 and n_ctx, consider the amount of RAM of your Apple Silicon Chip.\n",
-    "\n",
     "# Make sure the model path is correct for your system!\n",
     "llm = LlamaCpp(\n",
-    "    model_path=\"./ggml-model-q4_0.bin\",\n",
+    "    model_path=\"/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin\",\n",
     "    n_gpu_layers=n_gpu_layers,\n",
     "    n_batch=n_batch,\n",
     "    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls\n",
     "    callback_manager=callback_manager,\n",
-    "    verbose=True,\n",
+    "    verbose=True, # Verbose is required to pass to the callback manager\n",
     ")"
    ]
   },
@@ -531,6 +525,144 @@
     "\n",
     "For the first call to the LLM, the performance may be slow due to the model compilation in Metal GPU."
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Grammars\n",
+    "\n",
+    "\n",
+    "We can specify [grammars](https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md) to constrain model outputs.\n",
+    "\n",
+    "This will sample tokens according to the grammar.\n",
+    "  \n",
+    "For example, supply the path to the specifed `json.gbnf` file in order to produce JSON."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "n_gpu_layers = 1  # Metal set to 1 is enough.\n",
+    "n_batch = 512  # Should be between 1 and n_ctx, consider the amount of RAM of your Apple Silicon Chip.\n",
+    "# Make sure the model path is correct for your system!\n",
+    "llm = LlamaCpp(\n",
+    "    model_path=\"/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin\",\n",
+    "    n_gpu_layers=n_gpu_layers,\n",
+    "    n_batch=n_batch,\n",
+    "    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls\n",
+    "    callback_manager=callback_manager,\n",
+    "    verbose=True, # Verbose is required to pass to the callback manager\n",
+    "    grammar_path=\"/Users/rlm/Desktop/Code/langchain-main/langchain/libs/langchain/langchain/llms/grammars/json.gbnf\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\n",
+      "  \"name\": \"John Doe\",\n",
+      "  \"age\": 34,\n",
+      "  \"\": {\n",
+      "    \"title\": \"Software Developer\",\n",
+      "    \"company\": \"Google\"\n",
+      "  },\n",
+      "  \"interests\": [\n",
+      "    \"Sports\",\n",
+      "    \"Music\",\n",
+      "    \"Cooking\"\n",
+      "  ],\n",
+      "  \"address\": {\n",
+      "    \"street_number\": 123,\n",
+      "    \"street_name\": \"Oak Street\",\n",
+      "    \"city\": \"Mountain View\",\n",
+      "    \"state\": \"California\",\n",
+      "    \"postal_code\": 94040\n",
+      "  }}"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "llama_print_timings:        load time =   357.51 ms\n",
+      "llama_print_timings:      sample time =  1213.30 ms /   144 runs   (    8.43 ms per token,   118.68 tokens per second)\n",
+      "llama_print_timings: prompt eval time =   356.78 ms /     9 tokens (   39.64 ms per token,    25.23 tokens per second)\n",
+      "llama_print_timings:        eval time =  3947.16 ms /   143 runs   (   27.60 ms per token,    36.23 tokens per second)\n",
+      "llama_print_timings:       total time =  5846.21 ms\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%capture captured --no-stdout\n",
+    "result=llm(\"Describe a person in JSON format:\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can also supply `list.gbnf` to return a list."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "n_gpu_layers = 1 \n",
+    "n_batch = 512\n",
+    "llm = LlamaCpp(\n",
+    "    model_path=\"/Users/rlm/Desktop/Code/llama.cpp/models/openorca-platypus2-13b.gguf.q4_0.bin\",\n",
+    "    n_gpu_layers=n_gpu_layers,\n",
+    "    n_batch=n_batch,\n",
+    "    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls\n",
+    "    callback_manager=callback_manager,\n",
+    "    verbose=True,\n",
+    "    grammar_path=\"/Users/rlm/Desktop/Code/langchain-main/langchain/libs/langchain/langchain/llms/grammars/list.gbnf\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[\"The Catcher in the Rye\", \"Wuthering Heights\", \"Anna Karenina\"]\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "llama_print_timings:        load time =   322.34 ms\n",
+      "llama_print_timings:      sample time =   232.60 ms /    26 runs   (    8.95 ms per token,   111.78 tokens per second)\n",
+      "llama_print_timings: prompt eval time =   321.90 ms /    11 tokens (   29.26 ms per token,    34.17 tokens per second)\n",
+      "llama_print_timings:        eval time =   680.82 ms /    25 runs   (   27.23 ms per token,    36.72 tokens per second)\n",
+      "llama_print_timings:       total time =  1295.27 ms\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%capture captured --no-stdout\n",
+    "result=llm(\"List of top-3 my favourite books:\")"
+   ]
   }
  ],
  "metadata": {
@@ -549,7 +681,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.9.16"
   }
  },
  "nbformat": 4,

docs/extras/integrations/llms/llm_caching.ipynb

@@ -5,8 +5,9 @@
    "id": "f36d938c",
    "metadata": {},
    "source": [
-    "# Caching integrations\n",
-    "This notebook covers how to cache results of individual LLM calls."
+    "# LLM Caching integrations\n",
+    "\n",
+    "This notebook covers how to cache results of individual LLM calls using different caches."
    ]
   },
   {
@@ -26,9 +27,12 @@
   {
    "cell_type": "markdown",
    "id": "b50f0598",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
-    "## In Memory Cache"
+    "## `In Memory` Cache"
    ]
   },
   {
@@ -108,9 +112,12 @@
   {
    "cell_type": "markdown",
    "id": "4bf59c12",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
-    "## SQLite Cache"
+    "## `SQLite` Cache"
    ]
   },
   {
@@ -203,9 +210,12 @@
   {
    "cell_type": "markdown",
    "id": "278ad7ae",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
-    "## Redis Cache"
+    "## `Redis` Cache"
    ]
   },
   {
@@ -385,9 +395,12 @@
   {
    "cell_type": "markdown",
    "id": "684eab55",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
-    "## GPTCache\n",
+    "## `GPTCache`\n",
     "\n",
     "We can use [GPTCache](https://github.com/zilliztech/GPTCache) for exact match caching OR to cache results based on semantic similarity\n",
     "\n",
@@ -614,9 +627,12 @@
   {
    "cell_type": "markdown",
    "id": "726fe754",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
-    "## Momento Cache\n",
+    "## `Momento` Cache\n",
     "Use [Momento](/docs/ecosystem/integrations/momento.html) to cache prompts and responses.\n",
     "\n",
     "Requires momento to use, uncomment below to install:"
@@ -723,9 +739,14 @@
   {
    "cell_type": "markdown",
    "id": "934943dc",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
-    "## SQLAlchemy Cache"
+    "## `SQLAlchemy` Cache\n",
+    "\n",
+    "You can use `SQLAlchemyCache` to cache with any SQL database supported by `SQLAlchemy`."
    ]
   },
   {
@@ -735,8 +756,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# You can use SQLAlchemyCache to cache with any SQL database supported by SQLAlchemy.\n",
-    "\n",
     "# from langchain.cache import SQLAlchemyCache\n",
     "# from sqlalchemy import create_engine\n",
     "\n",
@@ -795,7 +814,10 @@
   {
    "cell_type": "markdown",
    "id": "0c69d84d",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
     "## Optional Caching\n",
     "You can also turn off caching for specific LLMs should you choose. In the example below, even though global caching is enabled, we turn it off for a specific LLM"
@@ -874,7 +896,10 @@
   {
    "cell_type": "markdown",
    "id": "5da41b77",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
     "## Optional Caching in Chains\n",
     "You can also turn off caching for particular nodes in chains. Note that because of certain interfaces, its often easier to construct the chain first, and then edit the LLM afterwards.\n",
@@ -1022,9 +1047,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "venv",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "venv"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -1036,7 +1061,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/integrations/llms/mosaicml.ipynb

@@ -63,7 +63,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "llm = MosaicML(inject_instruction_format=True, model_kwargs={\"do_sample\": False})"
+    "llm = MosaicML(inject_instruction_format=True, model_kwargs={\"max_new_tokens\": 128})"
    ]
   },
   {

docs/extras/integrations/memory/xata_chat_message_history.ipynb

@@ -0,0 +1,326 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Xata chat memory\n",
+    "\n",
+    "[Xata](https://xata.io) is a serverless data platform, based on PostgreSQL and Elasticsearch. It provides a Python SDK for interacting with your database, and a UI for managing your data. With the `XataChatMessageHistory` class, you can use Xata databases for longer-term persistence of chat sessions.\n",
+    "\n",
+    "This notebook covers:\n",
+    "\n",
+    "* A simple example showing what `XataChatMessageHistory` does.\n",
+    "* A more complex example using a REACT agent that answer questions based on a knowledge based or documentation (stored in Xata as a vector store) and also having a long-term searchable history of its past messages (stored in Xata as a memory store)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup\n",
+    "\n",
+    "### Create a database\n",
+    "\n",
+    "In the [Xata UI](https://app.xata.io) create a new database. You can name it whatever you want, in this notepad we'll use `langchain`. The Langchain integration can auto-create the table used for storying the memory, and this is what we'll use in this example. If you want to pre-create the table, ensure it has the right schema and set `create_table` to `False` when creating the class. Pre-creating the table saves one round-trip to the database during each session initialization."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's first install our dependencies:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install xata openai langchain"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we need to get the environment variables for Xata. You can create a new API key by visiting your [account settings](https://app.xata.io/settings). To find the database URL, go to the Settings page of the database that you have created. The database URL should look something like this: `https://demo-uni3q8.eu-west-1.xata.sh/db/langchain`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import getpass\n",
+    "\n",
+    "api_key = getpass.getpass(\"Xata API key: \")\n",
+    "db_url = input(\"Xata database URL (copy it from your DB settings):\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create a simple memory store\n",
+    "\n",
+    "To test the memory store functionality in isolation, let's use the following code snippet:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.memory import XataChatMessageHistory\n",
+    "\n",
+    "history = XataChatMessageHistory(\n",
+    "    session_id=\"session-1\",\n",
+    "    api_key=api_key,\n",
+    "    db_url=db_url,\n",
+    "    table_name=\"memory\"\n",
+    ")\n",
+    "\n",
+    "history.add_user_message(\"hi!\")\n",
+    "\n",
+    "history.add_ai_message(\"whats up?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The above code creates a session with the ID `session-1` and stores two messages in it. After running the above, if you visit the Xata UI, you should see a table named `memory` and the two messages added to it.\n",
+    "\n",
+    "You can retrieve the message history for a particular session with the following code:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "history.messages"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conversational Q&A chain on your data with memory\n",
+    "\n",
+    "Let's now see a more complex example in which we combine OpenAI, the Xata Vector Store integration, and the Xata memory store integration to create a Q&A chat bot on your data, with follow-up questions and history."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We're going to need to access the OpenAI API, so let's configure the API key:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To store the documents that the chatbot will search for answers, add a table named `docs` to your `langchain` database using the Xata UI, and add the following columns:\n",
+    "\n",
+    "* `content` of type \"Text\". This is used to store the `Document.pageContent` values.\n",
+    "* `embedding` of type \"Vector\". Use the dimension used by the model you plan to use. In this notebook we use OpenAI embeddings, which have 1536 dimensions."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's create the vector store and add some sample docs to it:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+    "from langchain.vectorstores.xata import XataVectorStore\n",
+    "\n",
+    "embeddings = OpenAIEmbeddings()\n",
+    "\n",
+    "texts = [\n",
+    "    \"Xata is a Serverless Data platform based on PostgreSQL\",\n",
+    "    \"Xata offers a built-in vector type that can be used to store and query vectors\",\n",
+    "    \"Xata includes similarity search\"\n",
+    "]\n",
+    "\n",
+    "vector_store = XataVectorStore.from_texts(texts, embeddings, api_key=api_key, db_url=db_url, table_name=\"docs\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After running the above command, if you go to the Xata UI, you should see the documents loaded together with their embeddings in the `docs` table."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's now create a ConversationBufferMemory to store the chat messages from both the user and the AI."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.memory import ConversationBufferMemory\n",
+    "from uuid import uuid4\n",
+    "\n",
+    "chat_memory = XataChatMessageHistory(\n",
+    "    session_id=str(uuid4()),   # needs to be unique per user session\n",
+    "    api_key=api_key,\n",
+    "    db_url=db_url,\n",
+    "    table_name=\"memory\"\n",
+    ")\n",
+    "memory = ConversationBufferMemory(memory_key=\"chat_history\", chat_memory=chat_memory, return_messages=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now it's time to create an Agent to use both the vector store and the chat memory together."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import initialize_agent, AgentType\n",
+    "from langchain.agents.agent_toolkits import create_retriever_tool\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "tool = create_retriever_tool(\n",
+    "    vector_store.as_retriever(), \n",
+    "    \"search_docs\",\n",
+    "    \"Searches and returns documents from the Xata manual. Useful when you need to answer questions about Xata.\"\n",
+    ")\n",
+    "tools = [tool]\n",
+    "\n",
+    "llm = ChatOpenAI(temperature=0)\n",
+    "\n",
+    "agent = initialize_agent(\n",
+    "    tools,\n",
+    "    llm,\n",
+    "    agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,\n",
+    "    verbose=True,\n",
+    "    memory=memory)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To test, let's tell the agent our name:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent.run(input=\"My name is bob\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now, let's now ask the agent some questions about Xata:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent.run(input=\"What is xata?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Notice that it answers based on the data stored in the document store. And now, let's ask a follow up question:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent.run(input=\"Does it support similarity search?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "And now let's test its memory:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent.run(input=\"Did I tell you my name? What is it?\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/extras/integrations/providers/ainetwork.mdx

@@ -0,0 +1,23 @@
+# AINetwork
+
+>[AI Network](https://www.ainetwork.ai/build-on-ain) is a layer 1 blockchain designed to accommodate 
+> large-scale AI models, utilizing a decentralized GPU network powered by the 
+> [$AIN token](https://www.ainetwork.ai/token), enriching AI-driven `NFTs` (`AINFTs`).
+
+
+## Installation and Setup
+
+You need to install `ain-py` python package.
+
+```bash
+pip install ain-py
+```
+You need to set the `AIN_BLOCKCHAIN_ACCOUNT_PRIVATE_KEY` environmental variable to your AIN Blockchain Account Private Key.
+## Toolkit
+
+See a [usage example](/docs/integrations/toolkits/ainetwork).
+
+```python
+from langchain.agents.agent_toolkits.ainetwork.toolkit import AINetworkToolkit
+```
+

docs/extras/integrations/providers/infino.mdx

@@ -2,10 +2,10 @@
 
 >[Infino](https://github.com/infinohq/infino) is an open-source observability platform that stores both metrics and application logs together.
 
-Key features of infino include:
-- Metrics Tracking: Capture time taken by LLM model to handle request, errors, number of tokens, and costing indication for the particular LLM.
-- Data Tracking: Log and store prompt, request, and response data for each LangChain interaction.
-- Graph Visualization: Generate basic graphs over time, depicting metrics such as request duration, error occurrences, token count, and cost.
+Key features of `Infino` include:
+- **Metrics Tracking**: Capture time taken by LLM model to handle request, errors, number of tokens, and costing indication for the particular LLM.
+- **Data Tracking**: Log and store prompt, request, and response data for each LangChain interaction.
+- **Graph Visualization**: Generate basic graphs over time, depicting metrics such as request duration, error occurrences, token count, and cost.
 
 ## Installation and Setup
 
@@ -15,7 +15,7 @@ First, you'll need to install the  `infinopy` Python package as follows:
 pip install infinopy
 ```
 
-If you already have an Infino Server running, then you're good to go; but if
+If you already have an `Infino Server` running, then you're good to go; but if
 you don't, follow the next steps to start it:
 
 - Make sure you have Docker installed
@@ -28,7 +28,7 @@ you don't, follow the next steps to start it:
 
 ## Using Infino
 
-See a [usage example of `InfinoCallbackHandler`](/docs/modules/callbacks/integrations/infino.html).
+See a [usage example of `InfinoCallbackHandler`](/docs/integrations/callbacks/infino.html).
 
 ```python
 from langchain.callbacks import InfinoCallbackHandler

docs/extras/integrations/providers/neo4j.mdx

@@ -0,0 +1,44 @@
+# Neo4j
+
+This page covers how to use the Neo4j ecosystem within LangChain.
+
+What is Neo4j?
+
+**Neo4j in a nutshell:**
+
+- Neo4j is an open-source database management system that specializes in graph database technology.
+- Neo4j allows you to represent and store data in nodes and edges, making it ideal for handling connected data and relationships.
+- Neo4j provides a Cypher Query Language, making it easy to interact with and query your graph data.
+- With Neo4j, you can achieve high-performance graph traversals and queries, suitable for production-level systems.
+- Get started quickly with Neo4j by visiting [their website](https://neo4j.com/).
+
+## Installation and Setup
+
+- Install the Python SDK with `pip install neo4j`
+
+## Wrappers
+
+### VectorStore
+
+There exists a wrapper around Neo4j vector index, allowing you to use it as a vectorstore,
+whether for semantic search or example selection.
+
+To import this vectorstore:
+
+```python
+from langchain.vectorstores import Neo4jVector
+```
+
+For a more detailed walkthrough of the Neo4j vector index wrapper, see [this notebook](/docs/integrations/vectorstores/neo4jvector.html)
+
+### GraphCypherQAChain
+
+There exists a wrapper around Neo4j graph database that allows you to generate Cypher statements based on the user input
+and use them to retrieve relevant information from the database.
+
+```python
+from langchain.graphs import Neo4jGraph
+from langchain.chains import GraphCypherQAChain
+```
+
+For a more detailed walkthrough of Cypher generating chain, see [this notebook](/docs/extras/use_cases/more/graph/graph_cypher_qa.html)

docs/extras/integrations/retrievers/google_drive.ipynb

@@ -0,0 +1,279 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b0ed136e-6983-4893-ae1b-b75753af05f8",
+   "metadata": {},
+   "source": [
+    "# Google Drive Retriever\n",
+    "This notebook covers how to retrieve documents from Google Drive.\n",
+    "\n",
+    "## Prerequisites\n",
+    "\n",
+    "1. Create a Google Cloud project or use an existing project\n",
+    "1. Enable the [Google Drive API](https://console.cloud.google.com/flows/enableapi?apiid=drive.googleapis.com)\n",
+    "1. [Authorize credentials for desktop app](https://developers.google.com/drive/api/quickstart/python#authorize_credentials_for_a_desktop_application)\n",
+    "1. `pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib`\n",
+    "\n",
+    "## Instructions for retrieving your Google Docs data\n",
+    "By default, the `GoogleDriveRetriever` expects the `credentials.json` file to be `~/.credentials/credentials.json`, but this is configurable using the `GOOGLE_ACCOUNT_FILE` environment variable. \n",
+    "The location of `token.json` use the same directory (or use the parameter `token_path`). Note that `token.json` will be created automatically the first time you use the retriever.\n",
+    "\n",
+    "`GoogleDriveRetriever` can retrieve a selection of files with some requests. \n",
+    "\n",
+    "By default, If you use a `folder_id`, all the files inside this folder can be retrieved to `Document`.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "35b94a93-97de-4af8-9cca-de9ffb7930c3",
+   "metadata": {},
+   "source": [
+    "You can obtain your folder and document id from the URL:\n",
+    "* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
+    "* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
+    "\n",
+    "The special value `root` is for your personal home."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9c9665c9-a023-4078-9d95-e43021cecb6f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "878928a6-a5ae-4f74-b351-64e3b01733fe",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-05-09T10:45:59.438650905Z",
+     "start_time": "2023-05-09T10:45:57.955900302Z"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.retrievers import GoogleDriveRetriever"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "755907c2-145d-4f0f-9b15-07a628a2d2d2",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-05-09T10:45:59.442890834Z",
+     "start_time": "2023-05-09T10:45:59.440941528Z"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "folder_id=\"root\"\n",
+    "#folder_id='1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5'"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2216c83f-68e4-4d2f-8ea2-5878fb18bbe7",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-05-09T10:45:59.795842403Z",
+     "start_time": "2023-05-09T10:45:59.445262457Z"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "retriever = GoogleDriveRetriever(\n",
+    "    num_results=2,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fa339ca0-f478-440c-ba80-0e5f41a19ce1",
+   "metadata": {},
+   "source": [
+    "By default, all files with these mime-type can be converted to `Document`.\n",
+    "- text/text\n",
+    "- text/plain\n",
+    "- text/html\n",
+    "- text/csv\n",
+    "- text/markdown\n",
+    "- image/png\n",
+    "- image/jpeg\n",
+    "- application/epub+zip\n",
+    "- application/pdf\n",
+    "- application/rtf\n",
+    "- application/vnd.google-apps.document (GDoc)\n",
+    "- application/vnd.google-apps.presentation (GSlide)\n",
+    "- application/vnd.google-apps.spreadsheet (GSheet)\n",
+    "- application/vnd.google.colaboratory (Notebook colab)\n",
+    "- application/vnd.openxmlformats-officedocument.presentationml.presentation (PPTX)\n",
+    "- application/vnd.openxmlformats-officedocument.wordprocessingml.document (DOCX)\n",
+    "\n",
+    "It's possible to update or customize this. See the documentation of `GDriveRetriever`.\n",
+    "\n",
+    "But, the corresponding packages must be installed."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9dadec48",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install unstructured"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8f3b6aa0-b45d-4e37-8c50-5bebe70fdb9d",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-05-09T10:46:00.990310466Z",
+     "start_time": "2023-05-09T10:45:59.798774595Z"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "retriever.get_relevant_documents(\"machine learning\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8ff33817-8619-4897-8742-2216b9934d2a",
+   "metadata": {},
+   "source": [
+    "You can customize the criteria to select the files. A set of predefined filter are proposed:\n",
+    "| template                               | description                                                           |\n",
+    "| -------------------------------------- | --------------------------------------------------------------------- |\n",
+    "| gdrive-all-in-folder                   | Return all compatible files from a `folder_id`                        |\n",
+    "| gdrive-query                           | Search `query` in all drives                                          |\n",
+    "| gdrive-by-name                         | Search file with name `query`)                                        |\n",
+    "| gdrive-query-in-folder                 | Search `query` in `folder_id` (and sub-folders in `_recursive=true`)  |\n",
+    "| gdrive-mime-type                       | Search a specific `mime_type`                                         |\n",
+    "| gdrive-mime-type-in-folder             | Search a specific `mime_type` in `folder_id`                          |\n",
+    "| gdrive-query-with-mime-type            | Search `query` with a specific `mime_type`                            |\n",
+    "| gdrive-query-with-mime-type-and-folder | Search `query` with a specific `mime_type` and in `folder_id`         |"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9977c712-9659-4959-b508-f59cc7d49d44",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "retriever = GoogleDriveRetriever(\n",
+    "    template=\"gdrive-query\", # Search everywhere\n",
+    "    num_results=2,  # But take only 2 documents\n",
+    ")\n",
+    "for doc in retriever.get_relevant_documents(\"machine learning\"):\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a5a0f3ef-26fb-4a5c-85f0-5aba90b682b1",
+   "metadata": {},
+   "source": [
+    "Else, you can customize the prompt with a specialized `PromptTemplate`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b0bbebde-0487-4d20-9d77-8070e4f0e0d6",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain import PromptTemplate\n",
+    "retriever = GoogleDriveRetriever(\n",
+    "    template=PromptTemplate(input_variables=['query'],\n",
+    "                            # See https://developers.google.com/drive/api/guides/search-files\n",
+    "                            template=\"(fullText contains '{query}') \"\n",
+    "                              \"and mimeType='application/vnd.google-apps.document' \"\n",
+    "                              \"and modifiedTime > '2000-01-01T00:00:00' \"\n",
+    "                              \"and trashed=false\"),\n",
+    "    num_results=2,\n",
+    "    # See https://developers.google.com/drive/api/v3/reference/files/list\n",
+    "    includeItemsFromAllDrives=False,\n",
+    "    supportsAllDrives=False,\n",
+    ")\n",
+    "for doc in retriever.get_relevant_documents(\"machine learning\"):\n",
+    "    print(f\"{doc.metadata['name']}:\")\n",
+    "    print(\"---\")\n",
+    "    print(doc.page_content.strip()[:60]+\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9b6fed29-1666-452e-b677-401613270388",
+   "metadata": {},
+   "source": [
+    "# Use GDrive 'description' metadata\n",
+    "Each Google Drive has a `description` field in metadata (see the *details of a file*).\n",
+    "Use the `snippets` mode to return the description of selected files.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "342dbe12-ed83-40f4-8957-0cc8c4609542",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "retriever = GoogleDriveRetriever(\n",
+    "    template='gdrive-mime-type-in-folder',\n",
+    "    folder_id=folder_id,\n",
+    "    mime_type='application/vnd.google-apps.document',  # Only Google Docs\n",
+    "    num_results=2,\n",
+    "    mode='snippets',\n",
+    "    includeItemsFromAllDrives=False,\n",
+    "    supportsAllDrives=False,\n",
+    ")\n",
+    "retriever.get_relevant_documents(\"machine learning\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/text_embedding/sagemaker-endpoint.ipynb

@@ -48,10 +48,31 @@
     "    accepts = \"application/json\"\n",
     "\n",
     "    def transform_input(self, inputs: list[str], model_kwargs: Dict) -> bytes:\n",
-    "        input_str = json.dumps({\"inputs\": inputs, **model_kwargs})\n",
+    "        \"\"\"\n",
+    "        Transforms the input into bytes that can be consumed by SageMaker endpoint.\n",
+    "        Args:\n",
+    "            inputs: List of input strings.\n",
+    "            model_kwargs: Additional keyword arguments to be passed to the endpoint.\n",
+    "        Returns:\n",
+    "            The transformed bytes input.\n",
+    "        \"\"\"\n",
+    "        # Example: inference.py expects a JSON string with a \"inputs\" key:\n",
+    "        input_str = json.dumps({\"inputs\": inputs, **model_kwargs})  \n",
     "        return input_str.encode(\"utf-8\")\n",
     "\n",
     "    def transform_output(self, output: bytes) -> List[List[float]]:\n",
+    "        \"\"\"\n",
+    "        Transforms the bytes output from the endpoint into a list of embeddings.\n",
+    "        Args:\n",
+    "            output: The bytes output from SageMaker endpoint.\n",
+    "        Returns:\n",
+    "            The transformed output - list of embeddings\n",
+    "        Note:\n",
+    "            The length of the outer list is the number of input strings.\n",
+    "            The length of the inner lists is the embedding dimension.\n",
+    "        \"\"\"\n",
+    "        # Example: inference.py returns a JSON string with the list of\n",
+    "        # embeddings in a \"vectors\" key:\n",
     "        response_json = json.loads(output.read().decode(\"utf-8\"))\n",
     "        return response_json[\"vectors\"]\n",
     "\n",
@@ -60,7 +81,6 @@
     "\n",
     "\n",
     "embeddings = SagemakerEndpointEmbeddings(\n",
-    "    # endpoint_name=\"endpoint-name\",\n",
     "    # credentials_profile_name=\"credentials-profile-name\",\n",
     "    endpoint_name=\"huggingface-pytorch-inference-2023-03-21-16-14-03-834\",\n",
     "    region_name=\"us-east-1\",\n",

docs/extras/integrations/toolkits/ainetwork.ipynb

@@ -1,17 +1,17 @@
 {
  "cells": [
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# AINetwork Toolkit\n",
+    "# AINetwork\n",
     "\n",
-    "The AINetwork Toolkit is a set of tools for interacting with the AINetwork Blockchain. These tools allow you to transfer AIN, read and write values, create apps, and set permissions for specific paths within the blockchain database."
+    ">[AI Network](https://www.ainetwork.ai/build-on-ain) is a layer 1 blockchain designed to accommodate large-scale AI models, utilizing a decentralized GPU network powered by the [$AIN token](https://www.ainetwork.ai/token), enriching AI-driven `NFTs` (`AINFTs`).\n",
+    ">\n",
+    ">The `AINetwork Toolkit` is a set of tools for interacting with the [AINetwork Blockchain](https://www.ainetwork.ai/public/whitepaper.pdf). These tools allow you to transfer `AIN`, read and write values, create apps, and set permissions for specific paths within the blockchain database."
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -30,7 +30,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -51,7 +50,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -96,7 +94,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -119,7 +116,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -147,7 +143,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -157,7 +152,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -174,7 +168,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -213,7 +206,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -250,7 +242,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -290,7 +281,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -337,7 +327,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -362,7 +351,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -397,7 +385,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -438,7 +425,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -453,9 +440,8 @@
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.10.12"
-  },
-  "orig_nbformat": 4
+  }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }

docs/extras/integrations/toolkits/google_drive.ipynb

@@ -0,0 +1,215 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Google Drive tool\n",
+    "\n",
+    "This notebook walks through connecting a LangChain to the Google Drive API.\n",
+    "\n",
+    "## Prerequisites\n",
+    "\n",
+    "1. Create a Google Cloud project or use an existing project\n",
+    "1. Enable the [Google Drive API](https://console.cloud.google.com/flows/enableapi?apiid=drive.googleapis.com)\n",
+    "1. [Authorize credentials for desktop app](https://developers.google.com/drive/api/quickstart/python#authorize_credentials_for_a_desktop_application)\n",
+    "1. `pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib`\n",
+    "\n",
+    "## Instructions for retrieving your Google Docs data\n",
+    "By default, the `GoogleDriveTools` and `GoogleDriveWrapper` expects the `credentials.json` file to be `~/.credentials/credentials.json`, but this is configurable using the `GOOGLE_ACCOUNT_FILE` environment variable. \n",
+    "The location of `token.json` use the same directory (or use the parameter `token_path`). Note that `token.json` will be created automatically the first time you use the tool.\n",
+    "\n",
+    "`GoogleDriveSearchTool` can retrieve a selection of files with some requests. \n",
+    "\n",
+    "By default, If you use a `folder_id`, all the files inside this folder can be retrieved to `Document`, if the name match the query.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can obtain your folder and document id from the URL:\n",
+    "* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",
+    "* Document: https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is `\"1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw\"`\n",
+    "\n",
+    "The special value `root` is for your personal home."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "folder_id=\"root\"\n",
+    "#folder_id='1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "By default, all files with these mime-type can be converted to `Document`.\n",
+    "- text/text\n",
+    "- text/plain\n",
+    "- text/html\n",
+    "- text/csv\n",
+    "- text/markdown\n",
+    "- image/png\n",
+    "- image/jpeg\n",
+    "- application/epub+zip\n",
+    "- application/pdf\n",
+    "- application/rtf\n",
+    "- application/vnd.google-apps.document (GDoc)\n",
+    "- application/vnd.google-apps.presentation (GSlide)\n",
+    "- application/vnd.google-apps.spreadsheet (GSheet)\n",
+    "- application/vnd.google.colaboratory (Notebook colab)\n",
+    "- application/vnd.openxmlformats-officedocument.presentationml.presentation (PPTX)\n",
+    "- application/vnd.openxmlformats-officedocument.wordprocessingml.document (DOCX)\n",
+    "\n",
+    "It's possible to update or customize this. See the documentation of `GoogleDriveAPIWrapper`.\n",
+    "\n",
+    "But, the corresponding packages must installed."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install unstructured"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.utilities.google_drive import GoogleDriveAPIWrapper\n",
+    "from langchain.tools.google_drive.tool import GoogleDriveSearchTool\n",
+    "\n",
+    "# By default, search only in the filename.\n",
+    "tool = GoogleDriveSearchTool(\n",
+    "    api_wrapper=GoogleDriveAPIWrapper(\n",
+    "        folder_id=folder_id,\n",
+    "        num_results=2,\n",
+    "        template=\"gdrive-query-in-folder\", # Search in the body of documents\n",
+    "    )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import logging\n",
+    "logging.basicConfig(level=logging.INFO)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool.run(\"machine learning\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool.description"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import load_tools\n",
+    "tools = load_tools([\"google-drive-search\"],\n",
+    "                   folder_id=folder_id,\n",
+    "                   template=\"gdrive-query-in-folder\",\n",
+    "                  )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Use within an Agent"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain import OpenAI\n",
+    "from langchain.agents import initialize_agent, AgentType\n",
+    "llm = OpenAI(temperature=0)\n",
+    "agent = initialize_agent(\n",
+    "    tools=tools,\n",
+    "    llm=llm,\n",
+    "    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "agent.run(\n",
+    "    \"Search in google drive, who is 'Yann LeCun' ?\"\n",
+    ")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/extras/integrations/vectorstores/azuresearch.ipynb

@@ -6,7 +6,9 @@
    "source": [
     "# Azure Cognitive Search\n",
     "\n",
-    "[Azure Cognitive Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search) (formerly known as `Azure Search`) is a cloud search service that gives developers infrastructure, APIs, and tools for building a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.\n"
+    "[Azure Cognitive Search](https://learn.microsoft.com/azure/search/search-what-is-azure-search) (formerly known as `Azure Search`) is a cloud search service that gives developers infrastructure, APIs, and tools for building a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.\n",
+    "\n",
+    "Vector search is currently in public preview. It's available through the Azure portal, preview REST API and beta client libraries. [More info](https://learn.microsoft.com/en-us/azure/search/vector-search-overview) Beta client libraries are subject to potential breaking changes, please be sure to use the SDK package version identified below. azure-search-documents==11.4.0b8"
    ]
   },
   {
@@ -22,7 +24,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install azure-search-documents==11.4.0b6\n",
+    "!pip install azure-search-documents==11.4.0b8\n",
     "!pip install azure-identity"
    ]
   },
@@ -36,13 +38,13 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
     "import openai\n",
     "import os\n",
-    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.vectorstores.azuresearch import AzureSearch"
    ]
   },
@@ -57,7 +59,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -79,7 +81,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -98,7 +100,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -151,7 +153,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -178,6 +180,41 @@
     "print(docs[0].page_content)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Perform a vector similarity search with relevance scores\n",
+    " \n",
+    "Execute a pure vector similarity search using the similarity_search_with_relevance_scores() method:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[(Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \\n\\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \\n\\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': 'C:\\\\repos\\\\langchain-fruocco-acs\\\\langchain\\\\docs\\\\extras\\\\modules\\\\state_of_the_union.txt'}),\n",
+      "  0.8441472),\n",
+      " (Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \\n\\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \\n\\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': 'C:\\\\repos\\\\langchain-fruocco-acs\\\\langchain\\\\docs\\\\extras\\\\modules\\\\state_of_the_union.txt'}),\n",
+      "  0.8441472),\n",
+      " (Document(page_content='A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. \\n\\nAnd if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. \\n\\nWe can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  \\n\\nWe’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  \\n\\nWe’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster. \\n\\nWe’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.', metadata={'source': 'C:\\\\repos\\\\langchain-fruocco-acs\\\\langchain\\\\docs\\\\extras\\\\modules\\\\state_of_the_union.txt'}),\n",
+      "  0.82153815),\n",
+      " (Document(page_content='A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. \\n\\nAnd if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. \\n\\nWe can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  \\n\\nWe’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  \\n\\nWe’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster. \\n\\nWe’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.', metadata={'source': 'C:\\\\repos\\\\langchain-fruocco-acs\\\\langchain\\\\docs\\\\extras\\\\modules\\\\state_of_the_union.txt'}),\n",
+      "  0.82153815)]\n"
+     ]
+    }
+   ],
+   "source": [
+    "docs_and_scores = vector_store.similarity_search_with_relevance_scores(query=\"What did the president say about Ketanji Brown Jackson\", k=4, score_threshold=0.80)\n",
+    "from pprint import pprint\n",
+    "pprint(docs_and_scores)"
+   ]
+  },
   {
    "attachments": {},
    "cell_type": "markdown",
@@ -190,7 +227,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 13,
    "metadata": {},
    "outputs": [
     {
@@ -219,7 +256,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 14,
    "metadata": {},
    "outputs": [
     {
@@ -254,7 +291,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 18,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -328,7 +365,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 19,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -348,7 +385,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 20,
    "metadata": {},
    "outputs": [
     {
@@ -371,7 +408,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 21,
    "metadata": {},
    "outputs": [
     {
@@ -400,7 +437,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 22,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -494,7 +531,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 23,
    "metadata": {},
    "outputs": [
     {
@@ -530,7 +567,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 23,
+   "execution_count": 24,
    "metadata": {},
    "outputs": [
     {

docs/extras/integrations/vectorstores/neo4jvector.ipynb

@@ -0,0 +1,440 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Neo4j Vector Index\n",
+    "\n",
+    ">[Neo4j](https://neo4j.com/) is an open-source graph database with integrated support for vector similarity search\n",
+    "\n",
+    "It supports:\n",
+    "- approximate nearest neighbor search\n",
+    "- L2 distance and cosine distance\n",
+    "\n",
+    "This notebook shows how to use the Neo4j vector index (`Neo4jVector`)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "See the [installation instruction](https://neo4j.com/docs/operations-manual/current/installation/)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: neo4j in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (5.11.0)\n",
+      "Requirement already satisfied: pytz in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from neo4j) (2023.3)\n",
+      "Requirement already satisfied: openai in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (0.27.6)\n",
+      "Requirement already satisfied: requests>=2.20 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from openai) (2.31.0)\n",
+      "Requirement already satisfied: tqdm in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from openai) (4.66.1)\n",
+      "Requirement already satisfied: aiohttp in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from openai) (3.8.5)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.20->openai) (3.2.0)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.20->openai) (3.4)\n",
+      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.20->openai) (2.0.4)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.20->openai) (2023.7.22)\n",
+      "Requirement already satisfied: attrs>=17.3.0 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from aiohttp->openai) (23.1.0)\n",
+      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from aiohttp->openai) (6.0.4)\n",
+      "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from aiohttp->openai) (4.0.3)\n",
+      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from aiohttp->openai) (1.9.2)\n",
+      "Requirement already satisfied: frozenlist>=1.1.1 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from aiohttp->openai) (1.4.0)\n",
+      "Requirement already satisfied: aiosignal>=1.1.2 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from aiohttp->openai) (1.3.1)\n",
+      "Requirement already satisfied: tiktoken in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (0.4.0)\n",
+      "Requirement already satisfied: regex>=2022.1.18 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from tiktoken) (2023.8.8)\n",
+      "Requirement already satisfied: requests>=2.26.0 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from tiktoken) (2.31.0)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (3.2.0)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (3.4)\n",
+      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (2.0.4)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in /home/tomaz/anaconda3/envs/myenv/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (2023.7.22)\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Pip install necessary package\n",
+    "!pip install neo4j\n",
+    "!pip install openai\n",
+    "!pip install tiktoken"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We want to use `OpenAIEmbeddings` so we have to get the OpenAI API Key."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "OpenAI API Key: ········\n"
+     ]
+    }
+   ],
+   "source": [
+    "import os\n",
+    "import getpass\n",
+    "\n",
+    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+    "from langchain.text_splitter import CharacterTextSplitter\n",
+    "from langchain.vectorstores import Neo4jVector\n",
+    "from langchain.document_loaders import TextLoader\n",
+    "from langchain.docstore.document import Document"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
+    "documents = loader.load()\n",
+    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
+    "docs = text_splitter.split_documents(documents)\n",
+    "\n",
+    "embeddings = OpenAIEmbeddings()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Neo4jVector requires the Neo4j database credentials\n",
+    "\n",
+    "url = \"bolt://localhost:7687\"\n",
+    "username = \"neo4j\"\n",
+    "password = \"pleaseletmein\"\n",
+    "\n",
+    "# You can also use environment variables instead of directly passing named parameters\n",
+    "#os.environ[\"NEO4J_URL\"] = \"bolt://localhost:7687\"\n",
+    "#os.environ[\"NEO4J_USERNAME\"] = \"neo4j\"\n",
+    "#os.environ[\"NEO4J_PASSWORD\"] = \"pleaseletmein\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Similarity Search with Cosine Distance (Default)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# The Neo4jVector Module will connect to Neo4j and create a vector index if needed.\n",
+    "\n",
+    "db = Neo4jVector.from_documents(\n",
+    "    docs, OpenAIEmbeddings(), url=url, username=username, password=password\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+    "docs_with_score = db.similarity_search_with_score(query)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--------------------------------------------------------------------------------\n",
+      "Score:  0.9077161550521851\n",
+      "Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n",
+      "\n",
+      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n",
+      "\n",
+      "One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n",
+      "\n",
+      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.\n",
+      "--------------------------------------------------------------------------------\n",
+      "--------------------------------------------------------------------------------\n",
+      "Score:  0.9077161550521851\n",
+      "Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n",
+      "\n",
+      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n",
+      "\n",
+      "One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n",
+      "\n",
+      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.\n",
+      "--------------------------------------------------------------------------------\n",
+      "--------------------------------------------------------------------------------\n",
+      "Score:  0.891287088394165\n",
+      "A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. \n",
+      "\n",
+      "And if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. \n",
+      "\n",
+      "We can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  \n",
+      "\n",
+      "We’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  \n",
+      "\n",
+      "We’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster. \n",
+      "\n",
+      "We’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.\n",
+      "--------------------------------------------------------------------------------\n",
+      "--------------------------------------------------------------------------------\n",
+      "Score:  0.891287088394165\n",
+      "A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. \n",
+      "\n",
+      "And if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. \n",
+      "\n",
+      "We can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  \n",
+      "\n",
+      "We’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  \n",
+      "\n",
+      "We’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster. \n",
+      "\n",
+      "We’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.\n",
+      "--------------------------------------------------------------------------------\n"
+     ]
+    }
+   ],
+   "source": [
+    "for doc, score in docs_with_score:\n",
+    "    print(\"-\" * 80)\n",
+    "    print(\"Score: \", score)\n",
+    "    print(doc.page_content)\n",
+    "    print(\"-\" * 80)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Working with vectorstore\n",
+    "\n",
+    "Above, we created a vectorstore from scratch. However, often times we want to work with an existing vectorstore.\n",
+    "In order to do that, we can initialize it directly."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "index_name = \"vector\"  # default index name\n",
+    "\n",
+    "store = Neo4jVector.from_existing_index(\n",
+    "    OpenAIEmbeddings(),\n",
+    "    url=url,\n",
+    "    username=username,\n",
+    "    password=password,\n",
+    "    index_name=index_name,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Add documents\n",
+    "We can add documents to the existing vectorstore."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['2f70679a-4416-11ee-b7c3-d46a6aa24f5b']"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "store.add_documents([Document(page_content=\"foo\")])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "docs_with_score = store.similarity_search_with_score(\"foo\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(Document(page_content='foo', metadata={}), 1.0)"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "docs_with_score[0]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Retriever options\n",
+    "\n",
+    "This section shows how to use `Neo4jVector` as a retriever."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \\n\\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \\n\\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': '../../modules/state_of_the_union.txt'})"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "retriever = store.as_retriever()\n",
+    "retriever.get_relevant_documents(query)[0]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Question Answering with Sources\n",
+    "\n",
+    "This section goes over how to do question-answering with sources over an Index. It does this by using the `RetrievalQAWithSourcesChain`, which does the lookup of the documents from an Index. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains import RetrievalQAWithSourcesChain\n",
+    "from langchain.chat_models import ChatOpenAI"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = RetrievalQAWithSourcesChain.from_chain_type(\n",
+    "    ChatOpenAI(temperature=0), chain_type=\"stuff\", retriever=retriever\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'answer': \"The president honored Justice Stephen Breyer, who is retiring from the United States Supreme Court, and thanked him for his service. The president also mentioned that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to continue Justice Breyer's legacy of excellence. \\n\",\n",
+       " 'sources': '../../modules/state_of_the_union.txt'}"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain(\n",
+    "    {\"question\": \"What did the president say about Justice Breyer\"},\n",
+    "    return_only_outputs=True,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/extras/integrations/vectorstores/vectara.ipynb

@@ -1,7 +1,6 @@
 {
  "cells": [
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "683953b3",
    "metadata": {},
@@ -60,7 +59,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "eeead681",
    "metadata": {},
@@ -73,7 +71,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "id": "04a1f1a0",
    "metadata": {},
    "outputs": [],
@@ -86,12 +84,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 2,
    "id": "be0a4973",
    "metadata": {},
    "outputs": [],
    "source": [
-    "loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
+    "loader = TextLoader(\"../../modules/state_of_the_union.txt\")\n",
     "documents = loader.load()\n",
     "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
     "docs = text_splitter.split_documents(documents)"
@@ -99,7 +97,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 3,
    "id": "8429667e",
    "metadata": {
     "ExecuteTime": {
@@ -118,7 +116,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "90dbf3e7",
    "metadata": {},
@@ -133,7 +130,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 4,
    "id": "85ef3468",
    "metadata": {},
    "outputs": [],
@@ -165,7 +162,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "1f9215c8",
    "metadata": {
@@ -182,7 +178,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 5,
    "id": "a8c513ab",
    "metadata": {
     "ExecuteTime": {
@@ -201,7 +197,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 6,
    "id": "fc516993",
    "metadata": {
     "ExecuteTime": {
@@ -215,13 +211,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n",
-      "\n",
-      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n",
-      "\n",
-      "One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n",
-      "\n",
-      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.\n"
+      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson.\n"
      ]
     }
    ],
@@ -230,7 +220,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "1bda9bf5",
    "metadata": {},
@@ -242,7 +231,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 7,
    "id": "8804a21d",
    "metadata": {
     "ExecuteTime": {
@@ -254,13 +243,13 @@
    "source": [
     "query = \"What did the president say about Ketanji Brown Jackson\"\n",
     "found_docs = vectara.similarity_search_with_score(\n",
-    "    query, filter=\"doc.speech = 'state-of-the-union'\"\n",
+    "    query, filter=\"doc.speech = 'state-of-the-union'\", score_threshold=0.2,\n",
     ")"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 8,
    "id": "756a6887",
    "metadata": {
     "ExecuteTime": {
@@ -273,15 +262,9 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n",
+      "Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice.\n",
       "\n",
-      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n",
-      "\n",
-      "One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n",
-      "\n",
-      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.\n",
-      "\n",
-      "Score: 0.4917977\n"
+      "Score: 0.786569\n"
      ]
     }
    ],
@@ -292,7 +275,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "1f9876a8",
    "metadata": {},
@@ -302,7 +284,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 9,
    "id": "47784de5",
    "metadata": {},
    "outputs": [
@@ -310,22 +292,43 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "(Document(page_content='We must forever conduct our struggle on the high plane of dignity and discipline.', metadata={'section': '1'}), 0.7962591)\n",
-      "(Document(page_content='We must not allow our\\ncreative protests to degenerate into physical violence. . . .', metadata={'section': '1'}), 0.25983918)\n"
+      "With this threshold of 1.2 we have 0 documents\n"
      ]
     }
    ],
    "source": [
     "query = \"We must forever conduct our struggle\"\n",
+    "min_score = 1.2\n",
     "found_docs = vectara.similarity_search_with_score(\n",
-    "    query, filter=\"doc.speech = 'I-have-a-dream'\"\n",
+    "    query, filter=\"doc.speech = 'I-have-a-dream'\", score_threshold=min_score,\n",
     ")\n",
-    "print(found_docs[0])\n",
-    "print(found_docs[1])"
+    "print(f\"With this threshold of {min_score} we have {len(found_docs)} documents\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "3e22949f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "With this threshold of 0.2 we have 3 documents\n"
+     ]
+    }
+   ],
+   "source": [
+    "query = \"We must forever conduct our struggle\"\n",
+    "min_score = 0.2\n",
+    "found_docs = vectara.similarity_search_with_score(\n",
+    "    query, filter=\"doc.speech = 'I-have-a-dream'\", score_threshold=min_score,\n",
+    ")\n",
+    "print(f\"With this threshold of {min_score} we have {len(found_docs)} documents\")\n"
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "691a82d6",
    "metadata": {},
@@ -349,7 +352,7 @@
     {
      "data": {
       "text/plain": [
-       "VectaraRetriever(vectorstore=<langchain.vectorstores.vectara.Vectara object at 0x12772caf0>, search_type='similarity', search_kwargs={'lambda_val': 0.025, 'k': 5, 'filter': '', 'n_sentence_context': '0'})"
+       "VectaraRetriever(tags=['Vectara'], metadata=None, vectorstore=<langchain.vectorstores.vectara.Vectara object at 0x1586bd330>, search_type='similarity', search_kwargs={'lambda_val': 0.025, 'k': 5, 'filter': '', 'n_sentence_context': '2'})"
       ]
      },
      "execution_count": 11,
@@ -376,7 +379,7 @@
     {
      "data": {
       "text/plain": [
-       "Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \\n\\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \\n\\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': '../../../state_of_the_union.txt'})"
+       "Document(page_content='Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice.', metadata={'source': 'langchain', 'lang': 'eng', 'offset': '596', 'len': '97', 'speech': 'state-of-the-union'})"
       ]
      },
      "execution_count": 12,
@@ -414,7 +417,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.9"
   }
  },
  "nbformat": 4,

docs/extras/integrations/vectorstores/xata.ipynb

@@ -52,7 +52,7 @@
    },
    "outputs": [],
    "source": [
-    "!pip install xata==1.0.0a7 openai tiktoken langchain"
+    "!pip install xata openai tiktoken langchain"
    ]
   },
   {

docs/extras/modules/data_connection/retrievers/multi_vector.ipynb

@@ -13,7 +13,10 @@
     "\n",
     "- smaller chunks: split a document into smaller chunks, and embed those (this is ParentDocumentRetriever)\n",
     "- summary: create a summary for each document, embed that along with (or instead of) the document\n",
-    "- hypothetical questions: create hypothetical questions that each document would be appropriate to answer, embed those along with (or instead of) the document"
+    "- hypothetical questions: create hypothetical questions that each document would be appropriate to answer, embed those along with (or instead of) the document\n",
+    "\n",
+    "\n",
+    "Note that this also enables another method of adding embeddings - manually. This is great because you can explicitly add questions or queries that should lead to a document being recovered, giving you more control"
    ]
   },
   {
@@ -106,7 +109,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 6,
    "id": "5d23247d",
    "metadata": {},
    "outputs": [],
@@ -122,7 +125,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "id": "92ed5861",
    "metadata": {},
    "outputs": [],
@@ -133,17 +136,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 8,
    "id": "8afed60c",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "Document(page_content='Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.', metadata={'doc_id': 'b4ca7817-e3fe-4103-ac81-574fb41439ef', 'source': '../../state_of_the_union.txt'})"
+       "Document(page_content='Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court.', metadata={'doc_id': '10e9cbc0-4ba5-4d79-a09b-c033d1ba7b01', 'source': '../../state_of_the_union.txt'})"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -155,7 +158,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 9,
    "id": "3c9017f1",
    "metadata": {},
    "outputs": [
@@ -165,7 +168,7 @@
        "9874"
       ]
      },
-     "execution_count": 10,
+     "execution_count": 9,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -187,7 +190,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 10,
    "id": "1433dff4",
    "metadata": {},
    "outputs": [],
@@ -201,7 +204,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 11,
    "id": "35b30390",
    "metadata": {},
    "outputs": [],
@@ -216,17 +219,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 12,
    "id": "41a2a738",
    "metadata": {},
    "outputs": [],
    "source": [
-    "summaries = [chain.invoke(d) for d in docs]"
+    "summaries = chain.batch(docs, {\"max_concurrency\": 5})"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 13,
    "id": "7ac5e4b1",
    "metadata": {},
    "outputs": [],
@@ -250,7 +253,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 15,
    "id": "0d93309f",
    "metadata": {},
    "outputs": [],
@@ -260,7 +263,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 16,
    "id": "6d5edf0d",
    "metadata": {},
    "outputs": [],
@@ -271,7 +274,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 17,
+   "id": "862ae920",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# # We can also add the original chunks to the vectorstore if we so want\n",
+    "# for i, doc in enumerate(docs):\n",
+    "#     doc.metadata[id_key] = doc_ids[i]\n",
+    "# retriever.vectorstore.add_documents(docs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
    "id": "299232d6",
    "metadata": {},
    "outputs": [],
@@ -281,17 +297,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 19,
    "id": "10e404c0",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "Document(page_content='The document discusses various topics and proposals put forth by the President in a State of the Union address. These include the nomination of a judge for the Supreme Court, securing the border and fixing the immigration system, advancing liberty and justice for women and LGBTQ+ individuals, passing bipartisan legislation, addressing the opioid epidemic and mental health issues, supporting veterans, and ending cancer. The President expresses optimism about the future of the country and emphasizes the strength of the American people.', metadata={'doc_id': '8c7a707d-615d-42d5-919d-bc5178dd1ae4'})"
+       "Document(page_content=\"The document is a transcript of a speech given by the President of the United States. The President discusses several important issues and initiatives, including the nomination of a Supreme Court Justice, border security and immigration reform, protecting women's rights, advancing LGBTQ+ equality, bipartisan legislation, addressing the opioid epidemic and mental health, supporting veterans, investigating the health effects of burn pits on military personnel, ending cancer, and the strength and resilience of the American people.\", metadata={'doc_id': '79fa2e9f-28d9-4372-8af3-2caf4f1de312'})"
       ]
      },
-     "execution_count": 20,
+     "execution_count": 19,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -302,7 +318,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 20,
    "id": "e4cce5c2",
    "metadata": {},
    "outputs": [],
@@ -312,7 +328,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 24,
+   "execution_count": 21,
    "id": "c8570dbb",
    "metadata": {},
    "outputs": [
@@ -322,7 +338,7 @@
        "9194"
       ]
      },
-     "execution_count": 24,
+     "execution_count": 21,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -340,6 +356,203 @@
     "\n",
     "An LLM can also be used to generate a list of hypothetical questions that could be asked of a particular document. These questions can then be embedded"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "5219b085",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "functions = [\n",
+    "    {\n",
+    "      \"name\": \"hypothetical_questions\",\n",
+    "      \"description\": \"Generate hypothetical questions\",\n",
+    "      \"parameters\": {\n",
+    "        \"type\": \"object\",\n",
+    "        \"properties\": {\n",
+    "          \"questions\": {\n",
+    "            \"type\": \"array\",\n",
+    "            \"items\": {\n",
+    "                \"type\": \"string\"\n",
+    "              },\n",
+    "          },\n",
+    "        },\n",
+    "        \"required\": [\"questions\"]\n",
+    "      }\n",
+    "    }\n",
+    "  ]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "523deb92",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.output_parsers.openai_functions import JsonKeyOutputFunctionsParser\n",
+    "chain = (\n",
+    "    {\"doc\": lambda x: x.page_content}\n",
+    "    # Only asking for 3 hypothetical questions, but this could be adjusted\n",
+    "    | ChatPromptTemplate.from_template(\"Generate a list of 3 hypothetical questions that the below document could be used to answer:\\n\\n{doc}\")\n",
+    "    | ChatOpenAI(max_retries=0, model=\"gpt-4\").bind(functions=functions, function_call={\"name\": \"hypothetical_questions\"})\n",
+    "    | JsonKeyOutputFunctionsParser(key_name=\"questions\")\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "11d30554",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[\"What was the author's initial impression of philosophy as a field of study, and how did it change when they got to college?\",\n",
+       " 'Why did the author decide to switch their focus to Artificial Intelligence (AI)?',\n",
+       " \"What led to the author's disillusionment with the field of AI as it was practiced at the time?\"]"
+      ]
+     },
+     "execution_count": 33,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke(docs[0])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "id": "3eb2e48c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "hypothetical_questions = chain.batch(docs, {\"max_concurrency\": 5})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 67,
+   "id": "b2cd6e75",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# The vectorstore to use to index the child chunks\n",
+    "vectorstore = Chroma(\n",
+    "    collection_name=\"hypo-questions\",\n",
+    "    embedding_function=OpenAIEmbeddings()\n",
+    ")\n",
+    "# The storage layer for the parent documents\n",
+    "store = InMemoryStore()\n",
+    "id_key = \"doc_id\"\n",
+    "# The retriever (empty to start)\n",
+    "retriever = MultiVectorRetriever(\n",
+    "    vectorstore=vectorstore, \n",
+    "    docstore=store, \n",
+    "    id_key=id_key,\n",
+    ")\n",
+    "doc_ids = [str(uuid.uuid4()) for _ in docs]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 68,
+   "id": "18831b3b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "question_docs = []\n",
+    "for i, question_list in enumerate(hypothetical_questions):\n",
+    "    question_docs.extend([Document(page_content=s,metadata={id_key: doc_ids[i]}) for s in question_list])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 69,
+   "id": "224b24c5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "retriever.vectorstore.add_documents(question_docs)\n",
+    "retriever.docstore.mset(list(zip(doc_ids, docs)))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 70,
+   "id": "7b442b90",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sub_docs = vectorstore.similarity_search(\"justice breyer\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 71,
+   "id": "089b5ad0",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content=\"What is the President's stance on immigration reform?\", metadata={'doc_id': '505d73e3-8350-46ec-a58e-3af032f04ab3'}),\n",
+       " Document(page_content=\"What is the President's stance on immigration reform?\", metadata={'doc_id': '1c9618f0-7660-4b4f-a37c-509cbbbf6dba'}),\n",
+       " Document(page_content=\"What is the President's stance on immigration reform?\", metadata={'doc_id': '82c08209-b904-46a8-9532-edd2380950b7'}),\n",
+       " Document(page_content='What measures is the President proposing to protect the rights of LGBTQ+ Americans?', metadata={'doc_id': '82c08209-b904-46a8-9532-edd2380950b7'})]"
+      ]
+     },
+     "execution_count": 71,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "sub_docs"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 72,
+   "id": "7594b24e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "retrieved_docs = retriever.get_relevant_documents(\"justice breyer\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 73,
+   "id": "4c120c65",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "9194"
+      ]
+     },
+     "execution_count": 73,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(retrieved_docs[0].page_content)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "616cfeeb",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {

docs/extras/modules/memory/adding_memory.ipynb

@@ -7,9 +7,11 @@
     "tags": []
    },
    "source": [
-    "# How to add Memory to an LLMChain\n",
+    "# Memory in LLMChain\n",
     "\n",
-    "This notebook goes over how to use the Memory class with an LLMChain. For the purposes of this walkthrough, we will add  the [ConversationBufferMemory](https://api.python.langchain.com/en/latest/memory/langchain.memory.buffer.ConversationBufferMemory.html#langchain.memory.buffer.ConversationBufferMemory) class, although this can be any memory class."
+    "This notebook goes over how to use the Memory class with an LLMChain. \n",
+    "\n",
+    "We will add  the [ConversationBufferMemory](https://api.python.langchain.com/en/latest/memory/langchain.memory.buffer.ConversationBufferMemory.html#langchain.memory.buffer.ConversationBufferMemory) class, although this can be any memory class."
    ]
   },
   {
@@ -203,7 +205,7 @@
     "prompt = ChatPromptTemplate.from_messages([\n",
     "    SystemMessage(content=\"You are a chatbot having a conversation with a human.\"), # The persistent system prompt\n",
     "    MessagesPlaceholder(variable_name=\"chat_history\"), # Where the memory will be stored.\n",
-    "    HumanMessagePromptTemplate.from_template(\"{human_input}\"), # Where the human input will injectd\n",
+    "    HumanMessagePromptTemplate.from_template(\"{human_input}\"), # Where the human input will injected\n",
     "])\n",
     "    \n",
     "memory = ConversationBufferMemory(memory_key=\"chat_history\", return_messages=True)"
@@ -321,7 +323,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.2"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/adding_memory_chain_multiple_inputs.ipynb

@@ -5,9 +5,9 @@
    "id": "e42733c5",
    "metadata": {},
    "source": [
-    "# How to add memory to a Multi-Input Chain\n",
+    "# Memory in the Multi-Input Chain\n",
     "\n",
-    "Most memory objects assume a single input. In this notebook, we go over how to add memory to a chain that has multiple inputs. As an example of such a chain, we will add memory to a question/answering chain. This chain takes as inputs both related documents and a user question."
+    "Most memory objects assume a single input. In this notebook, we go over how to add memory to a chain that has multiple inputs. We will add memory to a question/answering chain. This chain takes as inputs both related documents and a user question."
    ]
   },
   {
@@ -178,7 +178,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/agent_with_memory.ipynb

@@ -5,11 +5,11 @@
    "id": "fa6802ac",
    "metadata": {},
    "source": [
-    "# How to add Memory to an Agent\n",
+    "# Memory in Agent\n",
     "\n",
     "This notebook goes over adding memory to an Agent. Before going through this notebook, please walkthrough the following notebooks, as this will build on top of both of them:\n",
     "\n",
-    "- [Adding memory to an LLM Chain](/docs/modules/memory/how_to/adding_memory.html)\n",
+    "- [Memory in LLMChain](/docs/modules/memory/how_to/adding_memory.html)\n",
     "- [Custom Agents](/docs/modules/agents/how_to/custom_agent.html)\n",
     "\n",
     "In order to add a memory to an agent we are going to the the following steps:\n",
@@ -317,7 +317,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/agent_with_memory_in_db.ipynb

@@ -5,13 +5,13 @@
    "id": "fa6802ac",
    "metadata": {},
    "source": [
-    "# Adding Message Memory backed by a database to an Agent\n",
+    "# Message Memory in Agent backed by a database\n",
     "\n",
     "This notebook goes over adding memory to an Agent where the memory uses an external message store. Before going through this notebook, please walkthrough the following notebooks, as this will build on top of both of them:\n",
     "\n",
-    "- [Adding memory to an LLM Chain](/docs/modules/memory/how_to/adding_memory.html)\n",
+    "- [Memory in LLMChain](/docs/modules/memory/how_to/adding_memory.html)\n",
     "- [Custom Agents](/docs/modules/agents/how_to/custom_agent.html)\n",
-    "- [Agent with Memory](/docs/modules/memory/how_to/agent_with_memory.html)\n",
+    "- [Memory in Agent](/docs/modules/memory/how_to/agent_with_memory.html)\n",
     "\n",
     "In order to add a memory with an external message store to an agent we are going to do the following steps:\n",
     "\n",
@@ -348,7 +348,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/conversational_customization.ipynb

@@ -5,7 +5,7 @@
    "id": "69e35d6f",
    "metadata": {},
    "source": [
-    "# How to customize conversational memory\n",
+    "# Customizing Conversational Memory\n",
     "\n",
     "This notebook walks through a few ways to customize conversational memory."
    ]
@@ -373,7 +373,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/custom_memory.ipynb

@@ -5,7 +5,8 @@
    "id": "94e33ebe",
    "metadata": {},
    "source": [
-    "# How to create a custom Memory class\n",
+    "# Custom Memory\n",
+    "\n",
     "Although there are a few predefined types of memory in LangChain, it is highly possible you will want to add your own type of memory that is optimal for your application. This notebook covers how to do that."
    ]
   },
@@ -295,7 +296,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/multiple_memory.ipynb

@@ -5,8 +5,9 @@
    "id": "d9fec22e",
    "metadata": {},
    "source": [
-    "# How to use multiple memory classes in the same chain\n",
-    "It is also possible to use multiple memory classes in the same chain. To combine multiple memory classes, we can initialize the `CombinedMemory` class, and then use that."
+    "# Multiple Memory classes\n",
+    "\n",
+    "We can use multiple memory classes in the same chain. To combine multiple memory classes, we initialize and use the `CombinedMemory` class."
    ]
   },
   {
@@ -158,7 +159,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/types/kg.ipynb

@@ -5,11 +5,17 @@
    "id": "44c9933a",
    "metadata": {},
    "source": [
-    "# Conversation Knowledge Graph Memory\n",
+    "# Conversation Knowledge Graph\n",
     "\n",
-    "This type of memory uses a knowledge graph to recreate memory.\n",
-    "\n",
-    "Let's first walk through how to use the utilities"
+    "This type of memory uses a knowledge graph to recreate memory.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0c798006-ca04-4de3-83eb-cf167fb2bd01",
+   "metadata": {},
+   "source": [
+    "## Using memory with LLM"
    ]
   },
   {
@@ -162,6 +168,7 @@
    "metadata": {},
    "source": [
     "## Using in a chain\n",
+    "\n",
     "Let's now use this in a chain!"
    ]
   },
@@ -348,7 +355,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/types/summary_buffer.ipynb

@@ -5,13 +5,22 @@
    "id": "ff4be5f3",
    "metadata": {},
    "source": [
-    "# ConversationSummaryBufferMemory\n",
+    "# Conversation Summary Buffer\n",
     "\n",
-    "`ConversationSummaryBufferMemory` combines the last two ideas. It keeps a buffer of recent interactions in memory, but rather than just completely flushing old interactions it compiles them into a summary and uses both. Unlike the previous implementation though, it uses token length rather than number of interactions to determine when to flush interactions.\n",
+    "`ConversationSummaryBufferMemory` combines the two ideas. It keeps a buffer of recent interactions in memory, but rather than just completely flushing old interactions it compiles them into a summary and uses both. \n",
+    "It uses token length rather than number of interactions to determine when to flush interactions.\n",
     "\n",
     "Let's first walk through how to use the utilities"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "0309636e-a530-4d2a-ba07-0916ea18bb20",
+   "metadata": {},
+   "source": [
+    "## Using memory with LLM"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -320,7 +329,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/modules/memory/types/token_buffer.ipynb

@@ -5,13 +5,21 @@
    "id": "ff4be5f3",
    "metadata": {},
    "source": [
-    "# ConversationTokenBufferMemory\n",
+    "# Conversation Token Buffer\n",
     "\n",
     "`ConversationTokenBufferMemory` keeps a buffer of recent interactions in memory, and uses token length rather than number of interactions to determine when to flush interactions.\n",
     "\n",
     "Let's first walk through how to use the utilities"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "0e528ef0-7b04-4a4a-8ff2-493c02027e83",
+   "metadata": {},
+   "source": [
+    "## Using memory with LLM"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -286,7 +294,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/use_cases/agents/baby_agi.ipynb

@@ -1,565 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "517a9fd4",
-   "metadata": {},
-   "source": [
-    "# BabyAGI User Guide\n",
-    "\n",
-    "This notebook demonstrates how to implement [BabyAGI](https://github.com/yoheinakajima/babyagi/tree/main) by [Yohei Nakajima](https://twitter.com/yoheinakajima). BabyAGI is an AI agent that can generate and pretend to execute tasks based on a given objective.\n",
-    "\n",
-    "This guide will help you understand the components to create your own recursive agents.\n",
-    "\n",
-    "Although BabyAGI uses specific vectorstores/model providers (Pinecone, OpenAI), one of the benefits of implementing it with LangChain is that you can easily swap those out for different options. In this implementation we use a FAISS vectorstore (because it runs locally and is free)."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "556af556",
-   "metadata": {},
-   "source": [
-    "## Install and Import Required Modules"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 116,
-   "id": "c8a354b6",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "from collections import deque\n",
-    "from typing import Dict, List, Optional, Any\n",
-    "\n",
-    "from langchain import LLMChain, OpenAI, PromptTemplate\n",
-    "from langchain.embeddings import OpenAIEmbeddings\n",
-    "from langchain.llms import BaseLLM\n",
-    "from langchain.vectorstores.base import VectorStore\n",
-    "from pydantic import BaseModel, Field\n",
-    "from langchain.chains.base import Chain"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "09f70772",
-   "metadata": {},
-   "source": [
-    "## Connect to the Vector Store\n",
-    "\n",
-    "Depending on what vectorstore you use, this step may look different."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 71,
-   "id": "794045d4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.vectorstores import FAISS\n",
-    "from langchain.docstore import InMemoryDocstore"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 72,
-   "id": "6e0305eb",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Define your embedding model\n",
-    "embeddings_model = OpenAIEmbeddings()\n",
-    "# Initialize the vectorstore as empty\n",
-    "import faiss\n",
-    "\n",
-    "embedding_size = 1536\n",
-    "index = faiss.IndexFlatL2(embedding_size)\n",
-    "vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {})"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0f3b72bf",
-   "metadata": {},
-   "source": [
-    "## Define the Chains\n",
-    "\n",
-    "BabyAGI relies on three LLM chains:\n",
-    "- Task creation chain to select new tasks to add to the list\n",
-    "- Task prioritization chain to re-prioritize tasks\n",
-    "- Execution Chain to execute the tasks"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 73,
-   "id": "bf4bd5cd",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class TaskCreationChain(LLMChain):\n",
-    "    \"\"\"Chain to generates tasks.\"\"\"\n",
-    "\n",
-    "    @classmethod\n",
-    "    def from_llm(cls, llm: BaseLLM, verbose: bool = True) -> LLMChain:\n",
-    "        \"\"\"Get the response parser.\"\"\"\n",
-    "        task_creation_template = (\n",
-    "            \"You are a task creation AI that uses the result of an execution agent\"\n",
-    "            \" to create new tasks with the following objective: {objective},\"\n",
-    "            \" The last completed task has the result: {result}.\"\n",
-    "            \" This result was based on this task description: {task_description}.\"\n",
-    "            \" These are incomplete tasks: {incomplete_tasks}.\"\n",
-    "            \" Based on the result, create new tasks to be completed\"\n",
-    "            \" by the AI system that do not overlap with incomplete tasks.\"\n",
-    "            \" Return the tasks as an array.\"\n",
-    "        )\n",
-    "        prompt = PromptTemplate(\n",
-    "            template=task_creation_template,\n",
-    "            input_variables=[\n",
-    "                \"result\",\n",
-    "                \"task_description\",\n",
-    "                \"incomplete_tasks\",\n",
-    "                \"objective\",\n",
-    "            ],\n",
-    "        )\n",
-    "        return cls(prompt=prompt, llm=llm, verbose=verbose)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 74,
-   "id": "b6488ffe",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class TaskPrioritizationChain(LLMChain):\n",
-    "    \"\"\"Chain to prioritize tasks.\"\"\"\n",
-    "\n",
-    "    @classmethod\n",
-    "    def from_llm(cls, llm: BaseLLM, verbose: bool = True) -> LLMChain:\n",
-    "        \"\"\"Get the response parser.\"\"\"\n",
-    "        task_prioritization_template = (\n",
-    "            \"You are a task prioritization AI tasked with cleaning the formatting of and reprioritizing\"\n",
-    "            \" the following tasks: {task_names}.\"\n",
-    "            \" Consider the ultimate objective of your team: {objective}.\"\n",
-    "            \" Do not remove any tasks. Return the result as a numbered list, like:\"\n",
-    "            \" #. First task\"\n",
-    "            \" #. Second task\"\n",
-    "            \" Start the task list with number {next_task_id}.\"\n",
-    "        )\n",
-    "        prompt = PromptTemplate(\n",
-    "            template=task_prioritization_template,\n",
-    "            input_variables=[\"task_names\", \"next_task_id\", \"objective\"],\n",
-    "        )\n",
-    "        return cls(prompt=prompt, llm=llm, verbose=verbose)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 84,
-   "id": "b43cd580",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class ExecutionChain(LLMChain):\n",
-    "    \"\"\"Chain to execute tasks.\"\"\"\n",
-    "\n",
-    "    @classmethod\n",
-    "    def from_llm(cls, llm: BaseLLM, verbose: bool = True) -> LLMChain:\n",
-    "        \"\"\"Get the response parser.\"\"\"\n",
-    "        execution_template = (\n",
-    "            \"You are an AI who performs one task based on the following objective: {objective}.\"\n",
-    "            \" Take into account these previously completed tasks: {context}.\"\n",
-    "            \" Your task: {task}.\"\n",
-    "            \" Response:\"\n",
-    "        )\n",
-    "        prompt = PromptTemplate(\n",
-    "            template=execution_template,\n",
-    "            input_variables=[\"objective\", \"context\", \"task\"],\n",
-    "        )\n",
-    "        return cls(prompt=prompt, llm=llm, verbose=verbose)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3ad996c5",
-   "metadata": {},
-   "source": [
-    "### Define the BabyAGI Controller\n",
-    "\n",
-    "BabyAGI composes the chains defined above in a (potentially-)infinite loop."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 85,
-   "id": "0ada0636",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def get_next_task(\n",
-    "    task_creation_chain: LLMChain,\n",
-    "    result: Dict,\n",
-    "    task_description: str,\n",
-    "    task_list: List[str],\n",
-    "    objective: str,\n",
-    ") -> List[Dict]:\n",
-    "    \"\"\"Get the next task.\"\"\"\n",
-    "    incomplete_tasks = \", \".join(task_list)\n",
-    "    response = task_creation_chain.run(\n",
-    "        result=result,\n",
-    "        task_description=task_description,\n",
-    "        incomplete_tasks=incomplete_tasks,\n",
-    "        objective=objective,\n",
-    "    )\n",
-    "    new_tasks = response.split(\"\\n\")\n",
-    "    return [{\"task_name\": task_name} for task_name in new_tasks if task_name.strip()]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 86,
-   "id": "d35250ad",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def prioritize_tasks(\n",
-    "    task_prioritization_chain: LLMChain,\n",
-    "    this_task_id: int,\n",
-    "    task_list: List[Dict],\n",
-    "    objective: str,\n",
-    ") -> List[Dict]:\n",
-    "    \"\"\"Prioritize tasks.\"\"\"\n",
-    "    task_names = [t[\"task_name\"] for t in task_list]\n",
-    "    next_task_id = int(this_task_id) + 1\n",
-    "    response = task_prioritization_chain.run(\n",
-    "        task_names=task_names, next_task_id=next_task_id, objective=objective\n",
-    "    )\n",
-    "    new_tasks = response.split(\"\\n\")\n",
-    "    prioritized_task_list = []\n",
-    "    for task_string in new_tasks:\n",
-    "        if not task_string.strip():\n",
-    "            continue\n",
-    "        task_parts = task_string.strip().split(\".\", 1)\n",
-    "        if len(task_parts) == 2:\n",
-    "            task_id = task_parts[0].strip()\n",
-    "            task_name = task_parts[1].strip()\n",
-    "            prioritized_task_list.append({\"task_id\": task_id, \"task_name\": task_name})\n",
-    "    return prioritized_task_list"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 87,
-   "id": "e3f1840c",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def _get_top_tasks(vectorstore, query: str, k: int) -> List[str]:\n",
-    "    \"\"\"Get the top k tasks based on the query.\"\"\"\n",
-    "    results = vectorstore.similarity_search_with_score(query, k=k)\n",
-    "    if not results:\n",
-    "        return []\n",
-    "    sorted_results, _ = zip(*sorted(results, key=lambda x: x[1], reverse=True))\n",
-    "    return [str(item.metadata[\"task\"]) for item in sorted_results]\n",
-    "\n",
-    "\n",
-    "def execute_task(\n",
-    "    vectorstore, execution_chain: LLMChain, objective: str, task: str, k: int = 5\n",
-    ") -> str:\n",
-    "    \"\"\"Execute a task.\"\"\"\n",
-    "    context = _get_top_tasks(vectorstore, query=objective, k=k)\n",
-    "    return execution_chain.run(objective=objective, context=context, task=task)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 137,
-   "id": "1e978938",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class BabyAGI(Chain, BaseModel):\n",
-    "    \"\"\"Controller model for the BabyAGI agent.\"\"\"\n",
-    "\n",
-    "    task_list: deque = Field(default_factory=deque)\n",
-    "    task_creation_chain: TaskCreationChain = Field(...)\n",
-    "    task_prioritization_chain: TaskPrioritizationChain = Field(...)\n",
-    "    execution_chain: ExecutionChain = Field(...)\n",
-    "    task_id_counter: int = Field(1)\n",
-    "    vectorstore: VectorStore = Field(init=False)\n",
-    "    max_iterations: Optional[int] = None\n",
-    "\n",
-    "    class Config:\n",
-    "        \"\"\"Configuration for this pydantic object.\"\"\"\n",
-    "\n",
-    "        arbitrary_types_allowed = True\n",
-    "\n",
-    "    def add_task(self, task: Dict):\n",
-    "        self.task_list.append(task)\n",
-    "\n",
-    "    def print_task_list(self):\n",
-    "        print(\"\\033[95m\\033[1m\" + \"\\n*****TASK LIST*****\\n\" + \"\\033[0m\\033[0m\")\n",
-    "        for t in self.task_list:\n",
-    "            print(str(t[\"task_id\"]) + \": \" + t[\"task_name\"])\n",
-    "\n",
-    "    def print_next_task(self, task: Dict):\n",
-    "        print(\"\\033[92m\\033[1m\" + \"\\n*****NEXT TASK*****\\n\" + \"\\033[0m\\033[0m\")\n",
-    "        print(str(task[\"task_id\"]) + \": \" + task[\"task_name\"])\n",
-    "\n",
-    "    def print_task_result(self, result: str):\n",
-    "        print(\"\\033[93m\\033[1m\" + \"\\n*****TASK RESULT*****\\n\" + \"\\033[0m\\033[0m\")\n",
-    "        print(result)\n",
-    "\n",
-    "    @property\n",
-    "    def input_keys(self) -> List[str]:\n",
-    "        return [\"objective\"]\n",
-    "\n",
-    "    @property\n",
-    "    def output_keys(self) -> List[str]:\n",
-    "        return []\n",
-    "\n",
-    "    def _call(self, inputs: Dict[str, Any]) -> Dict[str, Any]:\n",
-    "        \"\"\"Run the agent.\"\"\"\n",
-    "        objective = inputs[\"objective\"]\n",
-    "        first_task = inputs.get(\"first_task\", \"Make a todo list\")\n",
-    "        self.add_task({\"task_id\": 1, \"task_name\": first_task})\n",
-    "        num_iters = 0\n",
-    "        while True:\n",
-    "            if self.task_list:\n",
-    "                self.print_task_list()\n",
-    "\n",
-    "                # Step 1: Pull the first task\n",
-    "                task = self.task_list.popleft()\n",
-    "                self.print_next_task(task)\n",
-    "\n",
-    "                # Step 2: Execute the task\n",
-    "                result = execute_task(\n",
-    "                    self.vectorstore, self.execution_chain, objective, task[\"task_name\"]\n",
-    "                )\n",
-    "                this_task_id = int(task[\"task_id\"])\n",
-    "                self.print_task_result(result)\n",
-    "\n",
-    "                # Step 3: Store the result in Pinecone\n",
-    "                result_id = f\"result_{task['task_id']}_{num_iters}\"\n",
-    "                self.vectorstore.add_texts(\n",
-    "                    texts=[result],\n",
-    "                    metadatas=[{\"task\": task[\"task_name\"]}],\n",
-    "                    ids=[result_id],\n",
-    "                )\n",
-    "\n",
-    "                # Step 4: Create new tasks and reprioritize task list\n",
-    "                new_tasks = get_next_task(\n",
-    "                    self.task_creation_chain,\n",
-    "                    result,\n",
-    "                    task[\"task_name\"],\n",
-    "                    [t[\"task_name\"] for t in self.task_list],\n",
-    "                    objective,\n",
-    "                )\n",
-    "                for new_task in new_tasks:\n",
-    "                    self.task_id_counter += 1\n",
-    "                    new_task.update({\"task_id\": self.task_id_counter})\n",
-    "                    self.add_task(new_task)\n",
-    "                self.task_list = deque(\n",
-    "                    prioritize_tasks(\n",
-    "                        self.task_prioritization_chain,\n",
-    "                        this_task_id,\n",
-    "                        list(self.task_list),\n",
-    "                        objective,\n",
-    "                    )\n",
-    "                )\n",
-    "            num_iters += 1\n",
-    "            if self.max_iterations is not None and num_iters == self.max_iterations:\n",
-    "                print(\n",
-    "                    \"\\033[91m\\033[1m\" + \"\\n*****TASK ENDING*****\\n\" + \"\\033[0m\\033[0m\"\n",
-    "                )\n",
-    "                break\n",
-    "        return {}\n",
-    "\n",
-    "    @classmethod\n",
-    "    def from_llm(\n",
-    "        cls, llm: BaseLLM, vectorstore: VectorStore, verbose: bool = False, **kwargs\n",
-    "    ) -> \"BabyAGI\":\n",
-    "        \"\"\"Initialize the BabyAGI Controller.\"\"\"\n",
-    "        task_creation_chain = TaskCreationChain.from_llm(llm, verbose=verbose)\n",
-    "        task_prioritization_chain = TaskPrioritizationChain.from_llm(\n",
-    "            llm, verbose=verbose\n",
-    "        )\n",
-    "        execution_chain = ExecutionChain.from_llm(llm, verbose=verbose)\n",
-    "        return cls(\n",
-    "            task_creation_chain=task_creation_chain,\n",
-    "            task_prioritization_chain=task_prioritization_chain,\n",
-    "            execution_chain=execution_chain,\n",
-    "            vectorstore=vectorstore,\n",
-    "            **kwargs,\n",
-    "        )"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "05ba762e",
-   "metadata": {},
-   "source": [
-    "### Run the BabyAGI\n",
-    "\n",
-    "Now it's time to create the BabyAGI controller and watch it try to accomplish your objective."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 138,
-   "id": "3d220b69",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "OBJECTIVE = \"Write a weather report for SF today\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 139,
-   "id": "8a8e5543",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm = OpenAI(temperature=0)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 140,
-   "id": "3d69899b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Logging of LLMChains\n",
-    "verbose = False\n",
-    "# If None, will keep on going forever\n",
-    "max_iterations: Optional[int] = 3\n",
-    "baby_agi = BabyAGI.from_llm(\n",
-    "    llm=llm, vectorstore=vectorstore, verbose=verbose, max_iterations=max_iterations\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 141,
-   "id": "f7957b51",
-   "metadata": {
-    "scrolled": false
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[95m\u001b[1m\n",
-      "*****TASK LIST*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "1: Make a todo list\n",
-      "\u001b[92m\u001b[1m\n",
-      "*****NEXT TASK*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "1: Make a todo list\n",
-      "\u001b[93m\u001b[1m\n",
-      "*****TASK RESULT*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "\n",
-      "\n",
-      "1. Check the temperature range for the day.\n",
-      "2. Gather temperature data for SF today.\n",
-      "3. Analyze the temperature data and create a weather report.\n",
-      "4. Publish the weather report.\n",
-      "\u001b[95m\u001b[1m\n",
-      "*****TASK LIST*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "2: Gather data on the expected temperature range for the day.\n",
-      "3: Collect data on the expected precipitation for the day.\n",
-      "4: Analyze the data and create a weather report.\n",
-      "5: Check the current weather conditions in SF.\n",
-      "6: Publish the weather report.\n",
-      "\u001b[92m\u001b[1m\n",
-      "*****NEXT TASK*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "2: Gather data on the expected temperature range for the day.\n",
-      "\u001b[93m\u001b[1m\n",
-      "*****TASK RESULT*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "\n",
-      "\n",
-      "I have gathered data on the expected temperature range for the day in San Francisco. The forecast is for temperatures to range from a low of 55 degrees Fahrenheit to a high of 68 degrees Fahrenheit.\n",
-      "\u001b[95m\u001b[1m\n",
-      "*****TASK LIST*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "3: Check the current weather conditions in SF.\n",
-      "4: Calculate the average temperature for the day in San Francisco.\n",
-      "5: Determine the probability of precipitation for the day in San Francisco.\n",
-      "6: Identify any potential weather warnings or advisories for the day in San Francisco.\n",
-      "7: Research any historical weather patterns for the day in San Francisco.\n",
-      "8: Compare the expected temperature range to the historical average for the day in San Francisco.\n",
-      "9: Collect data on the expected precipitation for the day.\n",
-      "10: Analyze the data and create a weather report.\n",
-      "11: Publish the weather report.\n",
-      "\u001b[92m\u001b[1m\n",
-      "*****NEXT TASK*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "3: Check the current weather conditions in SF.\n",
-      "\u001b[93m\u001b[1m\n",
-      "*****TASK RESULT*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "\n",
-      "\n",
-      "I am checking the current weather conditions in SF. According to the data I have gathered, the temperature in SF today is currently around 65 degrees Fahrenheit with clear skies. The temperature range for the day is expected to be between 60 and 70 degrees Fahrenheit.\n",
-      "\u001b[91m\u001b[1m\n",
-      "*****TASK ENDING*****\n",
-      "\u001b[0m\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "{'objective': 'Write a weather report for SF today'}"
-      ]
-     },
-     "execution_count": 141,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "baby_agi({\"objective\": OBJECTIVE})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "898a210b",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.1"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/extras/use_cases/agents/baby_agi_with_agent.ipynb

@@ -1,647 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "517a9fd4",
-   "metadata": {},
-   "source": [
-    "# BabyAGI with Tools\n",
-    "\n",
-    "This notebook builds on top of [baby agi](baby_agi.html), but shows how you can swap out the execution chain. The previous execution chain was just an LLM which made stuff up. By swapping it out with an agent that has access to tools, we can hopefully get real reliable information"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "556af556",
-   "metadata": {},
-   "source": [
-    "## Install and Import Required Modules"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "c8a354b6",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "from collections import deque\n",
-    "from typing import Dict, List, Optional, Any\n",
-    "\n",
-    "from langchain import LLMChain, OpenAI, PromptTemplate\n",
-    "from langchain.embeddings import OpenAIEmbeddings\n",
-    "from langchain.llms import BaseLLM\n",
-    "from langchain.vectorstores.base import VectorStore\n",
-    "from pydantic import BaseModel, Field\n",
-    "from langchain.chains.base import Chain"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "09f70772",
-   "metadata": {},
-   "source": [
-    "## Connect to the Vector Store\n",
-    "\n",
-    "Depending on what vectorstore you use, this step may look different."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "794045d4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%pip install faiss-cpu > /dev/null\n",
-    "%pip install google-search-results > /dev/null\n",
-    "from langchain.vectorstores import FAISS\n",
-    "from langchain.docstore import InMemoryDocstore"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "6e0305eb",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Define your embedding model\n",
-    "embeddings_model = OpenAIEmbeddings()\n",
-    "# Initialize the vectorstore as empty\n",
-    "import faiss\n",
-    "\n",
-    "embedding_size = 1536\n",
-    "index = faiss.IndexFlatL2(embedding_size)\n",
-    "vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {})"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0f3b72bf",
-   "metadata": {},
-   "source": [
-    "## Define the Chains\n",
-    "\n",
-    "BabyAGI relies on three LLM chains:\n",
-    "- Task creation chain to select new tasks to add to the list\n",
-    "- Task prioritization chain to re-prioritize tasks\n",
-    "- Execution Chain to execute the tasks\n",
-    "\n",
-    "\n",
-    "NOTE: in this notebook, the Execution chain will now be an agent."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "bf4bd5cd",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class TaskCreationChain(LLMChain):\n",
-    "    \"\"\"Chain to generates tasks.\"\"\"\n",
-    "\n",
-    "    @classmethod\n",
-    "    def from_llm(cls, llm: BaseLLM, verbose: bool = True) -> LLMChain:\n",
-    "        \"\"\"Get the response parser.\"\"\"\n",
-    "        task_creation_template = (\n",
-    "            \"You are an task creation AI that uses the result of an execution agent\"\n",
-    "            \" to create new tasks with the following objective: {objective},\"\n",
-    "            \" The last completed task has the result: {result}.\"\n",
-    "            \" This result was based on this task description: {task_description}.\"\n",
-    "            \" These are incomplete tasks: {incomplete_tasks}.\"\n",
-    "            \" Based on the result, create new tasks to be completed\"\n",
-    "            \" by the AI system that do not overlap with incomplete tasks.\"\n",
-    "            \" Return the tasks as an array.\"\n",
-    "        )\n",
-    "        prompt = PromptTemplate(\n",
-    "            template=task_creation_template,\n",
-    "            input_variables=[\n",
-    "                \"result\",\n",
-    "                \"task_description\",\n",
-    "                \"incomplete_tasks\",\n",
-    "                \"objective\",\n",
-    "            ],\n",
-    "        )\n",
-    "        return cls(prompt=prompt, llm=llm, verbose=verbose)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "b6488ffe",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class TaskPrioritizationChain(LLMChain):\n",
-    "    \"\"\"Chain to prioritize tasks.\"\"\"\n",
-    "\n",
-    "    @classmethod\n",
-    "    def from_llm(cls, llm: BaseLLM, verbose: bool = True) -> LLMChain:\n",
-    "        \"\"\"Get the response parser.\"\"\"\n",
-    "        task_prioritization_template = (\n",
-    "            \"You are an task prioritization AI tasked with cleaning the formatting of and reprioritizing\"\n",
-    "            \" the following tasks: {task_names}.\"\n",
-    "            \" Consider the ultimate objective of your team: {objective}.\"\n",
-    "            \" Do not remove any tasks. Return the result as a numbered list, like:\"\n",
-    "            \" #. First task\"\n",
-    "            \" #. Second task\"\n",
-    "            \" Start the task list with number {next_task_id}.\"\n",
-    "        )\n",
-    "        prompt = PromptTemplate(\n",
-    "            template=task_prioritization_template,\n",
-    "            input_variables=[\"task_names\", \"next_task_id\", \"objective\"],\n",
-    "        )\n",
-    "        return cls(prompt=prompt, llm=llm, verbose=verbose)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 46,
-   "id": "b43cd580",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.agents import ZeroShotAgent, Tool, AgentExecutor\n",
-    "from langchain import OpenAI, SerpAPIWrapper, LLMChain\n",
-    "\n",
-    "todo_prompt = PromptTemplate.from_template(\n",
-    "    \"You are a planner who is an expert at coming up with a todo list for a given objective. Come up with a todo list for this objective: {objective}\"\n",
-    ")\n",
-    "todo_chain = LLMChain(llm=OpenAI(temperature=0), prompt=todo_prompt)\n",
-    "search = SerpAPIWrapper()\n",
-    "tools = [\n",
-    "    Tool(\n",
-    "        name=\"Search\",\n",
-    "        func=search.run,\n",
-    "        description=\"useful for when you need to answer questions about current events\",\n",
-    "    ),\n",
-    "    Tool(\n",
-    "        name=\"TODO\",\n",
-    "        func=todo_chain.run,\n",
-    "        description=\"useful for when you need to come up with todo lists. Input: an objective to create a todo list for. Output: a todo list for that objective. Please be very clear what the objective is!\",\n",
-    "    ),\n",
-    "]\n",
-    "\n",
-    "\n",
-    "prefix = \"\"\"You are an AI who performs one task based on the following objective: {objective}. Take into account these previously completed tasks: {context}.\"\"\"\n",
-    "suffix = \"\"\"Question: {task}\n",
-    "{agent_scratchpad}\"\"\"\n",
-    "prompt = ZeroShotAgent.create_prompt(\n",
-    "    tools,\n",
-    "    prefix=prefix,\n",
-    "    suffix=suffix,\n",
-    "    input_variables=[\"objective\", \"task\", \"context\", \"agent_scratchpad\"],\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3ad996c5",
-   "metadata": {},
-   "source": [
-    "### Define the BabyAGI Controller\n",
-    "\n",
-    "BabyAGI composes the chains defined above in a (potentially-)infinite loop."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 47,
-   "id": "0ada0636",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def get_next_task(\n",
-    "    task_creation_chain: LLMChain,\n",
-    "    result: Dict,\n",
-    "    task_description: str,\n",
-    "    task_list: List[str],\n",
-    "    objective: str,\n",
-    ") -> List[Dict]:\n",
-    "    \"\"\"Get the next task.\"\"\"\n",
-    "    incomplete_tasks = \", \".join(task_list)\n",
-    "    response = task_creation_chain.run(\n",
-    "        result=result,\n",
-    "        task_description=task_description,\n",
-    "        incomplete_tasks=incomplete_tasks,\n",
-    "        objective=objective,\n",
-    "    )\n",
-    "    new_tasks = response.split(\"\\n\")\n",
-    "    return [{\"task_name\": task_name} for task_name in new_tasks if task_name.strip()]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 48,
-   "id": "d35250ad",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def prioritize_tasks(\n",
-    "    task_prioritization_chain: LLMChain,\n",
-    "    this_task_id: int,\n",
-    "    task_list: List[Dict],\n",
-    "    objective: str,\n",
-    ") -> List[Dict]:\n",
-    "    \"\"\"Prioritize tasks.\"\"\"\n",
-    "    task_names = [t[\"task_name\"] for t in task_list]\n",
-    "    next_task_id = int(this_task_id) + 1\n",
-    "    response = task_prioritization_chain.run(\n",
-    "        task_names=task_names, next_task_id=next_task_id, objective=objective\n",
-    "    )\n",
-    "    new_tasks = response.split(\"\\n\")\n",
-    "    prioritized_task_list = []\n",
-    "    for task_string in new_tasks:\n",
-    "        if not task_string.strip():\n",
-    "            continue\n",
-    "        task_parts = task_string.strip().split(\".\", 1)\n",
-    "        if len(task_parts) == 2:\n",
-    "            task_id = task_parts[0].strip()\n",
-    "            task_name = task_parts[1].strip()\n",
-    "            prioritized_task_list.append({\"task_id\": task_id, \"task_name\": task_name})\n",
-    "    return prioritized_task_list"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 49,
-   "id": "e3f1840c",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def _get_top_tasks(vectorstore, query: str, k: int) -> List[str]:\n",
-    "    \"\"\"Get the top k tasks based on the query.\"\"\"\n",
-    "    results = vectorstore.similarity_search_with_score(query, k=k)\n",
-    "    if not results:\n",
-    "        return []\n",
-    "    sorted_results, _ = zip(*sorted(results, key=lambda x: x[1], reverse=True))\n",
-    "    return [str(item.metadata[\"task\"]) for item in sorted_results]\n",
-    "\n",
-    "\n",
-    "def execute_task(\n",
-    "    vectorstore, execution_chain: LLMChain, objective: str, task: str, k: int = 5\n",
-    ") -> str:\n",
-    "    \"\"\"Execute a task.\"\"\"\n",
-    "    context = _get_top_tasks(vectorstore, query=objective, k=k)\n",
-    "    return execution_chain.run(objective=objective, context=context, task=task)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 50,
-   "id": "1e978938",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class BabyAGI(Chain, BaseModel):\n",
-    "    \"\"\"Controller model for the BabyAGI agent.\"\"\"\n",
-    "\n",
-    "    task_list: deque = Field(default_factory=deque)\n",
-    "    task_creation_chain: TaskCreationChain = Field(...)\n",
-    "    task_prioritization_chain: TaskPrioritizationChain = Field(...)\n",
-    "    execution_chain: AgentExecutor = Field(...)\n",
-    "    task_id_counter: int = Field(1)\n",
-    "    vectorstore: VectorStore = Field(init=False)\n",
-    "    max_iterations: Optional[int] = None\n",
-    "\n",
-    "    class Config:\n",
-    "        \"\"\"Configuration for this pydantic object.\"\"\"\n",
-    "\n",
-    "        arbitrary_types_allowed = True\n",
-    "\n",
-    "    def add_task(self, task: Dict):\n",
-    "        self.task_list.append(task)\n",
-    "\n",
-    "    def print_task_list(self):\n",
-    "        print(\"\\033[95m\\033[1m\" + \"\\n*****TASK LIST*****\\n\" + \"\\033[0m\\033[0m\")\n",
-    "        for t in self.task_list:\n",
-    "            print(str(t[\"task_id\"]) + \": \" + t[\"task_name\"])\n",
-    "\n",
-    "    def print_next_task(self, task: Dict):\n",
-    "        print(\"\\033[92m\\033[1m\" + \"\\n*****NEXT TASK*****\\n\" + \"\\033[0m\\033[0m\")\n",
-    "        print(str(task[\"task_id\"]) + \": \" + task[\"task_name\"])\n",
-    "\n",
-    "    def print_task_result(self, result: str):\n",
-    "        print(\"\\033[93m\\033[1m\" + \"\\n*****TASK RESULT*****\\n\" + \"\\033[0m\\033[0m\")\n",
-    "        print(result)\n",
-    "\n",
-    "    @property\n",
-    "    def input_keys(self) -> List[str]:\n",
-    "        return [\"objective\"]\n",
-    "\n",
-    "    @property\n",
-    "    def output_keys(self) -> List[str]:\n",
-    "        return []\n",
-    "\n",
-    "    def _call(self, inputs: Dict[str, Any]) -> Dict[str, Any]:\n",
-    "        \"\"\"Run the agent.\"\"\"\n",
-    "        objective = inputs[\"objective\"]\n",
-    "        first_task = inputs.get(\"first_task\", \"Make a todo list\")\n",
-    "        self.add_task({\"task_id\": 1, \"task_name\": first_task})\n",
-    "        num_iters = 0\n",
-    "        while True:\n",
-    "            if self.task_list:\n",
-    "                self.print_task_list()\n",
-    "\n",
-    "                # Step 1: Pull the first task\n",
-    "                task = self.task_list.popleft()\n",
-    "                self.print_next_task(task)\n",
-    "\n",
-    "                # Step 2: Execute the task\n",
-    "                result = execute_task(\n",
-    "                    self.vectorstore, self.execution_chain, objective, task[\"task_name\"]\n",
-    "                )\n",
-    "                this_task_id = int(task[\"task_id\"])\n",
-    "                self.print_task_result(result)\n",
-    "\n",
-    "                # Step 3: Store the result in Pinecone\n",
-    "                result_id = f\"result_{task['task_id']}_{num_iters}\"\n",
-    "                self.vectorstore.add_texts(\n",
-    "                    texts=[result],\n",
-    "                    metadatas=[{\"task\": task[\"task_name\"]}],\n",
-    "                    ids=[result_id],\n",
-    "                )\n",
-    "\n",
-    "                # Step 4: Create new tasks and reprioritize task list\n",
-    "                new_tasks = get_next_task(\n",
-    "                    self.task_creation_chain,\n",
-    "                    result,\n",
-    "                    task[\"task_name\"],\n",
-    "                    [t[\"task_name\"] for t in self.task_list],\n",
-    "                    objective,\n",
-    "                )\n",
-    "                for new_task in new_tasks:\n",
-    "                    self.task_id_counter += 1\n",
-    "                    new_task.update({\"task_id\": self.task_id_counter})\n",
-    "                    self.add_task(new_task)\n",
-    "                self.task_list = deque(\n",
-    "                    prioritize_tasks(\n",
-    "                        self.task_prioritization_chain,\n",
-    "                        this_task_id,\n",
-    "                        list(self.task_list),\n",
-    "                        objective,\n",
-    "                    )\n",
-    "                )\n",
-    "            num_iters += 1\n",
-    "            if self.max_iterations is not None and num_iters == self.max_iterations:\n",
-    "                print(\n",
-    "                    \"\\033[91m\\033[1m\" + \"\\n*****TASK ENDING*****\\n\" + \"\\033[0m\\033[0m\"\n",
-    "                )\n",
-    "                break\n",
-    "        return {}\n",
-    "\n",
-    "    @classmethod\n",
-    "    def from_llm(\n",
-    "        cls, llm: BaseLLM, vectorstore: VectorStore, verbose: bool = False, **kwargs\n",
-    "    ) -> \"BabyAGI\":\n",
-    "        \"\"\"Initialize the BabyAGI Controller.\"\"\"\n",
-    "        task_creation_chain = TaskCreationChain.from_llm(llm, verbose=verbose)\n",
-    "        task_prioritization_chain = TaskPrioritizationChain.from_llm(\n",
-    "            llm, verbose=verbose\n",
-    "        )\n",
-    "        llm_chain = LLMChain(llm=llm, prompt=prompt)\n",
-    "        tool_names = [tool.name for tool in tools]\n",
-    "        agent = ZeroShotAgent(llm_chain=llm_chain, allowed_tools=tool_names)\n",
-    "        agent_executor = AgentExecutor.from_agent_and_tools(\n",
-    "            agent=agent, tools=tools, verbose=True\n",
-    "        )\n",
-    "        return cls(\n",
-    "            task_creation_chain=task_creation_chain,\n",
-    "            task_prioritization_chain=task_prioritization_chain,\n",
-    "            execution_chain=agent_executor,\n",
-    "            vectorstore=vectorstore,\n",
-    "            **kwargs,\n",
-    "        )"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "05ba762e",
-   "metadata": {},
-   "source": [
-    "### Run the BabyAGI\n",
-    "\n",
-    "Now it's time to create the BabyAGI controller and watch it try to accomplish your objective."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 51,
-   "id": "3d220b69",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "OBJECTIVE = \"Write a weather report for SF today\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 52,
-   "id": "8a8e5543",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm = OpenAI(temperature=0)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 53,
-   "id": "3d69899b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Logging of LLMChains\n",
-    "verbose = False\n",
-    "# If None, will keep on going forever\n",
-    "max_iterations: Optional[int] = 3\n",
-    "baby_agi = BabyAGI.from_llm(\n",
-    "    llm=llm, vectorstore=vectorstore, verbose=verbose, max_iterations=max_iterations\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 54,
-   "id": "f7957b51",
-   "metadata": {
-    "scrolled": false
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[95m\u001b[1m\n",
-      "*****TASK LIST*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "1: Make a todo list\n",
-      "\u001b[92m\u001b[1m\n",
-      "*****NEXT TASK*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "1: Make a todo list\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
-      "\u001b[32;1m\u001b[1;3mThought: I need to gather data on the current weather conditions in SF\n",
-      "Action: Search\n",
-      "Action Input: Current weather conditions in SF\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3mHigh 67F. Winds WNW at 10 to 15 mph. Clear to partly cloudy.\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I need to make a todo list\n",
-      "Action: TODO\n",
-      "Action Input: Write a weather report for SF today\u001b[0m\n",
-      "Observation: \u001b[33;1m\u001b[1;3m\n",
-      "\n",
-      "1. Research current weather conditions in San Francisco\n",
-      "2. Gather data on temperature, humidity, wind speed, and other relevant weather conditions\n",
-      "3. Analyze data to determine current weather trends\n",
-      "4. Write a brief introduction to the weather report\n",
-      "5. Describe current weather conditions in San Francisco\n",
-      "6. Discuss any upcoming weather changes\n",
-      "7. Summarize the weather report\n",
-      "8. Proofread and edit the report\n",
-      "9. Submit the report\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: A weather report for SF today should include research on current weather conditions in San Francisco, gathering data on temperature, humidity, wind speed, and other relevant weather conditions, analyzing data to determine current weather trends, writing a brief introduction to the weather report, describing current weather conditions in San Francisco, discussing any upcoming weather changes, summarizing the weather report, proofreading and editing the report, and submitting the report.\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      "\u001b[93m\u001b[1m\n",
-      "*****TASK RESULT*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "A weather report for SF today should include research on current weather conditions in San Francisco, gathering data on temperature, humidity, wind speed, and other relevant weather conditions, analyzing data to determine current weather trends, writing a brief introduction to the weather report, describing current weather conditions in San Francisco, discussing any upcoming weather changes, summarizing the weather report, proofreading and editing the report, and submitting the report.\n",
-      "\u001b[95m\u001b[1m\n",
-      "*****TASK LIST*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "2: Gather data on temperature, humidity, wind speed, and other relevant weather conditions\n",
-      "3: Analyze data to determine current weather trends\n",
-      "4: Write a brief introduction to the weather report\n",
-      "5: Describe current weather conditions in San Francisco\n",
-      "6: Discuss any upcoming weather changes\n",
-      "7: Summarize the weather report\n",
-      "8: Proofread and edit the report\n",
-      "9: Submit the report\n",
-      "1: Research current weather conditions in San Francisco\n",
-      "\u001b[92m\u001b[1m\n",
-      "*****NEXT TASK*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "2: Gather data on temperature, humidity, wind speed, and other relevant weather conditions\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
-      "\u001b[32;1m\u001b[1;3mThought: I need to search for the current weather conditions in SF\n",
-      "Action: Search\n",
-      "Action Input: Current weather conditions in SF\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3mHigh 67F. Winds WNW at 10 to 15 mph. Clear to partly cloudy.\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I need to make a todo list\n",
-      "Action: TODO\n",
-      "Action Input: Create a weather report for SF today\u001b[0m\n",
-      "Observation: \u001b[33;1m\u001b[1;3m\n",
-      "\n",
-      "1. Gather current weather data for SF, including temperature, wind speed, humidity, and precipitation.\n",
-      "2. Research historical weather data for SF to compare current conditions.\n",
-      "3. Analyze current and historical data to determine any trends or patterns.\n",
-      "4. Create a visual representation of the data, such as a graph or chart.\n",
-      "5. Write a summary of the weather report, including key findings and any relevant information.\n",
-      "6. Publish the weather report on a website or other platform.\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: Today in San Francisco, the temperature is 67F with winds WNW at 10 to 15 mph. The sky is clear to partly cloudy.\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      "\u001b[93m\u001b[1m\n",
-      "*****TASK RESULT*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "Today in San Francisco, the temperature is 67F with winds WNW at 10 to 15 mph. The sky is clear to partly cloudy.\n",
-      "\u001b[95m\u001b[1m\n",
-      "*****TASK LIST*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "3: Research current weather conditions in San Francisco\n",
-      "4: Compare the current weather conditions in San Francisco to the average for this time of year.\n",
-      "5: Identify any potential weather-related hazards in the area.\n",
-      "6: Research any historical weather patterns in San Francisco.\n",
-      "7: Analyze data to determine current weather trends\n",
-      "8: Include any relevant data from nearby cities in the report.\n",
-      "9: Include any relevant data from the National Weather Service in the report.\n",
-      "10: Include any relevant data from local news sources in the report.\n",
-      "11: Include any relevant data from online weather sources in the report.\n",
-      "12: Include any relevant data from local meteorologists in the report.\n",
-      "13: Include any relevant data from local weather stations in the report.\n",
-      "14: Include any relevant data from satellite images in the report.\n",
-      "15: Describe current weather conditions in San Francisco\n",
-      "16: Discuss any upcoming weather changes\n",
-      "17: Write a brief introduction to the weather report\n",
-      "18: Summarize the weather report\n",
-      "19: Proofread and edit the report\n",
-      "20: Submit the report\n",
-      "\u001b[92m\u001b[1m\n",
-      "*****NEXT TASK*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "3: Research current weather conditions in San Francisco\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
-      "\u001b[32;1m\u001b[1;3mThought: I need to search for current weather conditions in San Francisco\n",
-      "Action: Search\n",
-      "Action Input: Current weather conditions in San Francisco\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3mTodaySun 04/09 High 67 · 1% Precip. ; TonightSun 04/09 Low 49 · 9% Precip. ; TomorrowMon 04/10 High 64 · 11% Precip.\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: Today in San Francisco, the high temperature is 67 degrees with 1% chance of precipitation. The low temperature tonight is 49 degrees with 9% chance of precipitation. Tomorrow's high temperature is 64 degrees with 11% chance of precipitation.\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      "\u001b[93m\u001b[1m\n",
-      "*****TASK RESULT*****\n",
-      "\u001b[0m\u001b[0m\n",
-      "Today in San Francisco, the high temperature is 67 degrees with 1% chance of precipitation. The low temperature tonight is 49 degrees with 9% chance of precipitation. Tomorrow's high temperature is 64 degrees with 11% chance of precipitation.\n",
-      "\u001b[91m\u001b[1m\n",
-      "*****TASK ENDING*****\n",
-      "\u001b[0m\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "{'objective': 'Write a weather report for SF today'}"
-      ]
-     },
-     "execution_count": 54,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "baby_agi({\"objective\": OBJECTIVE})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "898a210b",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.1"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/extras/use_cases/apis.ipynb

@@ -62,7 +62,7 @@
     "\n",
     "# Set env var OPENAI_API_KEY or load from a .env file:\n",
     "# import dotenv\n",
-    "# dotenv.load_env()"
+    "# dotenv.load_dotenv()"
    ]
   },
   {
@@ -145,7 +145,7 @@
    "source": [
     "## Functions \n",
     "\n",
-    "We can unpack what is hapening when we use the functions to calls external APIs.\n",
+    "We can unpack what is happening when we use the functions to call external APIs.\n",
     "\n",
     "Let's look at the [LangSmith trace](https://smith.langchain.com/public/76a58b85-193f-4eb7-ba40-747f0d5dd56e/r):\n",
     "\n",
@@ -155,10 +155,10 @@
     "https://www.klarna.com/us/shopping/public/openai/v0/api-docs/\n",
     "```\n",
     "\n",
-    "* The prompt then tells the LLM to use the API spec wiith input question:\n",
+    "* The prompt then tells the LLM to use the API spec with input question:\n",
     "\n",
     "```\n",
-    "Use the provided API's to respond to this user query:\n",
+    "Use the provided APIs to respond to this user query:\n",
     "What are some options for a men's large blue button down shirt\n",
     "```\n",
     "\n",
@@ -278,7 +278,7 @@
     "![Image description](/img/api_chain.png)\n",
     "\n",
     "* [Here](https://github.com/langchain-ai/langchain/blob/bbd22b9b761389a5e40fc45b0570e1830aabb707/libs/langchain/langchain/chains/api/base.py#L82) we make the API request with the API url.\n",
-    "* The `api_answer_chain` takes the response from the API and provides us with a natural langugae response:\n",
+    "* The `api_answer_chain` takes the response from the API and provides us with a natural language response:\n",
     "\n",
     "![Image description](/img/api_chain_response.png)"
    ]

docs/extras/use_cases/chatbots.ipynb

@@ -54,7 +54,7 @@
     "\n",
     "# Set env var OPENAI_API_KEY or load from a .env file:\n",
     "# import dotenv\n",
-    "# dotenv.load_env()"
+    "# dotenv.load_dotenv()"
    ]
   },
   {

docs/extras/use_cases/code_understanding.ipynb

@@ -25,7 +25,7 @@
     "In particular, we can employ a [splitting strategy](https://python.langchain.com/docs/integrations/document_loaders/source_code) that does a few things:\n",
     "\n",
     "* Keeps each top-level function and class in the code is loaded into separate documents. \n",
-    "* Puts remaining into a seperate document.\n",
+    "* Puts remaining into a separate document.\n",
     "* Retains metadata about where each split comes from\n",
     "\n",
     "## Quickstart"
@@ -42,7 +42,7 @@
     "# Set env var OPENAI_API_KEY or load from a .env file\n",
     "# import dotenv\n",
     "\n",
-    "# dotenv.load_env()"
+    "# dotenv.load_dotenv()"
    ]
   },
   {
@@ -66,11 +66,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
-    "from git import Repo\n",
+    "# from git import Repo\n",
     "from langchain.text_splitter import Language\n",
     "from langchain.document_loaders.generic import GenericLoader\n",
     "from langchain.document_loaders.parsers import LanguageParser"
@@ -78,13 +78,13 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
     "# Clone\n",
     "repo_path = \"/Users/rlm/Desktop/test_repo\"\n",
-    "repo = Repo.clone_from(\"https://github.com/hwchase17/langchain\", to_path=repo_path)"
+    "# repo = Repo.clone_from(\"https://github.com/hwchase17/langchain\", to_path=repo_path)"
    ]
   },
   {
@@ -94,13 +94,13 @@
     "We load the py code using [`LanguageParser`](https://python.langchain.com/docs/integrations/document_loaders/source_code), which will:\n",
     "\n",
     "* Keep top-level functions and classes together (into a single document)\n",
-    "* Put remaining code into a seperate document\n",
+    "* Put remaining code into a separate document\n",
     "* Retains metadata about where each split comes from"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -109,7 +109,7 @@
        "1293"
       ]
      },
-     "execution_count": 14,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -139,7 +139,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -148,7 +148,7 @@
        "3748"
       ]
      },
-     "execution_count": 17,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -187,7 +187,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -195,7 +195,7 @@
     "from langchain.embeddings.openai import OpenAIEmbeddings\n",
     "db = Chroma.from_documents(texts, OpenAIEmbeddings(disallowed_special=()))\n",
     "retriever = db.as_retriever(\n",
-    "    search_type=\"mmr\",  # Also test \"similarity\"\n",
+    "    search_type=\"mmr\", # Also test \"similarity\"\n",
     "    search_kwargs={\"k\": 8},\n",
     ")"
    ]
@@ -217,7 +217,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 32,
+   "execution_count": 42,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -231,22 +231,22 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 30,
+   "execution_count": 43,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "'To load a source code as documents for a QA over code, you can use the `CodeLoader` class. This class allows you to load source code files and split them into classes and functions.\\n\\nHere is an example of how to use the `CodeLoader` class:\\n\\n```python\\nfrom langchain.document_loaders.code import CodeLoader\\n\\n# Specify the path to the source code file\\ncode_file_path = \"path/to/code/file.py\"\\n\\n# Create an instance of the CodeLoader class\\ncode_loader = CodeLoader(code_file_path)\\n\\n# Load the code as documents\\ndocuments = code_loader.load()\\n\\n# Iterate over the documents\\nfor document in documents:\\n    # Access the class or function name\\n    name = document.metadata[\"name\"]\\n    \\n    # Access the code content\\n    code = document.page_content\\n    \\n    # Process the code as needed\\n    # ...\\n```\\n\\nIn the example above, `code_file_path` should be replaced with the actual path to your source code file. The `load()` method of the `CodeLoader` class will return a list of `Document` objects, where each document represents a class or function in the source code. You can access the class or function name using the `metadata[\"name\"]` attribute, and the code content using the `page_content` attribute of each `Document` object.\\n\\nYou can then process the code as needed for your QA task.'"
+       "'To initialize a ReAct agent, you need to follow these steps:\\n\\n1. Initialize a language model `llm` of type `BaseLanguageModel`.\\n\\n2. Initialize a document store `docstore` of type `Docstore`.\\n\\n3. Create a `DocstoreExplorer` with the initialized `docstore`. The `DocstoreExplorer` is used to search for and look up terms in the document store.\\n\\n4. Create an array of `Tool` objects. The `Tool` objects represent the actions that the agent can perform. In the case of `ReActDocstoreAgent`, the tools must be \"Search\" and \"Lookup\" with their corresponding functions from the `DocstoreExplorer`.\\n\\n5. Initialize the `ReActDocstoreAgent` using the `from_llm_and_tools` method with the `llm` (language model) and `tools` as parameters.\\n\\n6. Initialize the `ReActChain` (which is the `AgentExecutor`) using the `ReActDocstoreAgent` and `tools` as parameters.\\n\\nHere is an example of how to do this:\\n\\n```python\\nfrom langchain import ReActChain, OpenAI\\nfrom langchain.docstore.base import Docstore\\nfrom langchain.docstore.document import Document\\nfrom langchain.tools.base import BaseTool\\n\\n# Initialize the LLM and a docstore\\nllm = OpenAI()\\ndocstore = Docstore()\\n\\ndocstore_explorer = DocstoreExplorer(docstore)\\ntools = [\\n    Tool(\\n        name=\"Search\",\\n        func=docstore_explorer.search,\\n        description=\"Search for a term in the docstore.\",\\n    ),\\n    Tool(\\n        name=\"Lookup\",\\n        func=docstore_explorer.lookup,\\n        description=\"Lookup a term in the docstore.\",\\n    ),\\n]\\nagent = ReActDocstoreAgent.from_llm_and_tools(llm, tools)\\nreact = ReActChain(agent=agent, tools=tools)\\n```\\n\\nKeep in mind that this is a simplified example and you might need to adapt it to your specific needs.'"
       ]
      },
-     "execution_count": 30,
+     "execution_count": 43,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "question = \"How can I load a source code as documents, for a QA over code, spliting the code in classes and functions?\"\n",
+    "question = \"How can I initialize a ReAct agent?\"\n",
     "result = qa(question)\n",
     "result['answer']"
    ]
@@ -328,6 +328,679 @@
     "\n",
     "![Image description](/img/code_retrieval.png)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Open source LLMs\n",
+    "\n",
+    "We can use [Code LLaMA](https://about.fb.com/news/2023/08/code-llama-ai-for-coding/) via LLamaCPP or [Ollama integration](https://ollama.ai/blog/run-code-llama-locally).\n",
+    "\n",
+    "Note: be sure to upgrade `llama-cpp-python` in order to use the new `gguf` [file format](https://github.com/abetlen/llama-cpp-python/pull/633).\n",
+    "\n",
+    "```\n",
+    "CMAKE_ARGS=\"-DLLAMA_METAL=on\" FORCE_CMAKE=1 /Users/rlm/miniforge3/envs/llama2/bin/pip install -U llama-cpp-python --no-cache-dir\n",
+    "```\n",
+    " \n",
+    "Check out the latest code-llama models [here](https://huggingface.co/TheBloke/CodeLlama-13B-Instruct-GGUF/tree/main)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import LlamaCpp\n",
+    "from langchain import PromptTemplate, LLMChain\n",
+    "from langchain.callbacks.manager import CallbackManager\n",
+    "from langchain.memory import ConversationSummaryMemory\n",
+    "from langchain.chains import ConversationalRetrievalChain \n",
+    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "llama_model_loader: loaded meta data with 17 key-value pairs and 363 tensors from /Users/rlm/Desktop/Code/llama/code-llama/codellama-13b-instruct.Q4_K_M.gguf (version GGUF V1 (latest))\n",
+      "llama_model_loader: - tensor    0:                token_embd.weight q4_0     [  5120, 32016,     1,     1 ]\n",
+      "llama_model_loader: - tensor    1:               output_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor    2:                    output.weight f16      [  5120, 32016,     1,     1 ]\n",
+      "llama_model_loader: - tensor    3:              blk.0.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor    4:              blk.0.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor    5:              blk.0.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor    6:         blk.0.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor    7:            blk.0.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor    8:            blk.0.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor    9:              blk.0.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   10:           blk.0.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   11:            blk.0.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   12:              blk.1.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   13:              blk.1.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   14:              blk.1.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   15:         blk.1.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   16:            blk.1.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   17:            blk.1.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   18:              blk.1.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   19:           blk.1.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   20:            blk.1.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   21:              blk.2.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   22:              blk.2.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   23:              blk.2.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   24:         blk.2.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   25:            blk.2.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   26:            blk.2.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   27:              blk.2.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   28:           blk.2.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   29:            blk.2.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   30:              blk.3.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   31:              blk.3.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   32:              blk.3.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   33:         blk.3.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   34:            blk.3.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   35:            blk.3.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   36:              blk.3.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   37:           blk.3.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   38:            blk.3.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   39:              blk.4.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   40:              blk.4.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   41:              blk.4.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   42:         blk.4.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   43:            blk.4.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   44:            blk.4.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   45:              blk.4.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   46:           blk.4.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   47:            blk.4.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   48:              blk.5.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   49:              blk.5.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   50:              blk.5.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   51:         blk.5.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   52:            blk.5.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   53:            blk.5.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   54:              blk.5.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   55:           blk.5.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   56:            blk.5.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   57:              blk.6.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   58:              blk.6.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   59:              blk.6.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   60:         blk.6.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   61:            blk.6.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   62:            blk.6.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   63:              blk.6.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   64:           blk.6.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   65:            blk.6.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   66:              blk.7.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   67:              blk.7.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   68:              blk.7.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   69:         blk.7.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   70:            blk.7.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   71:            blk.7.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   72:              blk.7.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   73:           blk.7.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   74:            blk.7.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   75:              blk.8.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   76:              blk.8.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   77:              blk.8.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   78:         blk.8.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   79:            blk.8.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   80:            blk.8.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   81:              blk.8.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   82:           blk.8.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   83:            blk.8.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   84:              blk.9.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   85:              blk.9.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   86:              blk.9.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   87:         blk.9.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   88:            blk.9.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   89:            blk.9.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   90:              blk.9.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   91:           blk.9.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   92:            blk.9.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor   93:             blk.10.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   94:             blk.10.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   95:             blk.10.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   96:        blk.10.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   97:           blk.10.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor   98:           blk.10.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor   99:             blk.10.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  100:          blk.10.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  101:           blk.10.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  102:             blk.11.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  103:             blk.11.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  104:             blk.11.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  105:        blk.11.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  106:           blk.11.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  107:           blk.11.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  108:             blk.11.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  109:          blk.11.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  110:           blk.11.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  111:             blk.12.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  112:             blk.12.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  113:             blk.12.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  114:        blk.12.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  115:           blk.12.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  116:           blk.12.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  117:             blk.12.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  118:          blk.12.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  119:           blk.12.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  120:             blk.13.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  121:             blk.13.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  122:             blk.13.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  123:        blk.13.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  124:           blk.13.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  125:           blk.13.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  126:             blk.13.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  127:          blk.13.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  128:           blk.13.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  129:             blk.14.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  130:             blk.14.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  131:             blk.14.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  132:        blk.14.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  133:           blk.14.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  134:           blk.14.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  135:             blk.14.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  136:          blk.14.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  137:           blk.14.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  138:             blk.15.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  139:             blk.15.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  140:             blk.15.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  141:        blk.15.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  142:           blk.15.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  143:           blk.15.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  144:             blk.15.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  145:          blk.15.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  146:           blk.15.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  147:             blk.16.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  148:             blk.16.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  149:             blk.16.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  150:        blk.16.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  151:           blk.16.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  152:           blk.16.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  153:             blk.16.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  154:          blk.16.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  155:           blk.16.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  156:             blk.17.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  157:             blk.17.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  158:             blk.17.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  159:        blk.17.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  160:           blk.17.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  161:           blk.17.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  162:             blk.17.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  163:          blk.17.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  164:           blk.17.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  165:             blk.18.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  166:             blk.18.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  167:             blk.18.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  168:        blk.18.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  169:           blk.18.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  170:           blk.18.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  171:             blk.18.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  172:          blk.18.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  173:           blk.18.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  174:             blk.19.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  175:             blk.19.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  176:             blk.19.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  177:        blk.19.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  178:           blk.19.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  179:           blk.19.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  180:             blk.19.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  181:          blk.19.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  182:           blk.19.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  183:             blk.20.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  184:             blk.20.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  185:             blk.20.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  186:        blk.20.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  187:           blk.20.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  188:           blk.20.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  189:             blk.20.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  190:          blk.20.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  191:           blk.20.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  192:             blk.21.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  193:             blk.21.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  194:             blk.21.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  195:        blk.21.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  196:           blk.21.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  197:           blk.21.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  198:             blk.21.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  199:          blk.21.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  200:           blk.21.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  201:             blk.22.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  202:             blk.22.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  203:             blk.22.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  204:        blk.22.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  205:           blk.22.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  206:           blk.22.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  207:             blk.22.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  208:          blk.22.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  209:           blk.22.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  210:             blk.23.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  211:             blk.23.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  212:             blk.23.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  213:        blk.23.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  214:           blk.23.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  215:           blk.23.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  216:             blk.23.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  217:          blk.23.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  218:           blk.23.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  219:             blk.24.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  220:             blk.24.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  221:             blk.24.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  222:        blk.24.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  223:           blk.24.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  224:           blk.24.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  225:             blk.24.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  226:          blk.24.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  227:           blk.24.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  228:             blk.25.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  229:             blk.25.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  230:             blk.25.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  231:        blk.25.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  232:           blk.25.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  233:           blk.25.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  234:             blk.25.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  235:          blk.25.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  236:           blk.25.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  237:             blk.26.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  238:             blk.26.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  239:             blk.26.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  240:        blk.26.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  241:           blk.26.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  242:           blk.26.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  243:             blk.26.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  244:          blk.26.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  245:           blk.26.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  246:             blk.27.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  247:             blk.27.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  248:             blk.27.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  249:        blk.27.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  250:           blk.27.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  251:           blk.27.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  252:             blk.27.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  253:          blk.27.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  254:           blk.27.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  255:             blk.28.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  256:             blk.28.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  257:             blk.28.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  258:        blk.28.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  259:           blk.28.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  260:           blk.28.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  261:             blk.28.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  262:          blk.28.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  263:           blk.28.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  264:             blk.29.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  265:             blk.29.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  266:             blk.29.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  267:        blk.29.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  268:           blk.29.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  269:           blk.29.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  270:             blk.29.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  271:          blk.29.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  272:           blk.29.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  273:             blk.30.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  274:             blk.30.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  275:             blk.30.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  276:        blk.30.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  277:           blk.30.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  278:           blk.30.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  279:             blk.30.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  280:          blk.30.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  281:           blk.30.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  282:             blk.31.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  283:             blk.31.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  284:             blk.31.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  285:        blk.31.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  286:           blk.31.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  287:           blk.31.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  288:             blk.31.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  289:          blk.31.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  290:           blk.31.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  291:             blk.32.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  292:             blk.32.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  293:             blk.32.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  294:        blk.32.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  295:           blk.32.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  296:           blk.32.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  297:             blk.32.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  298:          blk.32.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  299:           blk.32.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  300:             blk.33.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  301:             blk.33.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  302:             blk.33.attn_v.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  303:        blk.33.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  304:           blk.33.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  305:           blk.33.ffn_down.weight q4_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  306:             blk.33.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  307:          blk.33.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  308:           blk.33.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  309:             blk.34.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  310:             blk.34.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  311:             blk.34.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  312:        blk.34.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  313:           blk.34.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  314:           blk.34.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  315:             blk.34.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  316:          blk.34.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  317:           blk.34.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  318:             blk.35.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  319:             blk.35.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  320:             blk.35.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  321:        blk.35.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  322:           blk.35.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  323:           blk.35.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  324:             blk.35.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  325:          blk.35.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  326:           blk.35.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  327:             blk.36.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  328:             blk.36.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  329:             blk.36.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  330:        blk.36.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  331:           blk.36.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  332:           blk.36.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  333:             blk.36.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  334:          blk.36.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  335:           blk.36.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  336:             blk.37.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  337:             blk.37.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  338:             blk.37.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  339:        blk.37.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  340:           blk.37.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  341:           blk.37.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  342:             blk.37.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  343:          blk.37.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  344:           blk.37.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  345:             blk.38.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  346:             blk.38.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  347:             blk.38.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  348:        blk.38.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  349:           blk.38.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  350:           blk.38.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  351:             blk.38.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  352:          blk.38.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  353:           blk.38.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  354:             blk.39.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  355:             blk.39.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  356:             blk.39.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  357:        blk.39.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  358:           blk.39.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  359:           blk.39.ffn_down.weight q6_K     [ 13824,  5120,     1,     1 ]\n",
+      "llama_model_loader: - tensor  360:             blk.39.ffn_up.weight q4_K     [  5120, 13824,     1,     1 ]\n",
+      "llama_model_loader: - tensor  361:          blk.39.attn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - tensor  362:           blk.39.ffn_norm.weight f32      [  5120,     1,     1,     1 ]\n",
+      "llama_model_loader: - kv   0:                       general.architecture str     \n",
+      "llama_model_loader: - kv   1:                               general.name str     \n",
+      "llama_model_loader: - kv   2:                       llama.context_length u32     \n",
+      "llama_model_loader: - kv   3:                     llama.embedding_length u32     \n",
+      "llama_model_loader: - kv   4:                          llama.block_count u32     \n",
+      "llama_model_loader: - kv   5:                  llama.feed_forward_length u32     \n",
+      "llama_model_loader: - kv   6:                 llama.rope.dimension_count u32     \n",
+      "llama_model_loader: - kv   7:                 llama.attention.head_count u32     \n",
+      "llama_model_loader: - kv   8:              llama.attention.head_count_kv u32     \n",
+      "llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32     \n",
+      "llama_model_loader: - kv  10:                       llama.rope.freq_base f32     \n",
+      "llama_model_loader: - kv  11:                          general.file_type u32     \n",
+      "llama_model_loader: - kv  12:                       tokenizer.ggml.model str     \n",
+      "llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr     \n",
+      "llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr     \n",
+      "llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr     \n",
+      "llama_model_loader: - kv  16:               general.quantization_version u32     \n",
+      "llama_model_loader: - type  f32:   81 tensors\n",
+      "llama_model_loader: - type  f16:    1 tensors\n",
+      "llama_model_loader: - type q4_0:    1 tensors\n",
+      "llama_model_loader: - type q4_K:  240 tensors\n",
+      "llama_model_loader: - type q6_K:   40 tensors\n",
+      "llm_load_print_meta: format         = GGUF V1 (latest)\n",
+      "llm_load_print_meta: arch           = llama\n",
+      "llm_load_print_meta: vocab type     = SPM\n",
+      "llm_load_print_meta: n_vocab        = 32016\n",
+      "llm_load_print_meta: n_merges       = 0\n",
+      "llm_load_print_meta: n_ctx_train    = 16384\n",
+      "llm_load_print_meta: n_ctx          = 5000\n",
+      "llm_load_print_meta: n_embd         = 5120\n",
+      "llm_load_print_meta: n_head         = 40\n",
+      "llm_load_print_meta: n_head_kv      = 40\n",
+      "llm_load_print_meta: n_layer        = 40\n",
+      "llm_load_print_meta: n_rot          = 128\n",
+      "llm_load_print_meta: n_gqa          = 1\n",
+      "llm_load_print_meta: f_norm_eps     = 1.0e-05\n",
+      "llm_load_print_meta: f_norm_rms_eps = 1.0e-05\n",
+      "llm_load_print_meta: n_ff           = 13824\n",
+      "llm_load_print_meta: freq_base      = 1000000.0\n",
+      "llm_load_print_meta: freq_scale     = 1\n",
+      "llm_load_print_meta: model type     = 13B\n",
+      "llm_load_print_meta: model ftype    = mostly Q4_K - Medium\n",
+      "llm_load_print_meta: model size     = 13.02 B\n",
+      "llm_load_print_meta: general.name   = LLaMA\n",
+      "llm_load_print_meta: BOS token = 1 '<s>'\n",
+      "llm_load_print_meta: EOS token = 2 '</s>'\n",
+      "llm_load_print_meta: UNK token = 0 '<unk>'\n",
+      "llm_load_print_meta: LF token  = 13 '<0x0A>'\n",
+      "llm_load_tensors: ggml ctx size =    0.11 MB\n",
+      "llm_load_tensors: mem required  = 7685.49 MB (+ 3906.25 MB per state)\n",
+      ".................................................................................................\n",
+      "llama_new_context_with_model: kv self size  = 3906.25 MB\n",
+      "ggml_metal_init: allocating\n",
+      "ggml_metal_init: loading '/Users/rlm/miniforge3/envs/llama2/lib/python3.9/site-packages/llama_cpp/ggml-metal.metal'\n",
+      "ggml_metal_init: loaded kernel_add                            0x12126dd00 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_add_row                        0x12126d610 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul                            0x12126f2a0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_row                        0x12126f500 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_scale                          0x12126f760 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_silu                           0x12126fe40 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_relu                           0x1212700a0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_gelu                           0x121270300 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_soft_max                       0x121270560 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_diag_mask_inf                  0x1212707c0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_f16                   0x121270a20 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q4_0                  0x121270c80 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q4_1                  0x121270ee0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q8_0                  0x121271140 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q2_K                  0x1212713a0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q3_K                  0x121271600 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q4_K                  0x121271860 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q5_K                  0x121271ac0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_get_rows_q6_K                  0x121271d20 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_rms_norm                       0x121271f80 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_norm                           0x1212721e0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_f16_f32                0x121272440 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q4_0_f32               0x1212726a0 | th_max =  896 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q4_1_f32               0x121272900 | th_max =  896 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q8_0_f32               0x121272b60 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q2_K_f32               0x121272dc0 | th_max =  640 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q3_K_f32               0x121273020 | th_max =  704 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q4_K_f32               0x121273280 | th_max =  576 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q5_K_f32               0x1212734e0 | th_max =  576 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mat_q6_K_f32               0x121273740 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_f16_f32                 0x1212739a0 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q4_0_f32                0x121273c00 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q8_0_f32                0x121273e60 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q4_1_f32                0x1212740c0 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q2_K_f32                0x121274320 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q3_K_f32                0x121274580 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q4_K_f32                0x1212747e0 | th_max =  768 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q5_K_f32                0x121274a40 | th_max =  704 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_mul_mm_q6_K_f32                0x121274ca0 | th_max =  704 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_rope                           0x121274f00 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_alibi_f32                      0x121275160 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_cpy_f32_f16                    0x1212753c0 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_cpy_f32_f32                    0x121275620 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: loaded kernel_cpy_f16_f16                    0x121275880 | th_max = 1024 | th_width =   32\n",
+      "ggml_metal_init: recommendedMaxWorkingSetSize  = 21845.34 MB\n",
+      "ggml_metal_init: hasUnifiedMemory              = true\n",
+      "ggml_metal_init: maxTransferRate               = built-in GPU\n",
+      "llama_new_context_with_model: compute buffer total size =  442.03 MB\n",
+      "llama_new_context_with_model: max tensor size =   312.66 MB\n",
+      "ggml_metal_add_buffer: allocated 'data            ' buffer, size =  7686.00 MB, (20243.77 / 21845.34)\n",
+      "ggml_metal_add_buffer: allocated 'eval            ' buffer, size =     1.42 MB, (20245.19 / 21845.34)\n",
+      "ggml_metal_add_buffer: allocated 'kv              ' buffer, size =  3908.25 MB, (24153.44 / 21845.34), warning: current allocated size is greater than the recommended max working set size\n",
+      "AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | \n",
+      "ggml_metal_add_buffer: allocated 'alloc           ' buffer, size =   440.64 MB, (24594.08 / 21845.34), warning: current allocated size is greater than the recommended max working set size\n"
+     ]
+    }
+   ],
+   "source": [
+    "callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])\n",
+    "llm = LlamaCpp(\n",
+    "    model_path=\"/Users/rlm/Desktop/Code/llama/code-llama/codellama-13b-instruct.Q4_K_M.gguf\",\n",
+    "    n_ctx=5000,\n",
+    "    n_gpu_layers=1,\n",
+    "    n_batch=512,\n",
+    "    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls\n",
+    "    callback_manager=callback_manager,\n",
+    "    verbose=True,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Llama.generate: prefix-match hit\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      " You can use the find command with a few options to this task. Here is an example of how you might go about it:\n",
+      "\n",
+      "find . -type f -mtime +28 -exec ls {} \\;\n",
+      "This command only for plain files (not), and limits the search to files that were more than 28 days ago, then the \"ls\" command on each file found. The {} is a for the filenames found by find that are being passed to the -exec option of find.\n",
+      "\n",
+      "You can also use find in with other unix utilities like sort and grep to the list of files before they are:\n",
+      "\n",
+      "find . -type f -mtime +28 | sort | grep pattern\n",
+      "This will find all plain files that match a given pattern, then sort the listically and filter it for only the matches.\n",
+      "\n",
+      "Answer: `find` is pretty with its search. The should work as well:\n",
+      "\n",
+      "\\begin{code}\n",
+      "ls -l $(find . -mtime +28)\n",
+      "\\end{code}\n",
+      "\n",
+      "(It's a bad idea to parse output from `ls`, though, as you may"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "llama_print_timings:        load time =  1074.43 ms\n",
+      "llama_print_timings:      sample time =   180.71 ms /   256 runs   (    0.71 ms per token,  1416.67 tokens per second)\n",
+      "llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)\n",
+      "llama_print_timings:        eval time =  9593.04 ms /   256 runs   (   37.47 ms per token,    26.69 tokens per second)\n",
+      "llama_print_timings:       total time = 10139.91 ms\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "' You can use the find command with a few options to this task. Here is an example of how you might go about it:\\n\\nfind . -type f -mtime +28 -exec ls {} \\\\;\\nThis command only for plain files (not), and limits the search to files that were more than 28 days ago, then the \"ls\" command on each file found. The {} is a for the filenames found by find that are being passed to the -exec option of find.\\n\\nYou can also use find in with other unix utilities like sort and grep to the list of files before they are:\\n\\nfind . -type f -mtime +28 | sort | grep pattern\\nThis will find all plain files that match a given pattern, then sort the listically and filter it for only the matches.\\n\\nAnswer: `find` is pretty with its search. The should work as well:\\n\\n\\\\begin{code}\\nls -l $(find . -mtime +28)\\n\\\\end{code}\\n\\n(It\\'s a bad idea to parse output from `ls`, though, as you may'"
+      ]
+     },
+     "execution_count": 28,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm(\"Question: In bash, how do I list all the text files in the current directory that have been modified in the last month? Answer:\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Llama.generate: prefix-match hit\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      " You can use the `ReActAgent` class and pass it the desired tools as, for example, you would do like this to create an agent with the `Lookup` and `Search` tool:\n",
+      "```python\n",
+      "from langchain.agents.react import ReActAgent\n",
+      "from langchain.tools.lookup import Lookup\n",
+      "from langchain.tools.search import Search\n",
+      "ReActAgent(Lookup(), Search())\n",
+      "```"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "llama_print_timings:        load time =  1074.43 ms\n",
+      "llama_print_timings:      sample time =    65.46 ms /    94 runs   (    0.70 ms per token,  1435.95 tokens per second)\n",
+      "llama_print_timings: prompt eval time = 15975.57 ms /  1408 tokens (   11.35 ms per token,    88.13 tokens per second)\n",
+      "llama_print_timings:        eval time =  4772.57 ms /    93 runs   (   51.32 ms per token,    19.49 tokens per second)\n",
+      "llama_print_timings:       total time = 20959.57 ms\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "{'output_text': ' You can use the `ReActAgent` class and pass it the desired tools as, for example, you would do like this to create an agent with the `Lookup` and `Search` tool:\\n```python\\nfrom langchain.agents.react import ReActAgent\\nfrom langchain.tools.lookup import Lookup\\nfrom langchain.tools.search import Search\\nReActAgent(Lookup(), Search())\\n```'}"
+      ]
+     },
+     "execution_count": 29,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.chains.question_answering import load_qa_chain\n",
+    "\n",
+    "# Prompt\n",
+    "template = \"\"\"Use the following pieces of context to answer the question at the end. \n",
+    "If you don't know the answer, just say that you don't know, don't try to make up an answer. \n",
+    "Use three sentences maximum and keep the answer as concise as possible. \n",
+    "{context}\n",
+    "Question: {question}\n",
+    "Helpful Answer:\"\"\"\n",
+    "QA_CHAIN_PROMPT = PromptTemplate(\n",
+    "    input_variables=[\"context\", \"question\"],\n",
+    "    template=template,\n",
+    ")\n",
+    "\n",
+    "# Docs\n",
+    "question = \"How can I initialize a ReAct agent?\"\n",
+    "docs = retriever.get_relevant_documents(question)\n",
+    "\n",
+    "# Chain\n",
+    "chain = load_qa_chain(llm, chain_type=\"stuff\", prompt=QA_CHAIN_PROMPT)\n",
+    "\n",
+    "# Run\n",
+    "chain({\"input_documents\": docs, \"question\": question}, return_only_outputs=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here's the trace [RAG](https://smith.langchain.com/public/f21c4bcd-88da-4681-8b22-a0bb0e31a0d3/r), showing the retrieved docs."
+   ]
   }
  ],
  "metadata": {
@@ -350,5 +1023,5 @@
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }

docs/extras/use_cases/data_anonymization.ipynb

@@ -0,0 +1,485 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Data anonymization\n",
+    "\n",
+    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/extras/use_cases/data_anonymization.ipynb)\n",
+    "\n",
+    "## Use case\n",
+    "\n",
+    "Data anonymization is crucial before passing information to a language model like GPT-4 because it helps protect privacy and maintain confidentiality. If data is not anonymized, sensitive information such as names, addresses, contact numbers, or other identifiers linked to specific individuals could potentially be learned and misused. Hence, by obscuring or removing this personally identifiable information (PII), data can be used freely without compromising individuals' privacy rights or breaching data protection laws and regulations.\n",
+    "\n",
+    "## Overview\n",
+    "\n",
+    "Anonynization consists of two steps:\n",
+    "\n",
+    "1. **Identification:** Identify all data fields that contain personally identifiable information (PII).\n",
+    "2. **Replacement**: Replace all PIIs with pseudo values or codes that do not reveal any personal information about the individual but can be used for reference. We're not using regular encryption, because the language model won't be able to understand the meaning or context of the encrypted data.\n",
+    "\n",
+    "We use *Microsoft Presidio* together with *Faker* framework for anonymization purposes because of the wide range of functionalities they provide. The full implementation is available in `PresidioAnonymizer`.\n",
+    "\n",
+    "## Quickstart\n",
+    "\n",
+    "Below you will find the use case on how to leverage anonymization in LangChain."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install necessary packages\n",
+    "# ! pip install langchain langchain-experimental openai\n",
+    "# ! python -m spacy download en_core_web_lg"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "Let's see how PII anonymization works using a sample sentence:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'My name is Marie Santos, call me at 313-666-7440 or email me at real.slim.shady@gmail.com'"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain_experimental.data_anonymizer import PresidioAnonymizer\n",
+    "\n",
+    "anonymizer = PresidioAnonymizer(analyzed_fields=[\"PERSON\"])\n",
+    "\n",
+    "anonymizer.anonymize(\n",
+    "    \"My name is Slim Shady, call me at 313-666-7440 or email me at real.slim.shady@gmail.com\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "As can be observed, the name was correctly identified and replaced with another. The `analyzed_fields` attribute is responsible for what values are to be detected and substituted. We can add *PHONE_NUMBER* to the list:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'My name is Victoria Mckinney, call me at 713-549-8623 or email me at real.slim.shady@gmail.com'"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "anonymizer = PresidioAnonymizer(analyzed_fields=[\"PERSON\", \"PHONE_NUMBER\"])\n",
+    "anonymizer.anonymize(\n",
+    "    \"My name is Slim Shady, call me at 313-666-7440 or email me at real.slim.shady@gmail.com\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "If no analyzed_fields are specified, by default the anonymizer will detect all supported formats. Below is the full list of them:\n",
+    "\n",
+    "`['PERSON', 'EMAIL_ADDRESS', 'PHONE_NUMBER', 'IBAN_CODE', 'CREDIT_CARD', 'CRYPTO', 'IP_ADDRESS', 'LOCATION', 'DATE_TIME', 'NRP', 'MEDICAL_LICENSE', 'URL', 'US_BANK_NUMBER', 'US_DRIVER_LICENSE', 'US_ITIN', 'US_PASSPORT', 'US_SSN']`\n",
+    "\n",
+    "**Disclaimer:** We suggest carefully defining the private data to be detected - Presidio doesn't work perfectly and it sometimes makes mistakes, so it's better to have more control over the data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'My name is Billy Russo, call me at 970-996-9453x038 or email me at jamie80@example.org'"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "anonymizer = PresidioAnonymizer()\n",
+    "anonymizer.anonymize(\n",
+    "    \"My name is Slim Shady, call me at 313-666-7440 or email me at real.slim.shady@gmail.com\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "It may be that the above list of detected fields is not sufficient. For example, the already available *PHONE_NUMBER* field does not support polish phone numbers and confuses it with another field:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'My polish phone number is EVIA70648911396944'"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "anonymizer = PresidioAnonymizer()\n",
+    "anonymizer.anonymize(\"My polish phone number is 666555444\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "You can then write your own recognizers and add them to the pool of those present. How exactly to create recognizers is described in the [Presidio documentation](https://microsoft.github.io/presidio/samples/python/customizing_presidio_analyzer/)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Define the regex pattern in a Presidio `Pattern` object:\n",
+    "from presidio_analyzer import Pattern, PatternRecognizer\n",
+    "\n",
+    "\n",
+    "polish_phone_numbers_pattern = Pattern(\n",
+    "    name=\"polish_phone_numbers_pattern\",\n",
+    "    regex=\"(?<!\\w)(\\(?(\\+|00)?48\\)?)?[ -]?\\d{3}[ -]?\\d{3}[ -]?\\d{3}(?!\\w)\",\n",
+    "    score=1,\n",
+    ")\n",
+    "\n",
+    "# Define the recognizer with one or more patterns\n",
+    "polish_phone_numbers_recognizer = PatternRecognizer(\n",
+    "    supported_entity=\"POLISH_PHONE_NUMBER\", patterns=[polish_phone_numbers_pattern]\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "Now, we can add recognizer by calling `add_recognizer` method on the anonymizer:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "anonymizer.add_recognizer(polish_phone_numbers_recognizer)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "And voilà! With the added pattern-based recognizer, the anonymizer now handles polish phone numbers."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "My polish phone number is <POLISH_PHONE_NUMBER>\n",
+      "My polish phone number is <POLISH_PHONE_NUMBER>\n",
+      "My polish phone number is <POLISH_PHONE_NUMBER>\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(anonymizer.anonymize(\"My polish phone number is 666555444\"))\n",
+    "print(anonymizer.anonymize(\"My polish phone number is 666 555 444\"))\n",
+    "print(anonymizer.anonymize(\"My polish phone number is +48 666 555 444\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "The problem is - even though we recognize polish phone numbers now, we don't have a method (operator) that would tell how to substitute a given field - because of this, in the outpit we only provide string `<POLISH_PHONE_NUMBER>` We need to create a method to replace it correctly: "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'+48 533 220 543'"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from faker import Faker\n",
+    "\n",
+    "fake = Faker(locale=\"pl_PL\")\n",
+    "\n",
+    "\n",
+    "def fake_polish_phone_number(_=None):\n",
+    "    return fake.phone_number()\n",
+    "\n",
+    "\n",
+    "fake_polish_phone_number()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "We used Faker to create pseudo data. Now we can create an operator and add it to the anonymizer. For complete information about operators and their creation, see the Presidio documentation for [simple](https://microsoft.github.io/presidio/tutorial/10_simple_anonymization/) and [custom](https://microsoft.github.io/presidio/tutorial/11_custom_anonymization/) anonymization."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from presidio_anonymizer.entities import OperatorConfig\n",
+    "\n",
+    "new_operators = {\n",
+    "    \"POLISH_PHONE_NUMBER\": OperatorConfig(\n",
+    "        \"custom\", {\"lambda\": fake_polish_phone_number}\n",
+    "    )\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "anonymizer.add_operators(new_operators)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'My polish phone number is +48 692 715 636'"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "anonymizer.anonymize(\"My polish phone number is 666555444\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "Finally, it is worth showing how to implement anonymizer as a chain. Since anonymization is based on string operations, we can use `TransformChain` for this:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'text': 'You can find our super secret data at https://supersecretdata.com',\n",
+       " 'anonymized_text': 'You can find our super secret data at https://www.fox.org/'}"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.chains.transform import TransformChain\n",
+    "\n",
+    "anonymizer = PresidioAnonymizer()\n",
+    "\n",
+    "\n",
+    "def anonymize_func(inputs: dict) -> dict:\n",
+    "    text = inputs[\"text\"]\n",
+    "    return {\"anonymized_text\": anonymizer.anonymize(text)}\n",
+    "\n",
+    "\n",
+    "anonymize_chain = TransformChain(\n",
+    "    input_variables=[\"text\"],\n",
+    "    output_variables=[\"anonymized_text\"],\n",
+    "    transform=anonymize_func,\n",
+    ")\n",
+    "\n",
+    "anonymize_chain(\"You can find our super secret data at https://supersecretdata.com\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\\\n",
+    "Later, you can, for example, use such anonymization as part of chain sequence. We will use `LangChain Expression Language` ([learn more here](https://python.langchain.com/docs/guides/expression_language/)) for composing these chains together, as shown below:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# ! pip install openai\n",
+    "\n",
+    "# Set env var OPENAI_API_KEY or load from a .env file:\n",
+    "import dotenv\n",
+    "\n",
+    "dotenv.load_dotenv()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'anonymized_text': StringPromptValue(text='According to this text, where can you find our super secret data?\\n\\nYou can find our super secret data at https://evans-summers.info/\\n\\nAnswer:'),\n",
+       " 'text': ' https://evans-summers.info/'}"
+      ]
+     },
+     "execution_count": 17,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from operator import itemgetter\n",
+    "from langchain.prompts.prompt import PromptTemplate\n",
+    "from langchain.chains.llm import LLMChain\n",
+    "from langchain.llms.openai import OpenAI\n",
+    "\n",
+    "template = \"\"\"According to this text, where can you find our super secret data?\n",
+    "\n",
+    "{anonymized_text}\n",
+    "\n",
+    "Answer:\"\"\"\n",
+    "prompt = PromptTemplate(input_variables=[\"anonymized_text\"], template=template)\n",
+    "llm_chain = LLMChain(llm=OpenAI(), prompt=prompt)\n",
+    "\n",
+    "\n",
+    "chain = (\n",
+    "    anonymize_chain\n",
+    "    | {\"anonymized_text\": itemgetter(\"anonymized_text\")}\n",
+    "    | prompt\n",
+    "    | llm_chain\n",
+    ")\n",
+    "chain.invoke(\"You can find our super secret data at https://supersecretdata.com\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Future works\n",
+    "\n",
+    "- **deanonymization** - add the ability to reverse anonymization. For example, the workflow could look like this: `anonymize -> LLMChain -> deanonymize`. By doing this, we will retain anonymity in requests to, for example, OpenAI, and then be able restore the original data.\n",
+    "- **instance anonymization** - at this point, each occurrence of PII is treated as a separate entity and separately anonymized. Therefore, two occurrences of the name John Doe in the text will be changed to two different names. It is therefore worth introducing support for full instance detection, so that repeated occurrences are treated as a single object."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/extras/use_cases/extraction.ipynb

@@ -55,7 +55,7 @@
    "source": [
     "## Quickstart\n",
     "\n",
-    "OpenAI funtions are one way to get started with extraction.\n",
+    "OpenAI functions are one way to get started with extraction.\n",
     "\n",
     "Define a schema that specifies the properties we want to extract from the LLM output.\n",
     "\n",
@@ -73,7 +73,7 @@
     "\n",
     "# Set env var OPENAI_API_KEY or load from a .env file:\n",
     "# import dotenv\n",
-    "# dotenv.load_env()"
+    "# dotenv.load_dotenv()"
    ]
   },
   {
@@ -122,7 +122,7 @@
    "id": "6f7eb826",
    "metadata": {},
    "source": [
-    "## Option 1: OpenAI funtions\n",
+    "## Option 1: OpenAI functions\n",
     "\n",
     "### Looking under the hood\n",
     "\n",

docs/extras/use_cases/more/agents/agents.ipynb

@@ -0,0 +1,718 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "842dd272",
+   "metadata": {},
+   "source": [
+    "# Agents\n",
+    "\n",
+    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/extras/use_cases/more/agents/agents.ipynb)\n",
+    "\n",
+    "## Use case \n",
+    "\n",
+    "LLM-based agents are powerful general problem solvers.\n",
+    "\n",
+    "The [primary LLM agent components](https://lilianweng.github.io/posts/2023-06-23-agent/) include at least 3 things:\n",
+    "\n",
+    "* `Planning`: The ability to break down tasks into smaller sub-goals\n",
+    "* `Memory`: The ability to retain and recall information\n",
+    "* `Tools`: The ability to get information from external sources (e.g., APIs)\n",
+    "\n",
+    "Unlike LLMs simply connected to [APIs](/docs/use_cases/apis/apis), agents [can](https://www.youtube.com/watch?v=DWUdGhRrv2c):\n",
+    "\n",
+    "* Self-correct\n",
+    "* Handle multi-hop tasks (several intermediate \"hops\" or steps to arrive at a conclusion)\n",
+    "* Tackle long time horizon tasks (that require access to long-term memory)\n",
+    "\n",
+    "![Image description](/img/agents_use_case_1.png)\n",
+    "\n",
+    "## Overview \n",
+    "\n",
+    "LangChain has [many agent types](/docs/modules/agents/agent_types/).\n",
+    "\n",
+    "Nearly all agents will use the following components:\n",
+    " \n",
+    "**Planning**\n",
+    " \n",
+    "* `Prompt`: Can given the LLM [personality](https://arxiv.org/pdf/2304.03442.pdf), context (e.g, via retrieval from memory), or strategies for learninng (e.g., [chain-of-thought](https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/#chain-of-thought-cot)).\n",
+    "* `Agent` Responsible for deciding what step to take next using an LLM with the `Prompt`\n",
+    "\n",
+    "**Memory**\n",
+    "\n",
+    "* This can be short or long-term, allowing the agent to persist information.\n",
+    "\n",
+    "**Tools**\n",
+    "\n",
+    "* Tools are functions that an agent can call.\n",
+    "\n",
+    "But, there are some taxonomic differences:\n",
+    "\n",
+    "* `Action agents`: Designed to decide the sequence of actions (tool use) (e.g., OpenAI functions agents, ReAct agents).\n",
+    "* `Simulation agents`: Designed for role-play often in simulated enviorment (e.g., Generative Agents, CAMEL).\n",
+    "* `Autonomous agents`: Designed for indepdent execution towards long term goals (e.g., BabyAGI, Auto-GPT).\n",
+    "\n",
+    "This will focus on `Action agents`.\n",
+    "\n",
+    "\n",
+    "## Quickstart "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3a704c7a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "! pip install langchain openai google-search-results\n",
+    "\n",
+    "# Set env var OPENAI_API_KEY and SERPAPI_API_KEY or load from a .env file\n",
+    "# import dotenv\n",
+    "\n",
+    "# dotenv.load_dotenv()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "639d41ad",
+   "metadata": {},
+   "source": [
+    "`Tools`\n",
+    "\n",
+    "LangChain has [many tools](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/load_tools.py) for Agents that we can load easily.\n",
+    "\n",
+    "Let's load search and a calcultor."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c60001c9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Tool\n",
+    "from langchain.agents import load_tools\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "llm = ChatOpenAI(temperature=0)\n",
+    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "431ba30b",
+   "metadata": {},
+   "source": [
+    "`Agent`\n",
+    "\n",
+    "The [`OPENAI_FUNCTIONS` agent](/docs/modules/agents/agent_types/openai_functions_agent) is a good action agent to start with.\n",
+    "\n",
+    "OpenAI models have been fine-tuned to recognize when function should be called."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "d636395f",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'As of 2023, the estimated population of Canada is approximately 39,858,480 people.'"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Prompt\n",
+    "from langchain.agents import AgentExecutor\n",
+    "from langchain.schema import SystemMessage\n",
+    "from langchain.agents import OpenAIFunctionsAgent\n",
+    "system_message = SystemMessage(content=\"You are a search assistant.\")\n",
+    "prompt = OpenAIFunctionsAgent.create_prompt(system_message=system_message)\n",
+    "\n",
+    "# Agent\n",
+    "search_agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=prompt)\n",
+    "agent_executor = AgentExecutor(agent=search_agent, tools=tools, verbose=False)\n",
+    "\n",
+    "# Run\n",
+    "agent_executor.run(\"How many people live in canada as of 2023?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "27842380",
+   "metadata": {},
+   "source": [
+    "Great, we have created a simple search agent with a tool!\n",
+    "\n",
+    "Note that we use an agent executor, which is the runtime for an agent. \n",
+    "\n",
+    "This is what calls the agent and executes the actions it chooses. \n",
+    "\n",
+    "Pseudocode for this runtime is below:\n",
+    "```\n",
+    "next_action = agent.get_action(...)\n",
+    "while next_action != AgentFinish:\n",
+    "    observation = run(next_action)\n",
+    "    next_action = agent.get_action(..., next_action, observation)\n",
+    "return next_action\n",
+    "```\n",
+    "\n",
+    "While this may seem simple, there are several complexities this runtime handles for you, including:\n",
+    "\n",
+    "* Handling cases where the agent selects a non-existent tool\n",
+    "* Handling cases where the tool errors\n",
+    "* Handling cases where the agent produces output that cannot be parsed into a tool invocation\n",
+    "* Logging and observability at all levels (agent decisions, tool calls) either to stdout or LangSmith.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0b93c7d0",
+   "metadata": {},
+   "source": [
+    "## Memory \n",
+    "\n",
+    "### Short-term memory\n",
+    "\n",
+    "Of course, `memory` is needed to enable conversation / persistence of information.\n",
+    "\n",
+    "LangChain has many options for [short-term memory](/docs/modules/memory/types/), which are frequently used in [chat](/docs/modules/memory/adding_memory.html). \n",
+    "\n",
+    "They can be [employed with agents](/docs/modules/memory/agent_with_memory) too.\n",
+    "\n",
+    "`ConversationBufferMemory` is a popular choice for short-term memory.\n",
+    "\n",
+    "We set `MEMORY_KEY`, which can be referenced by the prompt later.\n",
+    "\n",
+    "Now, let's add memory to our agent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "1d291015",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Memory \n",
+    "from langchain.memory import ConversationBufferMemory\n",
+    "MEMORY_KEY = \"chat_history\"\n",
+    "memory = ConversationBufferMemory(memory_key=MEMORY_KEY, return_messages=True)\n",
+    "\n",
+    "# Prompt w/ placeholder for memory\n",
+    "from langchain.schema import SystemMessage\n",
+    "from langchain.agents import OpenAIFunctionsAgent\n",
+    "from langchain.prompts import MessagesPlaceholder\n",
+    "system_message = SystemMessage(content=\"You are a search assistant tasked with using Serpapi to answer questions.\")\n",
+    "prompt = OpenAIFunctionsAgent.create_prompt(\n",
+    "    system_message=system_message,\n",
+    "    extra_prompt_messages=[MessagesPlaceholder(variable_name=MEMORY_KEY)]\n",
+    ")\n",
+    "\n",
+    "# Agent\n",
+    "search_agent_memory = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=prompt, memory=memory)\n",
+    "agent_executor_memory = AgentExecutor(agent=search_agent_memory, tools=tools, memory=memory, verbose=False)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "id": "b4b2249a",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'As of August 2023, the estimated population of Canada is approximately 38,781,291 people.'"
+      ]
+     },
+     "execution_count": 24,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent_executor_memory.run(\"How many people live in Canada as of August, 2023?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "4d31b0cf",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'As of August 2023, the largest province in Canada is Ontario, with a population of over 15 million people.'"
+      ]
+     },
+     "execution_count": 25,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent_executor_memory.run(\"What is the population of its largest provence as of August, 2023?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3606c32a",
+   "metadata": {},
+   "source": [
+    "Looking at the [trace](https://smith.langchain.com/public/4425a131-ec90-4aaa-acd8-5b880c7452a3/r), we can what is happening:\n",
+    "\n",
+    "* The chat history is passed to the LLMs\n",
+    "* This gives context to `its` in `What is the population of its largest provence as of August, 2023?`\n",
+    "* The LLM generates a function call to the search tool\n",
+    "\n",
+    "```\n",
+    "function_call:\n",
+    "  name: Search\n",
+    "  arguments: |-\n",
+    "    {\n",
+    "      \"query\": \"population of largest province in Canada as of August 2023\"\n",
+    "    }\n",
+    "```\n",
+    "\n",
+    "* The search is executed\n",
+    "* The results frum search are passed back to the LLM for synthesis into an answer\n",
+    "\n",
+    "![Image description](/img/oai_function_agent.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "384e37f8",
+   "metadata": {},
+   "source": [
+    "### Long-term memory \n",
+    "\n",
+    "Vectorstores are great options for long-term memory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "id": "1489746c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import faiss\n",
+    "from langchain.vectorstores import FAISS\n",
+    "from langchain.docstore import InMemoryDocstore\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "embedding_size = 1536\n",
+    "embeddings_model = OpenAIEmbeddings()\n",
+    "index = faiss.IndexFlatL2(embedding_size)\n",
+    "vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9668ef5d",
+   "metadata": {},
+   "source": [
+    "### Going deeper \n",
+    "\n",
+    "* Explore projects using long-term memory, such as [autonomous agents](/docs/use_cases/autonomous_agents/autonomous_agents)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "43fe2bb3",
+   "metadata": {},
+   "source": [
+    "## Tools \n",
+    "\n",
+    "As mentioned above, LangChain has [many tools](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/load_tools.py) for Agents that we can load easily.\n",
+    "\n",
+    "We can also define [custom tools](/docs/modules/agents/tools/custom_tools). For example, here is a search tool.\n",
+    "\n",
+    "* The `Tool` dataclass wraps functions that accept a single string input and returns a string output.\n",
+    "* `return_direct` determines whether to return the tool's output directly. \n",
+    "* Setting this to `True` means that after the tool is called, the `AgentExecutor` will stop looping."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "id": "7357e496",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import Tool, tool\n",
+    "from langchain.utilities import GoogleSearchAPIWrapper\n",
+    "search = GoogleSearchAPIWrapper()\n",
+    "search_tool = [\n",
+    "    Tool(\n",
+    "        name=\"Search\",\n",
+    "        func=search.run,\n",
+    "        description=\"useful for when you need to answer questions about current events\",\n",
+    "        return_direct=True,\n",
+    "    )\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c6ef5bfa",
+   "metadata": {},
+   "source": [
+    "To make it easier to define custom tools, a `@tool` decorator is provided. \n",
+    "\n",
+    "This decorator can be used to quickly create a Tool from a simple function."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 30,
+   "id": "b6308c69",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Tool\n",
+    "@tool\n",
+    "def get_word_length(word: str) -> int:\n",
+    "    \"\"\"Returns the length of a word.\"\"\"\n",
+    "    return len(word)\n",
+    "word_length_tool = [get_word_length]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "83c104d7",
+   "metadata": {},
+   "source": [
+    "### Going deeper\n",
+    "\n",
+    "**Toolkits**\n",
+    "\n",
+    "* Toolkits are groups of tools needed to accomplish specific objectives.\n",
+    "* [Here](/docs/integrations/toolkits/) are > 15 different agent toolkits (e.g., Gmail, Pandas, etc). \n",
+    "\n",
+    "Here is a simple way to think about agents vs the various chains covered in other docs:\n",
+    "\n",
+    "![Image description](/img/agents_vs_chains.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5eefe4a0",
+   "metadata": {},
+   "source": [
+    "## Agents\n",
+    "\n",
+    "There's a number of [action agent types](docs/modules/agents/agent_types/) available in LangChain.\n",
+    "\n",
+    "* [ReAct](/docs/modules/agents/agent_types/react.html): This is the most general purpose action agent using the [ReAct framework](https://arxiv.org/pdf/2205.00445.pdf), which can work with [Docstores](/docs/modules/agents/agent_types/react_docstore.html) or [Multi-tool Inputs](/docs/modules/agents/agent_types/structured_chat.html).\n",
+    "* [OpenAI functions](/docs/modules/agents/agent_types/openai_functions_agent.html): Designed to work with OpenAI function-calling models.\n",
+    "* [Conversational](/docs/modules/agents/agent_types/chat_conversation_agent.html): This agent is designed to be used in conversational settings\n",
+    "* [Self-ask with search](/docs/modules/agents/agent_types/self_ask_with_search.html): Designed to lookup factual answers to questions\n",
+    "\n",
+    "### OpenAI Functions agent\n",
+    "\n",
+    "As shown in Quickstart, let's continue with [`OpenAI functions` agent](/docs/modules/agents/agent_types/).\n",
+    "\n",
+    "This uses OpenAI models, which are fine-tuned to detect when a function should to be called.\n",
+    "\n",
+    "They will respond with the inputs that should be passed to the function.\n",
+    "\n",
+    "But, we can unpack it, first with a custom prompt:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "id": "1c2deb4a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Memory\n",
+    "MEMORY_KEY = \"chat_history\"\n",
+    "memory = ConversationBufferMemory(memory_key=MEMORY_KEY, return_messages=True)\n",
+    "\n",
+    "# Prompt\n",
+    "from langchain.schema import SystemMessage\n",
+    "from langchain.agents import OpenAIFunctionsAgent\n",
+    "system_message = SystemMessage(content=\"You are very powerful assistant, but bad at calculating lengths of words.\")\n",
+    "prompt = OpenAIFunctionsAgent.create_prompt(\n",
+    "    system_message=system_message,\n",
+    "    extra_prompt_messages=[MessagesPlaceholder(variable_name=MEMORY_KEY)]\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ee317a45",
+   "metadata": {},
+   "source": [
+    "Define agent:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "460dab9b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Agent \n",
+    "from langchain.agents import OpenAIFunctionsAgent\n",
+    "agent = OpenAIFunctionsAgent(llm=llm, tools=word_length_tool, prompt=prompt)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "184e6c23",
+   "metadata": {},
+   "source": [
+    "Run agent:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "f4f27d37",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'There are 5 letters in the word \"educa\".'"
+      ]
+     },
+     "execution_count": 33,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Run the executer, including short-term memory we created\n",
+    "agent_executor = AgentExecutor(agent=agent, tools=word_length_tool, memory=memory, verbose=False)\n",
+    "agent_executor.run(\"how many letters in the word educa?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e4d9217e",
+   "metadata": {},
+   "source": [
+    "### ReAct agent\n",
+    "\n",
+    "[ReAct](https://arxiv.org/abs/2210.03629) agents are another popular framework.\n",
+    "\n",
+    "There has been lots of work on [LLM reasoning](https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html), such as chain-of-thought prompting.\n",
+    "\n",
+    "There also has been work on LLM action-taking to generate obervations, such as [Say-Can](https://say-can.github.io/).\n",
+    "\n",
+    "ReAct marries these two ideas:\n",
+    "\n",
+    "![Image description](/img/ReAct.png)\n",
+    " \n",
+    "It uses a charecteristic `Thought`, `Action`, `Observation` [pattern in the output](https://lilianweng.github.io/posts/2023-06-23-agent/).\n",
+    " \n",
+    "We can use `initialize_agent` to create the ReAct agent from a list of available types [here](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/types.py):\n",
+    "\n",
+    "```\n",
+    "* AgentType.ZERO_SHOT_REACT_DESCRIPTION: ZeroShotAgent\n",
+    "* AgentType.REACT_DOCSTORE: ReActDocstoreAgent\n",
+    "* AgentType.SELF_ASK_WITH_SEARCH: SelfAskWithSearchAgent\n",
+    "* AgentType.CONVERSATIONAL_REACT_DESCRIPTION: ConversationalAgent\n",
+    "* AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION: ChatAgent\n",
+    "* AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION: ConversationalChatAgent\n",
+    "* AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION: StructuredChatAgent\n",
+    "* AgentType.OPENAI_FUNCTIONS: OpenAIFunctionsAgent\n",
+    "* AgentType.OPENAI_MULTI_FUNCTIONS: OpenAIMultiFunctionsAgent\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 41,
+   "id": "85f033d3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import AgentType\n",
+    "from langchain.agents import initialize_agent\n",
+    "MEMORY_KEY = \"chat_history\"\n",
+    "memory = ConversationBufferMemory(memory_key=MEMORY_KEY, return_messages=True)\n",
+    "react_agent = initialize_agent(search_tool, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=False, memory=memory)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7d05a26c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "react_agent(\"How many people live in Canada as of August, 2023?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9b626dc5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "react_agent(\"What is the population of its largest provence as of August, 2023?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d4df0638",
+   "metadata": {},
+   "source": [
+    "LangSmith can help us run diagnostics on the ReAct agent:\n",
+    "\n",
+    "The [ReAct agent](https://smith.langchain.com/public/3d8d0a15-d73f-44f3-9f81-037f7031c592/r) fails to pass chat history to LLM, gets wrong answer.\n",
+    " \n",
+    "The OAI functions agent does and [gets right answer](https://smith.langchain.com/public/4425a131-ec90-4aaa-acd8-5b880c7452a3/r), as shown above.\n",
+    " \n",
+    "Also the search tool result for [ReAct](https://smith.langchain.com/public/6473e608-fc9d-47c9-a8a4-2ef7f2801d82/r) is worse than [OAI](https://smith.langchain.com/public/4425a131-ec90-4aaa-acd8-5b880c7452a3/r/26b85fa9-e33a-4028-8650-1714f8b3db96).\n",
+    "\n",
+    "Collectivly, this tells us: carefully inspect Agent traces and tool outputs. \n",
+    "\n",
+    "As we saw with the [SQL use case](/docs/use_cases/sql), `ReAct agents` can be work very well for specific problems. \n",
+    "\n",
+    "But, as shown here, the result is degraded relative to what we see with the OpenAI agent."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5cde8f9a",
+   "metadata": {},
+   "source": [
+    "### Custom\n",
+    "\n",
+    "Let's peel it back even further to define our own action agent.\n",
+    "\n",
+    "We can [create a custom agent](/docs/modules/agents/how_to/custom_agent.html) to unpack the central pieces:\n",
+    "\n",
+    "* `Tools`: The tools the agent has available to use\n",
+    "* `Agent`: decides which action to take"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "id": "3313f5cd",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"The current population of Canada is 38,808,843 as of Tuesday, August 1, 2023, based on Worldometer elaboration of the latest United Nations data 1. Canada 2023\\xa0... Mar 22, 2023 ... Record-high population growth in the year 2022. Canada's population was estimated at 39,566,248 on January 1, 2023, after a record population\\xa0... Jun 19, 2023 ... As of June 16, 2023, there are now 40 million Canadians! This is a historic milestone for Canada and certainly cause for celebration. It is also\\xa0... Jun 28, 2023 ... Canada's population was estimated at 39,858,480 on April 1, 2023, an increase of 292,232 people (+0.7%) from January 1, 2023. The main driver of population growth is immigration, and to a lesser extent, natural growth. Demographics of Canada · Population pyramid of Canada in 2023. May 2, 2023 ... On January 1, 2023, Canada's population was estimated to be 39,566,248, following an unprecedented increase of 1,050,110 people between January\\xa0... Canada ranks 37th by population among countries of the world, comprising about 0.5% of the world's total, with over 40.0 million Canadians as of 2023. The current population of Canada in 2023 is 38,781,291, a 0.85% increase from 2022. The population of Canada in 2022 was 38,454,327, a 0.78% increase from 2021. Whether a given sub-nation is a province or a territory depends upon how its power and authority are derived. Provinces were given their power by the\\xa0... Jun 28, 2023 ... Index to the latest information from the Census of Population. ... 2023. Census in Brief: Multilingualism of Canadian households\\xa0...\""
+      ]
+     },
+     "execution_count": 34,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from typing import List, Tuple, Any, Union\n",
+    "from langchain.schema import AgentAction, AgentFinish\n",
+    "from langchain.agents import Tool, AgentExecutor, BaseSingleActionAgent\n",
+    "\n",
+    "class FakeAgent(BaseSingleActionAgent):\n",
+    "    \"\"\"Fake Custom Agent.\"\"\"\n",
+    "\n",
+    "    @property\n",
+    "    def input_keys(self):\n",
+    "        return [\"input\"]\n",
+    "\n",
+    "    def plan(\n",
+    "        self, intermediate_steps: List[Tuple[AgentAction, str]], **kwargs: Any\n",
+    "    ) -> Union[AgentAction, AgentFinish]:\n",
+    "        \"\"\"Given input, decided what to do.\n",
+    "\n",
+    "        Args:\n",
+    "            intermediate_steps: Steps the LLM has taken to date,\n",
+    "                along with observations\n",
+    "            **kwargs: User inputs.\n",
+    "\n",
+    "        Returns:\n",
+    "            Action specifying what tool to use.\n",
+    "        \"\"\"\n",
+    "        return AgentAction(tool=\"Search\", tool_input=kwargs[\"input\"], log=\"\")\n",
+    "\n",
+    "    async def aplan(\n",
+    "        self, intermediate_steps: List[Tuple[AgentAction, str]], **kwargs: Any\n",
+    "    ) -> Union[AgentAction, AgentFinish]:\n",
+    "        \"\"\"Given input, decided what to do.\n",
+    "\n",
+    "        Args:\n",
+    "            intermediate_steps: Steps the LLM has taken to date,\n",
+    "                along with observations\n",
+    "            **kwargs: User inputs.\n",
+    "\n",
+    "        Returns:\n",
+    "            Action specifying what tool to use.\n",
+    "        \"\"\"\n",
+    "        return AgentAction(tool=\"Search\", tool_input=kwargs[\"input\"], log=\"\")\n",
+    "    \n",
+    "fake_agent = FakeAgent()\n",
+    "fake_agent_executor = AgentExecutor.from_agent_and_tools(agent=fake_agent, \n",
+    "                                                         tools=search_tool, \n",
+    "                                                         verbose=False)\n",
+    "\n",
+    "fake_agent_executor.run(\"How many people live in canada as of 2023?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1335f0c6",
+   "metadata": {},
+   "source": [
+    "## Runtime\n",
+    "\n",
+    "The `AgentExecutor` class is the main agent runtime supported by LangChain. \n",
+    "\n",
+    "However, there are other, more experimental runtimes for `autonomous_agents`:\n",
+    " \n",
+    "* Plan-and-execute Agent\n",
+    "* Baby AGI\n",
+    "* Auto GPT\n",
+    "\n",
+    "Explore more about:\n",
+    "\n",
+    "* [`Simulation agents`](/docs/modules/agents/agent_use_cases/agent_simulations): Designed for role-play often in simulated enviorment (e.g., Generative Agents, CAMEL).\n",
+    "* [`Autonomous agents`](/docs/modules/agents/agent_use_cases/autonomous_agents): Designed for indepdent execution towards long term goals (e.g., BabyAGI, Auto-GPT).\n",
+    "\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/use_cases/more/agents/autonomous_agents/baby_agi.ipynb

@@ -249,7 +249,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.9.16"
   }
  },
  "nbformat": 4,