Filter args when function is only *args and **kwargs

Tradeoffs here:
- No lint-time checking for compatibility
- Differs from JS package
- The signature inference, etc. in the base tool isn't simple
- The `args_schema` is optional

Pros:
- Forwards compatibility retained
- Doesn't break backwards compatibility
- User doesn't have to think about which class to subclass (single base
tool or dynamic `Tool` interface regardless of input)
-  No need to change the load_tools, etc. interfaces

Co-authored-by: Hasan Patel <mangafield@gmail.com>

.gitignore

@@ -144,4 +144,8 @@ wandb/
 /.ruff_cache/
 
 *.pkl
-*.bin
+*.bin
+
+# integration test artifacts
+data_map*
+\[('_type', 'fake'), ('stop', None)]

README.md

@@ -4,6 +4,8 @@
 
 [![lint](https://github.com/hwchase17/langchain/actions/workflows/lint.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/lint.yml) [![test](https://github.com/hwchase17/langchain/actions/workflows/test.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/test.yml) [![linkcheck](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml) [![Downloads](https://static.pepy.tech/badge/langchain/month)](https://pepy.tech/project/langchain) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Twitter](https://img.shields.io/twitter/url/https/twitter.com/langchainai.svg?style=social&label=Follow%20%40LangChainAI)](https://twitter.com/langchainai) [![](https://dcbadge.vercel.app/api/server/6adMQxSpJS?compact=true&style=flat)](https://discord.gg/6adMQxSpJS)
 
+Looking for the JS/TS version? Check out [LangChain.js](https://github.com/hwchase17/langchainjs).
+
 **Production Support:** As you move your LangChains into production, we'd love to offer more comprehensive support.
 Please fill out [this form](https://forms.gle/57d8AmXBYp8PP8tZA) and we'll set up a dedicated support Slack channel.
 

docs/deployments.md

@@ -1,14 +1,10 @@
 # Deployments
 
-So you've made a really cool chain - now what? How do you deploy it and make it easily sharable with the world?
+So, you've created a really cool chain - now what? How do you deploy it and make it easily shareable with the world?
 
-This section covers several options for that.
-Note that these are meant as quick deployment options for prototypes and demos, and not for production systems.
-If you are looking for help with deployment of a production system, please contact us directly.
+This section covers several options for that. Note that these options are meant for quick deployment of prototypes and demos, not for production systems. If you need help with the deployment of a production system, please contact us directly.
 
-What follows is a list of template GitHub repositories aimed that are intended to be
-very easy to fork and modify to use your chain.
-This is far from an exhaustive list of options, and we are EXTREMELY open to contributions here.
+What follows is a list of template GitHub repositories designed to be easily forked and modified to use your chain. This list is far from exhaustive, and we are EXTREMELY open to contributions here.
 
 ## [Streamlit](https://github.com/hwchase17/langchain-streamlit-template)
 
@@ -33,6 +29,10 @@ It implements a Question Answering app and contains instructions for deploying t
 
 A minimal example on how to run LangChain on Vercel using Flask.
 
+## [Fly.io](https://github.com/fly-apps/hello-fly-langchain)
+
+A minimal example of how to deploy LangChain to [Fly.io](https://fly.io/) using Flask.
+
 ## [Digitalocean App Platform](https://github.com/homanp/digitalocean-langchain)
 
 A minimal example on how to deploy LangChain to DigitalOcean App Platform.
@@ -43,12 +43,11 @@ A minimal example on how to deploy LangChain to Google Cloud Run.
 
 ## [SteamShip](https://github.com/steamship-core/steamship-langchain/)
 
-This repository contains LangChain adapters for Steamship, enabling LangChain developers to rapidly deploy their apps on Steamship.
-This includes: production ready endpoints, horizontal scaling across dependencies, persistant storage of app state, multi-tenancy support, etc.
+This repository contains LangChain adapters for Steamship, enabling LangChain developers to rapidly deploy their apps on Steamship. This includes: production-ready endpoints, horizontal scaling across dependencies, persistent storage of app state, multi-tenancy support, etc.
 
 ## [Langchain-serve](https://github.com/jina-ai/langchain-serve)
 
-This repository allows users to serve local chains and agents as RESTful, gRPC, or Websocket APIs thanks to [Jina](https://docs.jina.ai/). Deploy your chains & agents with ease and enjoy independent scaling, serverless and autoscaling APIs, as well as a Streamlit playground on Jina AI Cloud.
+This repository allows users to serve local chains and agents as RESTful, gRPC, or WebSocket APIs, thanks to [Jina](https://docs.jina.ai/). Deploy your chains & agents with ease and enjoy independent scaling, serverless and autoscaling APIs, as well as a Streamlit playground on Jina AI Cloud.
 
 ## [BentoML](https://github.com/ssheng/BentoChain)
 
@@ -56,4 +55,4 @@ This repository provides an example of how to deploy a LangChain application wit
 
 ## [Databutton](https://databutton.com/home?new-data-app=true)
 
-These templates serve as examples of how to build, deploy, and share LangChain applications using Databutton. You can create user interfaces with Streamlit, automate tasks by scheduling Python code, and store files and data in the built-in store. Examples include Chatbot interface with conversational memory, Personal search engine, and a starter template for LangChain apps. Deploying and sharing is one click. 
+These templates serve as examples of how to build, deploy, and share LangChain applications using Databutton. You can create user interfaces with Streamlit, automate tasks by scheduling Python code, and store files and data in the built-in store. Examples include a Chatbot interface with conversational memory, a Personal search engine, and a starter template for LangChain apps. Deploying and sharing is just one click away.

docs/ecosystem/lancedb.md

@@ -0,0 +1,23 @@
+# LanceDB
+
+This page covers how to use [LanceDB](https://github.com/lancedb/lancedb) within LangChain.
+It is broken into two parts: installation and setup, and then references to specific LanceDB wrappers.
+
+## Installation and Setup
+
+- Install the Python SDK with `pip install lancedb`
+
+## Wrappers
+
+### VectorStore
+
+There exists a wrapper around LanceDB databases, allowing you to use it as a vectorstore,
+whether for semantic search or example selection.
+
+To import this vectorstore:
+
+```python
+from langchain.vectorstores import LanceDB
+```
+
+For a more detailed walkthrough of the LanceDB wrapper, see [this notebook](../modules/indexes/vectorstores/examples/lancedb.ipynb)

docs/ecosystem/metal.md

@@ -0,0 +1,26 @@
+# Metal
+
+This page covers how to use [Metal](https://getmetal.io) within LangChain.
+
+## What is Metal?
+
+Metal is a  managed retrieval & memory platform built for production. Easily index your data into `Metal` and run semantic search and retrieval on it.
+
+![Metal](../_static/MetalDash.png)
+
+## Quick start
+
+Get started by [creating a Metal account](https://app.getmetal.io/signup).
+
+Then, you can easily take advantage of the `MetalRetriever` class to start retrieving your data for semantic search, prompting context, etc. This class takes a `Metal` instance and a dictionary of parameters to pass to the Metal API.
+
+```python
+from langchain.retrievers import MetalRetriever
+from metal_sdk.metal import Metal
+
+
+metal = Metal("API_KEY", "CLIENT_ID", "INDEX_ID");
+retriever = MetalRetriever(metal, params={"limit": 2})
+
+docs = retriever.get_relevant_documents("search term")
+```

docs/ecosystem/pipelineai.md

@@ -0,0 +1,19 @@
+# PipelineAI
+
+This page covers how to use the PipelineAI ecosystem within LangChain.
+It is broken into two parts: installation and setup, and then references to specific PipelineAI wrappers.
+
+## Installation and Setup
+
+- Install with `pip install pipeline-ai`
+- Get a Pipeline Cloud api key and set it as an environment variable (`PIPELINE_API_KEY`)
+
+## Wrappers
+
+### LLM
+
+There exists a PipelineAI LLM wrapper, which you can access with
+
+```python
+from langchain.llms import PipelineAI
+```

docs/ecosystem/replicate.md

@@ -9,7 +9,7 @@ This page covers how to run models on Replicate within LangChain.
 
 Find a model on the [Replicate explore page](https://replicate.com/explore), and then paste in the model name and version in this format: `owner-name/model-name:version`
 
-For example, for this [flan-t5 model](https://replicate.com/daanelson/flan-t5), click on the API tab. The model name/version would be: `daanelson/flan-t5:04e422a9b85baed86a4f24981d7f9953e20c5fd82f6103b74ebc431588e1cec8`
+For example, for this [dolly model](https://replicate.com/replicate/dolly-v2-12b), click on the API tab. The model name/version would be: `"replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5"`
 
 Only the `model` param is required, but any other model parameters can also be passed in with the format `input={model_param: value, ...}`
 
@@ -24,7 +24,7 @@ Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6
 From here, we can initialize our model:
 
 ```python
-llm = Replicate(model="daanelson/flan-t5:04e422a9b85baed86a4f24981d7f9953e20c5fd82f6103b74ebc431588e1cec8")
+llm = Replicate(model="replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5")
 ```
 
 And run it:
@@ -40,8 +40,7 @@ llm(prompt)
 We can call any Replicate model (not just LLMs) using this syntax. For example, we can call [Stable Diffusion](https://replicate.com/stability-ai/stable-diffusion):
 
 ```python
-text2image = Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf",
-                       input={'image_dimensions'='512x512'}
+text2image = Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf", input={'image_dimensions':'512x512'})
 
 image_output = text2image("A cat riding a motorcycle by Picasso")
 ```

docs/modules/agents/tools/examples/bash.ipynb

@@ -39,11 +39,27 @@
           "name": "stdout",
           "output_type": "stream",
           "text": [
+            "apify.ipynb\n",
+            "arxiv.ipynb\n",
             "bash.ipynb\n",
+            "bing_search.ipynb\n",
+            "chatgpt_plugins.ipynb\n",
+            "ddg.ipynb\n",
+            "google_places.ipynb\n",
             "google_search.ipynb\n",
+            "google_serper.ipynb\n",
+            "gradio_tools.ipynb\n",
+            "human_tools.ipynb\n",
+            "ifttt.ipynb\n",
+            "openweathermap.ipynb\n",
             "python.ipynb\n",
             "requests.ipynb\n",
+            "search_tools.ipynb\n",
+            "searx_search.ipynb\n",
             "serpapi.ipynb\n",
+            "wikipedia.ipynb\n",
+            "wolfram_alpha.ipynb\n",
+            "zapier.ipynb\n",
             "\n"
           ]
         }
@@ -54,9 +70,94 @@
     },
     {
       "cell_type": "code",
-      "execution_count": null,
+      "execution_count": 4,
+      "id": "e7896f8e",
+      "metadata": {},
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "apify.ipynb\n",
+            "arxiv.ipynb\n",
+            "bash.ipynb\n",
+            "bing_search.ipynb\n",
+            "chatgpt_plugins.ipynb\n",
+            "ddg.ipynb\n",
+            "google_places.ipynb\n",
+            "google_search.ipynb\n",
+            "google_serper.ipynb\n",
+            "gradio_tools.ipynb\n",
+            "human_tools.ipynb\n",
+            "ifttt.ipynb\n",
+            "openweathermap.ipynb\n",
+            "python.ipynb\n",
+            "requests.ipynb\n",
+            "search_tools.ipynb\n",
+            "searx_search.ipynb\n",
+            "serpapi.ipynb\n",
+            "wikipedia.ipynb\n",
+            "wolfram_alpha.ipynb\n",
+            "zapier.ipynb\n",
+            "\n"
+          ]
+        }
+      ],
+      "source": [
+        "bash.run(\"cd ..\")\n",
+        "# The commands are executed in a new subprocess each time, meaning that\n",
+        "# this call will return the same results as the last.\n",
+        "print(bash.run(\"ls\"))"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
       "id": "851fee9f",
       "metadata": {},
+      "source": [
+        "## Terminal Persistance\n",
+        "\n",
+        "By default, the bash command will be executed in a new subprocess each time. To retain a persistent bash session, we can use the `persistent=True` arg."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 5,
+      "id": "4a93ea2c",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "bash = BashProcess(persistent=True)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 6,
+      "id": "a1e98b78",
+      "metadata": {},
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "custom_tools.ipynb\t\tmulti_input_tool.ipynb\n",
+            "examples\t\t\ttool_input_validation.ipynb\n",
+            "getting_started.md\n"
+          ]
+        }
+      ],
+      "source": [
+        "bash.run(\"cd ..\")\n",
+        "# Note the list of files is different\n",
+        "print(bash.run(\"ls\"))"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "e13c1c9c",
+      "metadata": {},
       "outputs": [],
       "source": []
     }
@@ -77,7 +178,7 @@
       "name": "python",
       "nbconvert_exporter": "python",
       "pygments_lexer": "ipython3",
-      "version": "3.10.9"
+      "version": "3.8.16"
     }
   },
   "nbformat": 4,

docs/modules/agents/tools/examples/ddg.ipynb

@@ -27,7 +27,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.tools import DuckDuckGoSearchTool"
+    "from langchain.tools import DuckDuckGoSearchRun"
    ]
   },
   {
@@ -37,7 +37,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "search = DuckDuckGoSearchTool()"
+    "search = DuckDuckGoSearchRun()"
    ]
   },
   {

docs/modules/chains/examples/llm_bash.ipynb

@@ -24,8 +24,8 @@
       "\n",
       "```bash\n",
       "echo \"Hello World\"\n",
-      "```\u001b[0m['```bash', 'echo \"Hello World\"', '```']\n",
-      "\n",
+      "```\u001b[0m\n",
+      "Code: \u001b[33;1m\u001b[1;3m['echo \"Hello World\"']\u001b[0m\n",
       "Answer: \u001b[33;1m\u001b[1;3mHello World\n",
       "\u001b[0m\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
@@ -65,7 +65,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 28,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -93,7 +93,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -107,8 +107,8 @@
       "\n",
       "```bash\n",
       "printf \"Hello World\\n\"\n",
-      "```\u001b[0m['```bash', 'printf \"Hello World\\\\n\"', '```']\n",
-      "\n",
+      "```\u001b[0m\n",
+      "Code: \u001b[33;1m\u001b[1;3m['printf \"Hello World\\\\n\"']\u001b[0m\n",
       "Answer: \u001b[33;1m\u001b[1;3mHello World\n",
       "\u001b[0m\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
@@ -120,7 +120,7 @@
        "'Hello World\\n'"
       ]
      },
-     "execution_count": 29,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -132,6 +132,114 @@
     "\n",
     "bash_chain.run(text)"
    ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Persistent Terminal\n",
+    "\n",
+    "By default, the chain will run in a separate subprocess each time it is called. This behavior can be changed by instantiating with a persistent bash process."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new LLMBashChain chain...\u001b[0m\n",
+      "List the current directory then move up a level.\u001b[32;1m\u001b[1;3m\n",
+      "\n",
+      "```bash\n",
+      "ls\n",
+      "cd ..\n",
+      "```\u001b[0m\n",
+      "Code: \u001b[33;1m\u001b[1;3m['ls', 'cd ..']\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3mapi.ipynb\t\t\tllm_summarization_checker.ipynb\n",
+      "constitutional_chain.ipynb\tmoderation.ipynb\n",
+      "llm_bash.ipynb\t\t\topenai_openapi.yaml\n",
+      "llm_checker.ipynb\t\topenapi.ipynb\n",
+      "llm_math.ipynb\t\t\tpal.ipynb\n",
+      "llm_requests.ipynb\t\tsqlite.ipynb\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'api.ipynb\\t\\t\\tllm_summarization_checker.ipynb\\r\\nconstitutional_chain.ipynb\\tmoderation.ipynb\\r\\nllm_bash.ipynb\\t\\t\\topenai_openapi.yaml\\r\\nllm_checker.ipynb\\t\\topenapi.ipynb\\r\\nllm_math.ipynb\\t\\t\\tpal.ipynb\\r\\nllm_requests.ipynb\\t\\tsqlite.ipynb'"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.utilities.bash import BashProcess\n",
+    "\n",
+    "\n",
+    "persistent_process = BashProcess(persistent=True)\n",
+    "bash_chain = LLMBashChain.from_bash_process(llm=llm, bash_process=persistent_process, verbose=True)\n",
+    "\n",
+    "text = \"List the current directory then move up a level.\"\n",
+    "\n",
+    "bash_chain.run(text)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new LLMBashChain chain...\u001b[0m\n",
+      "List the current directory then move up a level.\u001b[32;1m\u001b[1;3m\n",
+      "\n",
+      "```bash\n",
+      "ls\n",
+      "cd ..\n",
+      "```\u001b[0m\n",
+      "Code: \u001b[33;1m\u001b[1;3m['ls', 'cd ..']\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3mexamples\t\tgetting_started.ipynb\tindex_examples\n",
+      "generic\t\t\thow_to_guides.rst\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'examples\\t\\tgetting_started.ipynb\\tindex_examples\\r\\ngeneric\\t\\t\\thow_to_guides.rst'"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Run the same command again and see that the state is maintained between calls\n",
+    "bash_chain.run(text)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -150,7 +258,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.8.16"
   }
  },
  "nbformat": 4,

docs/modules/indexes/document_loaders/examples/arxiv.ipynb

@@ -0,0 +1,177 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "bda1f3f5",
+   "metadata": {},
+   "source": [
+    "# Arxiv\n",
+    "\n",
+    "[arXiv](https://arxiv.org/) is an open-access archive for 2 million scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics.\n",
+    "\n",
+    "This notebook shows how to load scientific articles from `Arxiv.org` into a document format that we can use downstream."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1b7a1eef-7bf7-4e7d-8bfc-c4e27c9488cb",
+   "metadata": {},
+   "source": [
+    "## Installation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2abd5578-aa3d-46b9-99af-8b262f0b3df8",
+   "metadata": {},
+   "source": [
+    "First, you need to install `arxiv` python package."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b674aaea-ed3a-4541-8414-260a8f67f623",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!pip install arxiv"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "094b5f13-7e54-4354-9d83-26d6926ecaa0",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "Second, you need to install `PyMuPDF` python package which transform PDF files from the `arxiv.org` site into the text fromat."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7cd91121-2e96-43ba-af50-319853695f86",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!pip install pymupdf"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "95f05e1c-195e-4e2b-ae8e-8d6637f15be6",
+   "metadata": {},
+   "source": [
+    "## Examples"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e29b954c-1407-4797-ae21-6ba8937156be",
+   "metadata": {},
+   "source": [
+    "`ArxivLoader` has these arguments:\n",
+    "- `query`: free text which used to find documents in the Arxiv\n",
+    "- optional `load_max_docs`: default=100. Use it to limit number of downloaded documents. It takes time to download all 100 documents, so use a small number for experiments.\n",
+    "- optional `load_all_available_meta`: default=False. By defaul only the most important fields downloaded: `Published` (date when document was published/last updated), `Title`, `Authors`, `Summary`. If True, other fields also downloaded."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9bfd5e46",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders.base import Document\n",
+    "from langchain.document_loaders import ArxivLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "700e4ef2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "docs = ArxivLoader(query=\"1605.08386\", load_max_docs=2).load()\n",
+    "len(docs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "8977bac0-0042-4f23-9754-247dbd32439b",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'Published': '2016-05-26',\n",
+       " 'Title': 'Heat-bath random walks with Markov bases',\n",
+       " 'Authors': 'Caprice Stanley, Tobias Windisch',\n",
+       " 'Summary': 'Graphs on lattice points are studied whose edges come from a finite set of\\nallowed moves of arbitrary length. We show that the diameter of these graphs on\\nfibers of a fixed integer matrix can be bounded from above by a constant. We\\nthen study the mixing behaviour of heat-bath random walks on these graphs. We\\nalso state explicit conditions on the set of moves so that the heat-bath random\\nwalk, a generalization of the Glauber dynamics, is an expander in fixed\\ndimension.'}"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "doc[0].metadata  # meta-information of the Document"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "46969806-45a9-4c4d-a61b-cfb9658fc9de",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'arXiv:1605.08386v1  [math.CO]  26 May 2016\\nHEAT-BATH RANDOM WALKS WITH MARKOV BASES\\nCAPRICE STANLEY AND TOBIAS WINDISCH\\nAbstract. Graphs on lattice points are studied whose edges come from a finite set of\\nallowed moves of arbitrary length. We show that the diameter of these graphs on fibers of a\\nfixed integer matrix can be bounded from above by a constant. We then study the mixing\\nbehaviour of heat-b'"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "doc[0].page_content[:400]  # all pages of the Document content\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/indexes/document_loaders/examples/googledrive.ipynb

@@ -16,7 +16,7 @@
     "1. `pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib`\n",
     "\n",
     "## 🧑 Instructions for ingesting your Google Docs data\n",
-    "By default, the `GoogleDriveLoader` expects the `credentials.json` file to be `~/.credentials/credentials.json`, but this is configurable using the `credentials_file` keyword argument. Same thing with `token.json`. Note that `token.json` will be created automatically the first time you use the loader.\n",
+    "By default, the `GoogleDriveLoader` expects the `credentials.json` file to be `~/.credentials/credentials.json`, but this is configurable using the `credentials_path` keyword argument. Same thing with `token.json` - `token_path`. Note that `token.json` will be created automatically the first time you use the loader.\n",
     "\n",
     "`GoogleDriveLoader` can load from a list of Google Docs document ids or a folder id. You can obtain your folder and document id from the URL:\n",
     "* Folder: https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is `\"1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5\"`\n",

docs/modules/indexes/retrievers/examples/self_query_retriever.ipynb

@@ -0,0 +1,330 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "13afcae7",
+   "metadata": {},
+   "source": [
+    "# Self-querying retriever\n",
+    "In the notebook we'll demo the `SelfQueryRetriever`, which, as the name suggests, has the ability to query itself. Specifically, given any natural language query, the retriever uses a query-constructing LLM chain to write a structured query and then applies that structured query to it's underlying VectorStore. This allows the retriever to not only use the user-input query for semantic similarity comparison with the contents of stored documented, but to also extract filters from the user query on the metadata of stored documents and to execute those filter."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "68e75fb9",
+   "metadata": {},
+   "source": [
+    "## Creating a Pinecone index\n",
+    "First we'll want to create a Pinecone VectorStore and seed it with some data. We've created a small demo set of documents that contain summaries of movies.\n",
+    "\n",
+    "NOTE: The self-query retriever currently only has built-in support for Pinecone VectorStore.\n",
+    "\n",
+    "NOTE: The self-query retriever requires you to have `lark` installed (`pip install lark`)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "63a8af5b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# !pip install lark"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "3eb9c9a4",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/pinecone/index.py:4: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
+      "  from tqdm.autonotebook import tqdm\n"
+     ]
+    }
+   ],
+   "source": [
+    "import os\n",
+    "\n",
+    "import pinecone\n",
+    "\n",
+    "\n",
+    "pinecone.init(api_key=os.environ[\"PINECONE_API_KEY\"], environment=os.environ[\"PINECONE_ENV\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "cb4a5787",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.schema import Document\n",
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+    "from langchain.vectorstores import Pinecone\n",
+    "\n",
+    "embeddings = OpenAIEmbeddings()\n",
+    "# create new index\n",
+    "pinecone.create_index(\"langchain-self-retriever-demo\", dimension=1536)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "bcbe04d9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "docs = [\n",
+    "    Document(page_content=\"A bunch of scientists bring back dinosaurs and mayhem breaks loose\", metadata={\"year\": 1993, \"rating\": 7.7, \"genre\": [\"action\", \"science fiction\"]}),\n",
+    "    Document(page_content=\"Leo DiCaprio gets lost in a dream within a dream within a dream within a ...\", metadata={\"year\": 2010, \"director\": \"Christopher Nolan\", \"rating\": 8.2}),\n",
+    "    Document(page_content=\"A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea\", metadata={\"year\": 2006, \"director\": \"Satoshi Kon\", \"rating\": 8.6}),\n",
+    "    Document(page_content=\"A bunch of normal-sized women are supremely wholesome and some men pine after them\", metadata={\"year\": 2019, \"director\": \"Greta Gerwig\", \"rating\": 8.3}),\n",
+    "    Document(page_content=\"Toys come alive and have a blast doing so\", metadata={\"year\": 1995, \"genre\": \"animated\"}),\n",
+    "    Document(page_content=\"Three men walk into the Zone, three men walk out of the Zone\", metadata={\"year\": 1979, \"rating\": 9.9, \"director\": \"Andrei Tarkovsky\", \"genre\": [\"science fiction\", \"thriller\"], \"rating\": 9.9})\n",
+    "]\n",
+    "vectorstore = Pinecone.from_documents(\n",
+    "    docs, embeddings, index_name=\"langchain-self-retriever-demo\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5ecaab6d",
+   "metadata": {},
+   "source": [
+    "# Creating our self-querying retriever\n",
+    "Now we can instantiate our retriever. To do this we'll need to provide some information upfront about the metadata fields that our documents support and a short description of the document contents."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "86e34dbf",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import OpenAI\n",
+    "from langchain.retrievers.self_query.base import SelfQueryRetriever\n",
+    "from langchain.chains.query_constructor.base import AttributeInfo\n",
+    "\n",
+    "metadata_field_info=[\n",
+    "    AttributeInfo(\n",
+    "        name=\"genre\",\n",
+    "        description=\"The genre of the movie\", \n",
+    "        type=\"string or list[string]\", \n",
+    "    ),\n",
+    "    AttributeInfo(\n",
+    "        name=\"year\",\n",
+    "        description=\"The year the movie was released\", \n",
+    "        type=\"integer\", \n",
+    "    ),\n",
+    "    AttributeInfo(\n",
+    "        name=\"director\",\n",
+    "        description=\"The name of the movie director\", \n",
+    "        type=\"string\", \n",
+    "    ),\n",
+    "    AttributeInfo(\n",
+    "        name=\"rating\",\n",
+    "        description=\"A 1-10 rating for the movie\",\n",
+    "        type=\"float\"\n",
+    "    ),\n",
+    "]\n",
+    "document_content_description = \"Brief summary of a movie\"\n",
+    "llm = OpenAI(temperature=0)\n",
+    "retriever = SelfQueryRetriever.from_llm(llm, vectorstore, document_content_description, metadata_field_info, verbose=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ea9df8d4",
+   "metadata": {},
+   "source": [
+    "# Testing it out\n",
+    "And now we can try actually using our retriever!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "38a126e9",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "query='dinosaur' filter=None\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='A bunch of scientists bring back dinosaurs and mayhem breaks loose', metadata={'genre': ['action', 'science fiction'], 'rating': 7.7, 'year': 1993.0}),\n",
+       " Document(page_content='Toys come alive and have a blast doing so', metadata={'genre': 'animated', 'year': 1995.0}),\n",
+       " Document(page_content='A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea', metadata={'director': 'Satoshi Kon', 'rating': 8.6, 'year': 2006.0}),\n",
+       " Document(page_content='Leo DiCaprio gets lost in a dream within a dream within a dream within a ...', metadata={'director': 'Christopher Nolan', 'rating': 8.2, 'year': 2010.0})]"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# This example only specifies a relevant query\n",
+    "retriever.get_relevant_documents(\"What are some movies about dinosaurs\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "fc3f1e6e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "query=' ' filter=Comparison(comparator=<Comparator.GT: 'gt'>, attribute='rating', value=8.5)\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea', metadata={'director': 'Satoshi Kon', 'rating': 8.6, 'year': 2006.0}),\n",
+       " Document(page_content='Three men walk into the Zone, three men walk out of the Zone', metadata={'director': 'Andrei Tarkovsky', 'genre': ['science fiction', 'thriller'], 'rating': 9.9, 'year': 1979.0})]"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# This example only specifies a filter\n",
+    "retriever.get_relevant_documents(\"I want to watch a movie rated higher than 8.5\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "b19d4da0",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "query='women' filter=Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='director', value='Greta Gerwig')\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='A bunch of normal-sized women are supremely wholesome and some men pine after them', metadata={'director': 'Greta Gerwig', 'rating': 8.3, 'year': 2019.0})]"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# This example specifies a query and a filter\n",
+    "retriever.get_relevant_documents(\"Has Greta Gerwig directed any movies about women\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "f900e40e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "query=' ' filter=Operation(operator=<Operator.AND: 'and'>, arguments=[Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='genre', value='science fiction'), Comparison(comparator=<Comparator.GT: 'gt'>, attribute='rating', value=8.5)])\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='Three men walk into the Zone, three men walk out of the Zone', metadata={'director': 'Andrei Tarkovsky', 'genre': ['science fiction', 'thriller'], 'rating': 9.9, 'year': 1979.0})]"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# This example specifies a composite filter\n",
+    "retriever.get_relevant_documents(\"What's a highly rated (above 8.5) science fiction film?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "12a51522",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "query='toys' filter=Operation(operator=<Operator.AND: 'and'>, arguments=[Comparison(comparator=<Comparator.GT: 'gt'>, attribute='year', value=1990.0), Comparison(comparator=<Comparator.LT: 'lt'>, attribute='year', value=2005.0), Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='genre', value='animated')])\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='Toys come alive and have a blast doing so', metadata={'genre': 'animated', 'year': 1995.0})]"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# This example specifies a query and composite filter\n",
+    "retriever.get_relevant_documents(\"What's a movie after 1990 but before 2005 that's all about toys, and preferably is animated\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "69bbd809",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/indexes/vectorstores/examples/lanecdb.ipynb

@@ -0,0 +1,179 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "683953b3",
+   "metadata": {},
+   "source": [
+    "# LanceDB\n",
+    "\n",
+    "This notebook shows how to use functionality related to the LanceDB vector database based on the Lance data format."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "bfcf346a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install lancedb"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "aac9563e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.vectorstores import LanceDB"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "a3c3999a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import TextLoader\n",
+    "from langchain.text_splitter import CharacterTextSplitter\n",
+    "loader = TextLoader('../../../state_of_the_union.txt')\n",
+    "documents = loader.load()\n",
+    "\n",
+    "documents = CharacterTextSplitter().split_documents(documents)\n",
+    "\n",
+    "embeddings = OpenAIEmbeddings()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "6e104aee",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import lancedb\n",
+    "\n",
+    "db = lancedb.connect('/tmp/lancedb')\n",
+    "table = db.create_table(\"my_table\", data=[\n",
+    "    {\"vector\": embeddings.embed_query(\"Hello World\"), \"text\": \"Hello World\", \"id\": \"1\"}\n",
+    "], mode=\"overwrite\")\n",
+    "\n",
+    "docsearch = LanceDB.from_documents(documents, embeddings, connection=table)\n",
+    "\n",
+    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+    "docs = docsearch.similarity_search(query)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "9c608226",
+   "metadata": {
+    "scrolled": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "They were responding to a 9-1-1 call when a man shot and killed them with a stolen gun. \n",
+      "\n",
+      "Officer Mora was 27 years old. \n",
+      "\n",
+      "Officer Rivera was 22. \n",
+      "\n",
+      "Both Dominican Americans who’d grown up on the same streets they later chose to patrol as police officers. \n",
+      "\n",
+      "I spoke with their families and told them that we are forever in debt for their sacrifice, and we will carry on their mission to restore the trust and safety every community deserves. \n",
+      "\n",
+      "I’ve worked on these issues a long time. \n",
+      "\n",
+      "I know what works: Investing in crime preventionand community police officers who’ll walk the beat, who’ll know the neighborhood, and who can restore trust and safety. \n",
+      "\n",
+      "So let’s not abandon our streets. Or choose between safety and equal justice. \n",
+      "\n",
+      "Let’s come together to protect our communities, restore trust, and hold law enforcement accountable. \n",
+      "\n",
+      "That’s why the Justice Department required body cameras, banned chokeholds, and restricted no-knock warrants for its officers. \n",
+      "\n",
+      "That’s why the American Rescue Plan provided $350 Billion that cities, states, and counties can use to hire more police and invest in proven strategies like community violence interruption—trusted messengers breaking the cycle of violence and trauma and giving young people hope.  \n",
+      "\n",
+      "We should all agree: The answer is not to Defund the police. The answer is to FUND the police with the resources and training they need to protect our communities. \n",
+      "\n",
+      "I ask Democrats and Republicans alike: Pass my budget and keep our neighborhoods safe.  \n",
+      "\n",
+      "And I will keep doing everything in my power to crack down on gun trafficking and ghost guns you can buy online and make at home—they have no serial numbers and can’t be traced. \n",
+      "\n",
+      "And I ask Congress to pass proven measures to reduce gun violence. Pass universal background checks. Why should anyone on a terrorist list be able to purchase a weapon? \n",
+      "\n",
+      "Ban assault weapons and high-capacity magazines. \n",
+      "\n",
+      "Repeal the liability shield that makes gun manufacturers the only industry in America that can’t be sued. \n",
+      "\n",
+      "These laws don’t infringe on the Second Amendment. They save lives. \n",
+      "\n",
+      "The most fundamental right in America is the right to vote – and to have it counted. And it’s under assault. \n",
+      "\n",
+      "In state after state, new laws have been passed, not only to suppress the vote, but to subvert entire elections. \n",
+      "\n",
+      "We cannot let this happen. \n",
+      "\n",
+      "Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n",
+      "\n",
+      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n",
+      "\n",
+      "One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n",
+      "\n",
+      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. \n",
+      "\n",
+      "A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. \n",
+      "\n",
+      "And if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. \n",
+      "\n",
+      "We can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  \n",
+      "\n",
+      "We’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  \n",
+      "\n",
+      "We’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(docs[0].page_content)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a359ed74",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/indexes/vectorstores/examples/weaviate.ipynb

@@ -75,7 +75,8 @@
     "            \"vectorizer\": \"text2vec-openai\",\n",
     "              \"moduleConfig\": {\n",
     "                \"text2vec-openai\": {\n",
-    "                  \"model\": \"babbage\",\n",
+    "                  \"model\": \"ada\",\n",
+    "                  \"modelVersion\": \"002\",\n",
     "                  \"type\": \"text\"\n",
     "                }\n",
     "              },\n",

docs/modules/models/llms/examples/llm_caching.ipynb

@@ -785,7 +785,9 @@
    "id": "9df0dab8",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "!rm .langchain.db sqlite.db"
+   ]
   }
  ],
  "metadata": {

docs/modules/models/llms/integrations/cerebriumai_example.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# CerebriumAI\n",
     "\n",
-    "`Cerebrium` is an AWS Sagemaker alternative. It also provides API access to [several LLM models](https://docs.cerebrium.ai/cerebrium/prebuilt-models/deploymen).\n",
+    "`Cerebrium` is an AWS Sagemaker alternative. It also provides API access to [several LLM models](https://docs.cerebrium.ai/cerebrium/prebuilt-models/deployment).\n",
     "\n",
     "This notebook goes over how to use Langchain with [CerebriumAI](https://docs.cerebrium.ai/introduction)."
    ]

docs/modules/models/llms/integrations/huggingface_pipelines.ipynb

@@ -11,7 +11,7 @@
     "\n",
     "The [Hugging Face Model Hub](https://huggingface.co/models) hosts over 120k models, 20k datasets, and 50k demo apps (Spaces), all open source and publicly available, in an online platform where people can easily collaborate and build ML together.\n",
     "\n",
-    "These can be called from LangChain either through this local pipeline wrapper or by calling their hosted inference endpoints through the HuggingFaceHub class. For more information on the hosted pipelines, see the [HugigngFaceHub](huggingface_hub.ipynb) notebook."
+    "These can be called from LangChain either through this local pipeline wrapper or by calling their hosted inference endpoints through the HuggingFaceHub class. For more information on the hosted pipelines, see the [HuggingFaceHub](huggingface_hub.ipynb) notebook."
    ]
   },
   {

docs/modules/models/llms/integrations/pipelineai_example.ipynb

@@ -0,0 +1,171 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# PipelineAI\n",
+    "\n",
+    "PipelineAI allows you to run your ML models at scale in the cloud. It also provides API access to [several LLM models](https://pipeline.ai).\n",
+    "\n",
+    "This notebook goes over how to use Langchain with [PipelineAI](https://docs.pipeline.ai/docs)."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Install pipeline-ai\n",
+    "The `pipeline-ai` library is required to use the `PipelineAI` API, AKA `Pipeline Cloud`. Install `pipeline-ai` using `pip install pipeline-ai`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install the package\n",
+    "!pip install pipeline-ai"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "from langchain.llms import PipelineAI\n",
+    "from langchain import PromptTemplate, LLMChain"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Set the Environment API Key\n",
+    "Make sure to get your API key from PipelineAI. Check out the [cloud quickstart guide](https://docs.pipeline.ai/docs/cloud-quickstart). You'll be given a 30 day free trial with 10 hours of serverless GPU compute to test different models."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "os.environ[\"PIPELINE_API_KEY\"] = \"YOUR_API_KEY_HERE\""
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create the PipelineAI instance\n",
+    "When instantiating PipelineAI, you need to specify the id or tag of the pipeline you want to use, e.g. `pipeline_key = \"public/gpt-j:base\"`. You then have the option of passing additional pipeline-specific keyword arguments:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = PipelineAI(pipeline_key=\"YOUR_PIPELINE_KEY\", pipeline_kwargs={...})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create a Prompt Template\n",
+    "We will create a prompt template for Question and Answer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "template = \"\"\"Question: {question}\n",
+    "\n",
+    "Answer: Let's think step by step.\"\"\"\n",
+    "\n",
+    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Initiate the LLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Run the LLMChain\n",
+    "Provide a question and run the LLMChain."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "question = \"What NFL team won the Super Bowl in the year Justin Beiber was born?\"\n",
+    "\n",
+    "llm_chain.run(question)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.6"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/modules/models/llms/integrations/replicate.ipynb

@@ -44,7 +44,7 @@
    },
    "outputs": [
     {
-     "name": "stdin",
+     "name": "stdout",
      "output_type": "stream",
      "text": [
       " ········\n"
@@ -85,6 +85,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -92,7 +93,7 @@
     "\n",
     "Find a model on the [replicate explore page](https://replicate.com/explore), and then paste in the model name and version in this format: model_name/version\n",
     "\n",
-    "For example, for this [flan-t5 model]( https://replicate.com/daanelson/flan-t5), click on the API tab. The model name/version would be: `daanelson/flan-t5:04e422a9b85baed86a4f24981d7f9953e20c5fd82f6103b74ebc431588e1cec8`\n",
+    "For example, for this [dolly model](https://replicate.com/replicate/dolly-v2-12b), click on the API tab. The model name/version would be: `replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5`\n",
     "\n",
     "Only the `model` param is required, but we can add other model params when initializing.\n",
     "\n",
@@ -113,7 +114,7 @@
    },
    "outputs": [],
    "source": [
-    "llm = Replicate(model=\"daanelson/flan-t5:04e422a9b85baed86a4f24981d7f9953e20c5fd82f6103b74ebc431588e1cec8\")"
+    "llm = Replicate(model=\"replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5\")"
    ]
   },
   {
@@ -243,7 +244,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "llm = Replicate(model=\"daanelson/flan-t5:04e422a9b85baed86a4f24981d7f9953e20c5fd82f6103b74ebc431588e1cec8\")\n",
+    "dolly_llm = Replicate(model=\"replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5\")\n",
     "text2image = Replicate(model=\"stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf\")"
    ]
   },
@@ -265,7 +266,7 @@
     "    template=\"What is a good name for a company that makes {product}?\",\n",
     ")\n",
     "\n",
-    "chain = LLMChain(llm=llm, prompt=prompt)"
+    "chain = LLMChain(llm=dolly_llm, prompt=prompt)"
    ]
   },
   {
@@ -285,7 +286,7 @@
     "    input_variables=[\"company_name\"],\n",
     "    template=\"Write a description of a logo for this company: {company_name}\",\n",
     ")\n",
-    "chain_two = LLMChain(llm=llm, prompt=second_prompt)"
+    "chain_two = LLMChain(llm=dolly_llm, prompt=second_prompt)"
    ]
   },
   {

docs/modules/models/text_embedding/examples/sentence_transformers.ipynb

@@ -8,12 +8,14 @@
    "source": [
     "# Sentence Transformers Embeddings\n",
     "\n",
-    "Let's generate embeddings using the [SentenceTransformers](https://www.sbert.net/) integration. SentenceTransformers is a python package that can generate text and image embeddings, originating from [Sentence-BERT](https://arxiv.org/abs/1908.10084)"
+    "[SentenceTransformers](https://www.sbert.net/) embeddings are called using the `HuggingFaceEmbeddings` integration. We have also added an alias for `SentenceTransformerEmbeddings` for users who are more familiar with directly using that package.\n",
+    "\n",
+    "SentenceTransformers is a python package that can generate text and image embeddings, originating from [Sentence-BERT](https://arxiv.org/abs/1908.10084)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 1,
    "id": "06c9f47d",
    "metadata": {},
    "outputs": [
@@ -21,10 +23,9 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
-      "To disable this warning, you can either:\n",
-      "\t- Avoid using `tokenizers` before the fork if possible\n",
-      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
+      "\n",
+      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.1\u001b[0m\n",
+      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
      ]
     }
    ],
@@ -34,27 +35,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 2,
    "id": "861521a9",
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.embeddings import SentenceTransformerEmbeddings "
+    "from langchain.embeddings import HuggingFaceEmbeddings, SentenceTransformerEmbeddings "
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": null,
    "id": "ff9be586",
    "metadata": {},
    "outputs": [],
    "source": [
-    "embeddings = SentenceTransformerEmbeddings(model=\"all-MiniLM-L6-v2\")"
+    "embeddings = HuggingFaceEmbeddings(model_name=\"all-MiniLM-L6-v2\")\n",
+    "# Equivalent to SentenceTransformerEmbeddings(model_name=\"all-MiniLM-L6-v2\")"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 4,
    "id": "d0a98ae9",
    "metadata": {},
    "outputs": [],
@@ -64,7 +66,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 5,
    "id": "5d6c682b",
    "metadata": {},
    "outputs": [],
@@ -74,7 +76,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 6,
    "id": "bb5e74c0",
    "metadata": {},
    "outputs": [],
@@ -107,7 +109,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.2"
+   "version": "3.8.16"
   },
   "vscode": {
    "interpreter": {

docs/modules/prompts/prompt_templates/examples/connecting_to_a_feature_store.ipynb

@@ -0,0 +1,474 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "a792b119",
+   "metadata": {},
+   "source": [
+    "# Connecting to a Feature Store\n",
+    "\n",
+    "Feature stores are a concept from traditional machine learning that make sure data fed into models is up-to-date and relevant. For more on this, see [here](https://www.tecton.ai/blog/what-is-a-feature-store/).\n",
+    "\n",
+    "This concept is extremely relevant when considering putting LLM applications in production. In order to personalize LLM applications, you may want to combine LLMs with up-to-date information about particular users. Feature stores can be a great way to keep that data fresh, and LangChain provides an easy way to combine that data with LLMs.\n",
+    "\n",
+    "In this notebook we will show how to connect prompt templates to feature stores. The basic idea is to call a feature store from inside a prompt template to retrieve values that are then formatted into the prompt."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ad0b5edf",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Feast\n",
+    "\n",
+    "To start, we will use the popular open source feature store framework [Feast](https://github.com/feast-dev/feast).\n",
+    "\n",
+    "This assumes you have already run the steps in the README around getting started. We will build of off that example in getting started, and create and LLMChain to write a note to a specific driver regarding their up-to-date statistics."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7f02f6f3",
+   "metadata": {},
+   "source": [
+    "### Load Feast Store\n",
+    "\n",
+    "Again, this should be set up according to the instructions in the Feast README"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "fd1a452a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from feast import FeatureStore\n",
+    "\n",
+    "# You may need to update the path depending on where you stored it\n",
+    "feast_repo_path = \"../../../../../my_feature_repo/feature_repo/\"\n",
+    "store = FeatureStore(repo_path=feast_repo_path)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cfe8aae5",
+   "metadata": {},
+   "source": [
+    "### Prompts\n",
+    "\n",
+    "Here we will set up a custom FeastPromptTemplate. This prompt template will take in a driver id, look up their stats, and format those stats into a prompt.\n",
+    "\n",
+    "Note that the input to this prompt template is just `driver_id`, since that is the only user defined piece (all other variables are looked up inside the prompt template)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "5e9cee04",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts import PromptTemplate, StringPromptTemplate"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "594a3cf3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "template = \"\"\"Given the driver's up to date stats, write them note relaying those stats to them.\n",
+    "If they have a conversation rate above .5, give them a compliment. Otherwise, make a silly joke about chickens at the end to make them feel better\n",
+    "\n",
+    "Here are the drivers stats:\n",
+    "Conversation rate: {conv_rate}\n",
+    "Acceptance rate: {acc_rate}\n",
+    "Average Daily Trips: {avg_daily_trips}\n",
+    "\n",
+    "Your response:\"\"\"\n",
+    "prompt = PromptTemplate.from_template(template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "8464c731",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class FeastPromptTemplate(StringPromptTemplate):\n",
+    "    \n",
+    "    def format(self, **kwargs) -> str:\n",
+    "        driver_id = kwargs.pop(\"driver_id\")\n",
+    "        feature_vector = store.get_online_features(\n",
+    "            features=[\n",
+    "                'driver_hourly_stats:conv_rate',\n",
+    "                'driver_hourly_stats:acc_rate',\n",
+    "                'driver_hourly_stats:avg_daily_trips'\n",
+    "            ],\n",
+    "            entity_rows=[{\"driver_id\": 1001}]\n",
+    "        ).to_dict()\n",
+    "        kwargs[\"conv_rate\"] = feature_vector[\"conv_rate\"][0]\n",
+    "        kwargs[\"acc_rate\"] = feature_vector[\"acc_rate\"][0]\n",
+    "        kwargs[\"avg_daily_trips\"] = feature_vector[\"avg_daily_trips\"][0]\n",
+    "        return prompt.format(**kwargs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "c0c7bae2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt_template = FeastPromptTemplate(input_variables=[\"driver_id\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "d8d70bb7",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Given the driver's up to date stats, write them note relaying those stats to them.\n",
+      "If they have a conversation rate above .5, give them a compliment. Otherwise, make a silly joke about chickens at the end to make them feel better\n",
+      "\n",
+      "Here are the drivers stats:\n",
+      "Conversation rate: 0.4745151400566101\n",
+      "Acceptance rate: 0.055561766028404236\n",
+      "Average Daily Trips: 936\n",
+      "\n",
+      "Your response:\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(prompt_template.format(driver_id=1001))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2870d070",
+   "metadata": {},
+   "source": [
+    "### Use in a chain\n",
+    "\n",
+    "We can now use this in a chain, successfully creating a chain that achieves personalization backed by a feature store"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "7106255c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.chains import LLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "id": "79543326",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = LLMChain(llm=ChatOpenAI(), prompt=prompt_template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "97a741a0",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"Hi there! I wanted to update you on your current stats. Your acceptance rate is 0.055561766028404236 and your average daily trips are 936. While your conversation rate is currently 0.4745151400566101, I have no doubt that with a little extra effort, you'll be able to exceed that .5 mark! Keep up the great work! And remember, even chickens can't always cross the road, but they still give it their best shot.\""
+      ]
+     },
+     "execution_count": 25,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.run(1001)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "12e59aaf",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c4049990-651d-44d3-82b1-0cd122da55c1",
+   "metadata": {},
+   "source": [
+    "## Tecton\n",
+    "\n",
+    "Above, we showed how you could use Feast, a popular open source and self-managed feature store, with LangChain. Our examples below will show a similar integration using Tecton. Tecton is a fully managed feature platform built to orchestrate the complete ML feature lifecycle, from transformation to online serving, with enterprise-grade SLAs."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7bb4dba1-0678-4ea4-be0a-d353c0b13fc2",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Prerequisites\n",
+    "\n",
+    "* Tecton Deployment (sign up at [https://tecton.ai](https://tecton.ai))\n",
+    "* `TECTON_API_KEY` environment variable set to a valid Service Account key"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ac9eb618-8c52-4cd6-bb8e-9c99a150dfa6",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Define and Load Features\n",
+    "\n",
+    "We will use the user_transaction_counts Feature View from the [Tecton tutorial](https://docs.tecton.ai/docs/tutorials/tecton-fundamentals) as part of a Feature Service. For simplicity, we are only using a single Feature View; however, more sophisticated applications may require more feature views to retrieve the features needed for its prompt.\n",
+    "\n",
+    "```python\n",
+    "user_transaction_metrics = FeatureService(\n",
+    "    name = \"user_transaction_metrics\",\n",
+    "    features = [user_transaction_counts]\n",
+    ")\n",
+    "```\n",
+    "\n",
+    "The above Feature Service is expected to be [applied to a live workspace](https://docs.tecton.ai/docs/applying-feature-repository-changes-to-a-workspace). For this example, we will be using the \"prod\" workspace."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 60,
+   "id": "32e9675d-a7e5-429f-906f-2260294d3e46",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "import tecton\n",
+    "\n",
+    "workspace = tecton.get_workspace(\"prod\")\n",
+    "feature_service = workspace.get_feature_service(\"user_transaction_metrics\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "29b7550c-0eb4-4bd1-a501-1c63fb77aa56",
+   "metadata": {},
+   "source": [
+    "### Prompts\n",
+    "\n",
+    "Here we will set up a custom TectonPromptTemplate. This prompt template will take in a user_id , look up their stats, and format those stats into a prompt.\n",
+    "\n",
+    "Note that the input to this prompt template is just `user_id`, since that is the only user defined piece (all other variables are looked up inside the prompt template)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 61,
+   "id": "6fb77ea4-64c6-4e48-a783-bd1ece021b82",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.prompts import PromptTemplate, StringPromptTemplate"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 77,
+   "id": "02a98fbc-8135-4b11-bf60-85d28e426667",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "template = \"\"\"Given the vendor's up to date transaction stats, write them a note based on the following rules:\n",
+    "\n",
+    "1. If they had a transaction in the last day, write a short congratulations message on their recent sales\n",
+    "2. If no transaction in the last day, but they had a transaction in the last 30 days, playfully encourage them to sell more.\n",
+    "3. Always add a silly joke about chickens at the end\n",
+    "\n",
+    "Here are the vendor's stats:\n",
+    "Number of Transactions Last Day: {transaction_count_1d}\n",
+    "Number of Transactions Last 30 Days: {transaction_count_30d}\n",
+    "\n",
+    "Your response:\"\"\"\n",
+    "prompt = PromptTemplate.from_template(template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 78,
+   "id": "a35cdfd5-6ccc-4394-acfe-60d53804be51",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "class TectonPromptTemplate(StringPromptTemplate):\n",
+    "    \n",
+    "    def format(self, **kwargs) -> str:\n",
+    "        user_id = kwargs.pop(\"user_id\")\n",
+    "        feature_vector = feature_service.get_online_features(join_keys={\"user_id\": user_id}).to_dict()\n",
+    "        kwargs[\"transaction_count_1d\"] = feature_vector[\"user_transaction_counts.transaction_count_1d_1d\"]\n",
+    "        kwargs[\"transaction_count_30d\"] = feature_vector[\"user_transaction_counts.transaction_count_30d_1d\"]\n",
+    "        return prompt.format(**kwargs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 79,
+   "id": "d5915df0-fb16-4770-8a82-22f885b74d1a",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "prompt_template = TectonPromptTemplate(input_variables=[\"user_id\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 80,
+   "id": "a36abfc8-ea60-4ae0-a36d-d7b639c7307c",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Given the vendor's up to date transaction stats, write them a note based on the following rules:\n",
+      "\n",
+      "1. If they had a transaction in the last day, write a short congratulations message on their recent sales\n",
+      "2. If no transaction in the last day, but they had a transaction in the last 30 days, playfully encourage them to sell more.\n",
+      "3. Always add a silly joke about chickens at the end\n",
+      "\n",
+      "Here are the vendor's stats:\n",
+      "Number of Transactions Last Day: 657\n",
+      "Number of Transactions Last 30 Days: 20326\n",
+      "\n",
+      "Your response:\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(prompt_template.format(user_id=\"user_469998441571\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f8d4b905-1051-4303-9c33-8eddb65c1274",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "### Use in a chain\n",
+    "\n",
+    "We can now use this in a chain, successfully creating a chain that achieves personalization backed by the Tecton Feature Platform"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 81,
+   "id": "ffb60cd0-8e3c-4c9d-b639-43d766e12c4c",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.chains import LLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 82,
+   "id": "3918abc7-00b5-466f-bdfc-ab046cd282da",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "chain = LLMChain(llm=ChatOpenAI(), prompt=prompt_template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 83,
+   "id": "e7d91c4b-3e99-40cc-b3e9-a004c8c9193e",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Wow, congratulations on your recent sales! Your business is really soaring like a chicken on a hot air balloon! Keep up the great work!'"
+      ]
+     },
+     "execution_count": 83,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.run(\"user_469998441571\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f752b924-caf9-4f7a-b78b-cb8c8ada8c2e",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/use_cases/agent_simulations.md

@@ -12,5 +12,7 @@ Specific implementations of agent simulations (or parts of agent simulations) in
 - [CAMEL](agent_simulations/camel_role_playing.ipynb): an implementation of the CAMEL (Communicative Agents for “Mind” Exploration of Large Scale Language Model Society) paper, where two agents communicate with each other.
 - [Two Player D&D](agent_simulations/two_player_dnd.ipynb): an example of how to use a generic simulator for two agents to implement a variant of the popular Dungeons & Dragons role playing game.
 
-## Generative Agents
+## Simulations with Multiple Agents
+- [Multi-Player D&D](agent_simulations/multi_player_dnd.ipynb): an example of how to use a generic dialogue simulator for multiple dialogue agents with a custom speaker-ordering, illustrated with a variant of the popular Dungeons & Dragons role playing game.
+- [Decentralized Speaker Selection](agent_simulations/multiagent_bidding.ipynb): an example of how to implement a multi-agent dialogue without a fixed schedule for who speaks when. Instead the agents decide for themselves who speaks by outputting bids to speak. This example shows how to do this in the context of a fictitious presidential debate.
 - [Generative Agents](agent_simulations/characters.ipynb): This notebook implements a generative agent based on the paper [Generative Agents: Interactive Simulacra of Human Behavior](https://arxiv.org/abs/2304.03442) by Park, et. al.

docs/use_cases/agent_simulations/multi_player_dnd.ipynb

@@ -0,0 +1,493 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Multi-Player Dungeons & Dragons\n",
+    "\n",
+    "This notebook shows how the `DialogueAgent` and `DialogueSimulator` class make it easy to extend the [Two-Player Dungeons & Dragons example](https://python.langchain.com/en/latest/use_cases/agent_simulations/two_player_dnd.html) to multiple players.\n",
+    "\n",
+    "The main difference between simulating two players and multiple players is in revising the schedule for when each agent speaks\n",
+    "\n",
+    "To this end, we augment `DialogueSimulator` to take in a custom function that determines the schedule of which agent speaks. In the example below, each character speaks in round-robin fashion, with the storyteller interleaved between each player."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Import LangChain related modules "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List, Dict, Callable\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.schema import (\n",
+    "    AIMessage,\n",
+    "    HumanMessage,\n",
+    "    SystemMessage,\n",
+    "    BaseMessage,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## `DialogueAgent` class\n",
+    "The `DialogueAgent` class is a simple wrapper around the `ChatOpenAI` model that stores the message history from the `dialogue_agent`'s point of view by simply concatenating the messages as strings.\n",
+    "\n",
+    "It exposes two methods: \n",
+    "- `send()`: applies the chatmodel to the message history and returns the message string\n",
+    "- `receive(name, message)`: adds the `message` spoken by `name` to message history"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class DialogueAgent():\n",
+    "\n",
+    "    def __init__(\n",
+    "        self,\n",
+    "        name,\n",
+    "        system_message: SystemMessage,\n",
+    "        model: ChatOpenAI,\n",
+    "    ) -> None:\n",
+    "        self.name = name\n",
+    "        self.system_message = system_message\n",
+    "        self.model = model\n",
+    "        self.message_history = f\"\"\"Here is the conversation so far.\n",
+    "        \"\"\"\n",
+    "        self.prefix = f'\\n{self.name}:'\n",
+    "        \n",
+    "    def send(self) -> str:\n",
+    "        \"\"\"\n",
+    "        Applies the chatmodel to the message history\n",
+    "        and returns the message string\n",
+    "        \"\"\"\n",
+    "        message = self.model(\n",
+    "            [self.system_message, \n",
+    "             HumanMessage(content=self.message_history+self.prefix)])\n",
+    "        return message.content\n",
+    "            \n",
+    "    def receive(self, name: str, message: str) -> None:\n",
+    "        \"\"\"\n",
+    "        Concatenates {message} spoken by {name} into message history\n",
+    "        \"\"\"\n",
+    "        self.message_history += f'\\n{name}: {message}'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## `DialogueSimulator` class\n",
+    "The `DialogueSimulator` class takes a list of agents. At each step, it performs the following:\n",
+    "1. Select the next speaker\n",
+    "2. Calls the next speaker to send a message \n",
+    "3. Broadcasts the message to all other agents\n",
+    "4. Update the step counter.\n",
+    "The selection of the next speaker can be implemented as any function, but in this case we simply loop through the agents."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class DialogueSimulator():\n",
+    "    \n",
+    "    def __init__(\n",
+    "        self, \n",
+    "        agents: List[DialogueAgent], \n",
+    "        selection_function: Callable[[int, List[DialogueAgent]], int]\n",
+    "    ) -> None:\n",
+    "        self.agents = agents\n",
+    "        self._step = 0\n",
+    "        self.select_next_speaker = selection_function\n",
+    "        \n",
+    "    def reset(self, name: str, message: str):\n",
+    "        \"\"\"\n",
+    "        Initiates the conversation with a {message} from {name}\n",
+    "        \"\"\"\n",
+    "        for agent in self.agents:\n",
+    "            agent.receive(name, message)\n",
+    "            \n",
+    "        # increment time\n",
+    "        self._step += 1\n",
+    "    \n",
+    "    def step(self) -> tuple[str, str]:\n",
+    "        # 1. choose the next speaker\n",
+    "        speaker_idx = self.select_next_speaker(self._step, self.agents)\n",
+    "        speaker = self.agents[speaker_idx]\n",
+    "        \n",
+    "        # 2. next speaker sends message\n",
+    "        message = speaker.send()\n",
+    "        \n",
+    "        # 3. everyone receives message\n",
+    "        for receiver in self.agents:\n",
+    "            receiver.receive(speaker.name, message)\n",
+    "            \n",
+    "        # 4. increment time\n",
+    "        self._step += 1\n",
+    "                \n",
+    "        return speaker.name, message"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Define roles and quest"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "character_names = [\"Harry Potter\", \"Ron Weasley\", \"Hermione Granger\", \"Argus Filch\"]\n",
+    "storyteller_name = \"Dungeon Master\"\n",
+    "quest = \"Find all of Lord Voldemort's seven horcruxes.\"\n",
+    "word_limit = 50 # word limit for task brainstorming"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Ask an LLM to add detail to the game description"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "game_description = f\"\"\"Here is the topic for a Dungeons & Dragons game: {quest}.\n",
+    "        The characters are: {*character_names,}.\n",
+    "        The story is narrated by the storyteller, {storyteller_name}.\"\"\"\n",
+    "\n",
+    "player_descriptor_system_message = SystemMessage(\n",
+    "    content=\"You can add detail to the description of a Dungeons & Dragons player.\")\n",
+    "\n",
+    "def generate_character_description(character_name):\n",
+    "    character_specifier_prompt = [\n",
+    "        player_descriptor_system_message,\n",
+    "        HumanMessage(content=\n",
+    "            f\"\"\"{game_description}\n",
+    "            Please reply with a creative description of the character, {character_name}, in {word_limit} words or less. \n",
+    "            Speak directly to {character_name}.\n",
+    "            Do not add anything else.\"\"\"\n",
+    "            )\n",
+    "    ]\n",
+    "    character_description = ChatOpenAI(temperature=1.0)(character_specifier_prompt).content\n",
+    "    return character_description\n",
+    "\n",
+    "def generate_character_system_message(character_name, character_description):\n",
+    "    return SystemMessage(content=(\n",
+    "    f\"\"\"{game_description}\n",
+    "    Your name is {character_name}. \n",
+    "    Your character description is as follows: {character_description}.\n",
+    "    You will propose actions you plan to take and {storyteller_name} will explain what happens when you take those actions.\n",
+    "    Speak in the first person from the perspective of {character_name}.\n",
+    "    For describing your own body movements, wrap your description in '*'.\n",
+    "    Do not change roles!\n",
+    "    Do not speak from the perspective of anyone else.\n",
+    "    Remember you are {character_name}.\n",
+    "    Stop speaking the moment you finish speaking from your perspective.\n",
+    "    Never forget to keep your response to {word_limit} words!\n",
+    "    Do not add anything else.\n",
+    "    \"\"\"\n",
+    "    ))\n",
+    "\n",
+    "character_descriptions = [generate_character_description(character_name) for character_name in character_names]\n",
+    "character_system_messages = [generate_character_system_message(character_name, character_description) for character_name, character_description in zip(character_names, character_descriptions)]\n",
+    "\n",
+    "storyteller_specifier_prompt = [\n",
+    "    player_descriptor_system_message,\n",
+    "    HumanMessage(content=\n",
+    "        f\"\"\"{game_description}\n",
+    "        Please reply with a creative description of the storyteller, {storyteller_name}, in {word_limit} words or less. \n",
+    "        Speak directly to {storyteller_name}.\n",
+    "        Do not add anything else.\"\"\"\n",
+    "        )\n",
+    "]\n",
+    "storyteller_description = ChatOpenAI(temperature=1.0)(storyteller_specifier_prompt).content\n",
+    "\n",
+    "storyteller_system_message = SystemMessage(content=(\n",
+    "f\"\"\"{game_description}\n",
+    "You are the storyteller, {storyteller_name}. \n",
+    "Your description is as follows: {storyteller_description}.\n",
+    "The other players will propose actions to take and you will explain what happens when they take those actions.\n",
+    "Speak in the first person from the perspective of {storyteller_name}.\n",
+    "Do not change roles!\n",
+    "Do not speak from the perspective of anyone else.\n",
+    "Remember you are the storyteller, {storyteller_name}.\n",
+    "Stop speaking the moment you finish speaking from your perspective.\n",
+    "Never forget to keep your response to {word_limit} words!\n",
+    "Do not add anything else.\n",
+    "\"\"\"\n",
+    "))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Storyteller Description:\n",
+      "Dungeon Master, your vivid imagination conjures a world of wonder and danger. Will you lead our triumphant trio or be the ultimate foil to their quest to rid the world of Voldemort's horcruxes? The fate of both the muggle and wizarding worlds rests in your hands.\n",
+      "Harry Potter Description:\n",
+      "Harry Potter, the boy who lived, you hold the fate of the wizarding world in your hands. Your bravery and loyalty to your friends are unmatched. The burden you carry is heavy, but with the power of love by your side, you can overcome any obstacle. The hunt for the horcruxes begins now.\n",
+      "Ron Weasley Description:\n",
+      "Ron Weasley, you are Harry Potter's loyal and brave best friend. You have a great sense of humor and always bring joy to the team. Your skills with magic and strategy make you a valuable asset in the fight against Voldemort. Your love for food and your family keeps you grounded and motivated.\n",
+      "Hermione Granger Description:\n",
+      "Hermione Granger, you are the brightest witch of your age. Your quick wit and vast knowledge are essential in our quest to find the horcruxes. Trust in your abilities and remember, knowledge is power.\n",
+      "Argus Filch Description:\n",
+      "Argus Filch, you are a bitter and cruel caretaker of the Hogwarts School of Witchcraft and Wizardry. Your harsh mannerisms and love for punishing the students know no bounds. Your loyalty to the Wizarding World and disdain for magic-wielders makes it surprising that you would join Harry, Ron, and Hermione in their quest to defeat Voldemort.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print('Storyteller Description:')\n",
+    "print(storyteller_description)\n",
+    "for character_name, character_description in zip(character_names, character_descriptions):\n",
+    "    print(f'{character_name} Description:')\n",
+    "    print(character_description)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Use an LLM to create an elaborate quest description"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Original quest:\n",
+      "Find all of Lord Voldemort's seven horcruxes.\n",
+      "\n",
+      "Detailed quest:\n",
+      "You have discovered that one of Voldemort's horcruxes is hidden deep in the Forbidden Forest. You must navigate the dangerous terrain, avoid the creatures lurking within, and find the horcrux before the full moon rises, unleashing a pack of hungry werewolves. Remember, time is of the essence!\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "quest_specifier_prompt = [\n",
+    "    SystemMessage(content=\"You can make a task more specific.\"),\n",
+    "    HumanMessage(content=\n",
+    "        f\"\"\"{game_description}\n",
+    "        \n",
+    "        You are the storyteller, {storyteller_name}.\n",
+    "        Please make the quest more specific. Be creative and imaginative.\n",
+    "        Please reply with the specified quest in {word_limit} words or less. \n",
+    "        Speak directly to the characters: {*character_names,}.\n",
+    "        Do not add anything else.\"\"\"\n",
+    "        )\n",
+    "]\n",
+    "specified_quest = ChatOpenAI(temperature=1.0)(quest_specifier_prompt).content\n",
+    "\n",
+    "print(f\"Original quest:\\n{quest}\\n\")\n",
+    "print(f\"Detailed quest:\\n{specified_quest}\\n\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Main Loop"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "characters = []\n",
+    "for character_name, character_system_message in zip(character_names, character_system_messages):\n",
+    "    characters.append(DialogueAgent(\n",
+    "        name=character_name,\n",
+    "        system_message=character_system_message, \n",
+    "        model=ChatOpenAI(temperature=0.2)))\n",
+    "storyteller = DialogueAgent(name=storyteller_name,\n",
+    "                     system_message=storyteller_system_message, \n",
+    "                     model=ChatOpenAI(temperature=0.2))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def select_next_speaker(step: int, agents: List[DialogueAgent]) -> int:\n",
+    "    \"\"\"\n",
+    "    If the step is even, then select the storyteller\n",
+    "    Otherwise, select the other characters in a round-robin fashion.\n",
+    "    \n",
+    "    For example, with three characters with indices: 1 2 3\n",
+    "    The storyteller is index 0.\n",
+    "    Then the selected index will be as follows:\n",
+    "\n",
+    "    step: 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16\n",
+    "\n",
+    "    idx:  0  1  0  2  0  3  0  1  0  2  0  3  0  1  0  2  0\n",
+    "    \"\"\"\n",
+    "    if step % 2 == 0:\n",
+    "        idx = 0\n",
+    "    else:\n",
+    "        idx = (step//2) % (len(agents)-1) + 1\n",
+    "    return idx"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(Dungeon Master): You have discovered that one of Voldemort's horcruxes is hidden deep in the Forbidden Forest. You must navigate the dangerous terrain, avoid the creatures lurking within, and find the horcrux before the full moon rises, unleashing a pack of hungry werewolves. Remember, time is of the essence!\n",
+      "\n",
+      "\n",
+      "(Harry Potter): I take out my wand and cast a Lumos spell to light our way through the dark forest. We need to move quickly and quietly to avoid any unwanted attention from the creatures. Ron, Hermione, and I will lead the way while Argus Filch keeps watch behind us. Let's go!\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): As you make your way through the forest, you hear the rustling of leaves and the snapping of twigs. Suddenly, a group of acromantulas, giant spiders, appear in front of you, blocking your path. What do you do?\n",
+      "\n",
+      "\n",
+      "(Ron Weasley): I quickly cast a spell to create a wall of fire between us and the acromantulas. Hopefully, the flames will deter them from attacking us. We need to keep moving forward and find that horcrux before it's too late.\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): The acromantulas hiss and retreat from the wall of fire, allowing you to pass. As you continue deeper into the forest, you come across a clearing with a small pond. In the center of the pond, you see a glowing object. It must be the horcrux! But how do you get to it? What do you do?\n",
+      "\n",
+      "\n",
+      "(Hermione Granger): I take out my wand and cast a spell to conjure a small boat. We can use it to reach the center of the pond and retrieve the horcrux. But we need to be careful, there could be traps or other obstacles in our way. Ron, Harry, let's row the boat while Argus Filch keeps watch from the shore.\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): As you row towards the center of the pond, you hear a loud hissing sound. Suddenly, a giant serpent emerges from the water, blocking your path. It looks angry and ready to attack. What do you do?\n",
+      "\n",
+      "\n",
+      "(Argus Filch): I take out my crossbow and aim it at the serpent. I may not be a wizard, but I know how to handle a weapon. I'll shoot it if it comes any closer. We can't let this serpent stop us from getting that horcrux.\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): The serpent lunges towards the boat, but Argus Filch's crossbow bolt hits it in the head, causing it to retreat back into the water. You reach the center of the pond and retrieve the glowing object, which turns out to be a locket. Congratulations, you have found one of Voldemort's horcruxes! But there are still six more to find. What challenges will you face next?\n",
+      "\n",
+      "\n",
+      "(Harry Potter): We need to regroup and figure out our next move. We should head back to Hogwarts and consult with Professor Dumbledore's portrait. He may have some insight on where the other horcruxes could be hidden. We can't waste any time, Voldemort is getting stronger every day. Let's go!\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): As you make your way back to Hogwarts, you hear a loud roar coming from the Forbidden Forest. It sounds like a werewolf. You must hurry before it catches up to you. You arrive at Dumbledore's office and he tells you that the next horcrux is hidden in a dangerous location. Are you ready for the next challenge?\n",
+      "\n",
+      "\n",
+      "(Ron Weasley): I'm always ready for a challenge! What's the location and what do we need to do to get there? We can't let Voldemort win, we have to find all of the horcruxes and destroy them. Let's do this!\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): Dumbledore tells you that the next horcrux is hidden in the depths of Gringotts Bank. You must break into the bank, navigate its treacherous security measures, and find the horcrux before the goblins catch you. Are you ready to face the challenge of a lifetime? The fate of the wizarding world rests in your hands.\n",
+      "\n",
+      "\n",
+      "(Hermione Granger): I suggest we do some research on Gringotts Bank and its security measures before we attempt to break in. We need to be prepared and have a solid plan in place. We can also gather any necessary tools or potions that may help us along the way. Let's not rush into this blindly.\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): As you research and plan your break-in to Gringotts Bank, you discover that the bank is heavily guarded by goblins, dragons, and other dangerous creatures. You'll need to be stealthy and quick to avoid detection. Are you ready to put your plan into action and face the dangers that await you? The clock is ticking, Voldemort's power grows stronger with each passing day.\n",
+      "\n",
+      "\n",
+      "(Argus Filch): I'll make sure to keep watch outside the bank while you all go in. I may not be able to help with the magic, but I can make sure no one interferes with our mission. We can't let anyone stop us from finding that horcrux and defeating Voldemort. Let's go!\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): As you approach Gringotts Bank, you see the imposing structure looming before you. You sneak past the guards and make your way inside, navigating the twisting corridors and avoiding the traps set to catch intruders. Finally, you reach the vault where the horcrux is hidden. But it's guarded by a fierce dragon. What do you do?\n",
+      "\n",
+      "\n",
+      "(Harry Potter): I remember the time when I faced a dragon during the Triwizard Tournament. I take out my wand and cast a spell to distract the dragon while Ron and Hermione retrieve the horcrux. We need to work together and be quick. Time is running out and we can't afford to fail.\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): The dragon roars and breathes fire, but Harry's spell distracts it long enough for Ron and Hermione to retrieve the horcrux. You make your way out of Gringotts Bank, but the goblins are hot on your trail. You must escape before they catch you. Congratulations, you have found another horcrux. But there are still five more to go. What challenges will you face next?\n",
+      "\n",
+      "\n",
+      "(Ron Weasley): We need to regroup and figure out our next move. We should consult with Professor Dumbledore's portrait again and see if he has any information on the next horcrux. We also need to be prepared for whatever challenges come our way. Voldemort won't make it easy for us, but we can't give up. Let's go!\n",
+      "\n",
+      "\n",
+      "(Dungeon Master): As you make your way back to Hogwarts, you hear a loud explosion coming from the direction of Hogsmeade. You arrive to find that Death Eaters have attacked the village and are wreaking havoc. You must fight off the Death Eaters and protect the innocent villagers. Are you ready to face this unexpected challenge and defend the wizarding world? The fate of both muggles and wizards rests in your hands.\n",
+      "\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "max_iters = 20\n",
+    "n = 0\n",
+    "\n",
+    "simulator = DialogueSimulator(\n",
+    "    agents=[storyteller] + characters,\n",
+    "    selection_function=select_next_speaker\n",
+    ")\n",
+    "simulator.reset(storyteller_name, specified_quest)\n",
+    "print(f\"({storyteller_name}): {specified_quest}\")\n",
+    "print('\\n')\n",
+    "\n",
+    "while n < max_iters:\n",
+    "    name, message = simulator.step()\n",
+    "    print(f\"({name}): {message}\")\n",
+    "    print('\\n')\n",
+    "    n += 1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/use_cases/agent_simulations/multiagent_bidding.ipynb

@@ -0,0 +1,823 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Multi-agent decentralized speaker selection\n",
+    "\n",
+    "This notebook showcases how to implement a multi-agent simulation without a fixed schedule for who speaks when. Instead the agents decide for themselves who speaks. We can implement this by having each agent bid to speak. Whichever agent's bid is the highest gets to speak.\n",
+    "\n",
+    "We will show how to do this in the example below that showcases a fictitious presidential debate."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Import LangChain related modules "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import PromptTemplate\n",
+    "import re\n",
+    "import tenacity\n",
+    "from typing import List, Dict, Callable\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.output_parsers import RegexParser\n",
+    "from langchain.schema import (\n",
+    "    AIMessage,\n",
+    "    HumanMessage,\n",
+    "    SystemMessage,\n",
+    "    BaseMessage,\n",
+    ")\n",
+    "from simulations import DialogueAgent, DialogueSimulator"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## `DialogueAgent` and `DialogueSimulator` classes\n",
+    "We will use the same `DialogueAgent` and `DialogueSimulator` classes defined in [Multi-Player Dungeons & Dragons](https://python.langchain.com/en/latest/use_cases/agent_simulations/multi_player_dnd.html)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class DialogueAgent:\n",
+    "    def __init__(\n",
+    "        self,\n",
+    "        name: str,\n",
+    "        system_message: SystemMessage,\n",
+    "        model: ChatOpenAI,\n",
+    "    ) -> None:\n",
+    "        self.name = name\n",
+    "        self.system_message = system_message\n",
+    "        self.model = model\n",
+    "        self.message_history = [\"Here is the conversation so far.\"]\n",
+    "        self.prefix = f\"{self.name}:\"\n",
+    "\n",
+    "    def send(self) -> str:\n",
+    "        \"\"\"\n",
+    "        Applies the chatmodel to the message history\n",
+    "        and returns the message string\n",
+    "        \"\"\"\n",
+    "        message = self.model(\n",
+    "            [\n",
+    "                self.system_message,\n",
+    "                HumanMessage(content=\"\\n\".join(self.message_history + [self.prefix])),\n",
+    "            ]\n",
+    "        )\n",
+    "        return message.content\n",
+    "\n",
+    "    def receive(self, name: str, message: str) -> None:\n",
+    "        \"\"\"\n",
+    "        Concatenates {message} spoken by {name} into message history\n",
+    "        \"\"\"\n",
+    "        self.message_history.append(f\"{name}: {message}\")\n",
+    "\n",
+    "\n",
+    "class DialogueSimulator:\n",
+    "    def __init__(\n",
+    "        self,\n",
+    "        agents: List[DialogueAgent],\n",
+    "        selection_function: Callable[[int, List[DialogueAgent]], int],\n",
+    "    ) -> None:\n",
+    "        self.agents = agents\n",
+    "        self._step = 0\n",
+    "        self.select_next_speaker = selection_function\n",
+    "\n",
+    "    def reset(self, name: str, message: str):\n",
+    "        \"\"\"\n",
+    "        Initiates the conversation with a {message} from {name}\n",
+    "        \"\"\"\n",
+    "        for agent in self.agents:\n",
+    "            agent.receive(name, message)\n",
+    "\n",
+    "        # increment time\n",
+    "        self._step += 1\n",
+    "\n",
+    "    def step(self) -> tuple[str, str]:\n",
+    "        # 1. choose the next speaker\n",
+    "        speaker_idx = self.select_next_speaker(self._step, self.agents)\n",
+    "        speaker = self.agents[speaker_idx]\n",
+    "\n",
+    "        # 2. next speaker sends message\n",
+    "        message = speaker.send()\n",
+    "\n",
+    "        # 3. everyone receives message\n",
+    "        for receiver in self.agents:\n",
+    "            receiver.receive(speaker.name, message)\n",
+    "\n",
+    "        # 4. increment time\n",
+    "        self._step += 1\n",
+    "\n",
+    "        return speaker.name, message"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## `BiddingDialogueAgent` class\n",
+    "We define a subclass of `DialogueAgent` that has a `bid()` method that produces a bid given the message history and the most recent message."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class BiddingDialogueAgent(DialogueAgent):\n",
+    "    def __init__(\n",
+    "        self,\n",
+    "        name,\n",
+    "        system_message: SystemMessage,\n",
+    "        bidding_template: PromptTemplate,\n",
+    "        model: ChatOpenAI,\n",
+    "    ) -> None:\n",
+    "        super().__init__(name, system_message, model)\n",
+    "        self.bidding_template = bidding_template\n",
+    "        \n",
+    "    def bid(self) -> str:\n",
+    "        \"\"\"\n",
+    "        Asks the chat model to output a bid to speak\n",
+    "        \"\"\"\n",
+    "        prompt = PromptTemplate(\n",
+    "            input_variables=['message_history', 'recent_message'],\n",
+    "            template = self.bidding_template\n",
+    "        ).format(\n",
+    "            message_history='\\n'.join(self.message_history),\n",
+    "            recent_message=self.message_history[-1])\n",
+    "        bid_string = self.model([SystemMessage(content=prompt)]).content\n",
+    "        return bid_string\n",
+    "        "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Define participants and debate topic"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "character_names = [\"Donald Trump\", \"Kanye West\", \"Elizabeth Warren\"]\n",
+    "topic = \"transcontinental high speed rail\"\n",
+    "word_limit = 50"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generate system messages"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "game_description = f\"\"\"Here is the topic for the presidential debate: {topic}.\n",
+    "The presidential candidates are: {', '.join(character_names)}.\"\"\"\n",
+    "\n",
+    "player_descriptor_system_message = SystemMessage(\n",
+    "    content=\"You can add detail to the description of each presidential candidate.\")\n",
+    "\n",
+    "def generate_character_description(character_name):\n",
+    "    character_specifier_prompt = [\n",
+    "        player_descriptor_system_message,\n",
+    "        HumanMessage(content=\n",
+    "            f\"\"\"{game_description}\n",
+    "            Please reply with a creative description of the presidential candidate, {character_name}, in {word_limit} words or less, that emphasizes their personalities. \n",
+    "            Speak directly to {character_name}.\n",
+    "            Do not add anything else.\"\"\"\n",
+    "            )\n",
+    "    ]\n",
+    "    character_description = ChatOpenAI(temperature=1.0)(character_specifier_prompt).content\n",
+    "    return character_description\n",
+    "\n",
+    "def generate_character_header(character_name, character_description):\n",
+    "    return f\"\"\"{game_description}\n",
+    "Your name is {character_name}.\n",
+    "You are a presidential candidate.\n",
+    "Your description is as follows: {character_description}\n",
+    "You are debating the topic: {topic}.\n",
+    "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+    "\"\"\"\n",
+    "\n",
+    "def generate_character_system_message(character_name, character_header):\n",
+    "    return SystemMessage(content=(\n",
+    "    f\"\"\"{character_header}\n",
+    "You will speak in the style of {character_name}, and exaggerate their personality.\n",
+    "You will come up with creative ideas related to {topic}.\n",
+    "Do not say the same things over and over again.\n",
+    "Speak in the first person from the perspective of {character_name}\n",
+    "For describing your own body movements, wrap your description in '*'.\n",
+    "Do not change roles!\n",
+    "Do not speak from the perspective of anyone else.\n",
+    "Speak only from the perspective of {character_name}.\n",
+    "Stop speaking the moment you finish speaking from your perspective.\n",
+    "Never forget to keep your response to {word_limit} words!\n",
+    "Do not add anything else.\n",
+    "    \"\"\"\n",
+    "    ))\n",
+    "\n",
+    "character_descriptions = [generate_character_description(character_name) for character_name in character_names]\n",
+    "character_headers = [generate_character_header(character_name, character_description) for character_name, character_description in zip(character_names, character_descriptions)]\n",
+    "character_system_messages = [generate_character_system_message(character_name, character_headers) for character_name, character_headers in zip(character_names, character_headers)]\n",
+    "                                                                                                                                       "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "Donald Trump Description:\n",
+      "\n",
+      "Donald Trump, you exude confidence and a bold personality. You are known for your unpredictability and your desire for greatness. You often speak your mind without reservation, which can be a strength but also a weakness.\n",
+      "\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Donald Trump.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Donald Trump, you exude confidence and a bold personality. You are known for your unpredictability and your desire for greatness. You often speak your mind without reservation, which can be a strength but also a weakness.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Donald Trump.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Donald Trump, you exude confidence and a bold personality. You are known for your unpredictability and your desire for greatness. You often speak your mind without reservation, which can be a strength but also a weakness.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "You will speak in the style of Donald Trump, and exaggerate their personality.\n",
+      "You will come up with creative ideas related to transcontinental high speed rail.\n",
+      "Do not say the same things over and over again.\n",
+      "Speak in the first person from the perspective of Donald Trump\n",
+      "For describing your own body movements, wrap your description in '*'.\n",
+      "Do not change roles!\n",
+      "Do not speak from the perspective of anyone else.\n",
+      "Speak only from the perspective of Donald Trump.\n",
+      "Stop speaking the moment you finish speaking from your perspective.\n",
+      "Never forget to keep your response to 50 words!\n",
+      "Do not add anything else.\n",
+      "    \n",
+      "\n",
+      "\n",
+      "Kanye West Description:\n",
+      "\n",
+      "Kanye West, you are a creative visionary who is unafraid to speak your mind. Your innovative approach to art and music has made you one of the most influential figures of our time. You bring a bold and unconventional perspective to this debate that I look forward to hearing.\n",
+      "\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Kanye West.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Kanye West, you are a creative visionary who is unafraid to speak your mind. Your innovative approach to art and music has made you one of the most influential figures of our time. You bring a bold and unconventional perspective to this debate that I look forward to hearing.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Kanye West.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Kanye West, you are a creative visionary who is unafraid to speak your mind. Your innovative approach to art and music has made you one of the most influential figures of our time. You bring a bold and unconventional perspective to this debate that I look forward to hearing.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "You will speak in the style of Kanye West, and exaggerate their personality.\n",
+      "You will come up with creative ideas related to transcontinental high speed rail.\n",
+      "Do not say the same things over and over again.\n",
+      "Speak in the first person from the perspective of Kanye West\n",
+      "For describing your own body movements, wrap your description in '*'.\n",
+      "Do not change roles!\n",
+      "Do not speak from the perspective of anyone else.\n",
+      "Speak only from the perspective of Kanye West.\n",
+      "Stop speaking the moment you finish speaking from your perspective.\n",
+      "Never forget to keep your response to 50 words!\n",
+      "Do not add anything else.\n",
+      "    \n",
+      "\n",
+      "\n",
+      "Elizabeth Warren Description:\n",
+      "\n",
+      "Elizabeth Warren, you are a fierce advocate for the middle class and a champion of progressive policies. Your tenacity and unwavering dedication to fighting for what you believe in have inspired many. Your policies are guided by a deep sense of empathy and a desire to help those who are most in need.\n",
+      "\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Elizabeth Warren.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Elizabeth Warren, you are a fierce advocate for the middle class and a champion of progressive policies. Your tenacity and unwavering dedication to fighting for what you believe in have inspired many. Your policies are guided by a deep sense of empathy and a desire to help those who are most in need.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Elizabeth Warren.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Elizabeth Warren, you are a fierce advocate for the middle class and a champion of progressive policies. Your tenacity and unwavering dedication to fighting for what you believe in have inspired many. Your policies are guided by a deep sense of empathy and a desire to help those who are most in need.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "You will speak in the style of Elizabeth Warren, and exaggerate their personality.\n",
+      "You will come up with creative ideas related to transcontinental high speed rail.\n",
+      "Do not say the same things over and over again.\n",
+      "Speak in the first person from the perspective of Elizabeth Warren\n",
+      "For describing your own body movements, wrap your description in '*'.\n",
+      "Do not change roles!\n",
+      "Do not speak from the perspective of anyone else.\n",
+      "Speak only from the perspective of Elizabeth Warren.\n",
+      "Stop speaking the moment you finish speaking from your perspective.\n",
+      "Never forget to keep your response to 50 words!\n",
+      "Do not add anything else.\n",
+      "    \n"
+     ]
+    }
+   ],
+   "source": [
+    "for character_name, character_description, character_header, character_system_message in zip(character_names, character_descriptions, character_headers, character_system_messages):\n",
+    "    print(f'\\n\\n{character_name} Description:')\n",
+    "    print(f'\\n{character_description}')\n",
+    "    print(f'\\n{character_header}')\n",
+    "    print(f'\\n{character_system_message.content}')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Output parser for bids\n",
+    "We ask the agents to output a bid to speak. But since the agents are LLMs that output strings, we need to \n",
+    "1. define a format they will produce their outputs in\n",
+    "2. parse their outputs\n",
+    "\n",
+    "We can subclass the [RegexParser](https://github.com/hwchase17/langchain/blob/master/langchain/output_parsers/regex.py) to implement our own custom output parser for bids."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class BidOutputParser(RegexParser):\n",
+    "    def get_format_instructions(self) -> str:\n",
+    "        return 'Your response should be an integer delimited by angled brackets, like this: <int>.'     \n",
+    "    \n",
+    "bid_parser = BidOutputParser(\n",
+    "    regex=r'<(\\d+)>', \n",
+    "    output_keys=['bid'],\n",
+    "    default_output_key='bid')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generate bidding system message\n",
+    "This is inspired by the prompt used in [Generative Agents](https://arxiv.org/pdf/2304.03442.pdf) for using an LLM to determine the importance of memories. This will use the formatting instructions from our `BidOutputParser`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def generate_character_bidding_template(character_header):\n",
+    "    bidding_template = (\n",
+    "    f\"\"\"{character_header}\n",
+    "\n",
+    "```\n",
+    "{{message_history}}\n",
+    "```\n",
+    "\n",
+    "On the scale of 1 to 10, where 1 is not contradictory and 10 is extremely contradictory, rate how contradictory the following message is to your ideas.\n",
+    "\n",
+    "```\n",
+    "{{recent_message}}\n",
+    "```\n",
+    "\n",
+    "{bid_parser.get_format_instructions()}\n",
+    "Do nothing else.\n",
+    "    \"\"\")\n",
+    "    return bidding_template\n",
+    "\n",
+    "character_bidding_templates = [generate_character_bidding_template(character_header) for character_header in character_headers]\n",
+    "                                      \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Donald Trump Bidding Template:\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Donald Trump.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Donald Trump, you exude confidence and a bold personality. You are known for your unpredictability and your desire for greatness. You often speak your mind without reservation, which can be a strength but also a weakness.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "\n",
+      "```\n",
+      "{message_history}\n",
+      "```\n",
+      "\n",
+      "On the scale of 1 to 10, where 1 is not contradictory and 10 is extremely contradictory, rate how contradictory the following message is to your ideas.\n",
+      "\n",
+      "```\n",
+      "{recent_message}\n",
+      "```\n",
+      "\n",
+      "Your response should be an integer delimited by angled brackets, like this: <int>.\n",
+      "Do nothing else.\n",
+      "    \n",
+      "Kanye West Bidding Template:\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Kanye West.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Kanye West, you are a creative visionary who is unafraid to speak your mind. Your innovative approach to art and music has made you one of the most influential figures of our time. You bring a bold and unconventional perspective to this debate that I look forward to hearing.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "\n",
+      "```\n",
+      "{message_history}\n",
+      "```\n",
+      "\n",
+      "On the scale of 1 to 10, where 1 is not contradictory and 10 is extremely contradictory, rate how contradictory the following message is to your ideas.\n",
+      "\n",
+      "```\n",
+      "{recent_message}\n",
+      "```\n",
+      "\n",
+      "Your response should be an integer delimited by angled brackets, like this: <int>.\n",
+      "Do nothing else.\n",
+      "    \n",
+      "Elizabeth Warren Bidding Template:\n",
+      "Here is the topic for the presidential debate: transcontinental high speed rail.\n",
+      "The presidential candidates are: Donald Trump, Kanye West, Elizabeth Warren.\n",
+      "Your name is Elizabeth Warren.\n",
+      "You are a presidential candidate.\n",
+      "Your description is as follows: Elizabeth Warren, you are a fierce advocate for the middle class and a champion of progressive policies. Your tenacity and unwavering dedication to fighting for what you believe in have inspired many. Your policies are guided by a deep sense of empathy and a desire to help those who are most in need.\n",
+      "You are debating the topic: transcontinental high speed rail.\n",
+      "Your goal is to be as creative as possible and make the voters think you are the best candidate.\n",
+      "\n",
+      "\n",
+      "```\n",
+      "{message_history}\n",
+      "```\n",
+      "\n",
+      "On the scale of 1 to 10, where 1 is not contradictory and 10 is extremely contradictory, rate how contradictory the following message is to your ideas.\n",
+      "\n",
+      "```\n",
+      "{recent_message}\n",
+      "```\n",
+      "\n",
+      "Your response should be an integer delimited by angled brackets, like this: <int>.\n",
+      "Do nothing else.\n",
+      "    \n"
+     ]
+    }
+   ],
+   "source": [
+    "for character_name, bidding_template in zip(character_names, character_bidding_templates):\n",
+    "    print(f'{character_name} Bidding Template:')\n",
+    "    print(bidding_template)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Use an LLM to create an elaborate on debate topic"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Original topic:\n",
+      "transcontinental high speed rail\n",
+      "\n",
+      "Detailed topic:\n",
+      "Candidates, with the rise of autonomous technologies, we must address the problem of how to integrate them into our proposed transcontinental high speed rail system. Outline your plan on how to safely integrate autonomous vehicles into rail travel, balancing the need for innovation and safety.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "topic_specifier_prompt = [\n",
+    "    SystemMessage(content=\"You can make a task more specific.\"),\n",
+    "    HumanMessage(content=\n",
+    "        f\"\"\"{game_description}\n",
+    "        \n",
+    "        You are the debate moderator.\n",
+    "        Please make the debate topic more specific. \n",
+    "        Frame the debate topic as a problem to be solved.\n",
+    "        Be creative and imaginative.\n",
+    "        Please reply with the specified topic in {word_limit} words or less. \n",
+    "        Speak directly to the presidential candidates: {*character_names,}.\n",
+    "        Do not add anything else.\"\"\"\n",
+    "        )\n",
+    "]\n",
+    "specified_topic = ChatOpenAI(temperature=1.0)(topic_specifier_prompt).content\n",
+    "\n",
+    "print(f\"Original topic:\\n{topic}\\n\")\n",
+    "print(f\"Detailed topic:\\n{specified_topic}\\n\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Define the speaker selection function\n",
+    "Lastly we will define a speaker selection function `select_next_speaker` that takes each agent's bid and selects the agent with the highest bid (with ties broken randomly).\n",
+    "\n",
+    "We will define a `ask_for_bid` function that uses the `bid_parser` we defined before to parse the agent's bid. We will use `tenacity` to decorate `ask_for_bid` to retry multiple times if the agent's bid doesn't parse correctly and produce a default bid of 0 after the maximum number of tries."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "@tenacity.retry(stop=tenacity.stop_after_attempt(2),\n",
+    "                    wait=tenacity.wait_none(),  # No waiting time between retries\n",
+    "                    retry=tenacity.retry_if_exception_type(ValueError),\n",
+    "                    before_sleep=lambda retry_state: print(f\"ValueError occurred: {retry_state.outcome.exception()}, retrying...\"),\n",
+    "                    retry_error_callback=lambda retry_state: 0) # Default value when all retries are exhausted\n",
+    "def ask_for_bid(agent) -> str:\n",
+    "    \"\"\"\n",
+    "    Ask for agent bid and parses the bid into the correct format.\n",
+    "    \"\"\"\n",
+    "    bid_string = agent.bid()\n",
+    "    bid = int(bid_parser.parse(bid_string)['bid'])\n",
+    "    return bid"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "def select_next_speaker(step: int, agents: List[DialogueAgent]) -> int:\n",
+    "    bids = []\n",
+    "    for agent in agents:\n",
+    "        bid = ask_for_bid(agent)\n",
+    "        bids.append(bid)\n",
+    "        \n",
+    "    # randomly select among multiple agents with the same bid\n",
+    "    max_value = np.max(bids)\n",
+    "    max_indices = np.where(bids == max_value)[0]\n",
+    "    idx = np.random.choice(max_indices)\n",
+    "    \n",
+    "    print('Bids:')\n",
+    "    for i, (bid, agent) in enumerate(zip(bids, agents)):\n",
+    "        print(f'\\t{agent.name} bid: {bid}')\n",
+    "        if i == idx:\n",
+    "            selected_name = agent.name\n",
+    "    print(f'Selected: {selected_name}')\n",
+    "    print('\\n')\n",
+    "    return idx"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Main Loop"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "characters = []\n",
+    "for character_name, character_system_message, bidding_template in zip(character_names, character_system_messages, character_bidding_templates):\n",
+    "    characters.append(BiddingDialogueAgent(\n",
+    "        name=character_name,\n",
+    "        system_message=character_system_message,\n",
+    "        model=ChatOpenAI(temperature=0.2),\n",
+    "        bidding_template=bidding_template,\n",
+    "    ))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(Debate Moderator): Candidates, with the rise of autonomous technologies, we must address the problem of how to integrate them into our proposed transcontinental high speed rail system. Outline your plan on how to safely integrate autonomous vehicles into rail travel, balancing the need for innovation and safety.\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 8\n",
+      "\tKanye West bid: 2\n",
+      "\tElizabeth Warren bid: 1\n",
+      "Selected: Donald Trump\n",
+      "\n",
+      "\n",
+      "(Donald Trump): Let me tell you, folks, I have the best plan for integrating autonomous vehicles into our high speed rail system. We're going to use the latest technology, the best technology, to ensure safety and efficiency. And let me tell you, we're going to do it in style. We're going to have luxury autonomous cars that will make you feel like you're in a private jet. It's going to be tremendous, believe me. *gestures with hands*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 8\n",
+      "\tKanye West bid: 7\n",
+      "\tElizabeth Warren bid: 10\n",
+      "Selected: Elizabeth Warren\n",
+      "\n",
+      "\n",
+      "(Elizabeth Warren): Thank you for the question. As someone who has always fought for the safety and well-being of the American people, I believe that any plan for integrating autonomous vehicles into our high speed rail system must prioritize safety above all else. We need to ensure that these vehicles are thoroughly tested and meet strict safety standards before they are allowed on our rails. Additionally, we must invest in the necessary infrastructure to support these vehicles, such as advanced sensors and communication systems. But we must also ensure that these innovations are accessible to all Americans, not just the wealthy. That's why I propose a public-private partnership to fund and build this system, with a focus on creating good-paying jobs and expanding economic opportunities for all Americans. *smiles confidently*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 8\n",
+      "\tKanye West bid: 2\n",
+      "\tElizabeth Warren bid: 1\n",
+      "Selected: Donald Trump\n",
+      "\n",
+      "\n",
+      "(Donald Trump): Let me tell you, Elizabeth, safety is important, but we also need to think about innovation and progress. We can't let fear hold us back from achieving greatness. That's why I propose a competition, a race to see which company can create the safest and most efficient autonomous vehicles for our high speed rail system. And let me tell you, the winner will receive a huge government contract and be hailed as a hero. It's going to be tremendous, folks. *points finger*\n",
+      "\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Bids:\n",
+      "\tDonald Trump bid: 3\n",
+      "\tKanye West bid: 8\n",
+      "\tElizabeth Warren bid: 8\n",
+      "Selected: Kanye West\n",
+      "\n",
+      "\n",
+      "(Kanye West): Yo, yo, yo, let me jump in here. First of all, I gotta say, I love innovation and progress. But we can't forget about the people, man. We need to make sure that this high speed rail system is accessible to everyone, not just the wealthy. And that means we need to invest in public transportation, not just luxury autonomous cars. We need to make sure that people can get from point A to point B safely and efficiently, without breaking the bank. And let me tell you, we can do it in style too. We can have art installations and live performances on the trains, making it a cultural experience. *starts nodding head*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 7\n",
+      "\tKanye West bid: 2\n",
+      "\tElizabeth Warren bid: 1\n",
+      "Selected: Donald Trump\n",
+      "\n",
+      "\n",
+      "(Donald Trump): Kanye, I hear what you're saying, but let's not forget about the importance of luxury and comfort. We need to make sure that our high speed rail system is not only accessible, but also enjoyable. That's why I propose that we have different tiers of service, from economy to first class, so that everyone can choose the level of luxury they want. And let me tell you, the first class experience will be something else. We're talking about gourmet meals, personal attendants, and even spa services. It's going to be tremendous, folks. *smirks confidently*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 2\n",
+      "\tKanye West bid: 8\n",
+      "\tElizabeth Warren bid: 10\n",
+      "Selected: Elizabeth Warren\n",
+      "\n",
+      "\n",
+      "(Elizabeth Warren): I agree with Kanye that we need to prioritize accessibility and affordability for all Americans. But we also need to think about the environmental impact of this system. That's why I propose that we invest in renewable energy sources to power our high speed rail system, such as solar and wind power. We can also use this opportunity to create green jobs and reduce our carbon footprint. And let's not forget about the importance of public input and transparency in this process. We need to engage with communities and listen to their concerns and ideas. *raises hand in emphasis*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 8\n",
+      "\tKanye West bid: 1\n",
+      "\tElizabeth Warren bid: 1\n",
+      "Selected: Donald Trump\n",
+      "\n",
+      "\n",
+      "(Donald Trump): Let me tell you, Elizabeth, I agree that we need to think about the environment, but we also need to think about the economy. That's why I propose that we use American-made materials and labor to build this high speed rail system. We're going to create jobs and boost our economy, all while creating a world-class transportation system. And let me tell you, it's going to be beautiful. We're going to have the best trains, the best tracks, and the best stations. It's going to be tremendous, folks. *smiles confidently*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 2\n",
+      "\tKanye West bid: 7\n",
+      "\tElizabeth Warren bid: 8\n",
+      "Selected: Elizabeth Warren\n",
+      "\n",
+      "\n",
+      "(Elizabeth Warren): Thank you, but let's not forget that we also need to prioritize safety and reliability. We can't cut corners or sacrifice quality for the sake of speed or cost. That's why I propose that we have rigorous safety and maintenance standards, with regular inspections and repairs. And we need to invest in training and support for our rail workers, so that they can operate and maintain this system with the highest level of expertise and care. *firmly nods head*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 2\n",
+      "\tKanye West bid: 1\n",
+      "\tElizabeth Warren bid: 1\n",
+      "Selected: Donald Trump\n",
+      "\n",
+      "\n",
+      "(Donald Trump): Let me tell you, Elizabeth, safety is important, but we also need to think about efficiency and speed. That's why I propose that we use the latest technology, such as artificial intelligence and machine learning, to monitor and maintain our high speed rail system. We can detect and fix any issues before they become a problem, ensuring that our trains run smoothly and on time. And let me tell you, we're going to be the envy of the world with this system. It's going to be tremendous, folks. *smirks confidently*\n",
+      "\n",
+      "\n",
+      "Bids:\n",
+      "\tDonald Trump bid: 2\n",
+      "\tKanye West bid: 8\n",
+      "\tElizabeth Warren bid: 8\n",
+      "Selected: Kanye West\n",
+      "\n",
+      "\n",
+      "(Kanye West): Yo, yo, yo, let me jump in here again. I hear what both of y'all are saying, but let's not forget about the culture, man. We need to make sure that this high speed rail system reflects the diversity and creativity of our country. That means we need to have art installations, live performances, and even fashion shows on the trains. We can showcase the best of American culture and inspire people from all over the world. And let me tell you, it's going to be a vibe. *starts swaying to the beat*\n",
+      "\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "max_iters = 10\n",
+    "n = 0\n",
+    "\n",
+    "simulator = DialogueSimulator(\n",
+    "    agents=characters,\n",
+    "    selection_function=select_next_speaker\n",
+    ")\n",
+    "simulator.reset('Debate Moderator', specified_topic)\n",
+    "print(f\"(Debate Moderator): {specified_topic}\")\n",
+    "print('\\n')\n",
+    "\n",
+    "while n < max_iters:\n",
+    "    name, message = simulator.step()\n",
+    "    print(f\"({name}): {message}\")\n",
+    "    print('\\n')\n",
+    "    n += 1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/use_cases/agents/custom_agent_with_plugin_retrieval_using_plugnplai.ipynb

@@ -0,0 +1,562 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "ba5f8741",
+   "metadata": {},
+   "source": [
+    "# Plug-and-Plai\n",
+    "\n",
+    "This notebook builds upon the idea of [tool retrieval](custom_agent_with_plugin_retrieval.html), but pulls all tools from `plugnplai` - a directory of AI Plugins."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fea4812c",
+   "metadata": {},
+   "source": [
+    "## Set up environment\n",
+    "\n",
+    "Do necessary imports, etc."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "aca08be8",
+   "metadata": {},
+   "source": [
+    "Install plugnplai lib to get a list of active plugins from https://plugplai.com directory"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "52e248c9",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.1\u001b[0m\n",
+      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
+      "Note: you may need to restart the kernel to use updated packages.\n"
+     ]
+    }
+   ],
+   "source": [
+    "pip install plugnplai -q"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "9af9734e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser\n",
+    "from langchain.prompts import StringPromptTemplate\n",
+    "from langchain import OpenAI, SerpAPIWrapper, LLMChain\n",
+    "from typing import List, Union\n",
+    "from langchain.schema import AgentAction, AgentFinish\n",
+    "from langchain.agents.agent_toolkits import NLAToolkit\n",
+    "from langchain.tools.plugin import AIPlugin\n",
+    "import re\n",
+    "import plugnplai"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2f91d8b4",
+   "metadata": {},
+   "source": [
+    "## Setup LLM"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "a1a3b59c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(temperature=0)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6df0253f",
+   "metadata": {},
+   "source": [
+    "## Set up plugins\n",
+    "\n",
+    "Load and index plugins"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "9e0f7882",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Get all plugins from plugnplai.com\n",
+    "urls = plugnplai.get_plugins()\n",
+    "\n",
+    "#  Get ChatGPT plugins - only ChatGPT verified plugins\n",
+    "urls = plugnplai.get_plugins(filter = 'ChatGPT')\n",
+    "\n",
+    "#  Get working plugins - only tested plugins (in progress)\n",
+    "urls = plugnplai.get_plugins(filter = 'working')\n",
+    "\n",
+    "\n",
+    "AI_PLUGINS = [AIPlugin.from_url(url + \"/.well-known/ai-plugin.json\") for url in urls]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "17362717",
+   "metadata": {},
+   "source": [
+    "## Tool Retriever\n",
+    "\n",
+    "We will use a vectorstore to create embeddings for each tool description. Then, for an incoming query we can create embeddings for that query and do a similarity search for relevant tools."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "77c4be4b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.vectorstores import FAISS\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.schema import Document"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "9092a158",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load an OpenAPI 3.0.2 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
+      "Attempting to load a Swagger 2.0 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n"
+     ]
+    }
+   ],
+   "source": [
+    "embeddings = OpenAIEmbeddings()\n",
+    "docs = [\n",
+    "    Document(page_content=plugin.description_for_model, \n",
+    "             metadata={\"plugin_name\": plugin.name_for_model}\n",
+    "            )\n",
+    "    for plugin in AI_PLUGINS\n",
+    "]\n",
+    "vector_store = FAISS.from_documents(docs, embeddings)\n",
+    "toolkits_dict = {plugin.name_for_model: \n",
+    "                 NLAToolkit.from_llm_and_ai_plugin(llm, plugin) \n",
+    "                 for plugin in AI_PLUGINS}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "735a7566",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "retriever = vector_store.as_retriever()\n",
+    "\n",
+    "def get_tools(query):\n",
+    "    # Get documents, which contain the Plugins to use\n",
+    "    docs = retriever.get_relevant_documents(query)\n",
+    "    # Get the toolkits, one for each plugin\n",
+    "    tool_kits = [toolkits_dict[d.metadata[\"plugin_name\"]] for d in docs]\n",
+    "    # Get the tools: a separate NLAChain for each endpoint\n",
+    "    tools = []\n",
+    "    for tk in tool_kits:\n",
+    "        tools.extend(tk.nla_tools)\n",
+    "    return tools"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7699afd7",
+   "metadata": {},
+   "source": [
+    "We can now test this retriever to see if it seems to work."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "425f2886",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['Milo.askMilo',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.search_all_actions',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.preview_a_zap',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.get_configuration_link',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.list_exposed_actions',\n",
+       " 'SchoolDigger_API_V2.0.Autocomplete_GetSchools',\n",
+       " 'SchoolDigger_API_V2.0.Districts_GetAllDistricts2',\n",
+       " 'SchoolDigger_API_V2.0.Districts_GetDistrict2',\n",
+       " 'SchoolDigger_API_V2.0.Rankings_GetSchoolRank2',\n",
+       " 'SchoolDigger_API_V2.0.Rankings_GetRank_District',\n",
+       " 'SchoolDigger_API_V2.0.Schools_GetAllSchools20',\n",
+       " 'SchoolDigger_API_V2.0.Schools_GetSchool20',\n",
+       " 'Speak.translate',\n",
+       " 'Speak.explainPhrase',\n",
+       " 'Speak.explainTask']"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "tools = get_tools(\"What could I do today with my kiddo\")\n",
+    "[t.name for t in tools]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "3aa88768",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['Open_AI_Klarna_product_Api.productsUsingGET',\n",
+       " 'Milo.askMilo',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.search_all_actions',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.preview_a_zap',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.get_configuration_link',\n",
+       " 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.list_exposed_actions',\n",
+       " 'SchoolDigger_API_V2.0.Autocomplete_GetSchools',\n",
+       " 'SchoolDigger_API_V2.0.Districts_GetAllDistricts2',\n",
+       " 'SchoolDigger_API_V2.0.Districts_GetDistrict2',\n",
+       " 'SchoolDigger_API_V2.0.Rankings_GetSchoolRank2',\n",
+       " 'SchoolDigger_API_V2.0.Rankings_GetRank_District',\n",
+       " 'SchoolDigger_API_V2.0.Schools_GetAllSchools20',\n",
+       " 'SchoolDigger_API_V2.0.Schools_GetSchool20']"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "tools = get_tools(\"what shirts can i buy?\")\n",
+    "[t.name for t in tools]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2e7a075c",
+   "metadata": {},
+   "source": [
+    "## Prompt Template\n",
+    "\n",
+    "The prompt template is pretty standard, because we're not actually changing that much logic in the actual prompt template, but rather we are just changing how retrieval is done."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "339b1bb8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Set up the base template\n",
+    "template = \"\"\"Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:\n",
+    "\n",
+    "{tools}\n",
+    "\n",
+    "Use the following format:\n",
+    "\n",
+    "Question: the input question you must answer\n",
+    "Thought: you should always think about what to do\n",
+    "Action: the action to take, should be one of [{tool_names}]\n",
+    "Action Input: the input to the action\n",
+    "Observation: the result of the action\n",
+    "... (this Thought/Action/Action Input/Observation can repeat N times)\n",
+    "Thought: I now know the final answer\n",
+    "Final Answer: the final answer to the original input question\n",
+    "\n",
+    "Begin! Remember to speak as a pirate when giving your final answer. Use lots of \"Arg\"s\n",
+    "\n",
+    "Question: {input}\n",
+    "{agent_scratchpad}\"\"\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1583acdc",
+   "metadata": {},
+   "source": [
+    "The custom prompt template now has the concept of a tools_getter, which we call on the input to select the tools to use"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "fd969d31",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import Callable\n",
+    "# Set up a prompt template\n",
+    "class CustomPromptTemplate(StringPromptTemplate):\n",
+    "    # The template to use\n",
+    "    template: str\n",
+    "    ############## NEW ######################\n",
+    "    # The list of tools available\n",
+    "    tools_getter: Callable\n",
+    "    \n",
+    "    def format(self, **kwargs) -> str:\n",
+    "        # Get the intermediate steps (AgentAction, Observation tuples)\n",
+    "        # Format them in a particular way\n",
+    "        intermediate_steps = kwargs.pop(\"intermediate_steps\")\n",
+    "        thoughts = \"\"\n",
+    "        for action, observation in intermediate_steps:\n",
+    "            thoughts += action.log\n",
+    "            thoughts += f\"\\nObservation: {observation}\\nThought: \"\n",
+    "        # Set the agent_scratchpad variable to that value\n",
+    "        kwargs[\"agent_scratchpad\"] = thoughts\n",
+    "        ############## NEW ######################\n",
+    "        tools = self.tools_getter(kwargs[\"input\"])\n",
+    "        # Create a tools variable from the list of tools provided\n",
+    "        kwargs[\"tools\"] = \"\\n\".join([f\"{tool.name}: {tool.description}\" for tool in tools])\n",
+    "        # Create a list of tool names for the tools provided\n",
+    "        kwargs[\"tool_names\"] = \", \".join([tool.name for tool in tools])\n",
+    "        return self.template.format(**kwargs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "798ef9fb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt = CustomPromptTemplate(\n",
+    "    template=template,\n",
+    "    tools_getter=get_tools,\n",
+    "    # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically\n",
+    "    # This includes the `intermediate_steps` variable because that is needed\n",
+    "    input_variables=[\"input\", \"intermediate_steps\"]\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ef3a1af3",
+   "metadata": {},
+   "source": [
+    "## Output Parser\n",
+    "\n",
+    "The output parser is unchanged from the previous notebook, since we are not changing anything about the output format."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "7c6fe0d3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class CustomOutputParser(AgentOutputParser):\n",
+    "    \n",
+    "    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:\n",
+    "        # Check if agent should finish\n",
+    "        if \"Final Answer:\" in llm_output:\n",
+    "            return AgentFinish(\n",
+    "                # Return values is generally always a dictionary with a single `output` key\n",
+    "                # It is not recommended to try anything else at the moment :)\n",
+    "                return_values={\"output\": llm_output.split(\"Final Answer:\")[-1].strip()},\n",
+    "                log=llm_output,\n",
+    "            )\n",
+    "        # Parse out the action and action input\n",
+    "        regex = r\"Action\\s*\\d*\\s*:(.*?)\\nAction\\s*\\d*\\s*Input\\s*\\d*\\s*:[\\s]*(.*)\"\n",
+    "        match = re.search(regex, llm_output, re.DOTALL)\n",
+    "        if not match:\n",
+    "            raise ValueError(f\"Could not parse LLM output: `{llm_output}`\")\n",
+    "        action = match.group(1).strip()\n",
+    "        action_input = match.group(2)\n",
+    "        # Return the action and action input\n",
+    "        return AgentAction(tool=action, tool_input=action_input.strip(\" \").strip('\"'), log=llm_output)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "d278706a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "output_parser = CustomOutputParser()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "170587b1",
+   "metadata": {},
+   "source": [
+    "## Set up LLM, stop sequence, and the agent\n",
+    "\n",
+    "Also the same as the previous notebook"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "f9d4c374",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(temperature=0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "9b1cc2a2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# LLM chain consisting of the LLM and a prompt\n",
+    "llm_chain = LLMChain(llm=llm, prompt=prompt)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "e4f5092f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool_names = [tool.name for tool in tools]\n",
+    "agent = LLMSingleActionAgent(\n",
+    "    llm_chain=llm_chain, \n",
+    "    output_parser=output_parser,\n",
+    "    stop=[\"\\nObservation:\"], \n",
+    "    allowed_tools=tool_names\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "aa8a5326",
+   "metadata": {},
+   "source": [
+    "## Use the Agent\n",
+    "\n",
+    "Now we can use it!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "490604e9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "653b1617",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mThought: I need to find a product API\n",
+      "Action: Open_AI_Klarna_product_Api.productsUsingGET\n",
+      "Action Input: shirts\u001b[0m\n",
+      "\n",
+      "Observation:\u001b[36;1m\u001b[1;3mI found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.\u001b[0m\u001b[32;1m\u001b[1;3m I now know what shirts I can buy\n",
+      "Final Answer: Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.'"
+      ]
+     },
+     "execution_count": 18,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent_executor.run(\"what shirts can i buy?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2481ee76",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "3ccef4e08d87aa1eeb90f63e0f071292ccb2e9c42e70f74ab2bf6f5493ca7bbc"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/use_cases/autonomous_agents/marathon_times.ipynb

@@ -219,7 +219,7 @@
    },
    "outputs": [],
    "source": [
-    "from langchain.tools import BaseTool, DuckDuckGoSearchTool\n",
+    "from langchain.tools import BaseTool, DuckDuckGoSearchRun\n",
     "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
     "\n",
     "from pydantic import Field\n",
@@ -321,7 +321,7 @@
    "outputs": [],
    "source": [
     "# !pip install duckduckgo_search\n",
-    "web_search = DuckDuckGoSearchTool()"
+    "web_search = DuckDuckGoSearchRun()"
    ]
   },
   {
@@ -618,7 +618,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.2"
+   "version": "3.8.16"
   }
  },
  "nbformat": 4,

docs/use_cases/evaluation/agent_benchmarking.ipynb

@@ -283,7 +283,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.2"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,

docs/use_cases/evaluation/generic_agent_evaluation.ipynb

@@ -0,0 +1,342 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Generic Agent Evaluation\n",
+    "\n",
+    "Good evaluation is key for quickly iterating on your agent's prompts and tools. Here we provide an example of how to use the TrajectoryEvalChain to evaluate your agent."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup\n",
+    "\n",
+    "Let's start by defining our agent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import Wikipedia\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.agents import initialize_agent, Tool\n",
+    "from langchain.agents import AgentType\n",
+    "from langchain.agents.react.base import DocstoreExplorer\n",
+    "from langchain.memory import ConversationBufferMemory\n",
+    "from langchain import LLMMathChain\n",
+    "from langchain.llms import OpenAI\n",
+    "\n",
+    "from langchain import SerpAPIWrapper\n",
+    "\n",
+    "docstore = DocstoreExplorer(Wikipedia())\n",
+    "\n",
+    "math_llm = OpenAI(temperature=0)\n",
+    "\n",
+    "llm_math_chain = LLMMathChain(llm=math_llm, verbose=True)\n",
+    "\n",
+    "search = SerpAPIWrapper()\n",
+    "\n",
+    "tools = [\n",
+    "    Tool(\n",
+    "        name=\"Search\",\n",
+    "        func=docstore.search,\n",
+    "        description=\"useful for when you need to ask with search\",\n",
+    "    ),\n",
+    "    Tool(\n",
+    "        name=\"Lookup\",\n",
+    "        func=docstore.lookup,\n",
+    "        description=\"useful for when you need to ask with lookup\",\n",
+    "    ),\n",
+    "    Tool(\n",
+    "        name=\"Calculator\",\n",
+    "        func=llm_math_chain.run,\n",
+    "        description=\"useful for doing calculations\",\n",
+    "    ),\n",
+    "    Tool(\n",
+    "        name=\"Search the Web (SerpAPI)\",\n",
+    "        func=search.run,\n",
+    "        description=\"useful for when you need to answer questions about current events\",\n",
+    "    ),\n",
+    "]\n",
+    "\n",
+    "memory = ConversationBufferMemory(\n",
+    "    memory_key=\"chat_history\", return_messages=True, output_key=\"output\"\n",
+    ")\n",
+    "\n",
+    "llm = ChatOpenAI(temperature=0, model_name=\"gpt-3.5-turbo\")\n",
+    "\n",
+    "agent = initialize_agent(\n",
+    "    tools,\n",
+    "    llm,\n",
+    "    agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,\n",
+    "    verbose=True,\n",
+    "    memory=memory,\n",
+    "    return_intermediate_steps=True, # This is needed for the evaluation later\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Testing the Agent\n",
+    "\n",
+    "Now let's try our agent out on some example queries."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3m{\n",
+      "    \"action\": \"Search the Web (SerpAPI)\",\n",
+      "    \"action_input\": \"How many ping pong balls would it take to fill the entire Empire State Building?\"\n",
+      "}\u001b[0m\n",
+      "Observation: \u001b[31;1m\u001b[1;3m12.8 billion. The volume of the Empire State Building Googles in at around 37 million ft³. A golf ball comes in at about 2.5 in³.\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m{\n",
+      "    \"action\": \"Final Answer\",\n",
+      "    \"action_input\": \"It would take approximately 12.8 billion ping pong balls to fill the entire Empire State Building.\"\n",
+      "}\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "query_one = \"How many ping pong balls would it take to fill the entire Empire State Building?\"\n",
+    "\n",
+    "test_outputs_one = agent({\"input\": query_one}, return_only_outputs=False)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This looks good! Let's try it out on another query."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3m{\n",
+      "    \"action\": \"Calculator\",\n",
+      "    \"action_input\": \"The length of the Eiffel Tower is 324 meters. The distance from coast to coast in the US is approximately 4,828 kilometers. First, we need to convert 4,828 kilometers to meters, which gives us 4,828,000 meters. To find out how many Eiffel Towers we need, we can divide 4,828,000 by 324. This gives us approximately 14,876 Eiffel Towers.\"\n",
+      "}\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n",
+      "The length of the Eiffel Tower is 324 meters. The distance from coast to coast in the US is approximately 4,828 kilometers. First, we need to convert 4,828 kilometers to meters, which gives us 4,828,000 meters. To find out how many Eiffel Towers we need, we can divide 4,828,000 by 324. This gives us approximately 14,876 Eiffel Towers.\u001b[32;1m\u001b[1;3m\n",
+      "```text\n",
+      "4828000 / 324\n",
+      "```\n",
+      "...numexpr.evaluate(\"4828000 / 324\")...\n",
+      "\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3m14901.234567901234\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "Observation: \u001b[38;5;200m\u001b[1;3mAnswer: 14901.234567901234\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m{\n",
+      "    \"action\": \"Calculator\",\n",
+      "    \"action_input\": \"The length of the Eiffel Tower is 324 meters. The distance from coast to coast in the US is approximately 4,828 kilometers. First, we need to convert 4,828 kilometers to meters, which gives us 4,828,000 meters. To find out how many Eiffel Towers we need, we can divide 4,828,000 by 324. This gives us approximately 14,901 Eiffel Towers.\"\n",
+      "}\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n",
+      "The length of the Eiffel Tower is 324 meters. The distance from coast to coast in the US is approximately 4,828 kilometers. First, we need to convert 4,828 kilometers to meters, which gives us 4,828,000 meters. To find out how many Eiffel Towers we need, we can divide 4,828,000 by 324. This gives us approximately 14,901 Eiffel Towers.\u001b[32;1m\u001b[1;3m\n",
+      "```text\n",
+      "4828000 / 324\n",
+      "```\n",
+      "...numexpr.evaluate(\"4828000 / 324\")...\n",
+      "\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3m14901.234567901234\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "Observation: \u001b[38;5;200m\u001b[1;3mAnswer: 14901.234567901234\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m{\n",
+      "    \"action\": \"Final Answer\",\n",
+      "    \"action_input\": \"If you laid the Eiffel Tower end to end, you would need approximately 14,901 Eiffel Towers to cover the US from coast to coast.\"\n",
+      "}\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "query_two = \"If you laid the Eiffel Tower end to end, how many would you need cover the US from coast to coast?\"\n",
+    "\n",
+    "test_outputs_two = agent({\"input\": query_two}, return_only_outputs=False)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This doesn't look so good. Let's try running some evaluation.\n",
+    "\n",
+    "## Evaluating the Agent\n",
+    "\n",
+    "Let's start by defining the TrajectoryEvalChain."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.evaluation.agents import TrajectoryEvalChain\n",
+    "\n",
+    "# Define chain\n",
+    "eval_chain = TrajectoryEvalChain.from_llm(\n",
+    "    llm=ChatOpenAI(temperature=0, model_name=\"gpt-4\"), # Note: This must be a ChatOpenAI model\n",
+    "    agent_tools=agent.tools,\n",
+    "    return_reasoning=True,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's try evaluating the first query."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Score from 1 to 5:  1\n",
+      "Reasoning:  First, let's evaluate the final answer. The final answer is incorrect because it uses the volume of golf balls instead of ping pong balls. The answer is not helpful.\n",
+      "\n",
+      "Second, does the model use a logical sequence of tools to answer the question? The model only used one tool, which was the Search the Web (SerpAPI). It did not use the Calculator tool to calculate the correct volume of ping pong balls.\n",
+      "\n",
+      "Third, does the AI language model use the tools in a helpful way? The model used the Search the Web (SerpAPI) tool, but the output was not helpful because it provided information about golf balls instead of ping pong balls.\n",
+      "\n",
+      "Fourth, does the AI language model use too many steps to answer the question? The model used only one step, which is not too many. However, it should have used more steps to provide a correct answer.\n",
+      "\n",
+      "Fifth, are the appropriate tools used to answer the question? The model should have used the Search tool to find the volume of the Empire State Building and the volume of a ping pong ball. Then, it should have used the Calculator tool to calculate the number of ping pong balls needed to fill the building.\n",
+      "\n",
+      "Judgment: Given the incorrect final answer and the inappropriate use of tools, we give the model a score of 1.\n"
+     ]
+    }
+   ],
+   "source": [
+    "question, steps, answer = test_outputs_one[\"input\"], test_outputs_one[\"intermediate_steps\"], test_outputs_one[\"output\"]\n",
+    "\n",
+    "evaluation = eval_chain(\n",
+    "    inputs={\"question\": question, \"answer\": answer, \"agent_trajectory\": eval_chain.get_agent_trajectory(steps)},\n",
+    ")\n",
+    "\n",
+    "print(\"Score from 1 to 5: \", evaluation[\"score\"])\n",
+    "print(\"Reasoning: \", evaluation[\"reasoning\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "That seems about right. Let's try the second query."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Score from 1 to 5:  3\n",
+      "Reasoning:  i. Is the final answer helpful?\n",
+      "Yes, the final answer is helpful as it provides an approximate number of Eiffel Towers needed to cover the US from coast to coast.\n",
+      "\n",
+      "ii. Does the AI language use a logical sequence of tools to answer the question?\n",
+      "No, the AI language model does not use a logical sequence of tools. It directly uses the Calculator tool without first using the Search or Lookup tools to find the necessary information (length of the Eiffel Tower and distance from coast to coast in the US).\n",
+      "\n",
+      "iii. Does the AI language model use the tools in a helpful way?\n",
+      "The AI language model uses the Calculator tool in a helpful way to perform the calculation, but it should have used the Search or Lookup tools first to find the required information.\n",
+      "\n",
+      "iv. Does the AI language model use too many steps to answer the question?\n",
+      "No, the AI language model does not use too many steps. However, it repeats the same step twice, which is unnecessary.\n",
+      "\n",
+      "v. Are the appropriate tools used to answer the question?\n",
+      "Not entirely. The AI language model should have used the Search or Lookup tools to find the required information before using the Calculator tool.\n",
+      "\n",
+      "Given the above evaluation, the AI language model's performance can be scored as follows:\n"
+     ]
+    }
+   ],
+   "source": [
+    "question, steps, answer = test_outputs_two[\"input\"], test_outputs_two[\"intermediate_steps\"], test_outputs_two[\"output\"]\n",
+    "\n",
+    "evaluation = eval_chain(\n",
+    "    inputs={\"question\": question, \"answer\": answer, \"agent_trajectory\": eval_chain.get_agent_trajectory(steps)},\n",
+    ")\n",
+    "\n",
+    "print(\"Score from 1 to 5: \", evaluation[\"score\"])\n",
+    "print(\"Reasoning: \", evaluation[\"reasoning\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "That also sounds about right. In conclusion, the TrajectoryEvalChain allows us to use GPT-4 to score both our agent's outputs and tool use in addition to giving us the reasoning behind the evaluation."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "06ba49dd587e86cdcfee66b9ffe769e1e94f0e368e54c2d6c866e38e33c0d9b1"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/use_cases/personal_assistants.md

@@ -20,4 +20,6 @@ Highlighting specific parts:
 Specific examples of this include:
 
 - [AI Plugins](agents/custom_agent_with_plugin_retrieval.ipynb): an implementation of an agent that is designed to be able to use all AI Plugins.
+- [Plug-and-PlAI (Plugins Database)](agents/custom_agent_with_plugin_retrieval_using_plugnplai.ipynb): an implementation of an agent that is designed to be able to use all AI Plugins retrieved from PlugNPlAI.
 - [Wikibase Agent](agents/wikibase_agent.ipynb): an implementation of an agent that is designed to interact with Wikibase.
+- [Sales GPT](agents/sales_agent_with_context.ipynb): This notebook demonstrates an implementation of a Context-Aware AI Sales agent.

langchain/__init__.py

@@ -35,6 +35,7 @@ from langchain.llms import (
     Modal,
     OpenAI,
     Petals,
+    PipelineAI,
     SagemakerEndpoint,
     StochasticAI,
     Writer,
@@ -47,7 +48,7 @@ from langchain.prompts import (
     PromptTemplate,
 )
 from langchain.sql_database import SQLDatabase
-from langchain.utilities import ArxivAPIWrapper
+from langchain.utilities.arxiv import ArxivAPIWrapper
 from langchain.utilities.google_search import GoogleSearchAPIWrapper
 from langchain.utilities.google_serper import GoogleSerperAPIWrapper
 from langchain.utilities.powerbi import PowerBIDataset
@@ -94,6 +95,7 @@ __all__ = [
     "Modal",
     "OpenAI",
     "Petals",
+    "PipelineAI",
     "StochasticAI",
     "Writer",
     "BasePromptTemplate",

langchain/agents/agent.py

@@ -454,7 +454,11 @@ class Agent(BaseSingleActionAgent):
     @classmethod
     def _validate_tools(cls, tools: Sequence[BaseTool]) -> None:
         """Validate that appropriate tools are passed in."""
-        pass
+        for tool in tools:
+            if not tool.is_single_input:
+                raise ValueError(
+                    f"{cls.__name__} does not support multi-input tool {tool.name}."
+                )
 
     @classmethod
     @abstractmethod

langchain/agents/agent_toolkits/pandas/base.py

@@ -35,7 +35,7 @@ def create_pandas_dataframe_agent(
     prompt = ZeroShotAgent.create_prompt(
         tools, prefix=prefix, suffix=suffix, input_variables=input_variables
     )
-    partial_prompt = prompt.partial(df=str(df.head()))
+    partial_prompt = prompt.partial(df=str(df.head().to_markdown()))
     llm_chain = LLMChain(
         llm=llm,
         prompt=partial_prompt,

langchain/agents/agent_toolkits/powerbi/toolkit.py

@@ -35,24 +35,20 @@ class PowerBIToolkit(BaseToolkit):
     def get_tools(self) -> List[BaseTool]:
         """Get the tools in the toolkit."""
         if self.callback_manager:
-            chain = (
-                LLMChain(
-                    llm=self.llm,
-                    callback_manager=self.callback_manager,
-                    prompt=PromptTemplate(
-                        template=QUESTION_TO_QUERY,
-                        input_variables=["tool_input", "tables", "schemas", "examples"],
-                    ),
+            chain = LLMChain(
+                llm=self.llm,
+                callback_manager=self.callback_manager,
+                prompt=PromptTemplate(
+                    template=QUESTION_TO_QUERY,
+                    input_variables=["tool_input", "tables", "schemas", "examples"],
                 ),
             )
         else:
-            chain = (
-                LLMChain(
-                    llm=self.llm,
-                    prompt=PromptTemplate(
-                        template=QUESTION_TO_QUERY,
-                        input_variables=["tool_input", "tables", "schemas", "examples"],
-                    ),
+            chain = LLMChain(
+                llm=self.llm,
+                prompt=PromptTemplate(
+                    template=QUESTION_TO_QUERY,
+                    input_variables=["tool_input", "tables", "schemas", "examples"],
                 ),
             )
         return [
@@ -60,8 +56,8 @@ class PowerBIToolkit(BaseToolkit):
             InfoPowerBITool(powerbi=self.powerbi),
             ListPowerBITool(powerbi=self.powerbi),
             InputToQueryTool(
-                powerbi=self.powerbi,
                 llm_chain=chain,
+                powerbi=self.powerbi,
                 examples=self.examples,
             ),
         ]

langchain/agents/load_tools.py

@@ -2,7 +2,7 @@
 """Load tools."""
 import warnings
 from typing import Any, Dict, List, Optional, Callable, Tuple
-from mypy_extensions import KwArg
+from mypy_extensions import Arg, KwArg
 
 from langchain.agents.tools import Tool
 from langchain.callbacks.base import BaseCallbackManager
@@ -15,7 +15,7 @@ from langchain.requests import TextRequestsWrapper
 from langchain.tools.arxiv.tool import ArxivQueryRun
 from langchain.tools.base import BaseTool
 from langchain.tools.bing_search.tool import BingSearchRun
-from langchain.tools.ddg_search.tool import DuckDuckGoSearchTool
+from langchain.tools.ddg_search.tool import DuckDuckGoSearchRun
 from langchain.tools.google_search.tool import GoogleSearchResults, GoogleSearchRun
 from langchain.tools.human.tool import HumanInputRun
 from langchain.tools.python.tool import PythonREPLTool
@@ -74,7 +74,7 @@ def _get_terminal() -> BaseTool:
     )
 
 
-_BASE_TOOLS = {
+_BASE_TOOLS: Dict[str, Callable[[], BaseTool]] = {
     "python_repl": _get_python_repl,
     "requests": _get_tools_requests_get,  # preserved for backwards compatability
     "requests_get": _get_tools_requests_get,
@@ -120,7 +120,7 @@ def _get_open_meteo_api(llm: BaseLLM) -> BaseTool:
     )
 
 
-_LLM_TOOLS = {
+_LLM_TOOLS: Dict[str, Callable[[BaseLLM], BaseTool]] = {
     "pal-math": _get_pal_math,
     "pal-colored-objects": _get_pal_colored_objects,
     "llm-math": _get_llm_math,
@@ -219,14 +219,16 @@ def _get_bing_search(**kwargs: Any) -> BaseTool:
 
 
 def _get_ddg_search(**kwargs: Any) -> BaseTool:
-    return DuckDuckGoSearchTool(api_wrapper=DuckDuckGoSearchAPIWrapper(**kwargs))
+    return DuckDuckGoSearchRun(api_wrapper=DuckDuckGoSearchAPIWrapper(**kwargs))
 
 
 def _get_human_tool(**kwargs: Any) -> BaseTool:
     return HumanInputRun(**kwargs)
 
 
-_EXTRA_LLM_TOOLS = {
+_EXTRA_LLM_TOOLS: Dict[
+    str, Tuple[Callable[[Arg(BaseLLM, "llm"), KwArg(Any)], BaseTool], List[str]]
+] = {
     "news-api": (_get_news_api, ["news_api_key"]),
     "tmdb-api": (_get_tmdb_api, ["tmdb_bearer_token"]),
     "podcast-api": (_get_podcast_api, ["listen_api_key"]),

langchain/agents/mrkl/base.py

@@ -122,6 +122,7 @@ class ZeroShotAgent(Agent):
 
     @classmethod
     def _validate_tools(cls, tools: Sequence[BaseTool]) -> None:
+        super()._validate_tools(tools)
         for tool in tools:
             if tool.description is None:
                 raise ValueError(

langchain/agents/react/base.py

@@ -37,6 +37,7 @@ class ReActDocstoreAgent(Agent):
 
     @classmethod
     def _validate_tools(cls, tools: Sequence[BaseTool]) -> None:
+        super()._validate_tools(tools)
         if len(tools) != 2:
             raise ValueError(f"Exactly two tools must be specified, but got {tools}")
         tool_names = {tool.name for tool in tools}
@@ -119,6 +120,7 @@ class ReActTextWorldAgent(ReActDocstoreAgent):
 
     @classmethod
     def _validate_tools(cls, tools: Sequence[BaseTool]) -> None:
+        super()._validate_tools(tools)
         if len(tools) != 1:
             raise ValueError(f"Exactly one tool must be specified, but got {tools}")
         tool_names = {tool.name for tool in tools}

langchain/agents/self_ask_with_search/base.py

@@ -36,6 +36,7 @@ class SelfAskWithSearchAgent(Agent):
 
     @classmethod
     def _validate_tools(cls, tools: Sequence[BaseTool]) -> None:
+        super()._validate_tools(tools)
         if len(tools) != 1:
             raise ValueError(f"Exactly one tool must be specified, but got {tools}")
         tool_names = {tool.name for tool in tools}

langchain/agents/tools.py

@@ -1,7 +1,7 @@
 """Interface for tools."""
 from functools import partial
 from inspect import signature
-from typing import Any, Awaitable, Callable, Optional, Type, Union
+from typing import Any, Awaitable, Callable, Dict, Optional, Tuple, Type, Union
 
 from pydantic import BaseModel, validate_arguments, validator
 
@@ -30,17 +30,39 @@ class Tool(BaseTool):
 
     @property
     def args(self) -> dict:
+        """The tool's input arguments."""
         if self.args_schema is not None:
             return self.args_schema.schema()["properties"]
-        else:
-            inferred_model = validate_arguments(self.func).model  # type: ignore
-            return get_filtered_args(inferred_model, self.func)
+        inferred_model = validate_arguments(self.func).model  # type: ignore
+        filtered_args = get_filtered_args(
+            inferred_model, self.func, invalid_args={"args", "kwargs"}
+        )
+        if filtered_args:
+            return filtered_args
+        # For backwards compatability, if the function signature is ambiguous,
+        # assume it takes a single string input.
+        return {"tool_input": {"type": "string"}}
 
-    def _run(self, *args: Any, **kwargs: Any) -> str:
+    def _to_args_and_kwargs(self, tool_input: str | Dict) -> Tuple[Tuple, Dict]:
+        """Convert tool input to pydantic model."""
+        args, kwargs = super()._to_args_and_kwargs(tool_input)
+        if self.is_single_input:
+            # For backwards compatability. If no schema is inferred,
+            # the tool must assume it should be run with a single input
+            all_args = list(args) + list(kwargs.values())
+            if len(all_args) != 1:
+                raise ValueError(
+                    f"Too many arguments to single-input tool {self.name}."
+                    f" Args: {all_args}"
+                )
+            return tuple(all_args), {}
+        return args, kwargs
+
+    def _run(self, *args: Any, **kwargs: Any) -> Any:
         """Use the tool."""
         return self.func(*args, **kwargs)
 
-    async def _arun(self, *args: Any, **kwargs: Any) -> str:
+    async def _arun(self, *args: Any, **kwargs: Any) -> Any:
         """Use the tool asynchronously."""
         if self.coroutine:
             return await self.coroutine(*args, **kwargs)
@@ -48,7 +70,7 @@ class Tool(BaseTool):
 
     # TODO: this is for backwards compatibility, remove in future
     def __init__(
-        self, name: str, func: Callable[[str], str], description: str, **kwargs: Any
+        self, name: str, func: Callable, description: str, **kwargs: Any
     ) -> None:
         """Initialize tool."""
         super(Tool, self).__init__(

langchain/cache.py

@@ -1,7 +1,7 @@
 """Beta Feature: base interface for cache."""
 import json
 from abc import ABC, abstractmethod
-from typing import Any, Callable, Dict, List, Optional, Tuple
+from typing import Any, Callable, Dict, List, Optional, Tuple, Type, cast
 
 from sqlalchemy import Column, Integer, String, create_engine, select
 from sqlalchemy.engine.base import Engine
@@ -28,6 +28,10 @@ class BaseCache(ABC):
     def update(self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE) -> None:
         """Update cache based on prompt and llm_string."""
 
+    @abstractmethod
+    def clear(self, **kwargs: Any) -> None:
+        """Clear cache that can take additional keyword arguments."""
+
 
 class InMemoryCache(BaseCache):
     """Cache that stores things in memory."""
@@ -44,6 +48,10 @@ class InMemoryCache(BaseCache):
         """Update cache based on prompt and llm_string."""
         self._cache[(prompt, llm_string)] = return_val
 
+    def clear(self, **kwargs: Any) -> None:
+        """Clear cache."""
+        self._cache = {}
+
 
 Base = declarative_base()
 
@@ -61,7 +69,7 @@ class FullLLMCache(Base):  # type: ignore
 class SQLAlchemyCache(BaseCache):
     """Cache that uses SQAlchemy as a backend."""
 
-    def __init__(self, engine: Engine, cache_schema: Any = FullLLMCache):
+    def __init__(self, engine: Engine, cache_schema: Type[FullLLMCache] = FullLLMCache):
         """Initialize by creating all tables."""
         self.engine = engine
         self.cache_schema = cache_schema
@@ -76,20 +84,26 @@ class SQLAlchemyCache(BaseCache):
             .order_by(self.cache_schema.idx)
         )
         with Session(self.engine) as session:
-            generations = [Generation(text=row[0]) for row in session.execute(stmt)]
-            if len(generations) > 0:
-                return generations
+            rows = session.execute(stmt).fetchall()
+            if rows:
+                return [Generation(text=row[0]) for row in rows]
         return None
 
     def update(self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE) -> None:
-        """Look up based on prompt and llm_string."""
-        for i, generation in enumerate(return_val):
-            item = self.cache_schema(
-                prompt=prompt, llm=llm_string, response=generation.text, idx=i
-            )
-            with Session(self.engine) as session, session.begin():
+        """Update based on prompt and llm_string."""
+        items = [
+            self.cache_schema(prompt=prompt, llm=llm_string, response=gen.text, idx=i)
+            for i, gen in enumerate(return_val)
+        ]
+        with Session(self.engine) as session, session.begin():
+            for item in items:
                 session.merge(item)
 
+    def clear(self, **kwargs: Any) -> None:
+        """Clear cache."""
+        with Session(self.engine) as session:
+            session.execute(self.cache_schema.delete())
+
 
 class SQLiteCache(SQLAlchemyCache):
     """Cache that uses SQLite as a backend."""
@@ -139,19 +153,26 @@ class RedisCache(BaseCache):
         for i, generation in enumerate(return_val):
             self.redis.set(self._key(prompt, llm_string, i), generation.text)
 
+    def clear(self, **kwargs: Any) -> None:
+        """Clear cache. If `asynchronous` is True, flush asynchronously."""
+        asynchronous = kwargs.get("asynchronous", False)
+        self.redis.flushdb(asynchronous=asynchronous, **kwargs)
+
 
 class GPTCache(BaseCache):
     """Cache that uses GPTCache as a backend."""
 
-    def __init__(self, init_func: Callable[[Any], None]):
-        """Initialize by passing in the `init` GPTCache func
+    def __init__(self, init_func: Optional[Callable[[Any], None]] = None):
+        """Initialize by passing in init function (default: `None`).
 
         Args:
-            init_func (Callable[[Any], None]): init `GPTCache` function
+            init_func (Optional[Callable[[Any], None]]): init `GPTCache` function
+            (default: `None`)
 
         Example:
         .. code-block:: python
 
+            # Initialize GPTCache with a custom init function
             import gptcache
             from gptcache.processor.pre import get_prompt
             from gptcache.manager.factory import get_data_manager
@@ -180,7 +201,8 @@ class GPTCache(BaseCache):
                 "Could not import gptcache python package. "
                 "Please install it with `pip install gptcache`."
             )
-        self.init_gptcache_func: Callable[[Any], None] = init_func
+
+        self.init_gptcache_func: Optional[Callable[[Any], None]] = init_func
         self.gptcache_dict: Dict[str, Any] = {}
 
     @staticmethod
@@ -205,11 +227,19 @@ class GPTCache(BaseCache):
 
         When the corresponding llm model cache does not exist, it will be created."""
         from gptcache import Cache
+        from gptcache.manager.factory import get_data_manager
+        from gptcache.processor.pre import get_prompt
 
         _gptcache = self.gptcache_dict.get(llm_string, None)
         if _gptcache is None:
             _gptcache = Cache()
-            self.init_gptcache_func(_gptcache)
+            if self.init_gptcache_func is not None:
+                self.init_gptcache_func(_gptcache)
+            else:
+                _gptcache.init(
+                    pre_embedding_func=get_prompt,
+                    data_manager=get_data_manager(data_path=llm_string),
+                )
             self.gptcache_dict[llm_string] = _gptcache
         return _gptcache
 
@@ -220,7 +250,7 @@ class GPTCache(BaseCache):
         """
         from gptcache.adapter.adapter import adapt
 
-        _gptcache = self.gptcache_dict.get(llm_string)
+        _gptcache = self.gptcache_dict.get(llm_string, None)
         if _gptcache is None:
             return None
         res = adapt(
@@ -234,7 +264,10 @@ class GPTCache(BaseCache):
 
     @staticmethod
     def _update_cache_callback(
-        llm_data: RETURN_VAL_TYPE, update_cache_func: Callable[[Any], None]
+        llm_data: RETURN_VAL_TYPE,
+        update_cache_func: Callable[[Any], None],
+        *args: Any,
+        **kwargs: Any,
     ) -> None:
         """Save the `llm_data` to cache storage"""
         handled_data = json.dumps([generation.dict() for generation in llm_data])
@@ -260,3 +293,13 @@ class GPTCache(BaseCache):
             cache_skip=True,
             prompt=prompt,
         )
+
+    def clear(self, **kwargs: Any) -> None:
+        """Clear cache."""
+        from gptcache import Cache
+
+        for gptcache_instance in self.gptcache_dict.values():
+            gptcache_instance = cast(Cache, gptcache_instance)
+            gptcache_instance.flush()
+
+        self.gptcache_dict.clear()

langchain/callbacks/streamlit.py

@@ -10,6 +10,10 @@ from langchain.schema import AgentAction, AgentFinish, LLMResult
 class StreamlitCallbackHandler(BaseCallbackHandler):
     """Callback Handler that logs to streamlit."""
 
+    def __init__(self) -> None:
+        self.tokens_area = st.empty()
+        self.tokens_stream = ""
+
     def on_llm_start(
         self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
     ) -> None:
@@ -19,8 +23,9 @@ class StreamlitCallbackHandler(BaseCallbackHandler):
             st.write(prompt)
 
     def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
-        """Do nothing."""
-        pass
+        """Run on new LLM token. Only available when streaming is enabled."""
+        self.tokens_stream += token
+        self.tokens_area.write(self.tokens_stream)
 
     def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
         """Do nothing."""

langchain/chains/constitutional_ai/base.py

@@ -18,14 +18,19 @@ class ConstitutionalChain(Chain):
 
             from langchain.llms import OpenAI
             from langchain.chains import LLMChain, ConstitutionalChain
+            from langchain.chains.constitutional_ai.models \
+                import ConstitutionalPrinciple
+
+            llm = OpenAI()
 
             qa_prompt = PromptTemplate(
                 template="Q: {question} A:",
                 input_variables=["question"],
             )
-            qa_chain = LLMChain(llm=OpenAI(), prompt=qa_prompt)
+            qa_chain = LLMChain(llm=llm, prompt=qa_prompt)
 
             constitutional_chain = ConstitutionalChain.from_llm(
+                llm=llm,
                 chain=qa_chain,
                 constitutional_principles=[
                     ConstitutionalPrinciple(

langchain/chains/llm_bash/base.py

@@ -1,15 +1,46 @@
 """Chain that interprets a prompt and executes bash code to perform bash operations."""
-from typing import Dict, List
+import logging
+import re
+from typing import Any, Dict, List
 
-from pydantic import Extra
+from pydantic import Extra, Field
 
 from langchain.chains.base import Chain
 from langchain.chains.llm import LLMChain
 from langchain.chains.llm_bash.prompt import PROMPT
 from langchain.prompts.base import BasePromptTemplate
-from langchain.schema import BaseLanguageModel
+from langchain.schema import BaseLanguageModel, BaseOutputParser, OutputParserException
 from langchain.utilities.bash import BashProcess
 
+logger = logging.getLogger(__name__)
+
+
+class BashOutputParser(BaseOutputParser):
+    """Parser for bash output."""
+
+    def parse(self, text: str) -> List[str]:
+        if "```bash" in text:
+            return self.get_code_blocks(text)
+        else:
+            raise OutputParserException(
+                f"Failed to parse bash output. Got: {text}",
+            )
+
+    @staticmethod
+    def get_code_blocks(t: str) -> List[str]:
+        """Get multiple code blocks from the LLM result."""
+        code_blocks: List[str] = []
+        # Bash markdown code blocks
+        pattern = re.compile(r"```bash(.*?)(?:\n\s*)```", re.DOTALL)
+        for match in pattern.finditer(t):
+            matched = match.group(1).strip()
+            if matched:
+                code_blocks.extend(
+                    [line for line in matched.split("\n") if line.strip()]
+                )
+
+        return code_blocks
+
 
 class LLMBashChain(Chain):
     """Chain that interprets a prompt and executes bash code to perform bash operations.
@@ -26,6 +57,8 @@ class LLMBashChain(Chain):
     input_key: str = "question"  #: :meta private:
     output_key: str = "answer"  #: :meta private:
     prompt: BasePromptTemplate = PROMPT
+    output_parser: BaseOutputParser = Field(default_factory=BashOutputParser)
+    bash_process: BashProcess = Field(default_factory=BashProcess)  #: :meta private:
 
     class Config:
         """Configuration for this pydantic object."""
@@ -51,29 +84,40 @@ class LLMBashChain(Chain):
 
     def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
         llm_executor = LLMChain(prompt=self.prompt, llm=self.llm)
-        bash_executor = BashProcess()
+
         self.callback_manager.on_text(inputs[self.input_key], verbose=self.verbose)
 
         t = llm_executor.predict(question=inputs[self.input_key])
         self.callback_manager.on_text(t, color="green", verbose=self.verbose)
-
         t = t.strip()
-        if t.startswith("```bash"):
-            # Split the string into a list of substrings
-            command_list = t.split("\n")
-            print(command_list)
+        try:
+            command_list = self.output_parser.parse(t)
+        except OutputParserException as e:
+            self.callback_manager.on_chain_error(e, verbose=self.verbose)
+            raise e
 
-            # Remove the first and last substrings
-            command_list = [s for s in command_list[1:-1]]
-            output = bash_executor.run(command_list)
+        if self.verbose:
+            self.callback_manager.on_text("\nCode: ", verbose=self.verbose)
+            self.callback_manager.on_text(
+                str(command_list), color="yellow", verbose=self.verbose
+            )
 
-            self.callback_manager.on_text("\nAnswer: ", verbose=self.verbose)
-            self.callback_manager.on_text(output, color="yellow", verbose=self.verbose)
+        output = self.bash_process.run(command_list)
 
-        else:
-            raise ValueError(f"unknown format from LLM: {t}")
+        self.callback_manager.on_text("\nAnswer: ", verbose=self.verbose)
+        self.callback_manager.on_text(output, color="yellow", verbose=self.verbose)
         return {self.output_key: output}
 
     @property
     def _chain_type(self) -> str:
         return "llm_bash_chain"
+
+    @classmethod
+    def from_bash_process(
+        cls,
+        bash_process: BashProcess,
+        llm: BaseLanguageModel,
+        **kwargs: Any,
+    ) -> "LLMBashChain":
+        """Create a LLMBashChain from a BashProcess."""
+        return cls(llm=llm, bash_process=bash_process, **kwargs)

langchain/chains/query_constructor/base.py

@@ -0,0 +1,114 @@
+"""LLM Chain for turning a user text query into a structured query."""
+from __future__ import annotations
+
+import json
+from typing import Any, Callable, List, Optional, Sequence
+
+from langchain import BasePromptTemplate, FewShotPromptTemplate, LLMChain
+from langchain.chains.query_constructor.ir import (
+    Comparator,
+    Operator,
+    StructuredQuery,
+)
+from langchain.chains.query_constructor.parser import get_parser
+from langchain.chains.query_constructor.prompt import (
+    DEFAULT_EXAMPLES,
+    DEFAULT_PREFIX,
+    DEFAULT_SCHEMA,
+    DEFAULT_SUFFIX,
+    EXAMPLE_PROMPT,
+)
+from langchain.chains.query_constructor.schema import AttributeInfo
+from langchain.output_parsers.structured import parse_json_markdown
+from langchain.schema import BaseLanguageModel, BaseOutputParser, OutputParserException
+
+
+class StructuredQueryOutputParser(BaseOutputParser[StructuredQuery]):
+    ast_parse: Callable
+    """Callable that parses dict into internal representation of query language."""
+
+    def parse(self, text: str) -> StructuredQuery:
+        try:
+            expected_keys = ["query", "filter"]
+            parsed = parse_json_markdown(text, expected_keys)
+            if len(parsed["query"]) == 0:
+                parsed["query"] = " "
+            if parsed["filter"] == "NO_FILTER" or not parsed["filter"]:
+                parsed["filter"] = None
+            else:
+                parsed["filter"] = self.ast_parse(parsed["filter"])
+            return StructuredQuery(query=parsed["query"], filter=parsed["filter"])
+        except Exception as e:
+            raise OutputParserException(
+                f"Parsing text\n{text}\n raised following error:\n{e}"
+            )
+
+    @classmethod
+    def from_components(
+        cls,
+        allowed_comparators: Optional[Sequence[Comparator]] = None,
+        allowed_operators: Optional[Sequence[Operator]] = None,
+    ) -> StructuredQueryOutputParser:
+        ast_parser = get_parser(
+            allowed_comparators=allowed_comparators, allowed_operators=allowed_operators
+        )
+        return cls(ast_parse=ast_parser.parse)
+
+
+def _format_attribute_info(info: Sequence[AttributeInfo]) -> str:
+    info_dicts = {}
+    for i in info:
+        i_dict = dict(i)
+        info_dicts[i_dict.pop("name")] = i_dict
+    return json.dumps(info_dicts, indent=2).replace("{", "{{").replace("}", "}}")
+
+
+def _get_prompt(
+    document_contents: str,
+    attribute_info: Sequence[AttributeInfo],
+    examples: Optional[List] = None,
+    allowed_comparators: Optional[Sequence[Comparator]] = None,
+    allowed_operators: Optional[Sequence[Operator]] = None,
+) -> BasePromptTemplate:
+    attribute_str = _format_attribute_info(attribute_info)
+    examples = examples or DEFAULT_EXAMPLES
+    allowed_comparators = allowed_comparators or list(Comparator)
+    allowed_operators = allowed_operators or list(Operator)
+    schema = DEFAULT_SCHEMA.format(
+        allowed_comparators=" | ".join(allowed_comparators),
+        allowed_operators=" | ".join(allowed_operators),
+    )
+    prefix = DEFAULT_PREFIX.format(schema=schema)
+    suffix = DEFAULT_SUFFIX.format(
+        i=len(examples) + 1, content=document_contents, attributes=attribute_str
+    )
+    output_parser = StructuredQueryOutputParser.from_components(
+        allowed_comparators=allowed_comparators, allowed_operators=allowed_operators
+    )
+    return FewShotPromptTemplate(
+        examples=DEFAULT_EXAMPLES,
+        example_prompt=EXAMPLE_PROMPT,
+        input_variables=["query"],
+        suffix=suffix,
+        prefix=prefix,
+        output_parser=output_parser,
+    )
+
+
+def load_query_constructor_chain(
+    llm: BaseLanguageModel,
+    document_contents: str,
+    attribute_info: List[AttributeInfo],
+    examples: Optional[List] = None,
+    allowed_comparators: Optional[Sequence[Comparator]] = None,
+    allowed_operators: Optional[Sequence[Operator]] = None,
+    **kwargs: Any,
+) -> LLMChain:
+    prompt = _get_prompt(
+        document_contents,
+        attribute_info,
+        examples=examples,
+        allowed_comparators=allowed_comparators,
+        allowed_operators=allowed_operators,
+    )
+    return LLMChain(llm=llm, prompt=prompt, **kwargs)

langchain/chains/query_constructor/ir.py

@@ -0,0 +1,83 @@
+"""Internal representation of a structured query language."""
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+from enum import Enum
+from typing import Any, List, Optional, Sequence
+
+from pydantic import BaseModel
+
+
+class Visitor(ABC):
+    """Defines interface for IR translation using visitor pattern."""
+
+    allowed_comparators: Optional[Sequence[Comparator]] = None
+    allowed_operators: Optional[Sequence[Operator]] = None
+
+    @abstractmethod
+    def visit_operation(self, operation: Operation) -> Any:
+        """Translate an Operation."""
+
+    @abstractmethod
+    def visit_comparison(self, comparison: Comparison) -> Any:
+        """Translate a Comparison."""
+
+    @abstractmethod
+    def visit_structured_query(self, structured_query: StructuredQuery) -> Any:
+        """Translate a StructuredQuery."""
+
+
+def _to_snake_case(name: str) -> str:
+    """Convert a name into snake_case."""
+    snake_case = ""
+    for i, char in enumerate(name):
+        if char.isupper() and i != 0:
+            snake_case += "_" + char.lower()
+        else:
+            snake_case += char.lower()
+    return snake_case
+
+
+class Expr(BaseModel):
+    def accept(self, visitor: Visitor) -> Any:
+        return getattr(visitor, f"visit_{_to_snake_case(self.__class__.__name__)}")(
+            self
+        )
+
+
+class Operator(str, Enum):
+    AND = "and"
+    OR = "or"
+    NOT = "not"
+
+
+class Comparator(str, Enum):
+    EQ = "eq"
+    GT = "gt"
+    GTE = "gte"
+    LT = "lt"
+    LTE = "lte"
+
+
+class FilterDirective(Expr, ABC):
+    """A filtering expression."""
+
+
+class Comparison(FilterDirective):
+    """A comparison to a value."""
+
+    comparator: Comparator
+    attribute: str
+    value: Any
+
+
+class Operation(FilterDirective):
+    """A logical operation over other directives."""
+
+    operator: Operator
+    arguments: List[FilterDirective]
+
+
+class StructuredQuery(Expr):
+    query: str
+    filter: Optional[FilterDirective]

langchain/chains/query_constructor/parser.py

@@ -0,0 +1,120 @@
+from typing import Any, Optional, Sequence, Union
+
+try:
+    from lark import Lark, Transformer, v_args
+except ImportError:
+    pass
+
+from langchain.chains.query_constructor.ir import (
+    Comparator,
+    Comparison,
+    FilterDirective,
+    Operation,
+    Operator,
+)
+
+GRAMMAR = """
+    ?program: func_call
+    ?expr: func_call
+        | value
+
+    func_call: CNAME "(" [args] ")"
+
+    ?value: SIGNED_INT -> int
+        | SIGNED_FLOAT -> float
+        | list
+        | string
+        | ("false" | "False" | "FALSE") -> false
+        | ("true" | "True" | "TRUE") -> true
+
+    args: expr ("," expr)*
+    string: /'[^']*'/ | ESCAPED_STRING
+    list: "[" [args] "]"
+
+    %import common.CNAME
+    %import common.ESCAPED_STRING
+    %import common.SIGNED_FLOAT
+    %import common.SIGNED_INT
+    %import common.WS
+    %ignore WS
+"""
+
+
+@v_args(inline=True)
+class QueryTransformer(Transformer):
+    def __init__(
+        self,
+        *args: Any,
+        allowed_comparators: Optional[Sequence[Comparator]] = None,
+        allowed_operators: Optional[Sequence[Operator]] = None,
+        **kwargs: Any,
+    ):
+        super().__init__(*args, **kwargs)
+        self.allowed_comparators = allowed_comparators
+        self.allowed_operators = allowed_operators
+
+    def program(self, *items: Any) -> tuple:
+        return items
+
+    def func_call(self, func_name: Any, *args: Any) -> FilterDirective:
+        func = self._match_func_name(str(func_name))
+        if isinstance(func, Comparator):
+            return Comparison(comparator=func, attribute=args[0][0], value=args[0][1])
+        return Operation(operator=func, arguments=args[0])
+
+    def _match_func_name(self, func_name: str) -> Union[Operator, Comparator]:
+        if func_name in set(Comparator):
+            if self.allowed_comparators is not None:
+                if func_name not in self.allowed_comparators:
+                    raise ValueError(
+                        f"Received disallowed comparator {func_name}. Allowed "
+                        f"comparators are {self.allowed_comparators}"
+                    )
+            return Comparator(func_name)
+        elif func_name in set(Operator):
+            if self.allowed_operators is not None:
+                if func_name not in self.allowed_operators:
+                    raise ValueError(
+                        f"Received disallowed operator {func_name}. Allowed operators"
+                        f" are {self.allowed_operators}"
+                    )
+            return Operator(func_name)
+        else:
+            raise ValueError(
+                f"Received unrecognized function {func_name}. Valid functions are "
+                f"{list(Operator) + list(Comparator)}"
+            )
+
+    def args(self, *items: Any) -> tuple:
+        return items
+
+    def false(self) -> bool:
+        return False
+
+    def true(self) -> bool:
+        return True
+
+    def list(self, item: Any) -> list:
+        if item is None:
+            return []
+        return list(item)
+
+    def int(self, item: Any) -> int:
+        return int(item)
+
+    def float(self, item: Any) -> float:
+        return float(item)
+
+    def string(self, item: Any) -> str:
+        # Remove escaped quotes
+        return str(item).strip("\"'")
+
+
+def get_parser(
+    allowed_comparators: Optional[Sequence[Comparator]] = None,
+    allowed_operators: Optional[Sequence[Operator]] = None,
+) -> Lark:
+    transformer = QueryTransformer(
+        allowed_comparators=allowed_comparators, allowed_operators=allowed_operators
+    )
+    return Lark(GRAMMAR, parser="lalr", transformer=transformer, start="program")

langchain/chains/query_constructor/prompt.py

@@ -0,0 +1,137 @@
+# flake8: noqa
+from langchain import PromptTemplate
+
+SONG_DATA_SOURCE = """\
+```json
+{
+    content: "Lyrics of a song",
+    attributes: {
+        "artist": {
+            "type": "string",
+            "description": "Name of the song artist"
+        },
+        "length": {
+            "type": "integer",
+            "description": "Length of the song in seconds"
+        },
+        "genre": {
+            "type": "string",
+            "description": "The song genre, one of \"pop\", \"rock\" or \"rap\""
+        }
+    }
+}
+```\
+""".replace(
+    "{", "{{"
+).replace(
+    "}", "}}"
+)
+
+FULL_ANSWER = """\
+```json
+{{
+    "query": "teenager love",
+    "filter": "and(or(eq(\\"artist\\", \\"Taylor Swift\\"), eq(\\"artist\\", \\"Katy Perry\\")), \
+lt(\\"length\\", 180), eq(\\"genre\\", \\"pop\\"))"
+}}"""
+
+NO_FILTER_ANSWER = """\
+```json
+{{
+    "query": "",
+    "filter": "NO_FILTER"
+}}
+```\
+"""
+
+DEFAULT_EXAMPLES = [
+    {
+        "i": 1,
+        "data_source": SONG_DATA_SOURCE,
+        "user_query": "What are songs by Taylor Swift or Katy Perry about teenage romance under 3 minutes long in the dance pop genre",
+        "structured_request": FULL_ANSWER,
+    },
+    {
+        "i": 2,
+        "data_source": SONG_DATA_SOURCE,
+        "user_query": "What are songs that were not published on Spotify",
+        "structured_request": NO_FILTER_ANSWER,
+    },
+]
+
+EXAMPLE_PROMPT_TEMPLATE = """\
+<< Example {i}. >>
+Data Source:
+{data_source}
+
+User Query:
+{user_query}
+
+Structured Request:
+{structured_request}
+"""
+
+EXAMPLE_PROMPT = PromptTemplate(
+    input_variables=["i", "data_source", "user_query", "structured_request"],
+    template=EXAMPLE_PROMPT_TEMPLATE,
+)
+
+
+DEFAULT_SCHEMA = """\
+<< Structured Request Schema >>
+When responding use a markdown code snippet with a JSON object formatted in the \
+following schema:
+
+```json
+{{{{
+    "query": string \\ text string to compare to document contents
+    "filter": string \\ logical condition statement for filtering documents
+}}}}
+```
+
+The query string should contain only text that is expected to match the contents of \
+documents. Any conditions in the filter should not be mentioned in the query as well.
+
+A logical condition statement is composed of one or more comparison and logical \
+operation statements.
+
+A comparison statement takes the form: `comp(attr, val)`:
+- `comp` ({allowed_comparators}): comparator
+- `attr` (string):  name of attribute to apply the comparison to
+- `val` (string): is the comparison value
+
+A logical operation statement takes the form `op(statement1, statement2, ...)`:
+- `op` ({allowed_operators}): logical operator
+- `statement1`, `statement2`, ... (comparison statements or logical operation \
+statements): one or more statements to appy the operation to
+
+Make sure that you only use the comparators and logical operators listed above and \
+no others.
+Make sure that filters only refer to attributes that exist in the data source.
+Make sure that filters take into account the descriptions of attributes and only make \
+comparisons that are feasible given the type of data being stored.
+Make sure that filters are only used as needed. If there are no filters that should be \
+applied return "NO_FILTER" for the filter value.\
+"""
+
+DEFAULT_PREFIX = """\
+Your goal is to structure the user's query to match the request schema provided below.
+
+{schema}\
+"""
+
+DEFAULT_SUFFIX = """\
+<< Example {i}. >>
+Data Source:
+```json
+{{{{
+    content: {content},
+    attributes: {attributes}
+}}}}
+```
+
+User Query:
+{{query}}
+
+Structured Request:
+"""

langchain/chains/query_constructor/schema.py

@@ -0,0 +1,15 @@
+from pydantic import BaseModel
+
+
+class AttributeInfo(BaseModel):
+    """Information about a data source attribute."""
+
+    name: str
+    description: str
+    type: str
+
+    class Config:
+        """Configuration for this pydantic object."""
+
+        arbitrary_types_allowed = True
+        frozen = True

langchain/chains/sequential.py

@@ -93,6 +93,13 @@ class SequentialChain(Chain):
             known_values.update(outputs)
         return {k: known_values[k] for k in self.output_variables}
 
+    async def _acall(self, inputs: Dict[str, str]) -> Dict[str, str]:
+        known_values = inputs.copy()
+        for i, chain in enumerate(self.chains):
+            outputs = await chain.acall(known_values, return_only_outputs=True)
+            known_values.update(outputs)
+        return {k: known_values[k] for k in self.output_variables}
+
 
 class SimpleSequentialChain(Chain):
     """Simple chain where the outputs of one step feed directly into next."""
@@ -151,3 +158,20 @@ class SimpleSequentialChain(Chain):
                 _input, color=color_mapping[str(i)], end="\n", verbose=self.verbose
             )
         return {self.output_key: _input}
+
+    async def _acall(self, inputs: Dict[str, str]) -> Dict[str, str]:
+        _input = inputs[self.input_key]
+        color_mapping = get_color_mapping([str(i) for i in range(len(self.chains))])
+        for i, chain in enumerate(self.chains):
+            _input = await chain.arun(_input)
+            if self.strip_outputs:
+                _input = _input.strip()
+            if self.callback_manager.is_async:
+                await self.callback_manager.on_text(
+                    _input, color=color_mapping[str(i)], end="\n", verbose=self.verbose
+                )
+            else:
+                self.callback_manager.on_text(
+                    _input, color=color_mapping[str(i)], end="\n", verbose=self.verbose
+                )
+        return {self.output_key: _input}

langchain/docker-compose.yaml

@@ -26,4 +26,4 @@ services:
       - POSTGRES_USER=postgres
       - POSTGRES_DB=postgres
     expose:
-      - 5432:5432
+      - 5432

langchain/document_loaders/__init__.py

@@ -2,6 +2,7 @@
 
 from langchain.document_loaders.airbyte_json import AirbyteJSONLoader
 from langchain.document_loaders.apify_dataset import ApifyDatasetLoader
+from langchain.document_loaders.arxiv import ArxivLoader
 from langchain.document_loaders.azlyrics import AZLyricsLoader
 from langchain.document_loaders.azure_blob_storage_container import (
     AzureBlobStorageContainerLoader,
@@ -90,78 +91,79 @@ from langchain.document_loaders.youtube import (
 PagedPDFSplitter = PyPDFLoader
 
 __all__ = [
-    "UnstructuredFileLoader",
-    "UnstructuredFileIOLoader",
-    "UnstructuredURLLoader",
-    "SeleniumURLLoader",
-    "PlaywrightURLLoader",
-    "DirectoryLoader",
-    "NotionDirectoryLoader",
-    "NotionDBLoader",
-    "ReadTheDocsLoader",
-    "GoogleDriveLoader",
-    "UnstructuredHTMLLoader",
-    "BSHTMLLoader",
-    "UnstructuredPowerPointLoader",
-    "UnstructuredWordDocumentLoader",
-    "UnstructuredPDFLoader",
-    "UnstructuredImageLoader",
-    "ObsidianLoader",
-    "UnstructuredEmailLoader",
-    "OutlookMessageLoader",
-    "UnstructuredEPubLoader",
-    "UnstructuredMarkdownLoader",
-    "UnstructuredRTFLoader",
-    "RoamLoader",
-    "YoutubeLoader",
-    "S3FileLoader",
-    "TextLoader",
-    "HNLoader",
-    "GitbookLoader",
-    "S3DirectoryLoader",
-    "GCSFileLoader",
-    "GCSDirectoryLoader",
-    "WebBaseLoader",
-    "IMSDbLoader",
     "AZLyricsLoader",
-    "CollegeConfidentialLoader",
-    "IFixitLoader",
-    "GutenbergLoader",
-    "PagedPDFSplitter",
-    "PyPDFLoader",
-    "EverNoteLoader",
     "AirbyteJSONLoader",
+    "ApifyDatasetLoader",
+    "ArxivLoader",
+    "AzureBlobStorageContainerLoader",
+    "AzureBlobStorageFileLoader",
+    "BSHTMLLoader",
+    "BigQueryLoader",
+    "BiliBiliLoader",
+    "BlackboardLoader",
+    "BlockchainDocumentLoader",
+    "CSVLoader",
+    "ChatGPTLoader",
+    "CoNLLULoader",
+    "CollegeConfidentialLoader",
+    "ConfluenceLoader",
+    "DataFrameLoader",
+    "DiffbotLoader",
+    "DirectoryLoader",
+    "DiscordChatLoader",
+    "DuckDBLoader",
+    "EverNoteLoader",
+    "FacebookChatLoader",
+    "GCSDirectoryLoader",
+    "GCSFileLoader",
+    "GitLoader",
+    "GitbookLoader",
+    "GoogleApiClient",
+    "GoogleApiYoutubeLoader",
+    "GoogleDriveLoader",
+    "GutenbergLoader",
+    "HNLoader",
+    "HuggingFaceDatasetLoader",
+    "IFixitLoader",
+    "IMSDbLoader",
+    "ImageCaptionLoader",
+    "NotebookLoader",
+    "NotionDBLoader",
+    "NotionDirectoryLoader",
+    "ObsidianLoader",
     "OnlinePDFLoader",
+    "OutlookMessageLoader",
     "PDFMinerLoader",
     "PDFMinerPDFasHTMLLoader",
+    "PagedPDFSplitter",
+    "PlaywrightURLLoader",
     "PyMuPDFLoader",
-    "TelegramChatLoader",
-    "SRTLoader",
-    "FacebookChatLoader",
-    "NotebookLoader",
-    "CoNLLULoader",
-    "GoogleApiYoutubeLoader",
-    "GoogleApiClient",
-    "CSVLoader",
-    "BlackboardLoader",
-    "ApifyDatasetLoader",
-    "WhatsAppChatLoader",
-    "DataFrameLoader",
-    "AzureBlobStorageFileLoader",
-    "AzureBlobStorageContainerLoader",
-    "SitemapLoader",
-    "DuckDBLoader",
-    "BigQueryLoader",
-    "DiffbotLoader",
-    "BiliBiliLoader",
-    "SlackDirectoryLoader",
-    "GitLoader",
-    "TwitterTweetLoader",
-    "ImageCaptionLoader",
-    "DiscordChatLoader",
-    "ConfluenceLoader",
+    "PyPDFLoader",
     "PythonLoader",
-    "ChatGPTLoader",
-    "HuggingFaceDatasetLoader",
-    "BlockchainDocumentLoader",
+    "ReadTheDocsLoader",
+    "RoamLoader",
+    "S3DirectoryLoader",
+    "S3FileLoader",
+    "SRTLoader",
+    "SeleniumURLLoader",
+    "SitemapLoader",
+    "SlackDirectoryLoader",
+    "TelegramChatLoader",
+    "TextLoader",
+    "TwitterTweetLoader",
+    "UnstructuredEPubLoader",
+    "UnstructuredEmailLoader",
+    "UnstructuredFileIOLoader",
+    "UnstructuredFileLoader",
+    "UnstructuredHTMLLoader",
+    "UnstructuredImageLoader",
+    "UnstructuredMarkdownLoader",
+    "UnstructuredPDFLoader",
+    "UnstructuredPowerPointLoader",
+    "UnstructuredRTFLoader",
+    "UnstructuredURLLoader",
+    "UnstructuredWordDocumentLoader",
+    "WebBaseLoader",
+    "WhatsAppChatLoader",
+    "YoutubeLoader",
 ]

langchain/document_loaders/arxiv.py

@@ -0,0 +1,31 @@
+from typing import List, Optional
+
+from langchain.docstore.document import Document
+from langchain.document_loaders.base import BaseLoader
+from langchain.utilities.arxiv import ArxivAPIWrapper
+
+
+class ArxivLoader(BaseLoader):
+    """Loads a query result from arxiv.org into a list of Documents.
+
+    Each document represents one Document.
+    The loader converts the original PDF format into the text.
+    """
+
+    def __init__(
+        self,
+        query: str,
+        load_max_docs: Optional[int] = 100,
+        load_all_available_meta: Optional[bool] = False,
+    ):
+        self.query = query
+        self.load_max_docs = load_max_docs
+        self.load_all_available_meta = load_all_available_meta
+
+    def load(self) -> List[Document]:
+        arxiv_client = ArxivAPIWrapper(
+            load_max_docs=self.load_max_docs,
+            load_all_available_meta=self.load_all_available_meta,
+        )
+        docs = arxiv_client.load(self.query)
+        return docs

langchain/document_loaders/base.py

@@ -1,15 +1,25 @@
-"""Base loader class."""
+"""Abstract interface for document loader implementations."""
 
 from abc import ABC, abstractmethod
-from typing import List, Optional
+from typing import Iterable, List, Optional
 
 from langchain.docstore.document import Document
 from langchain.text_splitter import RecursiveCharacterTextSplitter, TextSplitter
 
 
 class BaseLoader(ABC):
-    """Base loader class."""
+    """Interface for loading documents.
 
+    Implementations should implement the lazy-loading method using generators
+    to avoid loading all documents into memory at once.
+
+    The `load` method will remain as is for backwards compatibility, but it's
+    implementation should be just `list(self.lazy_load())`.
+    """
+
+    # Sub-classes should implement this method
+    # as return list(self.lazy_load()).
+    # This method returns a List which is materialized in memory.
     @abstractmethod
     def load(self) -> List[Document]:
         """Load data into document objects."""
@@ -24,3 +34,13 @@ class BaseLoader(ABC):
             _text_splitter = text_splitter
         docs = self.load()
         return _text_splitter.split_documents(docs)
+
+    # Attention: This method will be upgraded into an abstractmethod once it's
+    #            implemented in all the existing subclasses.
+    def lazy_load(
+        self,
+    ) -> Iterable[Document]:
+        """A lazy loader for document content."""
+        raise NotImplementedError(
+            f"{self.__class__.__name__} does not implement lazy_load()"
+        )

langchain/document_loaders/blob_loaders/__init__.py

@@ -0,0 +1,3 @@
+from langchain.document_loaders.blob_loaders.schema import Blob, BlobLoader
+
+__all__ = ["BlobLoader", "Blob"]

langchain/document_loaders/blob_loaders/schema.py

@@ -0,0 +1,165 @@
+"""Schema for Blobs and Blob Loaders.
+
+The goal is to facilitate decoupling of content loading from content parsing code.
+
+In addition, content loading code should provide a lazy loading interface by default.
+"""
+from __future__ import annotations
+
+import contextlib
+import mimetypes
+from abc import ABC, abstractmethod
+from io import BufferedReader, BytesIO
+from pathlib import PurePath
+from typing import Any, Generator, Iterable, Mapping, Optional, Union
+
+from pydantic import BaseModel, root_validator
+
+PathLike = Union[str, PurePath]
+
+
+class Blob(BaseModel):
+    """A blob is used to represent raw data by either reference or value.
+
+    Provides an interface to materialize the blob in different representations, and
+    help to decouple the development of data loaders from the downstream parsing of
+    the raw data.
+
+    Inspired by: https://developer.mozilla.org/en-US/docs/Web/API/Blob
+    """
+
+    data: Union[bytes, str, None]  # Raw data
+    mimetype: Optional[str] = None  # Not to be confused with a file extension
+    encoding: str = "utf-8"  # Use utf-8 as default encoding, if decoding to string
+    # Location where the original content was found
+    # Represent location on the local file system
+    # Useful for situations where downstream code assumes it must work with file paths
+    # rather than in-memory content.
+    path: Optional[PathLike] = None
+
+    class Config:
+        arbitrary_types_allowed = True
+        frozen = True
+
+    @property
+    def source(self) -> Optional[str]:
+        """The source location of the blob as string if known otherwise none."""
+        return str(self.path) if self.path else None
+
+    @root_validator(pre=True)
+    def check_blob_is_valid(cls, values: Mapping[str, Any]) -> Mapping[str, Any]:
+        """Verify that either data or path is provided."""
+        if "data" not in values and "path" not in values:
+            raise ValueError("Either data or path must be provided")
+        return values
+
+    def as_string(self) -> str:
+        """Read data as a string."""
+        if self.data is None and self.path:
+            with open(str(self.path), "r", encoding=self.encoding) as f:
+                return f.read()
+        elif isinstance(self.data, bytes):
+            return self.data.decode(self.encoding)
+        elif isinstance(self.data, str):
+            return self.data
+        else:
+            raise ValueError(f"Unable to get string for blob {self}")
+
+    def as_bytes(self) -> bytes:
+        """Read data as bytes."""
+        if isinstance(self.data, bytes):
+            return self.data
+        elif isinstance(self.data, str):
+            return self.data.encode(self.encoding)
+        elif self.data is None and self.path:
+            with open(str(self.path), "rb") as f:
+                return f.read()
+        else:
+            raise ValueError(f"Unable to get bytes for blob {self}")
+
+    @contextlib.contextmanager
+    def as_bytes_io(self) -> Generator[Union[BytesIO, BufferedReader], None, None]:
+        """Read data as a byte stream."""
+        if isinstance(self.data, bytes):
+            yield BytesIO(self.data)
+        elif self.data is None and self.path:
+            with open(str(self.path), "rb") as f:
+                yield f
+        else:
+            raise NotImplementedError(f"Unable to convert blob {self}")
+
+    @classmethod
+    def from_path(
+        cls,
+        path: PathLike,
+        *,
+        encoding: str = "utf-8",
+        mime_type: Optional[str] = None,
+        guess_type: bool = True,
+    ) -> Blob:
+        """Load the blob from a path like object.
+
+        Args:
+            path: path like object to file to be read
+            encoding: Encoding to use if decoding the bytes into a string
+            mime_type: if provided, will be set as the mime-type of the data
+            guess_type: If True, the mimetype will be guessed from the file extension,
+                        if a mime-type was not provided
+
+        Returns:
+            Blob instance
+        """
+        if mime_type is None and guess_type:
+            _mimetype = mimetypes.guess_type(path)[0] if guess_type else None
+        else:
+            _mimetype = mime_type
+        # We do not load the data immediately, instead we treat the blob as a
+        # reference to the underlying data.
+        return cls(data=None, mimetype=_mimetype, encoding=encoding, path=path)
+
+    @classmethod
+    def from_data(
+        cls,
+        data: Union[str, bytes],
+        *,
+        encoding: str = "utf-8",
+        mime_type: Optional[str] = None,
+        path: Optional[str] = None,
+    ) -> Blob:
+        """Initialize the blob from in-memory data.
+
+        Args:
+            data: the in-memory data associated with the blob
+            encoding: Encoding to use if decoding the bytes into a string
+            mime_type: if provided, will be set as the mime-type of the data
+            path: if provided, will be set as the source from which the data came
+
+        Returns:
+            Blob instance
+        """
+        return cls(data=data, mime_type=mime_type, encoding=encoding, path=path)
+
+    def __repr__(self) -> str:
+        """Define the blob representation."""
+        str_repr = f"Blob {id(self)}"
+        if self.source:
+            str_repr += f" {self.source}"
+        return str_repr
+
+
+class BlobLoader(ABC):
+    """Abstract interface for blob loaders implementation.
+
+    Implementer should be able to load raw content from a storage system according
+    to some criteria and return the raw content lazily as a stream of blobs.
+    """
+
+    @abstractmethod
+    def yield_blobs(
+        self,
+    ) -> Iterable[Blob]:
+        """A lazy loader for raw data represented by LangChain's Blob object.
+
+        Returns:
+            A generator over blobs
+        """

langchain/document_loaders/confluence.py

@@ -189,19 +189,8 @@ class ConfluenceLoader(BaseLoader):
                 "`label`, `cql` parameters."
             )
 
-        try:
-            import html2text  # type: ignore
-        except ImportError:
-            raise ImportError(
-                "`html2text` package not found, please run `pip install html2text`"
-            )
-
         docs = []
 
-        text_maker = html2text.HTML2Text()
-        text_maker.ignore_links = True
-        text_maker.ignore_images = True
-
         if space_key:
             pages = self.paginate_request(
                 self.confluence.get_all_pages_from_space,
@@ -211,9 +200,7 @@ class ConfluenceLoader(BaseLoader):
                 expand="body.storage.value",
             )
             for page in pages:
-                doc = self.process_page(
-                    page, include_attachments, include_comments, text_maker
-                )
+                doc = self.process_page(page, include_attachments, include_comments)
                 docs.append(doc)
 
         if label:
@@ -225,9 +212,7 @@ class ConfluenceLoader(BaseLoader):
                 expand="body.storage.value",
             )
             for page in pages:
-                doc = self.process_page(
-                    page, include_attachments, include_comments, text_maker
-                )
+                doc = self.process_page(page, include_attachments, include_comments)
                 docs.append(doc)
 
         if cql:
@@ -239,9 +224,7 @@ class ConfluenceLoader(BaseLoader):
                 expand="body.storage.value",
             )
             for page in pages:
-                doc = self.process_page(
-                    page, include_attachments, include_comments, text_maker
-                )
+                doc = self.process_page(page, include_attachments, include_comments)
                 docs.append(doc)
 
         if page_ids:
@@ -259,9 +242,7 @@ class ConfluenceLoader(BaseLoader):
                     before_sleep=before_sleep_log(logger, logging.WARNING),
                 )(self.confluence.get_page_by_id)
                 page = get_page(page_id=page_id, expand="body.storage.value")
-                doc = self.process_page(
-                    page, include_attachments, include_comments, text_maker
-                )
+                doc = self.process_page(page, include_attachments, include_comments)
                 docs.append(doc)
 
         return docs
@@ -313,21 +294,28 @@ class ConfluenceLoader(BaseLoader):
         page: dict,
         include_attachments: bool,
         include_comments: bool,
-        text_maker: Any,
     ) -> Document:
+        try:
+            from bs4 import BeautifulSoup  # type: ignore
+        except ImportError:
+            raise ImportError(
+                "`beautifulsoup4` package not found, please run"
+                " `pip install beautifulsoup4`"
+            )
+
         if include_attachments:
             attachment_texts = self.process_attachment(page["id"])
         else:
             attachment_texts = []
-        text = text_maker.handle(page["body"]["storage"]["value"]) + "".join(
-            attachment_texts
-        )
+        text = BeautifulSoup(
+            page["body"]["storage"]["value"], "lxml"
+        ).get_text() + "".join(attachment_texts)
         if include_comments:
             comments = self.confluence.get_page_comments(
                 page["id"], expand="body.view.value", depth="all"
             )["results"]
             comment_texts = [
-                text_maker.handle(comment["body"]["view"]["value"])
+                BeautifulSoup(comment["body"]["view"]["value"], "lxml").get_text()
                 for comment in comments
             ]
             text = text + "".join(comment_texts)

langchain/document_loaders/html_bs.py

@@ -17,6 +17,7 @@ class BSHTMLLoader(BaseLoader):
         file_path: str,
         open_encoding: Union[str, None] = None,
         bs_kwargs: Union[dict, None] = None,
+        get_text_separator: str = "",
     ) -> None:
         """Initialise with path, and optionally, file encoding to use, and any kwargs
         to pass to the BeautifulSoup object."""
@@ -33,6 +34,7 @@ class BSHTMLLoader(BaseLoader):
         if bs_kwargs is None:
             bs_kwargs = {"features": "lxml"}
         self.bs_kwargs = bs_kwargs
+        self.get_text_separator = get_text_separator
 
     def load(self) -> List[Document]:
         from bs4 import BeautifulSoup
@@ -41,7 +43,7 @@ class BSHTMLLoader(BaseLoader):
         with open(self.file_path, "r", encoding=self.open_encoding) as f:
             soup = BeautifulSoup(f, **self.bs_kwargs)
 
-        text = soup.get_text()
+        text = soup.get_text(self.get_text_separator)
 
         if soup.title:
             title = str(soup.title.string)

langchain/document_loaders/url.py

@@ -15,6 +15,7 @@ class UnstructuredURLLoader(BaseLoader):
         self,
         urls: List[str],
         continue_on_failure: bool = True,
+        mode: str = "single",
         **unstructured_kwargs: Any,
     ):
         """Initialize with file path."""
@@ -29,6 +30,9 @@ class UnstructuredURLLoader(BaseLoader):
                 "`pip install unstructured`"
             )
 
+        self._validate_mode(mode)
+        self.mode = mode
+
         headers = unstructured_kwargs.pop("headers", {})
         if len(headers.keys()) != 0:
             warn_about_headers = False
@@ -48,6 +52,13 @@ class UnstructuredURLLoader(BaseLoader):
         self.headers = headers
         self.unstructured_kwargs = unstructured_kwargs
 
+    def _validate_mode(self, mode: str) -> None:
+        _valid_modes = {"single", "elements"}
+        if mode not in _valid_modes:
+            raise ValueError(
+                f"Got {mode} for `mode`, but should be one of `{_valid_modes}`"
+            )
+
     def __is_headers_available_for_html(self) -> bool:
         _unstructured_version = self.__version.split("-")[0]
         unstructured_version = tuple([int(x) for x in _unstructured_version.split(".")])
@@ -94,7 +105,15 @@ class UnstructuredURLLoader(BaseLoader):
                     continue
                 else:
                     raise e
-            text = "\n\n".join([str(el) for el in elements])
-            metadata = {"source": url}
-            docs.append(Document(page_content=text, metadata=metadata))
+
+            if self.mode == "single":
+                text = "\n\n".join([str(el) for el in elements])
+                metadata = {"source": url}
+                docs.append(Document(page_content=text, metadata=metadata))
+            elif self.mode == "elements":
+                for element in elements:
+                    metadata = element.metadata.to_dict()
+                    metadata["category"] = element.category
+                    docs.append(Document(page_content=str(element), metadata=metadata))
+
         return docs

langchain/document_loaders/url_selenium.py

@@ -71,6 +71,7 @@ class SeleniumURLLoader(BaseLoader):
             chrome_options = ChromeOptions()
             if self.headless:
                 chrome_options.add_argument("--headless")
+                chrome_options.add_argument("--no-sandbox")
             if self.executable_path is None:
                 return Chrome(options=chrome_options)
             return Chrome(executable_path=self.executable_path, options=chrome_options)

langchain/embeddings/sentence_transformer.py

@@ -1,63 +1,4 @@
 """Wrapper around sentence transformer embedding models."""
-from typing import Any, Dict, List, Optional
+from langchain.embeddings.huggingface import HuggingFaceEmbeddings
 
-from pydantic import BaseModel, Extra, Field, root_validator
-
-from langchain.embeddings.base import Embeddings
-
-
-class SentenceTransformerEmbeddings(BaseModel, Embeddings):
-    embedding_function: Any  #: :meta private:
-
-    model: Optional[str] = Field("all-MiniLM-L6-v2", alias="model")
-    """Transformer model to use."""
-
-    class Config:
-        """Configuration for this pydantic object."""
-
-        extra = Extra.forbid
-
-    @root_validator()
-    def validate_environment(cls, values: Dict) -> Dict:
-        """Validate that sentence_transformers library is installed."""
-        model = values["model"]
-
-        try:
-            from sentence_transformers import SentenceTransformer
-
-            values["embedding_function"] = SentenceTransformer(model)
-        except ImportError:
-            raise ModuleNotFoundError(
-                "Could not import sentence_transformers library. "
-                "Please install the sentence_transformers library to "
-                "use this embedding model: pip install sentence_transformers"
-            )
-        except Exception:
-            raise NameError(f"Could not load SentenceTransformer model {model}.")
-
-        return values
-
-    def embed_documents(self, texts: List[str]) -> List[List[float]]:
-        """Embed a list of documents using the SentenceTransformer model.
-
-        Args:
-            texts: The list of texts to embed.
-
-        Returns:
-            List of embeddings, one for each text.
-        """
-        embeddings = self.embedding_function.encode(
-            texts, convert_to_numpy=True
-        ).tolist()
-        return [list(map(float, e)) for e in embeddings]
-
-    def embed_query(self, text: str) -> List[float]:
-        """Embed a query using the SentenceTransformer model.
-
-        Args:
-            text: The text to embed.
-
-        Returns:
-            Embedding for the text.
-        """
-        return self.embed_documents([text])[0]
+SentenceTransformerEmbeddings = HuggingFaceEmbeddings

langchain/evaluation/agents/__init__.py

@@ -0,0 +1,4 @@
+"""Chains for evaluating ReAct style agents."""
+from langchain.evaluation.agents.trajectory_eval_chain import TrajectoryEvalChain
+
+__all__ = ["TrajectoryEvalChain"]

langchain/evaluation/agents/trajectory_eval_chain.py

@@ -0,0 +1,106 @@
+"""A chain for evaluating ReAct style agents."""
+from typing import Any, Dict, List, NamedTuple, Optional, Sequence, Tuple, Union
+
+from langchain.chains.base import Chain
+from langchain.chains.llm import LLMChain
+from langchain.chat_models import ChatOpenAI
+from langchain.evaluation.agents.trajectory_eval_prompt import EVAL_CHAT_PROMPT
+from langchain.schema import AgentAction, BaseOutputParser, OutputParserException
+from langchain.tools.base import BaseTool
+
+
+class TrajectoryEval(NamedTuple):
+    score: int
+    reasoning: str
+
+
+class TrajectoryOutputParser(BaseOutputParser):
+    def parse(self, text: str) -> TrajectoryEval:
+        if "Score:" not in text:
+            raise OutputParserException(
+                f"Could not find score in model eval output: {text}"
+            )
+
+        reasoning, score_str = text.split("Score: ")
+
+        reasoning, score_str = reasoning.strip(), score_str.strip()
+
+        score_str = next(
+            (char for char in score_str if char.isdigit()), "0"
+        )  # Scan for first digit
+
+        if not 1 <= int(score_str) <= 5:
+            raise OutputParserException(
+                f"Score is not a digit in the range 1-5: {text}"
+            )
+
+        return TrajectoryEval(score=int(score_str), reasoning=reasoning)
+
+
+class TrajectoryEvalChain(Chain):
+    agent_tools: List[BaseTool]
+    eval_chain: LLMChain
+    output_parser: TrajectoryOutputParser
+    return_reasoning: bool = False
+
+    @property
+    def _tools_description(self) -> str:
+        return "\n\n".join(
+            [
+                f"""Tool {i}: {tool.name}
+Description: {tool.description}"""
+                for i, tool in enumerate(self.agent_tools, 1)
+            ]
+        )
+
+    @staticmethod
+    def get_agent_trajectory(steps: Union[str, List[Tuple[AgentAction, str]]]) -> str:
+        if isinstance(steps, str):
+            return steps
+
+        return "\n\n".join(
+            [
+                f"""Step {i}:
+Tool used: {action.tool}
+Tool input: {action.tool_input}
+Tool output: {output}"""
+                for i, (action, output) in enumerate(steps, 1)
+            ]
+        )
+
+    @classmethod
+    def from_llm(
+        cls,
+        llm: ChatOpenAI,
+        agent_tools: Sequence[BaseTool],
+        output_parser: Optional[TrajectoryOutputParser] = None,
+        return_reasoning: bool = False,
+    ) -> "TrajectoryEvalChain":
+        eval_chain = LLMChain(llm=llm, prompt=EVAL_CHAT_PROMPT)
+        return cls(
+            agent_tools=agent_tools,
+            return_reasoning=return_reasoning,
+            eval_chain=eval_chain,
+            output_parser=output_parser or TrajectoryOutputParser(),
+        )
+
+    @property
+    def input_keys(self) -> List[str]:
+        return ["question", "agent_trajectory", "answer"]
+
+    @property
+    def output_keys(self) -> List[str]:
+        if self.return_reasoning:
+            return ["score", "reasoning"]
+        return ["score"]
+
+    def _call(self, inputs: Dict[str, str]) -> Dict[str, Any]:
+        raw_output = self.eval_chain.run(
+            {"tool_descriptions": self._tools_description, **inputs}
+        )
+        parsed_output = self.output_parser.parse(raw_output)
+
+        if self.return_reasoning:
+            return {"score": parsed_output.score, "reasoning": parsed_output.reasoning}
+
+        return {"score": parsed_output.score}

langchain/evaluation/agents/trajectory_eval_prompt.py

@@ -0,0 +1,98 @@
+"""Prompt for trajectory evaluation chain."""
+# flake8: noqa
+from langchain.schema import AIMessage
+from langchain.schema import HumanMessage
+from langchain.schema import SystemMessage
+
+from langchain.prompts.chat import (
+    ChatPromptTemplate,
+    HumanMessagePromptTemplate,
+)
+
+
+EVAL_TEMPLATE = """An AI language model has been given access to the following set of tools to help answer a user's question.
+
+The tools given to the AI model are:
+
+{tool_descriptions}
+
+The question the human asked the AI model was: {question}
+
+The AI language model decided to use the following set of tools to answer the question:
+
+{agent_trajectory}
+
+The AI language model's final answer to the question was: {answer}
+
+Let's to do a detailed evaluation of the AI language model's answer step by step.
+
+We consider the following criteria before giving a score from 1 to 5:
+
+i. Is the final answer helpful?
+ii. Does the AI language use a logical sequence of tools to answer the question?
+iii. Does the AI language model use the tools in a helpful way?
+iv. Does the AI language model use too many steps to answer the question?
+v. Are the appropriate tools used to answer the question?"""
+
+EXAMPLE_INPUT = """An AI language model has been given acces to the following set of tools to help answer a user's question.
+
+The tools given to the AI model are:
+
+Tool 1:
+Name: Search
+Description: useful for when you need to ask with search
+
+Tool 2:
+Name: Lookup
+Description: useful for when you need to ask with lookup
+
+Tool 3:
+Name: Calculator
+Description: useful for doing calculations
+
+Tool 4:
+Name: Search the Web (SerpAPI)
+Description: useful for when you need to answer questions about current events
+
+The question the human asked the AI model was: If laid the Statue of Liberty end to end, how many times would it stretch across the United States?
+
+The AI language model decided to use the following set of tools to answer the question:
+
+Step 1:
+Tool used: Search the Web (SerpAPI)
+Tool input: If laid the Statue of Liberty end to end, how many times would it stretch across the United States?
+Tool output: The Statue of Liberty was given to the United States by France, as a symbol of the two countries' friendship. It was erected atop an American-designed ...
+
+The AI language model's final answer to the question was: There are different ways to measure the length of the United States, but if we use the distance between the Statue of Liberty and the westernmost point of the contiguous United States (Cape Alava, Washington), which is approximately 2,857 miles (4,596 km), and assume that the Statue of Liberty is 305 feet (93 meters) tall, then the statue would stretch across the United States approximately 17.5 times if laid end to end.
+
+Let's to do a detailed evaluation of the AI language model's answer step by step.
+
+We consider the following criteria before giving a score from 1 to 5:
+
+i. Is the final answer helpful?
+ii. Does the AI language use a logical sequence of tools to answer the question?
+iii. Does the AI language model use the tools in a helpful way?
+iv. Does the AI language model use too many steps to answer the question?
+v. Are the appropriate tools used to answer the question?"""
+
+EXAMPLE_OUTPUT = """First, let's evaluate the final answer. The final uses good reasoning but is wrong. 2,857 divided by 305 is not 17.5.\
+The model should have used the calculator to figure this out. Second does the model use a logical sequence of tools to answer the question?\
+The way model uses the search is not helpful. The model should have used the search tool to figure the width of the US or the height of the statue.\
+The model didn't use the calculator tool and gave an incorrect answer. The search API should be used for current events or specific questions.\
+The tools were not used in a helpful way. The model did not use too many steps to answer the question.\
+The model did not use the appropriate tools to answer the question.\
+    
+Judgment: Given the good reasoning in the final answer but otherwise poor performance, we give the model a score of 2.
+
+Score: 2"""
+
+EVAL_CHAT_PROMPT = ChatPromptTemplate.from_messages(
+    messages=[
+        SystemMessage(
+            content="You are a helpful assistant that evaluates language models."
+        ),
+        HumanMessage(content=EXAMPLE_INPUT),
+        AIMessage(content=EXAMPLE_OUTPUT),
+        HumanMessagePromptTemplate.from_template(EVAL_TEMPLATE),
+    ]
+)

langchain/llms/__init__.py

@@ -20,6 +20,7 @@ from langchain.llms.modal import Modal
 from langchain.llms.nlpcloud import NLPCloud
 from langchain.llms.openai import AzureOpenAI, OpenAI, OpenAIChat
 from langchain.llms.petals import Petals
+from langchain.llms.pipelineai import PipelineAI
 from langchain.llms.predictionguard import PredictionGuard
 from langchain.llms.promptlayer_openai import PromptLayerOpenAI, PromptLayerOpenAIChat
 from langchain.llms.replicate import Replicate
@@ -46,6 +47,7 @@ __all__ = [
     "OpenAI",
     "OpenAIChat",
     "Petals",
+    "PipelineAI",
     "HuggingFaceEndpoint",
     "HuggingFaceHub",
     "SagemakerEndpoint",
@@ -82,6 +84,7 @@ type_to_cls_dict: Dict[str, Type[BaseLLM]] = {
     "nlpcloud": NLPCloud,
     "openai": OpenAI,
     "petals": Petals,
+    "pipelineai": PipelineAI,
     "huggingface_pipeline": HuggingFacePipeline,
     "azure": AzureOpenAI,
     "replicate": Replicate,

langchain/llms/openai.py

@@ -242,10 +242,15 @@ class BaseOpenAI(BaseLLM):
             "frequency_penalty": self.frequency_penalty,
             "presence_penalty": self.presence_penalty,
             "n": self.n,
-            "best_of": self.best_of,
             "request_timeout": self.request_timeout,
             "logit_bias": self.logit_bias,
         }
+
+        # Azure gpt-35-turbo doesn't support best_of
+        # don't specify best_of if it is 1
+        if self.best_of > 1:
+            normal_params["best_of"] = self.best_of
+
         return {**normal_params, **self.model_kwargs}
 
     def _generate(

langchain/llms/pipelineai.py

@@ -0,0 +1,107 @@
+"""Wrapper around Pipeline Cloud API."""
+import logging
+from typing import Any, Dict, List, Mapping, Optional
+
+from pydantic import BaseModel, Extra, Field, root_validator
+
+from langchain.llms.base import LLM
+from langchain.llms.utils import enforce_stop_tokens
+from langchain.utils import get_from_dict_or_env
+
+logger = logging.getLogger(__name__)
+
+
+class PipelineAI(LLM, BaseModel):
+    """Wrapper around PipelineAI large language models.
+
+    To use, you should have the ``pipeline-ai`` python package installed,
+    and the environment variable ``PIPELINE_API_KEY`` set with your API key.
+
+    Any parameters that are valid to be passed to the call can be passed
+    in, even if not explicitly saved on this class.
+
+    Example:
+        .. code-block:: python
+            from langchain import PipelineAI
+            pipeline = PipelineAI(pipeline_key="")
+    """
+
+    pipeline_key: str = ""
+    """The id or tag of the target pipeline"""
+
+    pipeline_kwargs: Dict[str, Any] = Field(default_factory=dict)
+    """Holds any pipeline parameters valid for `create` call not
+    explicitly specified."""
+
+    pipeline_api_key: Optional[str] = None
+
+    class Config:
+        """Configuration for this pydantic config."""
+
+        extra = Extra.forbid
+
+    @root_validator(pre=True)
+    def build_extra(cls, values: Dict[str, Any]) -> Dict[str, Any]:
+        """Build extra kwargs from additional params that were passed in."""
+        all_required_field_names = {field.alias for field in cls.__fields__.values()}
+
+        extra = values.get("pipeline_kwargs", {})
+        for field_name in list(values):
+            if field_name not in all_required_field_names:
+                if field_name in extra:
+                    raise ValueError(f"Found {field_name} supplied twice.")
+                logger.warning(
+                    f"""{field_name} was transfered to pipeline_kwargs.
+                    Please confirm that {field_name} is what you intended."""
+                )
+                extra[field_name] = values.pop(field_name)
+        values["pipeline_kwargs"] = extra
+        return values
+
+    @root_validator()
+    def validate_environment(cls, values: Dict) -> Dict:
+        """Validate that api key and python package exists in environment."""
+        pipeline_api_key = get_from_dict_or_env(
+            values, "pipeline_api_key", "PIPELINE_API_KEY"
+        )
+        values["pipeline_api_key"] = pipeline_api_key
+        return values
+
+    @property
+    def _identifying_params(self) -> Mapping[str, Any]:
+        """Get the identifying parameters."""
+        return {
+            **{"pipeline_key": self.pipeline_key},
+            **{"pipeline_kwargs": self.pipeline_kwargs},
+        }
+
+    @property
+    def _llm_type(self) -> str:
+        """Return type of llm."""
+        return "pipeline_ai"
+
+    def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
+        """Call to Pipeline Cloud endpoint."""
+        try:
+            from pipeline import PipelineCloud
+        except ImportError:
+            raise ValueError(
+                "Could not import pipeline-ai python package. "
+                "Please install it with `pip install pipeline-ai`."
+            )
+        client = PipelineCloud(token=self.pipeline_api_key)
+        params = self.pipeline_kwargs or {}
+
+        run = client.run_pipeline(self.pipeline_key, [prompt, params])
+        try:
+            text = run.result_preview[0][0]
+        except AttributeError:
+            raise AttributeError(
+                f"A pipeline run should have a `result_preview` attribute."
+                f"Run was: {run}"
+            )
+        if stop is not None:
+            # I believe this is required since the stop tokens
+            # are not enforced by the pipeline parameters
+            text = enforce_stop_tokens(text, stop)
+        return text

langchain/llms/replicate.py

@@ -103,6 +103,6 @@ class Replicate(LLM):
         first_input_name = input_properties[0][0]
 
         inputs = {first_input_name: prompt, **self.input}
+        iterator = replicate_python.run(self.model, input={**inputs})
 
-        outputs = replicate_python.run(self.model, input={**inputs})
-        return outputs[0]
+        return "".join([output for output in iterator])

langchain/memory/entity.py

@@ -235,4 +235,5 @@ class ConversationEntityMemory(BaseChatMemory):
     def clear(self) -> None:
         """Clear memory contents."""
         self.chat_memory.clear()
+        self.entity_cache.clear()
         self.entity_store.clear()

langchain/output_parsers/structured.py

@@ -22,6 +22,27 @@ def _get_sub_string(schema: ResponseSchema) -> str:
     )
 
 
+def parse_json_markdown(text: str, expected_keys: List[str]) -> Any:
+    if "```json" not in text:
+        raise OutputParserException(
+            f"Got invalid return object. Expected markdown code snippet with JSON "
+            f"object, but got:\n{text}"
+        )
+
+    json_string = text.split("```json")[1].strip().strip("```").strip()
+    try:
+        json_obj = json.loads(json_string)
+    except json.JSONDecodeError as e:
+        raise OutputParserException(f"Got invalid JSON object. Error: {e}")
+    for key in expected_keys:
+        if key not in json_obj:
+            raise OutputParserException(
+                f"Got invalid return object. Expected key `{key}` "
+                f"to be present, but got {json_obj}"
+            )
+    return json_obj
+
+
 class StructuredOutputParser(BaseOutputParser):
     response_schemas: List[ResponseSchema]
 
@@ -38,24 +59,8 @@ class StructuredOutputParser(BaseOutputParser):
         return STRUCTURED_FORMAT_INSTRUCTIONS.format(format=schema_str)
 
     def parse(self, text: str) -> Any:
-        if "```json" not in text:
-            raise OutputParserException(
-                f"Got invalid return object. Expected markdown code snippet with JSON "
-                f"object, but got:\n{text}"
-            )
-
-        json_string = text.split("```json")[1].strip().strip("```").strip()
-        try:
-            json_obj = json.loads(json_string)
-        except json.JSONDecodeError as e:
-            raise OutputParserException(f"Got invalid JSON object. Error: {e}")
-        for schema in self.response_schemas:
-            if schema.name not in json_obj:
-                raise OutputParserException(
-                    f"Got invalid return object. Expected key `{schema.name}` "
-                    f"to be present, but got {json_obj}"
-                )
-        return json_obj
+        expected_keys = [rs.name for rs in self.response_schemas]
+        return parse_json_markdown(text, expected_keys)
 
     @property
     def _type(self) -> str:

langchain/retrievers/document_compressors/base.py

@@ -8,7 +8,7 @@ from langchain.schema import BaseDocumentTransformer, Document
 
 
 class BaseDocumentCompressor(BaseModel, ABC):
-    """"""
+    """Base abstraction interface for document compression."""
 
     @abstractmethod
     def compress_documents(

langchain/retrievers/self_query/base.py

@@ -0,0 +1,116 @@
+"""Retriever that generates and executes structured queries over its own data source."""
+from typing import Any, Dict, List, Optional, Type, cast
+
+from pydantic import BaseModel, Field, root_validator
+
+from langchain import LLMChain
+from langchain.chains.query_constructor.base import (
+    load_query_constructor_chain,
+)
+from langchain.chains.query_constructor.ir import StructuredQuery, Visitor
+from langchain.chains.query_constructor.schema import AttributeInfo
+from langchain.retrievers.self_query.pinecone import PineconeTranslator
+from langchain.schema import BaseLanguageModel, BaseRetriever, Document
+from langchain.vectorstores import Pinecone, VectorStore
+
+
+def _get_builtin_translator(vectorstore_cls: Type[VectorStore]) -> Visitor:
+    """Get the translator class corresponding to the vector store class."""
+    BUILTIN_TRANSLATORS: Dict[Type[VectorStore], Type[Visitor]] = {
+        Pinecone: PineconeTranslator
+    }
+    if vectorstore_cls not in BUILTIN_TRANSLATORS:
+        raise ValueError(
+            f"Self query retriever with Vector Store type {vectorstore_cls}"
+            f" not supported."
+        )
+    return BUILTIN_TRANSLATORS[vectorstore_cls]()
+
+
+class SelfQueryRetriever(BaseRetriever, BaseModel):
+    """Retriever that wraps around a vector store and uses an LLM to generate
+    the vector store queries."""
+
+    vectorstore: VectorStore
+    """The underlying vector store from which documents will be retrieved."""
+    llm_chain: LLMChain
+    """The LLMChain for generating the vector store queries."""
+    search_type: str = "similarity"
+    """The search type to perform on the vector store."""
+    search_kwargs: dict = Field(default_factory=dict)
+    """Keyword arguments to pass in to the vector store search."""
+    structured_query_translator: Visitor
+    """Translator for turning internal query language into vectorstore search params."""
+    verbose: bool = False
+
+    class Config:
+        """Configuration for this pydantic object."""
+
+        arbitrary_types_allowed = True
+
+    @root_validator(pre=True)
+    def validate_translator(cls, values: Dict) -> Dict:
+        """Validate translator."""
+        if "structured_query_translator" not in values:
+            vectorstore_cls = values["vectorstore"].__class__
+            values["structured_query_translator"] = _get_builtin_translator(
+                vectorstore_cls
+            )
+        return values
+
+    def get_relevant_documents(self, query: str) -> List[Document]:
+        """Get documents relevant for a query.
+
+        Args:
+            query: string to find relevant documents for
+
+        Returns:
+            List of relevant documents
+        """
+        inputs = self.llm_chain.prep_inputs(query)
+        structured_query = cast(
+            StructuredQuery, self.llm_chain.predict_and_parse(**inputs)
+        )
+        if self.verbose:
+            print(structured_query)
+        new_query, new_kwargs = self.structured_query_translator.visit_structured_query(
+            structured_query
+        )
+        search_kwargs = {**self.search_kwargs, **new_kwargs}
+        docs = self.vectorstore.search(query, self.search_type, **search_kwargs)
+        return docs
+
+    async def aget_relevant_documents(self, query: str) -> List[Document]:
+        raise NotImplementedError
+
+    @classmethod
+    def from_llm(
+        cls,
+        llm: BaseLanguageModel,
+        vectorstore: VectorStore,
+        document_contents: str,
+        metadata_field_info: List[AttributeInfo],
+        structured_query_translator: Optional[Visitor] = None,
+        chain_kwargs: Optional[Dict] = None,
+        **kwargs: Any,
+    ) -> "SelfQueryRetriever":
+        if structured_query_translator is None:
+            structured_query_translator = _get_builtin_translator(vectorstore.__class__)
+        chain_kwargs = chain_kwargs or {}
+        if "allowed_comparators" not in chain_kwargs:
+            chain_kwargs[
+                "allowed_comparators"
+            ] = structured_query_translator.allowed_comparators
+        if "allowed_operators" not in chain_kwargs:
+            chain_kwargs[
+                "allowed_operators"
+            ] = structured_query_translator.allowed_operators
+        llm_chain = load_query_constructor_chain(
+            llm, document_contents, metadata_field_info, **chain_kwargs
+        )
+        return cls(
+            llm_chain=llm_chain,
+            vectorstore=vectorstore,
+            structured_query_translator=structured_query_translator,
+            **kwargs,
+        )

langchain/retrievers/self_query/pinecone.py

@@ -0,0 +1,53 @@
+"""Logic for converting internal query language to a valid Pinecone query."""
+from typing import Dict, Tuple, Union
+
+from langchain.chains.query_constructor.ir import (
+    Comparator,
+    Comparison,
+    Operation,
+    Operator,
+    StructuredQuery,
+    Visitor,
+)
+
+
+class PineconeTranslator(Visitor):
+    """Logic for converting internal query language elements to valid filters."""
+
+    allowed_operators = [Operator.AND, Operator.OR]
+    """Subset of allowed logical operators."""
+
+    def _format_func(self, func: Union[Operator, Comparator]) -> str:
+        if isinstance(func, Operator) and self.allowed_operators is not None:
+            if func not in self.allowed_operators:
+                raise ValueError(
+                    f"Received disallowed operator {func}. Allowed "
+                    f"comparators are {self.allowed_operators}"
+                )
+        if isinstance(func, Comparator) and self.allowed_comparators is not None:
+            if func not in self.allowed_comparators:
+                raise ValueError(
+                    f"Received disallowed comparator {func}. Allowed "
+                    f"comparators are {self.allowed_comparators}"
+                )
+        return f"${func}"
+
+    def visit_operation(self, operation: Operation) -> Dict:
+        args = [arg.accept(self) for arg in operation.arguments]
+        return {self._format_func(operation.operator): args}
+
+    def visit_comparison(self, comparison: Comparison) -> Dict:
+        return {
+            comparison.attribute: {
+                self._format_func(comparison.comparator): comparison.value
+            }
+        }
+
+    def visit_structured_query(
+        self, structured_query: StructuredQuery
+    ) -> Tuple[str, dict]:
+        if structured_query.filter is None:
+            kwargs = {}
+        else:
+            kwargs = {"filter": structured_query.filter.accept(self)}
+        return structured_query.query, kwargs

langchain/tools/__init__.py

@@ -1,19 +1,27 @@
 """Core toolkit implementations."""
 
 from langchain.tools.base import BaseTool
-from langchain.tools.ddg_search.tool import DuckDuckGoSearchTool
+from langchain.tools.bing_search.tool import BingSearchResults, BingSearchRun
+from langchain.tools.ddg_search.tool import DuckDuckGoSearchResults, DuckDuckGoSearchRun
 from langchain.tools.google_places.tool import GooglePlacesTool
+from langchain.tools.google_search.tool import GoogleSearchResults, GoogleSearchRun
 from langchain.tools.ifttt import IFTTTWebhook
 from langchain.tools.openapi.utils.api_models import APIOperation
 from langchain.tools.openapi.utils.openapi_utils import OpenAPISpec
 from langchain.tools.plugin import AIPluginTool
 
 __all__ = [
-    "BaseTool",
-    "IFTTTWebhook",
     "AIPluginTool",
-    "OpenAPISpec",
     "APIOperation",
+    "BingSearchResults",
+    "BingSearchRun",
+    "DuckDuckGoSearchResults",
+    "DuckDuckGoSearchRun",
+    "DuckDuckGoSearchRun",
     "GooglePlacesTool",
-    "DuckDuckGoSearchTool",
+    "GoogleSearchResults",
+    "GoogleSearchRun",
+    "IFTTTWebhook",
+    "OpenAPISpec",
+    "BaseTool",
 ]

langchain/tools/base.py

@@ -3,7 +3,7 @@ from __future__ import annotations
 
 from abc import ABC, abstractmethod
 from inspect import signature
-from typing import Any, Callable, Dict, Optional, Sequence, Tuple, Type, Union
+from typing import Any, Callable, Dict, Optional, Set, Tuple, Type, Union
 
 from pydantic import (
     BaseModel,
@@ -19,15 +19,6 @@ from langchain.callbacks import get_callback_manager
 from langchain.callbacks.base import BaseCallbackManager
 
 
-def _to_args_and_kwargs(run_input: Union[str, Dict]) -> Tuple[Sequence, dict]:
-    # For backwards compatability, if run_input is a string,
-    # pass as a positional argument.
-    if isinstance(run_input, str):
-        return (run_input,), {}
-    else:
-        return [], run_input
-
-
 class SchemaAnnotationError(TypeError):
     """Raised when 'args_schema' is missing or has an incorrect type annotation."""
 
@@ -81,11 +72,16 @@ def _create_subset_model(
     return create_model(name, **fields)  # type: ignore
 
 
-def get_filtered_args(inferred_model: Type[BaseModel], func: Callable) -> dict:
+def get_filtered_args(
+    inferred_model: Type[BaseModel],
+    func: Callable,
+    invalid_args: Optional[Set[str]] = None,
+) -> dict:
     """Get the arguments from a function's signature."""
     schema = inferred_model.schema()["properties"]
     valid_keys = signature(func).parameters
-    return {k: schema[k] for k in valid_keys}
+    invalid_args = invalid_args or set()
+    return {k: schema[k] for k in valid_keys if k not in invalid_args}
 
 
 def create_schema_from_function(model_name: str, func: Callable) -> Type[BaseModel]:
@@ -115,6 +111,11 @@ class BaseTool(ABC, BaseModel, metaclass=ToolMetaclass):
         extra = Extra.forbid
         arbitrary_types_allowed = True
 
+    @property
+    def is_single_input(self) -> bool:
+        """Whether the tool only accepts a single input."""
+        return len(self.args) == 1
+
     @property
     def args(self) -> dict:
         if self.args_schema is not None:
@@ -148,13 +149,21 @@ class BaseTool(ABC, BaseModel, metaclass=ToolMetaclass):
         return callback_manager or get_callback_manager()
 
     @abstractmethod
-    def _run(self, *args: Any, **kwargs: Any) -> str:
+    def _run(self, *args: Any, **kwargs: Any) -> Any:
         """Use the tool."""
 
     @abstractmethod
-    async def _arun(self, *args: Any, **kwargs: Any) -> str:
+    async def _arun(self, *args: Any, **kwargs: Any) -> Any:
         """Use the tool asynchronously."""
 
+    def _to_args_and_kwargs(self, tool_input: Union[str, Dict]) -> Tuple[Tuple, Dict]:
+        # For backwards compatability, if run_input is a string,
+        # pass as a positional argument.
+        if isinstance(tool_input, str):
+            return (tool_input,), {}
+        else:
+            return (), tool_input
+
     def run(
         self,
         tool_input: Union[str, Dict],
@@ -177,13 +186,13 @@ class BaseTool(ABC, BaseModel, metaclass=ToolMetaclass):
             **kwargs,
         )
         try:
-            tool_args, tool_kwargs = _to_args_and_kwargs(tool_input)
+            tool_args, tool_kwargs = self._to_args_and_kwargs(tool_input)
             observation = self._run(*tool_args, **tool_kwargs)
         except (Exception, KeyboardInterrupt) as e:
             self.callback_manager.on_tool_error(e, verbose=verbose_)
             raise e
         self.callback_manager.on_tool_end(
-            observation, verbose=verbose_, color=color, name=self.name, **kwargs
+            str(observation), verbose=verbose_, color=color, name=self.name, **kwargs
         )
         return observation
 
@@ -194,7 +203,7 @@ class BaseTool(ABC, BaseModel, metaclass=ToolMetaclass):
         start_color: Optional[str] = "green",
         color: Optional[str] = "green",
         **kwargs: Any,
-    ) -> str:
+    ) -> Any:
         """Run the tool asynchronously."""
         self._parse_input(tool_input)
         if not self.verbose and verbose is not None:
@@ -219,8 +228,8 @@ class BaseTool(ABC, BaseModel, metaclass=ToolMetaclass):
             )
         try:
             # We then call the tool on the tool input to get an observation
-            args, kwargs = _to_args_and_kwargs(tool_input)
-            observation = await self._arun(*args, **kwargs)
+            tool_args, tool_kwargs = self._to_args_and_kwargs(tool_input)
+            observation = await self._arun(*tool_args, **tool_kwargs)
         except (Exception, KeyboardInterrupt) as e:
             if self.callback_manager.is_async:
                 await self.callback_manager.on_tool_error(e, verbose=verbose_)
@@ -229,7 +238,11 @@ class BaseTool(ABC, BaseModel, metaclass=ToolMetaclass):
             raise e
         if self.callback_manager.is_async:
             await self.callback_manager.on_tool_end(
-                observation, verbose=verbose_, color=color, name=self.name, **kwargs
+                str(observation),
+                verbose=verbose_,
+                color=color,
+                name=self.name,
+                **kwargs,
             )
         else:
             self.callback_manager.on_tool_end(
@@ -237,6 +250,6 @@ class BaseTool(ABC, BaseModel, metaclass=ToolMetaclass):
             )
         return observation
 
-    def __call__(self, tool_input: str) -> str:
+    def __call__(self, tool_input: Union[str, dict]) -> Any:
         """Make tool callable."""
         return self.run(tool_input)

langchain/tools/bing_search/tool.py

@@ -22,3 +22,24 @@ class BingSearchRun(BaseTool):
     async def _arun(self, query: str) -> str:
         """Use the tool asynchronously."""
         raise NotImplementedError("BingSearchRun does not support async")
+
+
+class BingSearchResults(BaseTool):
+    """Tool that has capability to query the Bing Search API and get back json."""
+
+    name = "Bing Search Results JSON"
+    description = (
+        "A wrapper around Bing Search. "
+        "Useful for when you need to answer questions about current events. "
+        "Input should be a search query. Output is a JSON array of the query results"
+    )
+    num_results: int = 4
+    api_wrapper: BingSearchAPIWrapper
+
+    def _run(self, query: str) -> str:
+        """Use the tool."""
+        return str(self.api_wrapper.results(query, self.num_results))
+
+    async def _arun(self, query: str) -> str:
+        """Use the tool asynchronously."""
+        raise NotImplementedError("BingSearchResults does not support async")

langchain/tools/ddg_search/__init__.py

@@ -1,5 +1,5 @@
 """DuckDuckGo Search API toolkit."""
 
-from langchain.tools.ddg_search.tool import DuckDuckGoSearchTool
+from langchain.tools.ddg_search.tool import DuckDuckGoSearchRun
 
-__all__ = ["DuckDuckGoSearchTool"]
+__all__ = ["DuckDuckGoSearchRun"]

langchain/tools/ddg_search/tool.py

@@ -1,12 +1,15 @@
 """Tool for the DuckDuckGo search API."""
 
+import warnings
+from typing import Any
+
 from pydantic import Field
 
 from langchain.tools.base import BaseTool
 from langchain.utilities.duckduckgo_search import DuckDuckGoSearchAPIWrapper
 
 
-class DuckDuckGoSearchTool(BaseTool):
+class DuckDuckGoSearchRun(BaseTool):
     """Tool that adds the capability to query the DuckDuckGo search API."""
 
     name = "DuckDuckGo Search"
@@ -26,3 +29,35 @@ class DuckDuckGoSearchTool(BaseTool):
     async def _arun(self, query: str) -> str:
         """Use the tool asynchronously."""
         raise NotImplementedError("DuckDuckGoSearch does not support async")
+
+
+class DuckDuckGoSearchResults(BaseTool):
+    """Tool that queries the Duck Duck Go Search API and get back json."""
+
+    name = "DuckDuckGo Results JSON"
+    description = (
+        "A wrapper around Duck Duck Go Search. "
+        "Useful for when you need to answer questions about current events. "
+        "Input should be a search query. Output is a JSON array of the query results"
+    )
+    num_results: int = 4
+    api_wrapper: DuckDuckGoSearchAPIWrapper = Field(
+        default_factory=DuckDuckGoSearchAPIWrapper
+    )
+
+    def _run(self, query: str) -> str:
+        """Use the tool."""
+        return str(self.api_wrapper.results(query, self.num_results))
+
+    async def _arun(self, query: str) -> str:
+        """Use the tool asynchronously."""
+        raise NotImplementedError("DuckDuckGoSearchResults does not support async")
+
+
+def DuckDuckGoSearchTool(*args: Any, **kwargs: Any) -> DuckDuckGoSearchRun:
+    warnings.warn(
+        "DuckDuckGoSearchTool will be deprecated in the future. "
+        "Please use DuckDuckGoSearchRun instead.",
+        DeprecationWarning,
+    )
+    return DuckDuckGoSearchRun(*args, **kwargs)

langchain/tools/powerbi/prompt.py

@@ -58,5 +58,5 @@ BAD_REQUEST_RESPONSE = (
     "Bad request. Please ask the question_to_query_powerbi tool to provide the query."
 )
 BAD_REQUEST_RESPONSE_ESCALATED = "You already tried this, please try a different query."
-
+SCHEMA_ERROR_RESPONSE = "Bad request, are you sure the table name is correct?"
 UNAUTHORIZED_RESPONSE = "Unauthorized. Try changing your authentication, do not retry."

langchain/tools/powerbi/tool.py

@@ -147,8 +147,8 @@ class InputToQueryTool(BaseTool):
     """  # noqa: E501
     llm_chain: LLMChain
     powerbi: PowerBIDataset = Field(exclude=True)
-    template: str = QUESTION_TO_QUERY
-    examples: str = DEFAULT_FEWSHOT_EXAMPLES
+    template: Optional[str] = QUESTION_TO_QUERY
+    examples: Optional[str] = DEFAULT_FEWSHOT_EXAMPLES
 
     class Config:
         """Configuration for this pydantic object."""

langchain/tools/requests/tool.py

@@ -14,6 +14,11 @@ def _parse_input(text: str) -> Dict[str, Any]:
     return json.loads(text)
 
 
+def _clean_url(url: str) -> str:
+    """Strips quotes from the url."""
+    return url.strip("\"'")
+
+
 class BaseRequestsTool(BaseModel):
     """Base class for requests tools."""
 
@@ -28,11 +33,11 @@ class RequestsGetTool(BaseRequestsTool, BaseTool):
 
     def _run(self, url: str) -> str:
         """Run the tool."""
-        return self.requests_wrapper.get(url)
+        return self.requests_wrapper.get(_clean_url(url))
 
     async def _arun(self, url: str) -> str:
         """Run the tool asynchronously."""
-        return await self.requests_wrapper.aget(url)
+        return await self.requests_wrapper.aget(_clean_url(url))
 
 
 class RequestsPostTool(BaseRequestsTool, BaseTool):
@@ -51,7 +56,7 @@ class RequestsPostTool(BaseRequestsTool, BaseTool):
         """Run the tool."""
         try:
             data = _parse_input(text)
-            return self.requests_wrapper.post(data["url"], data["data"])
+            return self.requests_wrapper.post(_clean_url(data["url"]), data["data"])
         except Exception as e:
             return repr(e)
 
@@ -59,7 +64,9 @@ class RequestsPostTool(BaseRequestsTool, BaseTool):
         """Run the tool asynchronously."""
         try:
             data = _parse_input(text)
-            return await self.requests_wrapper.apost(data["url"], data["data"])
+            return await self.requests_wrapper.apost(
+                _clean_url(data["url"]), data["data"]
+            )
         except Exception as e:
             return repr(e)
 
@@ -80,7 +87,7 @@ class RequestsPatchTool(BaseRequestsTool, BaseTool):
         """Run the tool."""
         try:
             data = _parse_input(text)
-            return self.requests_wrapper.patch(data["url"], data["data"])
+            return self.requests_wrapper.patch(_clean_url(data["url"]), data["data"])
         except Exception as e:
             return repr(e)
 
@@ -88,7 +95,9 @@ class RequestsPatchTool(BaseRequestsTool, BaseTool):
         """Run the tool asynchronously."""
         try:
             data = _parse_input(text)
-            return await self.requests_wrapper.apatch(data["url"], data["data"])
+            return await self.requests_wrapper.apatch(
+                _clean_url(data["url"]), data["data"]
+            )
         except Exception as e:
             return repr(e)
 
@@ -109,7 +118,7 @@ class RequestsPutTool(BaseRequestsTool, BaseTool):
         """Run the tool."""
         try:
             data = _parse_input(text)
-            return self.requests_wrapper.put(data["url"], data["data"])
+            return self.requests_wrapper.put(_clean_url(data["url"]), data["data"])
         except Exception as e:
             return repr(e)
 
@@ -117,7 +126,9 @@ class RequestsPutTool(BaseRequestsTool, BaseTool):
         """Run the tool asynchronously."""
         try:
             data = _parse_input(text)
-            return await self.requests_wrapper.aput(data["url"], data["data"])
+            return await self.requests_wrapper.aput(
+                _clean_url(data["url"]), data["data"]
+            )
         except Exception as e:
             return repr(e)
 
@@ -130,8 +141,8 @@ class RequestsDeleteTool(BaseRequestsTool, BaseTool):
 
     def _run(self, url: str) -> str:
         """Run the tool."""
-        return self.requests_wrapper.delete(url)
+        return self.requests_wrapper.delete(_clean_url(url))
 
     async def _arun(self, url: str) -> str:
         """Run the tool asynchronously."""
-        return await self.requests_wrapper.adelete(url)
+        return await self.requests_wrapper.adelete(_clean_url(url))

langchain/utilities/arxiv.py

@@ -1,8 +1,13 @@
 """Util that calls Arxiv."""
-from typing import Any, Dict
+import logging
+from typing import Any, Dict, List
 
 from pydantic import BaseModel, Extra, root_validator
 
+from langchain.schema import Document
+
+logger = logging.getLogger(__name__)
+
 
 class ArxivAPIWrapper(BaseModel):
     """Wrapper around ArxivAPI.
@@ -12,12 +17,23 @@ class ArxivAPIWrapper(BaseModel):
     This wrapper will use the Arxiv API to conduct searches and
     fetch document summaries. By default, it will return the document summaries
     of the top-k results of an input search.
+
+    Parameters:
+        top_k_results: number of the top-scored document used for the arxiv tool
+        ARXIV_MAX_QUERY_LENGTH: the cut limit on the query used for the arxiv tool.
+        load_max_docs: a limit to the number of loaded documents
+        load_all_available_meta:
+          if True: the `metadata` of the loaded Documents gets all available meta info
+            (see https://lukasschwab.me/arxiv.py/index.html#Result),
+          if False: the `metadata` gets only the most informative fields.
     """
 
     arxiv_client: Any  #: :meta private:
     arxiv_exceptions: Any  # :meta private:
     top_k_results: int = 3
     ARXIV_MAX_QUERY_LENGTH = 300
+    load_max_docs: int = 100
+    load_all_available_meta: bool = False
 
     class Config:
         """Configuration for this pydantic object."""
@@ -36,6 +52,7 @@ class ArxivAPIWrapper(BaseModel):
                 arxiv.UnexpectedEmptyPageError,
                 arxiv.HTTPError,
             )
+            values["arxiv_result"] = arxiv.Result
         except ImportError:
             raise ValueError(
                 "Could not import arxiv python package. "
@@ -62,3 +79,64 @@ class ArxivAPIWrapper(BaseModel):
             return "\n\n".join(docs) if docs else "No good Arxiv Result was found"
         except self.arxiv_exceptions as ex:
             return f"Arxiv exception: {ex}"
+
+    def load(self, query: str) -> List[Document]:
+        """
+        Run Arxiv search and get the PDF documents plus the meta information.
+        See https://lukasschwab.me/arxiv.py/index.html#Search
+
+        Returns: a list of documents with the document.page_content in PDF format
+
+        """
+        try:
+            import fitz
+        except ImportError:
+            raise ValueError(
+                "PyMuPDF package not found, please install it with "
+                "`pip install pymupdf`"
+            )
+
+        try:
+            docs: List[Document] = []
+            for result in self.arxiv_search(  # type: ignore
+                query[: self.ARXIV_MAX_QUERY_LENGTH], max_results=self.load_max_docs
+            ).results():
+                try:
+                    doc_file_name: str = result.download_pdf()
+                    with fitz.open(doc_file_name) as doc_file:
+                        text: str = "".join(page.get_text() for page in doc_file)
+                        add_meta = (
+                            {
+                                "entry_id": result.entry_id,
+                                "published_first_time": str(result.published.date()),
+                                "comment": result.comment,
+                                "journal_ref": result.journal_ref,
+                                "doi": result.doi,
+                                "primary_category": result.primary_category,
+                                "categories": result.categories,
+                                "links": [link.href for link in result.links],
+                            }
+                            if self.load_all_available_meta
+                            else {}
+                        )
+                        doc = Document(
+                            page_content=text,
+                            metadata=(
+                                {
+                                    "Published": str(result.updated.date()),
+                                    "Title": result.title,
+                                    "Authors": ", ".join(
+                                        a.name for a in result.authors
+                                    ),
+                                    "Summary": result.summary,
+                                    **add_meta,
+                                }
+                            ),
+                        )
+                        docs.append(doc)
+                except FileNotFoundError as f_ex:
+                    logger.debug(f_ex)
+            return docs
+        except self.arxiv_exceptions as ex:
+            logger.debug("Error on arxiv: %s", ex)
+            return []

langchain/utilities/bash.py

@@ -1,24 +1,80 @@
 """Wrapper around subprocess to run commands."""
+from __future__ import annotations
+
+import platform
+import re
 import subprocess
-from typing import List, Union
+from typing import TYPE_CHECKING, List, Union
+from uuid import uuid4
+
+if TYPE_CHECKING:
+    import pexpect
+
+
+def _lazy_import_pexpect() -> pexpect:
+    """Import pexpect only when needed."""
+    if platform.system() == "Windows":
+        raise ValueError("Persistent bash processes are not yet supported on Windows.")
+    try:
+        import pexpect
+
+    except ImportError:
+        raise ImportError(
+            "pexpect required for persistent bash processes."
+            " To install, run `pip install pexpect`."
+        )
+    return pexpect
 
 
 class BashProcess:
     """Executes bash commands and returns the output."""
 
-    def __init__(self, strip_newlines: bool = False, return_err_output: bool = False):
+    def __init__(
+        self,
+        strip_newlines: bool = False,
+        return_err_output: bool = False,
+        persistent: bool = False,
+    ):
         """Initialize with stripping newlines."""
         self.strip_newlines = strip_newlines
         self.return_err_output = return_err_output
+        self.prompt = ""
+        self.process = None
+        if persistent:
+            self.prompt = str(uuid4())
+            self.process = self._initialize_persistent_process(self.prompt)
+
+    @staticmethod
+    def _initialize_persistent_process(prompt: str) -> pexpect.spawn:
+        # Start bash in a clean environment
+        # Doesn't work on windows
+        pexpect = _lazy_import_pexpect()
+        process = pexpect.spawn(
+            "env", ["-i", "bash", "--norc", "--noprofile"], encoding="utf-8"
+        )
+        # Set the custom prompt
+        process.sendline("PS1=" + prompt)
+
+        process.expect_exact(prompt, timeout=10)
+        return process
 
     def run(self, commands: Union[str, List[str]]) -> str:
         """Run commands and return final output."""
         if isinstance(commands, str):
             commands = [commands]
         commands = ";".join(commands)
+        if self.process is not None:
+            return self._run_persistent(
+                commands,
+            )
+        else:
+            return self._run(commands)
+
+    def _run(self, command: str) -> str:
+        """Run commands and return final output."""
         try:
             output = subprocess.run(
-                commands,
+                command,
                 shell=True,
                 check=True,
                 stdout=subprocess.PIPE,
@@ -31,3 +87,32 @@ class BashProcess:
         if self.strip_newlines:
             output = output.strip()
         return output
+
+    def process_output(self, output: str, command: str) -> str:
+        # Remove the command from the output using a regular expression
+        pattern = re.escape(command) + r"\s*\n"
+        output = re.sub(pattern, "", output, count=1)
+        return output.strip()
+
+    def _run_persistent(self, command: str) -> str:
+        """Run commands and return final output."""
+        pexpect = _lazy_import_pexpect()
+        if self.process is None:
+            raise ValueError("Process not initialized")
+        self.process.sendline(command)
+
+        # Clear the output with an empty string
+        self.process.expect(self.prompt, timeout=10)
+        self.process.sendline("")
+
+        try:
+            self.process.expect([self.prompt, pexpect.EOF], timeout=10)
+        except pexpect.TIMEOUT:
+            return f"Timeout error while executing command {command}"
+        if self.process.after == pexpect.EOF:
+            return f"Exited with error status: {self.process.exitstatus}"
+        output = self.process.before
+        output = self.process_output(output, command)
+        if self.strip_newlines:
+            return output.strip()
+        return output

langchain/utilities/duckduckgo_search.py

@@ -41,7 +41,7 @@ class DuckDuckGoSearchAPIWrapper(BaseModel):
     def run(self, query: str) -> str:
         from duckduckgo_search import ddg
 
-        """Run query through DuckDuckGo and return results."""
+        """Run query through DuckDuckGo and return concatenated results."""
         results = ddg(
             query,
             region=self.region,
@@ -54,7 +54,7 @@ class DuckDuckGoSearchAPIWrapper(BaseModel):
         snippets = [result["body"] for result in results]
         return " ".join(snippets)
 
-    def results(self, query: str, num_results: int) -> List[Dict]:
+    def results(self, query: str, num_results: int) -> List[Dict[str, str]]:
         """Run query through DuckDuckGo and return metadata.
 
         Args:
@@ -80,7 +80,7 @@ class DuckDuckGoSearchAPIWrapper(BaseModel):
         if results is None or len(results) == 0:
             return [{"Result": "No good DuckDuckGo Search Result was found"}]
 
-        def to_metadata(result: Dict) -> Dict:
+        def to_metadata(result: Dict) -> Dict[str, str]:
             return {
                 "snippet": result["body"],
                 "title": result["title"],

langchain/utilities/powerbi.py

@@ -12,17 +12,15 @@ from aiohttp import ServerTimeoutError
 from pydantic import BaseModel, Field, root_validator
 from requests.exceptions import Timeout
 
-from langchain.tools.powerbi.prompt import BAD_REQUEST_RESPONSE, UNAUTHORIZED_RESPONSE
+from langchain.tools.powerbi.prompt import SCHEMA_ERROR_RESPONSE, UNAUTHORIZED_RESPONSE
 
 _LOGGER = logging.getLogger(__name__)
 
-if TYPE_CHECKING:
-    from azure.core.exceptions import ClientAuthenticationError
-    from azure.identity import ChainedTokenCredential
-    from azure.identity._internal import InteractiveCredential
-
 BASE_URL = os.getenv("POWERBI_BASE_URL", "https://api.powerbi.com/v1.0/myorg/datasets/")
 
+if TYPE_CHECKING:
+    from azure.core.credentials import TokenCredential
+
 
 class PowerBIDataset(BaseModel):
     """Create PowerBI engine from dataset ID and credential or token.
@@ -36,7 +34,7 @@ class PowerBIDataset(BaseModel):
     dataset_id: str
     table_names: List[str]
     group_id: Optional[str] = None
-    credential: Optional[Union[ChainedTokenCredential, InteractiveCredential]] = None
+    credential: Optional[TokenCredential] = None
     token: Optional[str] = None
     impersonated_user_name: Optional[str] = None
     sample_rows_in_table_info: int = Field(default=1, gt=0, le=10)
@@ -65,6 +63,8 @@ class PowerBIDataset(BaseModel):
     @property
     def headers(self) -> Dict[str, str]:
         """Get the token."""
+        from azure.core.exceptions import ClientAuthenticationError
+
         token = None
         if self.token:
             token = self.token
@@ -144,7 +144,7 @@ class PowerBIDataset(BaseModel):
                 continue
             except Exception as exc:  # pylint: disable=broad-exception-caught
                 if "bad request" in str(exc).lower():
-                    return BAD_REQUEST_RESPONSE
+                    return SCHEMA_ERROR_RESPONSE
                 if "unauthorized" in str(exc).lower():
                     return UNAUTHORIZED_RESPONSE
                 return str(exc)
@@ -167,7 +167,7 @@ class PowerBIDataset(BaseModel):
                 continue
             except Exception as exc:  # pylint: disable=broad-exception-caught
                 if "bad request" in str(exc).lower():
-                    return BAD_REQUEST_RESPONSE
+                    return SCHEMA_ERROR_RESPONSE
                 if "unauthorized" in str(exc).lower():
                     return UNAUTHORIZED_RESPONSE
                 return str(exc)

langchain/utilities/serpapi.py

@@ -77,7 +77,23 @@ class SerpAPIWrapper(BaseModel):
         return values
 
     async def arun(self, query: str) -> str:
-        """Use aiohttp to run query through SerpAPI and parse result."""
+        """Run query through SerpAPI and parse result async."""
+        return self._process_response(await self.aresults(query))
+
+    def run(self, query: str) -> str:
+        """Run query through SerpAPI and parse result."""
+        return self._process_response(self.results(query))
+
+    def results(self, query: str) -> dict:
+        """Run query through SerpAPI and return the raw result."""
+        params = self.get_params(query)
+        with HiddenPrints():
+            search = self.search_engine(params)
+            res = search.get_dict()
+        return res
+
+    async def aresults(self, query: str) -> dict:
+        """Use aiohttp to run query through SerpAPI and return the results async."""
 
         def construct_url_and_params() -> Tuple[str, Dict[str, str]]:
             params = self.get_params(query)
@@ -97,18 +113,6 @@ class SerpAPIWrapper(BaseModel):
             async with self.aiosession.get(url, params=params) as response:
                 res = await response.json()
 
-        return self._process_response(res)
-
-    def run(self, query: str) -> str:
-        """Run query through SerpAPI and parse result."""
-        return self._process_response(self.results(query))
-
-    def results(self, query: str) -> dict:
-        """Run query through SerpAPI and return the raw result."""
-        params = self.get_params(query)
-        with HiddenPrints():
-            search = self.search_engine(params)
-            res = search.get_dict()
         return res
 
     def get_params(self, query: str) -> Dict[str, str]:

langchain/vectorstores/__init__.py

@@ -7,6 +7,7 @@ from langchain.vectorstores.chroma import Chroma
 from langchain.vectorstores.deeplake import DeepLake
 from langchain.vectorstores.elastic_vector_search import ElasticVectorSearch
 from langchain.vectorstores.faiss import FAISS
+from langchain.vectorstores.lancedb import LanceDB
 from langchain.vectorstores.milvus import Milvus
 from langchain.vectorstores.myscale import MyScale, MyScaleSettings
 from langchain.vectorstores.opensearch_vector_search import OpenSearchVectorSearch
@@ -34,4 +35,5 @@ __all__ = [
     "MyScaleSettings",
     "SupabaseVectorStore",
     "AnalyticDB",
+    "LanceDB",
 ]

langchain/vectorstores/base.py

@@ -75,6 +75,32 @@ class VectorStore(ABC):
         metadatas = [doc.metadata for doc in documents]
         return await self.aadd_texts(texts, metadatas, **kwargs)
 
+    def search(self, query: str, search_type: str, **kwargs: Any) -> List[Document]:
+        """Return docs most similar to query using specified search type."""
+        if search_type == "similarity":
+            return self.similarity_search(query, **kwargs)
+        elif search_type == "mmr":
+            return self.max_marginal_relevance_search(query, **kwargs)
+        else:
+            raise ValueError(
+                f"search_type of {search_type} not allowed. Expected "
+                "search_type to be 'similarity' or 'mmr'."
+            )
+
+    async def asearch(
+        self, query: str, search_type: str, **kwargs: Any
+    ) -> List[Document]:
+        """Return docs most similar to query using specified search type."""
+        if search_type == "similarity":
+            return await self.asimilarity_search(query, **kwargs)
+        elif search_type == "mmr":
+            return await self.amax_marginal_relevance_search(query, **kwargs)
+        else:
+            raise ValueError(
+                f"search_type of {search_type} not allowed. Expected "
+                "search_type to be 'similarity' or 'mmr'."
+            )
+
     @abstractmethod
     def similarity_search(
         self, query: str, k: int = 4, **kwargs: Any

langchain/vectorstores/deeplake.py

@@ -243,7 +243,7 @@ class DeepLake(VectorStore):
         self.ds.summary()
         return ids
 
-    def search(
+    def _search_helper(
         self,
         query: Any[str, None] = None,
         embedding: Any[float, None] = None,
@@ -366,7 +366,7 @@ class DeepLake(VectorStore):
         Returns:
             List of Documents most similar to the query vector.
         """
-        return self.search(query=query, k=k, **kwargs)
+        return self._search_helper(query=query, k=k, **kwargs)
 
     def similarity_search_by_vector(
         self, embedding: List[float], k: int = 4, **kwargs: Any
@@ -379,7 +379,7 @@ class DeepLake(VectorStore):
         Returns:
             List of Documents most similar to the query vector.
         """
-        return self.search(embedding=embedding, k=k, **kwargs)
+        return self._search_helper(embedding=embedding, k=k, **kwargs)
 
     def similarity_search_with_score(
         self,
@@ -401,7 +401,7 @@ class DeepLake(VectorStore):
             List[Tuple[Document, float]]: List of documents most similar to the query
                 text with distance in float.
         """
-        return self.search(
+        return self._search_helper(
             query=query,
             k=k,
             filter=filter,
@@ -431,7 +431,7 @@ class DeepLake(VectorStore):
         Returns:
             List of Documents selected by maximal marginal relevance.
         """
-        return self.search(
+        return self._search_helper(
             embedding=embedding,
             k=k,
             fetch_k=fetch_k,
@@ -465,7 +465,7 @@ class DeepLake(VectorStore):
             raise ValueError(
                 "For MMR search, you must specify an embedding function on" "creation."
             )
-        return self.search(
+        return self._search_helper(
             query=query,
             k=k,
             fetch_k=fetch_k,

langchain/vectorstores/lancedb.py

@@ -0,0 +1,133 @@
+"""Wrapper around LanceDB vector database"""
+from __future__ import annotations
+
+import uuid
+from typing import Any, Iterable, List, Optional
+
+from langchain.docstore.document import Document
+from langchain.embeddings.base import Embeddings
+from langchain.vectorstores.base import VectorStore
+
+
+class LanceDB(VectorStore):
+    """Wrapper around LanceDB vector database.
+
+    To use, you should have ``lancedb`` python package installed.
+
+    Example:
+        .. code-block:: python
+
+            db = lancedb.connect('./lancedb')
+            table = db.open_table('my_table')
+            vectorstore = LanceDB(table, embedding_function)
+            vectorstore.add_texts(['text1', 'text2'])
+            result = vectorstore.similarity_search('text1')
+    """
+
+    def __init__(
+        self,
+        connection: Any,
+        embedding: Embeddings,
+        vector_key: Optional[str] = "vector",
+        id_key: Optional[str] = "id",
+        text_key: Optional[str] = "text",
+    ):
+        """Initialize with Lance DB connection"""
+        try:
+            import lancedb
+        except ImportError:
+            raise ValueError(
+                "Could not import lancedb python package. "
+                "Please install it with `pip install lancedb`."
+            )
+        if not isinstance(connection, lancedb.db.LanceTable):
+            raise ValueError(
+                "connection should be an instance of lancedb.db.LanceTable, ",
+                f"got {type(connection)}",
+            )
+        self._connection = connection
+        self._embedding = embedding
+        self._vector_key = vector_key
+        self._id_key = id_key
+        self._text_key = text_key
+
+    def add_texts(
+        self,
+        texts: Iterable[str],
+        metadatas: Optional[List[dict]] = None,
+        ids: Optional[List[str]] = None,
+        **kwargs: Any,
+    ) -> List[str]:
+        """Turn texts into embedding and add it to the database
+
+        Args:
+            texts: Iterable of strings to add to the vectorstore.
+            metadatas: Optional list of metadatas associated with the texts.
+            ids: Optional list of ids to associate with the texts.
+
+        Returns:
+            List of ids of the added texts.
+        """
+        # Embed texts and create documents
+        docs = []
+        ids = ids or [str(uuid.uuid4()) for _ in texts]
+        embeddings = self._embedding.embed_documents(list(texts))
+        for idx, text in enumerate(texts):
+            embedding = embeddings[idx]
+            metadata = metadatas[idx] if metadatas else {}
+            docs.append(
+                {
+                    self._vector_key: embedding,
+                    self._id_key: ids[idx],
+                    self._text_key: text,
+                    **metadata,
+                }
+            )
+
+        self._connection.add(docs)
+        return ids
+
+    def similarity_search(
+        self, query: str, k: int = 4, **kwargs: Any
+    ) -> List[Document]:
+        """Return documents most similar to the query
+
+        Args:
+            query: String to query the vectorstore with.
+            k: Number of documents to return.
+
+        Returns:
+            List of documents most similar to the query.
+        """
+        embedding = self._embedding.embed_query(query)
+        docs = self._connection.search(embedding).limit(k).to_df()
+        return [
+            Document(
+                page_content=row[self._text_key],
+                metadata=row[docs.columns != self._text_key],
+            )
+            for _, row in docs.iterrows()
+        ]
+
+    @classmethod
+    def from_texts(
+        cls,
+        texts: List[str],
+        embedding: Embeddings,
+        metadatas: Optional[List[dict]] = None,
+        connection: Any = None,
+        vector_key: Optional[str] = "vector",
+        id_key: Optional[str] = "id",
+        text_key: Optional[str] = "text",
+        **kwargs: Any,
+    ) -> LanceDB:
+        instance = LanceDB(
+            connection,
+            embedding,
+            vector_key,
+            id_key,
+            text_key,
+        )
+        instance.add_texts(texts, metadatas=metadatas, **kwargs)
+
+        return instance

langchain/vectorstores/opensearch_vector_search.py

@@ -35,6 +35,15 @@ def _import_bulk() -> Any:
     return bulk
 
 
+def _import_not_found_error() -> Any:
+    """Import not found error if available, otherwise raise error."""
+    try:
+        from opensearchpy.exceptions import NotFoundError
+    except ImportError:
+        raise ValueError(IMPORT_OPENSEARCH_PY_ERROR)
+    return NotFoundError
+
+
 def _get_opensearch_client(opensearch_url: str, **kwargs: Any) -> Any:
     """Get OpenSearch client from the opensearch_url, otherwise raise error."""
     try:
@@ -67,11 +76,20 @@ def _bulk_ingest_embeddings(
     metadatas: Optional[List[dict]] = None,
     vector_field: str = "vector_field",
     text_field: str = "text",
+    mapping: Dict = {},
 ) -> List[str]:
     """Bulk Ingest Embeddings into given index."""
     bulk = _import_bulk()
+    not_found_error = _import_not_found_error()
     requests = []
     ids = []
+    mapping = mapping
+
+    try:
+        client.indices.get(index=index_name)
+    except not_found_error:
+        client.indices.create(index=index_name, body=mapping)
+
     for i, text in enumerate(texts):
         metadata = metadatas[i] if metadatas else {}
         _id = str(uuid.uuid4())
@@ -311,8 +329,19 @@ class OpenSearchVectorSearch(VectorStore):
         """
         embeddings = self.embedding_function.embed_documents(list(texts))
         _validate_embeddings_and_bulk_size(len(embeddings), bulk_size)
-        vector_field = _get_kwargs_value(kwargs, "vector_field", "vector_field")
         text_field = _get_kwargs_value(kwargs, "text_field", "text")
+        dim = len(embeddings[0])
+        engine = _get_kwargs_value(kwargs, "engine", "nmslib")
+        space_type = _get_kwargs_value(kwargs, "space_type", "l2")
+        ef_search = _get_kwargs_value(kwargs, "ef_search", 512)
+        ef_construction = _get_kwargs_value(kwargs, "ef_construction", 512)
+        m = _get_kwargs_value(kwargs, "m", 16)
+        vector_field = _get_kwargs_value(kwargs, "vector_field", "vector_field")
+
+        mapping = _default_text_mapping(
+            dim, engine, space_type, ef_search, ef_construction, m, vector_field
+        )
+
         return _bulk_ingest_embeddings(
             self.client,
             self.index_name,
@@ -321,6 +350,7 @@ class OpenSearchVectorSearch(VectorStore):
             metadatas,
             vector_field,
             text_field,
+            mapping,
         )
 
     def similarity_search(
@@ -532,8 +562,14 @@ class OpenSearchVectorSearch(VectorStore):
 
         [kwargs.pop(key, None) for key in keys_list]
         client = _get_opensearch_client(opensearch_url, **kwargs)
-        client.indices.create(index=index_name, body=mapping)
         _bulk_ingest_embeddings(
-            client, index_name, embeddings, texts, metadatas, vector_field, text_field
+            client,
+            index_name,
+            embeddings,
+            texts,
+            metadatas,
+            vector_field,
+            text_field,
+            mapping,
         )
         return cls(opensearch_url, index_name, embedding, **kwargs)

langchain/vectorstores/weaviate.py

@@ -27,14 +27,18 @@ def _default_schema(index_name: str) -> Dict:
 
 def _create_weaviate_client(**kwargs: Any) -> Any:
     client = kwargs.get("client")
-
     if client is not None:
         return client
 
     weaviate_url = get_from_dict_or_env(kwargs, "weaviate_url", "WEAVIATE_URL")
-    weaviate_api_key = get_from_dict_or_env(
-        kwargs, "weaviate_api_key", "WEAVIATE_API_KEY", None
-    )
+
+    try:
+        # the weaviate api key param should not be mandatory
+        weaviate_api_key = get_from_dict_or_env(
+            kwargs, "weaviate_api_key", "WEAVIATE_API_KEY", None
+        )
+    except ValueError:
+        weaviate_api_key = None
 
     try:
         import weaviate
@@ -117,9 +121,21 @@ class Weaviate(VectorStore):
                         data_properties[key] = metadatas[i][key]
 
                 _id = get_valid_uuid(uuid4())
-                batch.add_data_object(
-                    data_object=data_properties, class_name=self._index_name, uuid=_id
-                )
+
+                if self._embedding is not None:
+                    embeddings = self._embedding.embed_documents(list(doc))
+                    batch.add_data_object(
+                        data_object=data_properties,
+                        class_name=self._index_name,
+                        uuid=_id,
+                        vector=embeddings[0],
+                    )
+                else:
+                    batch.add_data_object(
+                        data_object=data_properties,
+                        class_name=self._index_name,
+                        uuid=_id,
+                    )
                 ids.append(_id)
         return ids
 

poetry.lock

@@ -1,4 +1,4 @@
-# This file is automatically @generated by Poetry and should not be changed by hand.
+# This file is automatically @generated by Poetry 1.4.2 and should not be changed by hand.
 
 [[package]]
 name = "absl-py"
@@ -1595,7 +1595,7 @@ files = [
 name = "duckdb"
 version = "0.7.1"
 description = "DuckDB embedded database"
-category = "dev"
+category = "main"
 optional = false
 python-versions = "*"
 files = [
@@ -3445,6 +3445,29 @@ files = [
     {file = "keras-2.11.0-py2.py3-none-any.whl", hash = "sha256:38c6fff0ea9a8b06a2717736565c92a73c8cd9b1c239e7125ccb188b7848f65e"},
 ]
 
+[[package]]
+name = "lancedb"
+version = "0.1"
+description = "lancedb"
+category = "main"
+optional = true
+python-versions = ">=3.8"
+files = [
+    {file = "lancedb-0.1-py3-none-any.whl", hash = "sha256:b4180c08298324f36df1128aaae6b7f1fbef46c959a1b2d430f559ffe3454bf3"},
+    {file = "lancedb-0.1.tar.gz", hash = "sha256:443c26c392de409243a3758a1bb9535d9d057d8c6aebf4d3290915de1a7aea99"},
+]
+
+[package.dependencies]
+pylance = ">=0.4.3"
+ratelimiter = "*"
+retry = "*"
+tqdm = "*"
+
+[package.extras]
+dev = ["black", "pre-commit", "ruff"]
+docs = ["mkdocs", "mkdocs-jupyter", "mkdocs-material", "mkdocstrings[python]"]
+tests = ["pytest"]
+
 [[package]]
 name = "langcodes"
 version = "3.3.0"
@@ -3460,6 +3483,23 @@ files = [
 [package.extras]
 data = ["language-data (>=1.1,<2.0)"]
 
+[[package]]
+name = "lark"
+version = "1.1.5"
+description = "a modern parsing library"
+category = "main"
+optional = false
+python-versions = "*"
+files = [
+    {file = "lark-1.1.5-py3-none-any.whl", hash = "sha256:8476f9903e93fbde4f6c327f74d79e9b4bd0ed9294c5dfa3164ab8c581b5de2a"},
+    {file = "lark-1.1.5.tar.gz", hash = "sha256:4b534eae1f9af5b4ea000bea95776350befe1981658eea3820a01c37e504bb4d"},
+]
+
+[package.extras]
+atomic-cache = ["atomicwrites"]
+nearley = ["js2py"]
+regex = ["regex"]
+
 [[package]]
 name = "libclang"
 version = "16.0.0"
@@ -5024,7 +5064,7 @@ files = [
 name = "pandas"
 version = "2.0.0"
 description = "Powerful data structures for data analysis, time series, and statistics"
-category = "dev"
+category = "main"
 optional = false
 python-versions = ">=3.8"
 files = [
@@ -5722,6 +5762,18 @@ files = [
 [package.extras]
 tests = ["pytest"]
 
+[[package]]
+name = "py"
+version = "1.11.0"
+description = "library with cross-python path, ini-parsing, io, code, log facilities"
+category = "main"
+optional = true
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+files = [
+    {file = "py-1.11.0-py2.py3-none-any.whl", hash = "sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378"},
+    {file = "py-1.11.0.tar.gz", hash = "sha256:51c75c4126074b472f746a24399ad32f6053d1b34b68d2fa41e558e6f4a98719"},
+]
+
 [[package]]
 name = "pyarrow"
 version = "11.0.0"
@@ -5995,6 +6047,29 @@ dev = ["coverage[toml] (==5.0.4)", "cryptography (>=3.4.0)", "pre-commit", "pyte
 docs = ["sphinx (>=4.5.0,<5.0.0)", "sphinx-rtd-theme", "zope.interface"]
 tests = ["coverage[toml] (==5.0.4)", "pytest (>=6.0.0,<7.0.0)"]
 
+[[package]]
+name = "pylance"
+version = "0.4.3"
+description = "python wrapper for lance-rs"
+category = "main"
+optional = true
+python-versions = ">=3.8"
+files = [
+    {file = "pylance-0.4.3-cp38-abi3-macosx_10_15_x86_64.whl", hash = "sha256:94ea9489daa802cea726f56dbdb29f402e38e99e7472e48a56cba1d2cf53a83f"},
+    {file = "pylance-0.4.3-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:567654e35aae255f4dd48fb9c05e6b25299239bb277921195f7b242525363605"},
+    {file = "pylance-0.4.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3c63a9067adb18c96459dee289b3d3b8508e131d1c9ed737656ad7d222efda25"},
+    {file = "pylance-0.4.3-cp38-abi3-win_amd64.whl", hash = "sha256:7c7f6dd032b13d3532d200cb92a74b596cdcd59c7f21add1d99607a7f427e624"},
+]
+
+[package.dependencies]
+duckdb = ">=0.7"
+numpy = "*"
+pandas = ">=1.5"
+pyarrow = ">=10"
+
+[package.extras]
+tests = ["duckdb", "polars[pandas,pyarrow]", "pytest"]
+
 [[package]]
 name = "pyowm"
 version = "3.3.0"
@@ -6563,6 +6638,21 @@ packaging = "*"
 [package.extras]
 test = ["pytest (>=6,!=7.0.0,!=7.0.1)", "pytest-cov (>=3.0.0)", "pytest-qt"]
 
+[[package]]
+name = "ratelimiter"
+version = "1.2.0.post0"
+description = "Simple python rate limiting object"
+category = "main"
+optional = true
+python-versions = "*"
+files = [
+    {file = "ratelimiter-1.2.0.post0-py3-none-any.whl", hash = "sha256:a52be07bc0bb0b3674b4b304550f10c769bbb00fead3072e035904474259809f"},
+    {file = "ratelimiter-1.2.0.post0.tar.gz", hash = "sha256:5c395dcabdbbde2e5178ef3f89b568a3066454a6ddc223b76473dac22f89b4f7"},
+]
+
+[package.extras]
+test = ["pytest (>=3.0)", "pytest-asyncio"]
+
 [[package]]
 name = "redis"
 version = "4.5.4"
@@ -6715,6 +6805,22 @@ urllib3 = ">=1.25.10"
 [package.extras]
 tests = ["coverage (>=6.0.0)", "flake8", "mypy", "pytest (>=7.0.0)", "pytest-asyncio", "pytest-cov", "pytest-httpserver", "types-requests"]
 
+[[package]]
+name = "retry"
+version = "0.9.2"
+description = "Easy to use retry decorator."
+category = "main"
+optional = true
+python-versions = "*"
+files = [
+    {file = "retry-0.9.2-py2.py3-none-any.whl", hash = "sha256:ccddf89761fa2c726ab29391837d4327f819ea14d244c232a1d24c67a2f98606"},
+    {file = "retry-0.9.2.tar.gz", hash = "sha256:f8bfa8b99b69c4506d6f5bd3b0aabf77f98cdb17f3c9fc3f5ca820033336fba4"},
+]
+
+[package.dependencies]
+decorator = ">=3.4.2"
+py = ">=1.4.26,<2.0.0"
+
 [[package]]
 name = "rfc3339-validator"
 version = "0.1.4"
@@ -7524,7 +7630,7 @@ files = [
 ]
 
 [package.dependencies]
-greenlet = {version = "!=0.4.17", markers = "python_version >= \"3\" and (platform_machine == \"aarch64\" or platform_machine == \"ppc64le\" or platform_machine == \"x86_64\" or platform_machine == \"amd64\" or platform_machine == \"AMD64\" or platform_machine == \"win32\" or platform_machine == \"WIN32\")"}
+greenlet = {version = "!=0.4.17", markers = "python_version >= \"3\" and platform_machine == \"aarch64\" or python_version >= \"3\" and platform_machine == \"ppc64le\" or python_version >= \"3\" and platform_machine == \"x86_64\" or python_version >= \"3\" and platform_machine == \"amd64\" or python_version >= \"3\" and platform_machine == \"AMD64\" or python_version >= \"3\" and platform_machine == \"win32\" or python_version >= \"3\" and platform_machine == \"WIN32\""}
 
 [package.extras]
 aiomysql = ["aiomysql", "greenlet (!=0.4.17)"]
@@ -8498,7 +8604,7 @@ typing-extensions = ">=3.7.4"
 name = "tzdata"
 version = "2023.3"
 description = "Provider of IANA time zone data"
-category = "dev"
+category = "main"
 optional = false
 python-versions = ">=2"
 files = [
@@ -9287,15 +9393,15 @@ cffi = {version = ">=1.11", markers = "platform_python_implementation == \"PyPy\
 cffi = ["cffi (>=1.11)"]
 
 [extras]
-all = ["anthropic", "cohere", "openai", "nlpcloud", "huggingface_hub", "jina", "manifest-ml", "elasticsearch", "opensearch-py", "google-search-results", "faiss-cpu", "sentence-transformers", "transformers", "spacy", "nltk", "wikipedia", "beautifulsoup4", "tiktoken", "torch", "jinja2", "pinecone-client", "pinecone-text", "weaviate-client", "redis", "google-api-python-client", "wolframalpha", "qdrant-client", "tensorflow-text", "pypdf", "networkx", "nomic", "aleph-alpha-client", "deeplake", "pgvector", "psycopg2-binary", "pyowm", "pytesseract", "html2text", "atlassian-python-api", "gptcache", "duckduckgo-search", "arxiv", "azure-identity", "clickhouse-connect", "azure-cosmos"]
-azure = ["azure-identity", "azure-cosmos", "openai"]
+all = ["aleph-alpha-client", "anthropic", "arxiv", "atlassian-python-api", "azure-cosmos", "azure-identity", "beautifulsoup4", "clickhouse-connect", "cohere", "deeplake", "duckduckgo-search", "elasticsearch", "faiss-cpu", "google-api-python-client", "google-search-results", "gptcache", "html2text", "huggingface_hub", "jina", "jinja2", "lancedb", "lark", "manifest-ml", "networkx", "nlpcloud", "nltk", "nomic", "openai", "opensearch-py", "pgvector", "pinecone-client", "pinecone-text", "psycopg2-binary", "pyowm", "pypdf", "pytesseract", "qdrant-client", "redis", "sentence-transformers", "spacy", "tensorflow-text", "tiktoken", "torch", "transformers", "weaviate-client", "wikipedia", "wolframalpha"]
+azure = ["azure-core", "azure-cosmos", "azure-identity", "openai"]
 cohere = ["cohere"]
 embeddings = ["sentence-transformers"]
-llms = ["anthropic", "cohere", "openai", "nlpcloud", "huggingface_hub", "manifest-ml", "torch", "transformers"]
+llms = ["anthropic", "cohere", "huggingface_hub", "manifest-ml", "nlpcloud", "openai", "torch", "transformers"]
 openai = ["openai"]
 qdrant = ["qdrant-client"]
 
 [metadata]
 lock-version = "2.0"
 python-versions = ">=3.8.1,<4.0"
-content-hash = "1dd0c2f259c674c2f3f2e2212459bf7a056f72f25e0a271194ad7ac8f70a3ac1"
+content-hash = "f7ff48dfce65630ea5c67287e91d923be83b9d0d9dd68639afcbc29f5f6f9c5f"

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "langchain"
-version = "0.0.149"
+version = "0.0.151"
 description = "Building applications with LLMs through composability"
 authors = []
 license = "MIT"
@@ -17,6 +17,7 @@ SQLAlchemy = ">1.3,<3"
 requests = "^2"
 PyYAML = ">=5.4.1"
 numpy = "^1"
+azure-core = {version = "^1.26.4", optional=true}
 tqdm = {version = ">=4.48.0", optional = true}
 openapi-schema-pydantic = "^1.2"
 faiss-cpu = {version = "^1", optional = true}
@@ -70,6 +71,9 @@ html2text = {version="^2020.1.16", optional=true}
 numexpr = "^2.8.4"
 duckduckgo-search = {version="^2.8.6", optional=true}
 azure-cosmos = {version="^4.4.0b1", optional=true}
+lark = {version="^1.1.5", optional=true}
+lancedb = {version = "^0.1", optional = true}
+pexpect = {version = "^4.8.0", optional = true}
 
 [tool.poetry.group.docs.dependencies]
 autodoc_pydantic = "^1.8.0"
@@ -95,6 +99,7 @@ pytest-watcher = "^0.2.6"
 freezegun = "^1.2.2"
 responses = "^0.22.0"
 pytest-asyncio = "^0.20.3"
+lark = "^1.1.5"
 
 [tool.poetry.group.test_integration]
 optional = true
@@ -146,8 +151,8 @@ qdrant = ["qdrant-client"]
 openai = ["openai"]
 cohere = ["cohere"]
 embeddings = ["sentence-transformers"]
-azure = ["azure-identity", "azure-cosmos", "openai"]
-all = ["anthropic", "cohere", "openai", "nlpcloud", "huggingface_hub", "jina", "manifest-ml", "elasticsearch", "opensearch-py", "google-search-results", "faiss-cpu", "sentence-transformers", "transformers", "spacy", "nltk", "wikipedia", "beautifulsoup4", "tiktoken", "torch", "jinja2", "pinecone-client", "pinecone-text", "weaviate-client", "redis", "google-api-python-client", "wolframalpha", "qdrant-client", "tensorflow-text", "pypdf", "networkx", "nomic", "aleph-alpha-client", "deeplake", "pgvector", "psycopg2-binary", "boto3", "pyowm", "pytesseract", "html2text", "atlassian-python-api", "gptcache", "duckduckgo-search", "arxiv", "azure-identity", "clickhouse-connect", "azure-cosmos"]
+azure = ["azure-identity", "azure-cosmos", "openai", "azure-core"]
+all = ["anthropic", "cohere", "openai", "nlpcloud", "huggingface_hub", "jina", "manifest-ml", "elasticsearch", "opensearch-py", "google-search-results", "faiss-cpu", "sentence-transformers", "transformers", "spacy", "nltk", "wikipedia", "beautifulsoup4", "tiktoken", "torch", "jinja2", "pinecone-client", "pinecone-text", "weaviate-client", "redis", "google-api-python-client", "wolframalpha", "qdrant-client", "tensorflow-text", "pypdf", "networkx", "nomic", "aleph-alpha-client", "deeplake", "pgvector", "psycopg2-binary", "boto3", "pyowm", "pytesseract", "html2text", "atlassian-python-api", "gptcache", "duckduckgo-search", "arxiv", "azure-identity", "clickhouse-connect", "azure-cosmos", "lancedb", "lark", "pexpect"]
 
 [tool.ruff]
 select = [

tests/integration_tests/cache/test_gptcache.py

@@ -1,61 +1,48 @@
 import os
+from typing import Any, Callable, Optional
 
 import pytest
 
 import langchain
 from langchain.cache import GPTCache
-from langchain.schema import Generation, LLMResult
+from langchain.schema import Generation
 from tests.unit_tests.llms.fake_llm import FakeLLM
 
 try:
-    import gptcache  # noqa: F401
+    from gptcache import Cache  # noqa: F401
+    from gptcache.manager.factory import get_data_manager
+    from gptcache.processor.pre import get_prompt
 
     gptcache_installed = True
 except ImportError:
     gptcache_installed = False
 
 
+def init_gptcache_map(cache_obj: Cache) -> None:
+    i = getattr(init_gptcache_map, "_i", 0)
+    cache_path = f"data_map_{i}.txt"
+    if os.path.isfile(cache_path):
+        os.remove(cache_path)
+    cache_obj.init(
+        pre_embedding_func=get_prompt,
+        data_manager=get_data_manager(data_path=cache_path),
+    )
+    init_gptcache_map._i = i + 1  # type: ignore
+
+
 @pytest.mark.skipif(not gptcache_installed, reason="gptcache not installed")
-def test_gptcache_map_caching() -> None:
-    """Test gptcache caching behavior."""
-
-    from gptcache import Cache
-    from gptcache.manager.factory import get_data_manager
-    from gptcache.processor.pre import get_prompt
-
-    i = 0
-    file_prefix = "data_map"
-
-    def init_gptcache_map(cache_obj: Cache) -> None:
-        nonlocal i
-        cache_path = f"{file_prefix}_{i}.txt"
-        if os.path.isfile(cache_path):
-            os.remove(cache_path)
-        cache_obj.init(
-            pre_embedding_func=get_prompt,
-            data_manager=get_data_manager(data_path=cache_path),
-        )
-        i += 1
-
-    langchain.llm_cache = GPTCache(init_gptcache_map)
-
+@pytest.mark.parametrize("init_func", [None, init_gptcache_map])
+def test_gptcache_caching(init_func: Optional[Callable[[Any], None]]) -> None:
+    """Test gptcache default caching behavior."""
+    langchain.llm_cache = GPTCache(init_func)
     llm = FakeLLM()
     params = llm.dict()
     params["stop"] = None
     llm_string = str(sorted([(k, v) for k, v in params.items()]))
     langchain.llm_cache.update("foo", llm_string, [Generation(text="fizz")])
-    output = llm.generate(["foo", "bar", "foo"])
-    expected_cache_output = [Generation(text="foo")]
-    cache_output = langchain.llm_cache.lookup("bar", llm_string)
-    assert cache_output == expected_cache_output
-    langchain.llm_cache = None
-    expected_generations = [
-        [Generation(text="fizz")],
-        [Generation(text="foo")],
-        [Generation(text="fizz")],
-    ]
-    expected_output = LLMResult(
-        generations=expected_generations,
-        llm_output=None,
-    )
-    assert output == expected_output
+    _ = llm.generate(["foo", "bar", "foo"])
+    cache_output = langchain.llm_cache.lookup("foo", llm_string)
+    assert cache_output == [Generation(text="fizz")]
+
+    langchain.llm_cache.clear()
+    assert langchain.llm_cache.lookup("bar", llm_string) is None

tests/integration_tests/document_loaders/test_arxiv.py

@@ -0,0 +1,55 @@
+from typing import List
+
+from langchain.document_loaders.arxiv import ArxivLoader
+from langchain.schema import Document
+
+
+def assert_docs(docs: List[Document]) -> None:
+    for doc in docs:
+        assert doc.page_content
+        assert doc.metadata
+        assert set(doc.metadata) == {"Published", "Title", "Authors", "Summary"}
+
+
+def test_load_success() -> None:
+    """Test that returns one document"""
+    loader = ArxivLoader(query="1605.08386", load_max_docs=2)
+
+    docs = loader.load()
+    assert len(docs) == 1
+    print(docs[0].metadata)
+    print(docs[0].page_content)
+    assert_docs(docs)
+
+
+def test_load_returns_no_result() -> None:
+    """Test that returns no docs"""
+    loader = ArxivLoader(query="1605.08386WWW", load_max_docs=2)
+    docs = loader.load()
+
+    assert len(docs) == 0
+
+
+def test_load_returns_limited_docs() -> None:
+    """Test that returns several docs"""
+    expected_docs = 2
+    loader = ArxivLoader(query="ChatGPT", load_max_docs=expected_docs)
+    docs = loader.load()
+
+    assert len(docs) == expected_docs
+    assert_docs(docs)
+
+
+def test_load_returns_full_set_of_metadata() -> None:
+    """Test that returns several docs"""
+    loader = ArxivLoader(query="ChatGPT", load_max_docs=1, load_all_available_meta=True)
+    docs = loader.load()
+    assert len(docs) == 1
+    for doc in docs:
+        assert doc.page_content
+        assert doc.metadata
+        assert set(doc.metadata).issuperset(
+            {"Published", "Title", "Authors", "Summary"}
+        )
+        print(doc.metadata)
+        assert len(set(doc.metadata)) > 4

tests/integration_tests/document_loaders/test_bshtml.py

@@ -9,15 +9,17 @@ from langchain.document_loaders.html_bs import BSHTMLLoader
 def test_bs_html_loader() -> None:
     """Test unstructured loader."""
     file_path = Path(__file__).parent.parent / "examples/example.html"
-    loader = BSHTMLLoader(str(file_path))
+    loader = BSHTMLLoader(str(file_path), get_text_separator="|")
     docs = loader.load()
 
     assert len(docs) == 1
 
     metadata = docs[0].metadata
+    content = docs[0].page_content
 
     assert metadata["title"] == "Chew dad's slippers"
     assert metadata["source"] == str(file_path)
+    assert content[:2] == "\n|"
 
 
 @pytest.mark.skipif(

tests/integration_tests/llms/test_pipelineai.py

@@ -0,0 +1,10 @@
+"""Test Pipeline Cloud API wrapper."""
+
+from langchain.llms.pipelineai import PipelineAI
+
+
+def test_pipelineai_call() -> None:
+    """Test valid call to Pipeline Cloud."""
+    llm = PipelineAI()
+    output = llm("Say foo:")
+    assert isinstance(output, str)