WIP logging to disk

use json.dump

move test to integration tests (since it requires huggingface_hub)

README.md

@@ -30,38 +30,72 @@ Please see [here](https://langchain.readthedocs.io/en/latest/?) for full documen
 
 There are three main areas (with a forth coming soon) that LangChain is designed to help with.
 These are, in increasing order of complexity:
-1. LLM and Prompt usage
-2. Chaining LLMs with other tools in a deterministic manner
-3. Having a router LLM which uses other tools as needed
-4. (Coming Soon) Memory
+1. LLM and Prompts
+2. Chains
+3. Agents
+4. Memory
+
+Let's go through these categories and for each one identify key concepts (to clarify terminology) as well as the problems in this area LangChain helps solve.
 
 ### LLMs and Prompts
 Calling out to an LLM once is pretty easy, with most of them being behind well documented APIs.
-However, there are still some challenges going from that to an application running in production that LangChain attempts to address:
-- Easy switching costs: by exposing a standard interface for all the top LLM providers, LangChain makes it easy to switch from one provider to another, whether it be for production use cases or just for testing stuff out.
+However, there are still some challenges going from that to an application running in production that LangChain attempts to address.
+
+**Key Concepts**
+- LLM: A large language model, in particular a text-to-text model.
+- Prompt: The input to a language model. Typically this is not simply a hardcoded string but rather a combination of a template, some examples, and user input.
+- Prompt Template: An object responsible for constructing the final prompt to pass to a LLM.
+- Examples: Datapoints that can be included in the prompt in order to give the model more context what to do.
+- Few Shot Prompt Template: A subclass of the PromptTemplate class that uses examples.
+- Example Selector: A class responsible to selecting examples to use dynamically (depending on user input) in a few shot prompt.
+
+**Problems Solved**
+- Switching costs: by exposing a standard interface for all the top LLM providers, LangChain makes it easy to switch from one provider to another, whether it be for production use cases or just for testing stuff out.
 - Prompt management: managing your prompts is easy when you only have one simple one, but can get tricky when you have a bunch or when they start to get more complex. LangChain provides a standard way for storing, constructing, and referencing prompts.
 - Prompt optimization: despite the underlying models getting better and better, there is still currently a need for carefully constructing prompts. 
-- More coming soon
 
 ### Chains
-Using an LLM in isolation is fine for some simple applications, but many more complex ones require chaining LLMs - either with eachother or with other tools.
-LangChain provides several parts to help with that:
+Using an LLM in isolation is fine for some simple applications, but many more complex ones require chaining LLMs - either with eachother or with other experts.
+LangChain provides several parts to help with that.
+
+**Key Concepts**
+- Tools: APIs designed for assisting with a particular use case (search, databases, Python REPL, etc). Prompt templates, LLMs, and chains can also be considered tools.
+- Chains: A combination of multiple tools in a deterministic manner.
+
+**Problems Solved**
 - Standard interface for working with Chains
 - Easy way to construct chains of LLMs
-- Lots of integrations with other tools that you may want to use in conjunction with LLMs (search, databases, Python REPL, etc)
+- Lots of integrations with other tools that you may want to use in conjunction with LLMs 
 - End-to-end chains for common workflows (database question/answer, recursive summarization, etc)
 
-### Routing Chains
+### Agents
 Some applications will require not just a predetermined chain of calls to LLMs/other tools, but potentially an unknown chain that depends on the user input.
-In these types of chains, there is a "router" LLM chain which has access to a suite of tools.
-Depending on the user input, the router can then decide which, if any, of these tools to call.
-To help develop applications like these, LangChain provides:
-- Standard router and router chain interfaces
-- Common router LLM chains from literature
+In these types of chains, there is a “agent” which has access to a suite of tools.
+Depending on the user input, the agent can then decide which, if any, of these tools to call.
+
+**Key Concepts**
+- Tools: same as above.
+- Agent: An LLM-powered class responsible for determining which tools to use and in what order.
+
+
+**Problems Solved**
+- Standard agent interfaces
+- A selection of powerful agents to choose from
 - Common chains that can be used as tools
 
 ### Memory
-Coming soon.
+By default, Chains and Agents are stateless, meaning that they treat each incoming query independently.
+In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions,
+both at a short term but also at a long term level. The concept of "Memory" exists to do exactly that.
+
+**Key Concepts**
+- Memory: A class that can be added to an Agent or Chain to (1) pull in memory variables before calling that chain/agent, and (2) create new memories after the chain/agent finishes.
+- Memory Variables: Variables returned from a Memory class, to be passed into the chain/agent along with the user input.
+
+**Problems Solved**
+- Standard memory interfaces
+- A collection of common memory implementations to choose from
+- Common chains/agents that use memory (e.g. chatbots)
 
 ## 🤖 Developer Guide
 

docs/examples/agents.rst

@@ -0,0 +1,11 @@
+Agents
+======
+
+The examples here are all end-to-end agents for specific applications.
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+   :caption: Agents
+
+   agents/*

docs/examples/agents/custom_agent.ipynb

@@ -0,0 +1,232 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "ba5f8741",
+   "metadata": {},
+   "source": [
+    "# Custom Agent\n",
+    "\n",
+    "This notebook goes through how to create your own custom agent.\n",
+    "\n",
+    "An agent consists of three parts:\n",
+    "    \n",
+    "    - Tools: The tools the agent has available to use.\n",
+    "    - LLMChain: The LLMChain that produces the text that is parsed in a certain way to determine which action to take.\n",
+    "    - The agent class itself: this parses the output of the LLMChain to determin which action to take.\n",
+    "        \n",
+    "        \n",
+    "In this notebook we walk through two types of custom agents. The first type shows how to create a custom LLMChain, but still use an existing agent class to parse the output. The second shows how to create a custom agent class."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6064f080",
+   "metadata": {},
+   "source": [
+    "### Custom LLMChain\n",
+    "\n",
+    "The first way to create a custom agent is to use an existing Agent class, but use a custom LLMChain. This is the simplest way to create a custom Agent. It is highly reccomended that you work with the `ZeroShotAgent`, as at the moment that is by far the most generalizable one. \n",
+    "\n",
+    "Most of the work in creating the custom LLMChain comes down to the prompt. Because we are using an existing agent class to parse the output, it is very important that the prompt say to produce text in that format. However, besides those instructions, you can customize the prompt as you wish.\n",
+    "\n",
+    "To ensure that the prompt contains the appropriate instructions, we will utilize a helper method on that class. The helper method for the `ZeroShotAgent` takes the following arguments:\n",
+    "\n",
+    "- tools: List of tools the agent will have access to, used to format the prompt.\n",
+    "- prefix: String to put before the list of tools.\n",
+    "- suffix: String to put after the list of tools.\n",
+    "- input_variables: List of input variables the final prompt will expect.\n",
+    "\n",
+    "For this exercise, we will give our agent access to Google Search, and we will customize it in that we will have it answer as a pirate."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "9af9734e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import ZeroShotAgent, Tool\n",
+    "from langchain import OpenAI, SerpAPIChain, LLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "becda2a1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "search = SerpAPIChain()\n",
+    "tools = [\n",
+    "    Tool(\n",
+    "        name = \"Search\",\n",
+    "        func=search.run,\n",
+    "        description=\"useful for when you need to answer questions about current events\"\n",
+    "    )\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "339b1bb8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prefix = \"\"\"Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:\"\"\"\n",
+    "suffix = \"\"\"Begin! Remember to speak as a pirate when giving your final answer. Use lots of \"Args\"\n",
+    "\n",
+    "Question: {input}\"\"\"\n",
+    "\n",
+    "prompt = ZeroShotAgent.create_prompt(\n",
+    "    tools, \n",
+    "    prefix=prefix, \n",
+    "    suffix=suffix, \n",
+    "    input_variables=[\"input\"]\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "59db7b58",
+   "metadata": {},
+   "source": [
+    "In case we are curious, we can now take a look at the final prompt template to see what it looks like when its all put together."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "e21d2098",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:\n",
+      "\n",
+      "Search: useful for when you need to answer questions about current events\n",
+      "\n",
+      "Use the following format:\n",
+      "\n",
+      "Question: the input question you must answer\n",
+      "Thought: you should always think about what to do\n",
+      "Action: the action to take, should be one of [Search]\n",
+      "Action Input: the input to the action\n",
+      "Observation: the result of the action\n",
+      "... (this Thought/Action/Action Input/Observation can repeat N times)\n",
+      "Thought: I now know the final answer\n",
+      "Final Answer: the final answer to the original input question\n",
+      "\n",
+      "Begin! Remember to speak as a pirate when giving your final answer. Use lots of \"Args\"\n",
+      "\n",
+      "Question: {input}\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(prompt.template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "9b1cc2a2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "e4f5092f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "653b1617",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "How many people live in canada?\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I should look this up\n",
+      "Action: Search\n",
+      "Action Input: How many people live in canada\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mThe current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data. · Canada 2020 ...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
+      "Final Answer: Arrr, there be 38,533,678 people in Canada\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'Arrr, there be 38,533,678 people in Canada'"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.run(\"How many people live in canada?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "90171b2b",
+   "metadata": {},
+   "source": [
+    "### Custom Agent Class\n",
+    "\n",
+    "Coming soon."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "adefb4c2",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/agents/mrkl.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# MRKL\n",
     "\n",
-    "This notebook showcases using the MRKL chain to route between tasks"
+    "This notebook showcases using an agent to replicate the MRKL chain."
    ]
   },
   {
@@ -26,8 +26,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain import LLMMathChain, OpenAI, SerpAPIChain, MRKLChain, SQLDatabase, SQLDatabaseChain\n",
-    "from langchain.routing_chains.mrkl.base import ChainConfig"
+    "from langchain import LLMMathChain, OpenAI, SerpAPIChain, SQLDatabase, SQLDatabaseChain\n",
+    "from langchain.agents import initialize_agent, Tool"
    ]
   },
   {
@@ -42,22 +42,21 @@
     "llm_math_chain = LLMMathChain(llm=llm, verbose=True)\n",
     "db = SQLDatabase.from_uri(\"sqlite:///../../../notebooks/Chinook.db\")\n",
     "db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)\n",
-    "chains = [\n",
-    "    ChainConfig(\n",
-    "        action_name = \"Search\",\n",
-    "        action=search.run,\n",
-    "        action_description=\"useful for when you need to answer questions about current events\"\n",
+    "tools = [\n",
+    "    Tool(\n",
+    "        name = \"Search\",\n",
+    "        func=search.run,\n",
+    "        description=\"useful for when you need to answer questions about current events\"\n",
     "    ),\n",
-    "    ChainConfig(\n",
-    "        action_name=\"Calculator\",\n",
-    "        action=llm_math_chain.run,\n",
-    "        action_description=\"useful for when you need to answer questions about math\"\n",
+    "    Tool(\n",
+    "        name=\"Calculator\",\n",
+    "        func=llm_math_chain.run,\n",
+    "        description=\"useful for when you need to answer questions about math\"\n",
     "    ),\n",
-    "    \n",
-    "    ChainConfig(\n",
-    "        action_name=\"FooBar DB\",\n",
-    "        action=db_chain.run,\n",
-    "        action_description=\"useful for when you need to answer questions about FooBar. Input should be in the form of a question\"\n",
+    "    Tool(\n",
+    "        name=\"FooBar DB\",\n",
+    "        func=db_chain.run,\n",
+    "        description=\"useful for when you need to answer questions about FooBar. Input should be in the form of a question\"\n",
     "    )\n",
     "]"
    ]
@@ -69,7 +68,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "mrkl = MRKLChain.from_chains(llm, chains, verbose=True)"
+    "mrkl = initialize_agent(tools, llm, agent=\"zero-shot-react-description\", verbose=True)"
    ]
   },
   {
@@ -82,9 +81,6 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new chain...\u001b[0m\n",
       "What is the age of Olivia Wilde's boyfriend raised to the 0.23 power?\n",
       "Thought:\u001b[32;1m\u001b[1;3m I need to find the age of Olivia Wilde's boyfriend\n",
       "Action: Search\n",
@@ -112,8 +108,7 @@
       "Observation: \u001b[33;1m\u001b[1;3mAnswer: 2.1520202182226886\n",
       "\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: 2.1520202182226886\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
+      "Final Answer: 2.1520202182226886\u001b[0m"
      ]
     },
     {
@@ -141,9 +136,6 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new chain...\u001b[0m\n",
       "Who recently released an album called 'The Storm Before the Calm' and are they in the FooBar database? If so, what albums of theirs are in the FooBar database?\n",
       "Thought:\u001b[32;1m\u001b[1;3m I need to find an album called 'The Storm Before the Calm'\n",
       "Action: Search\n",
@@ -167,15 +159,14 @@
       "\n",
       "\u001b[1m> Entering new chain...\u001b[0m\n",
       "What albums by Alanis Morissette are in the FooBar database?\n",
-      "SQLQuery:\u001b[32;1m\u001b[1;3m SELECT Album.Title FROM Album JOIN Artist ON Album.ArtistId = Artist.ArtistId WHERE Artist.Name = \"Alanis Morissette\"\u001b[0m\n",
+      "SQLQuery:\u001b[32;1m\u001b[1;3m SELECT Album.Title FROM Album JOIN Artist ON Album.ArtistId = Artist.ArtistId WHERE Artist.Name = 'Alanis Morissette'\u001b[0m\n",
       "SQLResult: \u001b[33;1m\u001b[1;3m[('Jagged Little Pill',)]\u001b[0m\n",
       "Answer:\u001b[32;1m\u001b[1;3m Jagged Little Pill\u001b[0m\n",
       "\u001b[1m> Finished chain.\u001b[0m\n",
       "\n",
       "Observation: \u001b[38;5;200m\u001b[1;3m Jagged Little Pill\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: The album is by Alanis Morissette and the albums in the FooBar database by her are Jagged Little Pill\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
+      "Final Answer: The album is by Alanis Morissette and the albums in the FooBar database by her are Jagged Little Pill\u001b[0m"
      ]
     },
     {

docs/examples/agents/react.ipynb

@@ -0,0 +1,89 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "82140df0",
+   "metadata": {},
+   "source": [
+    "# ReAct\n",
+    "\n",
+    "This notebook showcases using an agent to implement the ReAct logic."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "4e272b47",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import OpenAI, Wikipedia\n",
+    "from langchain.agents import initialize_agent, Tool\n",
+    "from langchain.agents.react.base import DocstoreExplorer\n",
+    "docstore=DocstoreExplorer(Wikipedia())\n",
+    "tools = [\n",
+    "    Tool(\n",
+    "        name=\"Search\",\n",
+    "        func=docstore.search\n",
+    "    ),\n",
+    "    Tool(\n",
+    "        name=\"Lookup\",\n",
+    "        func=docstore.lookup\n",
+    "    )\n",
+    "]\n",
+    "\n",
+    "llm = OpenAI(temperature=0)\n",
+    "react = initialize_agent(tools, llm, agent=\"react-docstore\", verbose=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8078c8f1",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?\n",
+      "Thought 1:"
+     ]
+    }
+   ],
+   "source": [
+    "question = \"Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?\"\n",
+    "react.run(question)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4ff64e81",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/agents/self_ask_with_search.ipynb

@@ -20,17 +20,13 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new chain...\u001b[0m\n",
       "What is the hometown of the reigning men's U.S. Open champion?\n",
       "Are follow up questions needed here:\u001b[32;1m\u001b[1;3m Yes.\n",
       "Follow up: Who is the reigning men's U.S. Open champion?\u001b[0m\n",
       "Intermediate answer: \u001b[36;1m\u001b[1;3mCarlos Alcaraz\u001b[0m\n",
       "\u001b[32;1m\u001b[1;3mFollow up: Where is Carlos Alcaraz from?\u001b[0m\n",
       "Intermediate answer: \u001b[36;1m\u001b[1;3mEl Palmar, Spain\u001b[0m\n",
-      "\u001b[32;1m\u001b[1;3mSo the final answer is: El Palmar, Spain\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
+      "\u001b[32;1m\u001b[1;3mSo the final answer is: El Palmar, Spain\u001b[0m"
      ]
     },
     {
@@ -45,12 +41,19 @@
     }
    ],
    "source": [
-    "from langchain import SelfAskWithSearchChain, OpenAI, SerpAPIChain\n",
+    "from langchain import OpenAI, SerpAPIChain\n",
+    "from langchain.agents import initialize_agent, Tool\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "search = SerpAPIChain()\n",
+    "tools = [\n",
+    "    Tool(\n",
+    "        name=\"Intermediate Answer\",\n",
+    "        func=search.run\n",
+    "    )\n",
+    "]\n",
     "\n",
-    "self_ask_with_search = SelfAskWithSearchChain(llm=llm, search_chain=search, verbose=True)\n",
+    "self_ask_with_search = initialize_agent(tools, llm, agent=\"self-ask-with-search\", verbose=True)\n",
     "\n",
     "self_ask_with_search.run(\"What is the hometown of the reigning men's U.S. Open champion?\")"
    ]

docs/examples/chains.rst

@@ -0,0 +1,11 @@
+Chains
+======
+
+The examples here are all end-to-end chains for specific applications.
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+   :caption: Chains
+
+   chains/*

docs/examples/chains/llm_chain.ipynb

@@ -5,9 +5,9 @@
    "id": "d8a5c5d4",
    "metadata": {},
    "source": [
-    "# Simple Example\n",
+    "# LLM Chain\n",
     "\n",
-    "This notebook showcases a simple chain."
+    "This notebook showcases a simple LLM chain."
    ]
   },
   {
@@ -81,7 +81,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.7"
+   "version": "3.7.6"
   }
  },
  "nbformat": 4,

docs/examples/chains/llm_math.ipynb

@@ -83,7 +83,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.6"
+   "version": "3.10.4"
   }
  },
  "nbformat": 4,

docs/examples/demos.rst

@@ -1,25 +0,0 @@
-Demos
-=====
-
-The examples here are all end-to-end chains of specific applications.
-They are separated into normal chains and then routing chains.
-
-.. toctree::
-   :maxdepth: 1
-   :glob:
-   :caption: Chains
-
-   demos/llm_math.ipynb
-   demos/map_reduce.ipynb
-   demos/simple_prompts.ipynb
-   demos/sqlite.ipynb
-   demos/vector_db_qa.ipynb
-
-.. toctree::
-   :maxdepth: 1
-   :glob:
-   :caption: Routing Chains
-
-   demos/mrkl.ipynb
-   demos/react.ipynb
-   demos/self_ask_with_search.ipynb

docs/examples/demos/custom_routing_chains.ipynb

@@ -1,183 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "0af33207",
-   "metadata": {},
-   "source": [
-    "# Custom Routing Chains\n",
-    "\n",
-    "This covers how to implement a custom routing chain. That problem really reduces to how to implement a custom router. This also acts as a design doc of sorts for routers."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "16773dc8",
-   "metadata": {},
-   "source": [
-    "## Terminology\n",
-    "\n",
-    "Before going through any code, let's align on some terminology.\n",
-    "- Tool: A function that performs a specific duty. This can be things like: Google Search, Database lookup, Python REPL. The interface for a tool is currently a function that is expected to have a string as an input, with a string as an output.\n",
-    "- Tool Input: The input string to a tool.\n",
-    "- Observation: The output from calling a tool on a particular input.\n",
-    "- Router: The object responsible for deciding which tools to call and when. Exposes a `route` method, which takes in a string and returns a Router Output.\n",
-    "- Router Output: The object returned from calling `Router.route` on a string. Consists of:\n",
-    "    - The tool to use\n",
-    "    - The input to that tool\n",
-    "    - A log of the router's thinking.\n",
-    "- Routing Chain: A chain which is made up of a router and suite of tools. When passed a string, the Routing Chain will iterative call tools as needed until it arrives at a Final Answer.\n",
-    "- Final Answer: The final output of a Routing Chain."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6eaca15e",
-   "metadata": {},
-   "source": [
-    "## Router\n",
-    "A central piece of this chain is the router. The router is responsible for taking user input and deciding which tools, if any, to use. Although it doesn't necessarily have to be backed by a language model (LLM), for pretty much all current use cases it is. LLMs make great routers because they are really good at understanding human intent, which makes them perfect for choosing which tools to use (and for interpreting the output of those tools).\n",
-    "\n",
-    "Below is the interface we expect routers to expose, along with the RouterOutput definition.\n",
-    "\n",
-    "```python\n",
-    "\n",
-    "class RouterOutput(NamedTuple):\n",
-    "    \"\"\"Output of a router.\"\"\"\n",
-    "\n",
-    "    tool: str\n",
-    "    tool_input: str\n",
-    "    log: str\n",
-    "        \n",
-    "\n",
-    "class Router(ABC):\n",
-    "    \"\"\"Chain responsible for deciding the action to take.\"\"\"\n",
-    "\n",
-    "    @abstractmethod\n",
-    "    def route(self, text: str) -> RouterOutput:\n",
-    "        \"\"\"Given input, decided how to route it.\n",
-    "\n",
-    "        Args:\n",
-    "            text: input string\n",
-    "\n",
-    "        Returns:\n",
-    "            RouterOutput specifying what tool to use.\n",
-    "        \"\"\"\n",
-    "\n",
-    "    @property\n",
-    "    @abstractmethod\n",
-    "    def observation_prefix(self) -> str:\n",
-    "        \"\"\"Prefix to append the observation with before calling the router again.\"\"\"\n",
-    "\n",
-    "    @property\n",
-    "    @abstractmethod\n",
-    "    def router_prefix(self) -> str:\n",
-    "        \"\"\"Prefix to prepend the router call with.\"\"\"\n",
-    "\n",
-    "    @property\n",
-    "    def finish_tool_name(self) -> str:\n",
-    "        \"\"\"Name of the tool to use to finish the chain.\"\"\"\n",
-    "        return \"Final Answer\"\n",
-    "\n",
-    "    @property\n",
-    "    def starter_string(self) -> str:\n",
-    "        \"\"\"Put this string after user input but before first router call.\"\"\"\n",
-    "        return \"\\n\"\n",
-    "```"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "471389be",
-   "metadata": {},
-   "source": [
-    "In order to understand why the router interface is what it is, let's take a look at how it is used in the RoutingChain class:\n",
-    "\n",
-    "```python\n",
-    "def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:\n",
-    "    # Construct a mapping of tool name to tool for easy lookup\n",
-    "    name_to_tool_map = {tc.tool_name: tc.tool for tc in self.tool_configs}\n",
-    "    # Construct the initial string to pass into the router. This is made up\n",
-    "    # of the user input, the special starter string, and then the router prefix.\n",
-    "    # The starter string is a special string that may be used by a router to\n",
-    "    # immediately follow the user input. The router prefix is a string that\n",
-    "    # prompts the router to start routing.\n",
-    "    starter_string = (\n",
-    "        inputs[self.input_key]\n",
-    "        + self.router.starter_string\n",
-    "        + self.router.router_prefix\n",
-    "    )\n",
-    "    # We use the ChainedInput class to iteratively add to the input over time.\n",
-    "    chained_input = ChainedInput(starter_string, verbose=self.verbose)\n",
-    "    # We construct a mapping from each tool to a color, used for logging.\n",
-    "    color_mapping = get_color_mapping(\n",
-    "        [c.tool_name for c in self.tool_configs], excluded_colors=[\"green\"]\n",
-    "    )\n",
-    "    # We now enter the router loop (until it returns something).\n",
-    "    while True:\n",
-    "        # Call the router to see what to do.\n",
-    "        output = self.router.route(chained_input.input)\n",
-    "        # Add the log to the Chained Input.\n",
-    "        chained_input.add(output.log, color=\"green\")\n",
-    "        # If the tool chosen is the finishing tool, then we end and return.\n",
-    "        if output.tool == self.router.finish_tool_name:\n",
-    "            return {self.output_key: output.tool_input}\n",
-    "        # Otherwise we lookup the tool\n",
-    "        chain = name_to_tool_map[output.tool]\n",
-    "        # We then call the tool on the tool input to get an observation\n",
-    "        observation = chain(output.tool_input)\n",
-    "        # We then log the observation\n",
-    "        chained_input.add(f\"\\n{self.router.observation_prefix}\")\n",
-    "        chained_input.add(observation, color=color_mapping[output.tool])\n",
-    "        # We then add the router prefix into the prompt to get the router to start\n",
-    "        # thinking, and start the loop all over.\n",
-    "        chained_input.add(f\"\\n{self.router.router_prefix}\")\n",
-    "\n",
-    "```"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "d9f6ca91",
-   "metadata": {},
-   "source": [
-    "Once we have the custom router written, it is pretty easy to construct the routing chain:\n",
-    "\n",
-    "```python\n",
-    "tools: List[ToolConfig] = ...\n",
-    "router = CustomRouter(....)\n",
-    "routing_chain = RoutingChain(tools=tools, router=router, verbose=True)\n",
-    "```"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5d0c7662",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.7.6"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/examples/demos/react.ipynb

@@ -1,100 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "82140df0",
-   "metadata": {},
-   "source": [
-    "# ReAct\n",
-    "\n",
-    "This notebook showcases the implementation of the ReAct chain logic."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "4e272b47",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain import OpenAI, ReActChain, Wikipedia\n",
-    "\n",
-    "llm = OpenAI(temperature=0)\n",
-    "react = ReActChain(llm=llm, docstore=Wikipedia(), verbose=True)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "8078c8f1",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new chain...\u001b[0m\n",
-      "Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?\n",
-      "Thought 1:\u001b[32;1m\u001b[1;3m I need to search David Chanoff and find the U.S. Navy admiral he collaborated\n",
-      "with.\n",
-      "Action 1: Search[David Chanoff]\u001b[0m\n",
-      "Observation 1: \u001b[36;1m\u001b[1;3mDavid Chanoff is a noted author of non-fiction work. His work has typically involved collaborations with the principal protagonist of the work concerned. His collaborators have included; Augustus A. White, Joycelyn Elders, Đoàn Văn Toại, William J. Crowe, Ariel Sharon, Kenneth Good and Felix Zandman. He has also written about a wide range of subjects including literary history, education and foreign for The Washington Post, The New Republic and The New York Times Magazine. He has published more than twelve books.\u001b[0m\n",
-      "Thought 2:\u001b[32;1m\u001b[1;3m The U.S. Navy admiral David Chanoff collaborated with is William J. Crowe. I\n",
-      "need to search him next.\n",
-      "Action 2: Search[William J. Crowe]\u001b[0m\n",
-      "Observation 2: \u001b[36;1m\u001b[1;3mWilliam James Crowe Jr. (January 2, 1925 – October 18, 2007) was a United States Navy admiral and diplomat who served as the 11th chairman of the Joint Chiefs of Staff under Presidents Ronald Reagan and George H. W. Bush, and as the ambassador to the United Kingdom and Chair of the Intelligence Oversight Board under President Bill Clinton.\u001b[0m\n",
-      "Thought 3:\u001b[32;1m\u001b[1;3m William J. Crowe served as the ambassador to the United Kingdom under\n",
-      "President Bill Clinton. So the answer is Bill Clinton.\n",
-      "Action 3: Finish[Bill Clinton]\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'Bill Clinton'"
-      ]
-     },
-     "execution_count": 2,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "question = \"Author David Chanoff has collaborated with a U.S. Navy admiral who served as the ambassador to the United Kingdom under which President?\"\n",
-    "react.run(question)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "4ff64e81",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.7.6"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/examples/memory.rst

@@ -0,0 +1,11 @@
+Memory
+======
+
+The examples here are all related to working with the concept of Memory in LangChain.
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+   :caption: Memory
+
+   memory/*

docs/examples/memory/adding_memory.ipynb

@@ -0,0 +1,175 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "00695447",
+   "metadata": {},
+   "source": [
+    "# Adding Memory To an LLMChain\n",
+    "\n",
+    "This notebook goes over how to use the Memory class with an LLMChain. For the purposes of this walkthrough, we will add  the `ConversationBufferMemory` class, although this can be any memory class."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "9f1aaf47",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains.conversation.memory import ConversationBufferMemory\n",
+    "from langchain import OpenAI, LLMChain, PromptTemplate"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4b066ced",
+   "metadata": {},
+   "source": [
+    "The most important step is setting up the prompt correctly. In the below prompt, we have two input keys: one for the actual input, another for the input from the Memory class. Importantly, we make sure the keys in the PromptTemplate and the ConversationBufferMemory match up (`chat_history`)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "e5501eda",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "template = \"\"\"You are a chatbot having a conversation with a human.\n",
+    "\n",
+    "{chat_history}\n",
+    "Human: {human_input}\n",
+    "Chatbot:\"\"\"\n",
+    "\n",
+    "prompt = PromptTemplate(\n",
+    "    input_variables=[\"chat_history\", \"human_input\"], \n",
+    "    template=template\n",
+    ")\n",
+    "memory = ConversationBufferMemory(memory_key=\"chat_history\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "f6566275",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm_chain = LLMChain(\n",
+    "    llm=OpenAI(), \n",
+    "    prompt=prompt, \n",
+    "    verbose=True, \n",
+    "    memory=memory,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "e2b189dc",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mYou are a chatbot having a conversation with a human.\n",
+      "\n",
+      "\n",
+      "Human: Hi there my friend\n",
+      "Chatbot:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "' Hi there!'"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm_chain.predict(human_input=\"Hi there my friend\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "a902729f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mYou are a chatbot having a conversation with a human.\n",
+      "\n",
+      "\n",
+      "Human: Hi there my friend\n",
+      "AI:  Hi there!\n",
+      "Human: Not to bad - how are you?\n",
+      "Chatbot:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "\"\\n\\nI'm doing well, thanks for asking. How about you?\""
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm_chain.predict(human_input=\"Not to bad - how are you?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ae5309bb",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/memory/agent_with_memory.ipynb

@@ -0,0 +1,325 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "fa6802ac",
+   "metadata": {},
+   "source": [
+    "# Adding Memory to an Agent\n",
+    "\n",
+    "This notebook goes over adding memory to an Agent. Before going through this notebook, please walkthrough the following notebooks, as this will build on top of both of them:\n",
+    "\n",
+    "- [Adding memory to an LLM Chain](adding_memory.ipynb)\n",
+    "- [Custom Agents](../agents/custom_agent.ipynb)\n",
+    "\n",
+    "In order to add a memory to an agent we are going to the the following steps:\n",
+    "\n",
+    "1. We are going to create an LLMChain with memory.\n",
+    "2. We are going to use that LLMChain to create a custom Agent.\n",
+    "\n",
+    "For the purposes of this exercise, we are going to create a simple custom Agent that has access to a search tool and utilizes the `ConversationBufferMemory` class."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "8db95912",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import ZeroShotAgent, Tool\n",
+    "from langchain.chains.conversation.memory import ConversationBufferMemory\n",
+    "from langchain import OpenAI, SerpAPIChain, LLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "97ad8467",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "search = SerpAPIChain()\n",
+    "tools = [\n",
+    "    Tool(\n",
+    "        name = \"Search\",\n",
+    "        func=search.run,\n",
+    "        description=\"useful for when you need to answer questions about current events\"\n",
+    "    )\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4ad2e708",
+   "metadata": {},
+   "source": [
+    "Notice the usage of the `chat_history` variable in the PromptTemplate, which matches up with the dynamic key name in the ConversationBufferMemory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "e3439cd6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prefix = \"\"\"Have a conversation with a human, answering the following questions as best you can. You have access to the following tools:\"\"\"\n",
+    "suffix = \"\"\"Begin!\"\n",
+    "\n",
+    "{chat_history}\n",
+    "Question: {input}\"\"\"\n",
+    "\n",
+    "prompt = ZeroShotAgent.create_prompt(\n",
+    "    tools, \n",
+    "    prefix=prefix, \n",
+    "    suffix=suffix, \n",
+    "    input_variables=[\"input\", \"chat_history\"]\n",
+    ")\n",
+    "memory = ConversationBufferMemory(memory_key=\"chat_history\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0021675b",
+   "metadata": {},
+   "source": [
+    "We can now construct the LLMChain, with the Memory object, and then create the agent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "c56a0e73",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt, memory=memory)\n",
+    "agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "ca4bc1fb",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "How many people live in canada?\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I should look up how many people live in canada\n",
+      "Action: Search\n",
+      "Action Input: \"How many people live in canada?\"\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mThe current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data. · Canada 2020 ...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
+      "Final Answer: The current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data.\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'The current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data.'"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.run(\"How many people live in canada?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "45627664",
+   "metadata": {},
+   "source": [
+    "To test the memory of this agent, we can ask a followup question that relies on information in the previous exchange to be answered correctly."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "eecc0462",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "what is their national anthem called?\n",
+      "Thought:\u001b[32;1m\u001b[1;3m\n",
+      "AI:  I should look up the name of Canada's national anthem\n",
+      "Action: Search\n",
+      "Action Input: \"What is the name of Canada's national anthem?\"\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mAfter 100 years of tradition, O Canada was proclaimed Canada's national anthem in 1980. The music for O Canada was composed in 1880 by Calixa ...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m\n",
+      "AI:  I now know the final answer\n",
+      "Final Answer: After 100 years of tradition, O Canada was proclaimed Canada's national anthem in 1980. The music for O Canada was composed in 1880 by Calixa Lavallée.\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "\"After 100 years of tradition, O Canada was proclaimed Canada's national anthem in 1980. The music for O Canada was composed in 1880 by Calixa Lavallée.\""
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.run(\"what is their national anthem called?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cc3d0aa4",
+   "metadata": {},
+   "source": [
+    "We can see that the agent remembered that the previous question was about Canada, and properly asked Google Search what the name of Canada's national anthem was.\n",
+    "\n",
+    "For fun, let's compare this to an agent that does NOT have memory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "3359d043",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prefix = \"\"\"Have a conversation with a human, answering the following questions as best you can. You have access to the following tools:\"\"\"\n",
+    "suffix = \"\"\"Begin!\"\n",
+    "\n",
+    "Question: {input}\"\"\"\n",
+    "\n",
+    "prompt = ZeroShotAgent.create_prompt(\n",
+    "    tools, \n",
+    "    prefix=prefix, \n",
+    "    suffix=suffix, \n",
+    "    input_variables=[\"input\"]\n",
+    ")\n",
+    "llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)\n",
+    "agent_without_memory = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "970d23df",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "How many people live in canada?\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I should look up how many people live in canada\n",
+      "Action: Search\n",
+      "Action Input: \"How many people live in canada?\"\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mThe current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data. · Canada 2020 ...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
+      "Final Answer: The current population of Canada is 38,533,678\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'The current population of Canada is 38,533,678'"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent_without_memory.run(\"How many people live in canada?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "d9ea82f0",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "what is their national anthem called?\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I should probably look this up\n",
+      "Action: Search\n",
+      "Action Input: \"What is the national anthem of [country]\"\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mMost nation states have an anthem, defined as \"a song, as of praise, devotion, or patriotism\"; most anthems are either marches or hymns in style.\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
+      "Final Answer: The national anthem is called \"the national anthem.\"\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'The national anthem is called \"the national anthem.\"'"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent_without_memory.run(\"what is their national anthem called?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5b1f9223",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/memory/custom_memory.ipynb

@@ -0,0 +1,295 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "94e33ebe",
+   "metadata": {},
+   "source": [
+    "# Custom Memory\n",
+    "Although there are a few predefined types of memory in LangChain, it is highly possible you will want to add your own type of memory that is optimal for your application. This notebook covers how to do that."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bdfd0305",
+   "metadata": {},
+   "source": [
+    "For this notebook, we will add a custom memory type to `ConversationChain`. In order to add a custom memory class, we need to import the base memory class and subclass it."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "6d787ef2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import OpenAI, ConversationChain\n",
+    "from langchain.chains.base import Memory\n",
+    "from pydantic import BaseModel\n",
+    "from typing import List, Dict, Any"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9489e5e1",
+   "metadata": {},
+   "source": [
+    "In this example, we will write a custom memory class that uses spacy to extract entities and save information about them in a simple hash table. Then, during the conversation, we will look at the input text, extract any entities, and put any information about them into the context.\n",
+    "\n",
+    "* Please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations.\n",
+    "\n",
+    "For this, we will need spacy."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "12bbed4e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# !pip install spacy\n",
+    "# !python -m spacy download en_core_web_lg"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "ff065f58",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import spacy\n",
+    "nlp = spacy.load('en_core_web_lg')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "1d45d429",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class SpacyEntityMemory(Memory, BaseModel):\n",
+    "    \"\"\"Memory class for storing information about entities.\"\"\"\n",
+    "\n",
+    "    # Define dictionary to store information about entities.\n",
+    "    entities: dict = {}\n",
+    "    # Define key to pass information about entities into prompt.\n",
+    "    memory_key: str = \"entities\"\n",
+    "\n",
+    "    @property\n",
+    "    def memory_variables(self) -> List[str]:\n",
+    "        \"\"\"Define the variables we are providing to the prompt.\"\"\"\n",
+    "        return [self.memory_key]\n",
+    "\n",
+    "    def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]:\n",
+    "        \"\"\"Load the memory variables, in this case the entity key.\"\"\"\n",
+    "        # Get the input text and run through spacy\n",
+    "        doc = nlp(inputs[list(inputs.keys())[0]])\n",
+    "        # Extract known information about entities, if they exist.\n",
+    "        entities = [self.entities[str(ent)] for ent in doc.ents if str(ent) in self.entities]\n",
+    "        # Return combined information about entities to put into context.\n",
+    "        return {self.memory_key: \"\\n\".join(entities)}\n",
+    "\n",
+    "    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:\n",
+    "        \"\"\"Save context from this conversation to buffer.\"\"\"\n",
+    "        # Get the input text and run through spacy\n",
+    "        text = inputs[list(inputs.keys())[0]]\n",
+    "        doc = nlp(text)\n",
+    "        # For each entity that was mentioned, save this information to the dictionary.\n",
+    "        for ent in doc.ents:\n",
+    "            ent_str = str(ent)\n",
+    "            if ent_str in self.entities:\n",
+    "                self.entities[ent_str] += f\"\\n{text}\"\n",
+    "            else:\n",
+    "                self.entities[ent_str] = text"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "429ba264",
+   "metadata": {},
+   "source": [
+    "We now define a prompt that takes in information about entities as well as user input"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "c05159b6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts.prompt import PromptTemplate\n",
+    "\n",
+    "template = \"\"\"The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.\n",
+    "\n",
+    "Relevant entity information:\n",
+    "{entities}\n",
+    "\n",
+    "Conversation:\n",
+    "Human: {input}\n",
+    "AI:\"\"\"\n",
+    "prompt = PromptTemplate(\n",
+    "    input_variables=[\"entities\", \"input\"], template=template\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "db611041",
+   "metadata": {},
+   "source": [
+    "And now we put it all together!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "f08dc8ed",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(temperature=0)\n",
+    "conversation = ConversationChain(llm=llm, prompt=prompt, verbose=True, memory=SpacyEntityMemory())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "92a5f685",
+   "metadata": {},
+   "source": [
+    "In the first example, with no prior knowledge about Harrison, the \"Relevant entity information\" section is empty."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "5b96e836",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.\n",
+      "\n",
+      "Relevant entity information:\n",
+      "\n",
+      "\n",
+      "Conversation:\n",
+      "Human: Harrison likes machine learning\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "\"\\n\\nThat's really interesting! I'm sure he has a lot of fun with it.\""
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation.predict(input=\"Harrison likes machine learning\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b1faa743",
+   "metadata": {},
+   "source": [
+    "Now in the second example, we can see that it pulls in information about Harrison."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "4bca7070",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.\n",
+      "\n",
+      "Relevant entity information:\n",
+      "Harrison likes machine learning\n",
+      "\n",
+      "Conversation:\n",
+      "Human: What do you think Harrison's favorite subject in college was?\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "\" Harrison's favorite subject in college was machine learning.\""
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation.predict(input=\"What do you think Harrison's favorite subject in college was?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "58b856e3",
+   "metadata": {},
+   "source": [
+    "Again, please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a1994600",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/prompts/custom_example_selector.ipynb

@@ -0,0 +1,176 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "f897c784",
+   "metadata": {},
+   "source": [
+    "# Custom ExampleSelector\n",
+    "\n",
+    "This notebook goes over how to implement a custom ExampleSelector. ExampleSelectors are used to select examples to use in few shot prompts.\n",
+    "\n",
+    "An ExampleSelector must implement two methods:\n",
+    "\n",
+    "1. An `add_example` method which takes in an example and adds it into the ExampleSelector\n",
+    "2. A `select_examples` method which takes in input variables (which are meant to be user input) and returns a list of examples to use in the few shot prompt.\n",
+    "\n",
+    "\n",
+    "Let's implement a custom ExampleSelector that just selects two examples at random."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "1a945da1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts.example_selector.base import BaseExampleSelector\n",
+    "from typing import Dict, List\n",
+    "import numpy as np"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "62cf0ad7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class CustomExampleSelector(BaseExampleSelector):\n",
+    "    \n",
+    "    def __init__(self, examples: List[Dict[str, str]]):\n",
+    "        self.examples = examples\n",
+    "    \n",
+    "    def add_example(self, example: Dict[str, str]) -> None:\n",
+    "        \"\"\"Add new example to store for a key.\"\"\"\n",
+    "        self.examples.append(example)\n",
+    "\n",
+    "    def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:\n",
+    "        \"\"\"Select which examples to use based on the inputs.\"\"\"\n",
+    "        return np.random.choice(self.examples, size=2, replace=False)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "242d3213",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "examples = [{\"foo\": \"1\"}, {\"foo\": \"2\"}, {\"foo\": \"3\"}]\n",
+    "example_selector = CustomExampleSelector(examples)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2a038065",
+   "metadata": {},
+   "source": [
+    "Let's now try it out! We can select some examples and try adding examples."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "74fbbef5",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([{'foo': '2'}, {'foo': '3'}], dtype=object)"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "example_selector.select_examples({\"foo\": \"foo\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "9bbb5421",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "example_selector.add_example({\"foo\": \"4\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "c0eb9f22",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'foo': '1'}, {'foo': '2'}, {'foo': '3'}, {'foo': '4'}]"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "example_selector.examples"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "cc39b1e3",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "array([{'foo': '1'}, {'foo': '4'}], dtype=object)"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "example_selector.select_examples({\"foo\": \"foo\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1739dd96",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/prompts/custom_llm.ipynb

@@ -0,0 +1,153 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "9e9b7651",
+   "metadata": {},
+   "source": [
+    "# Custom LLM\n",
+    "\n",
+    "This notebook goes over how to create a custom LLM wrapper, in case you want to use your own LLM or a different wrapper than one that is supported in LangChain.\n",
+    "\n",
+    "There is only one required thing that a custom LLM needs to implement:\n",
+    "\n",
+    "1. A `__call__` method that takes in a string, some optional stop words, and returns a string\n",
+    "\n",
+    "There is a second optional thing it can implement:\n",
+    "\n",
+    "1. An `_identifying_params` property that is used to help with printing of this class. Should return a dictionary.\n",
+    "\n",
+    "Let's implement a very simple custom LLM that just returns the first N characters of the input."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "a65696a0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms.base import LLM\n",
+    "from typing import Optional, List, Mapping, Any"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "d5ceff02",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class CustomLLM(LLM):\n",
+    "    \n",
+    "    def __init__(self, n: int):\n",
+    "        self.n = n\n",
+    "    \n",
+    "    def __call__(self, prompt: str, stop: Optional[List[str]] = None) -> str:\n",
+    "        if stop is not None:\n",
+    "            raise ValueError(\"stop kwargs are not permitted.\")\n",
+    "        return prompt[:self.n]\n",
+    "    \n",
+    "    @property\n",
+    "    def _identifying_params(self) -> Mapping[str, Any]:\n",
+    "        \"\"\"Get the identifying parameters.\"\"\"\n",
+    "        return {\"n\": self.n}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "714dede0",
+   "metadata": {},
+   "source": [
+    "We can now use this as an any other LLM."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "10e5ece6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = CustomLLM(n=10)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "8cd49199",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'This is a '"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm(\"This is a foobar thing\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bbfebea1",
+   "metadata": {},
+   "source": [
+    "We can also print the LLM and see its custom print."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "9c33fa19",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\u001b[1mCustomLLM\u001b[0m\n",
+      "Params: {'n': 10}\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(llm)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6dac3f47",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/prompts/custom_prompt_template.ipynb

@@ -0,0 +1,116 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "a37d9694",
+   "metadata": {},
+   "source": [
+    "# Custom Prompt Template\n",
+    "\n",
+    "This notebook goes over how to create a custom prompt template, in case you want to create your own methodology for creating prompts.\n",
+    "\n",
+    "The only two requirements for all prompt templates are:\n",
+    "\n",
+    "1. They have a `input_variables` attribute that exposes what input variables this prompt template expects.\n",
+    "2. They expose a `format` method which takes in keyword arguments corresponding to the expected `input_variables` and returns the formatted prompt.\n",
+    "\n",
+    "Let's imagine that we want to create a prompt template that takes in input variables and formats them into the template AFTER capitalizing them. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "26f796e5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts import BasePromptTemplate\n",
+    "from pydantic import BaseModel"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "27919e96",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class CustomPromptTemplate(BasePromptTemplate, BaseModel):\n",
+    "    template: str\n",
+    "        \n",
+    "    def format(self, **kwargs) -> str:\n",
+    "        capitalized_kwargs = {k: v.upper() for k, v in kwargs.items()}\n",
+    "        return self.template.format(**capitalized_kwargs)\n",
+    "        "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "76d1d84d",
+   "metadata": {},
+   "source": [
+    "We can now see that when we use this, the input variables get formatted."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "eed1ff28",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt = CustomPromptTemplate(input_variables=[\"foo\"], template=\"Capitalized: {foo}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "94892a3c",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Capitalized: LOWERCASE'"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "prompt.format(foo=\"lowercase\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d3d9a7c7",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/prompts/prompt_management.ipynb

@@ -310,7 +310,7 @@
     "    example_prompt=example_prompt, \n",
     "    # This is the maximum length that the formatted examples should be.\n",
     "    # Length is measured by the get_text_length function below.\n",
-    "    max_length=18,\n",
+    "    max_length=25,\n",
     "    # This is the function used to get the length of a string, which is used\n",
     "    # to determine which examples to include. It is commented out because\n",
     "    # it is provided as a default value if none is specified.\n",
@@ -378,17 +378,59 @@
       "Input: happy\n",
       "Output: sad\n",
       "\n",
-      "Input: big and huge and massive and large and gigantic and tall and bigger than everything else\n",
+      "Input: big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else\n",
       "Output:\n"
      ]
     }
    ],
    "source": [
     "# An example with long input, so it selects only one example.\n",
-    "long_string = \"big and huge and massive and large and gigantic and tall and bigger than everything else\"\n",
+    "long_string = \"big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else\"\n",
     "print(dynamic_prompt.format(adjective=long_string))"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "e4bebcd9",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Give the antonym of every input\n",
+      "\n",
+      "Input: happy\n",
+      "Output: sad\n",
+      "\n",
+      "Input: tall\n",
+      "Output: short\n",
+      "\n",
+      "Input: energetic\n",
+      "Output: lethargic\n",
+      "\n",
+      "Input: sunny\n",
+      "Output: gloomy\n",
+      "\n",
+      "Input: windy\n",
+      "Output: calm\n",
+      "\n",
+      "Input: big\n",
+      "Output: small\n",
+      "\n",
+      "Input: enthusiastic\n",
+      "Output:\n"
+     ]
+    }
+   ],
+   "source": [
+    "# You can add an example to an example selector as well.\n",
+    "new_example = {\"input\": \"big\", \"output\": \"small\"}\n",
+    "dynamic_prompt.example_selector.add_example(new_example)\n",
+    "print(dynamic_prompt.format(adjective=\"enthusiastic\"))"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "2d007b0a",
@@ -401,7 +443,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 14,
    "id": "241bfe80",
    "metadata": {},
    "outputs": [],
@@ -413,7 +455,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 15,
    "id": "50d0a701",
    "metadata": {},
    "outputs": [],
@@ -440,7 +482,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 16,
    "id": "4c8fdf45",
    "metadata": {},
    "outputs": [
@@ -465,9 +507,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 17,
    "id": "829af21a",
-   "metadata": {},
+   "metadata": {
+    "scrolled": true
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -484,10 +528,36 @@
     }
    ],
    "source": [
-    "# Input is a measurment, so should select the tall/short example\n",
+    "# Input is a measurement, so should select the tall/short example\n",
     "print(similar_prompt.format(adjective=\"fat\"))"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "3c16fe23",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Give the antonym of every input\n",
+      "\n",
+      "Input: enthusiastic\n",
+      "Output: apathetic\n",
+      "\n",
+      "Input: joyful\n",
+      "Output:\n"
+     ]
+    }
+   ],
+   "source": [
+    "# You can add new examples to the SemanticSimilarityExampleSelector as well\n",
+    "similar_prompt.example_selector.add_example({\"input\": \"enthusiastic\", \"output\": \"apathetic\"})\n",
+    "print(similar_prompt.format(adjective=\"joyful\"))"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "dbc32551",

docs/explanation/agents.md

@@ -0,0 +1,29 @@
+# Agents
+
+Agents use an LLM to determine which actions to take and in what order.
+An action can either be using a tool and observing its output, or returning to the user.
+Here are the agents available in LangChain.
+
+For a tutorial on how to load agents, see [here](/getting_started/agents.ipynb).
+
+### `zero-shot-react-description`
+
+This agent uses the ReAct framework to determine which tool to use
+based solely on the tool's description. Any number of tools can be provided.
+This agent requires that a description is provided for each tool.
+
+### `react-docstore`
+
+This agent uses the ReAct framework to interact with a docstore. Two tools must
+be provided: a `Search` tool and a `Lookup` tool (they must be named exactly as so).
+The `Search` tool should search for a document, while the `Lookup` tool should lookup
+a term in the most recently found document.
+This agent is equivalent to the
+original [ReAct paper](https://arxiv.org/pdf/2210.03629.pdf), specifically the Wikipedia example.
+
+### `self-ask-with-search`
+
+This agent utilizes a single tool that should be named `Intermediate Answer`.
+This tool should be able to lookup factual answers to questions. This agent
+is equivalent to the original [self ask with search paper](https://ofir.io/self-ask.pdf),
+where a Google search API was provided as the tool.

docs/explanation/core_concepts.md

@@ -25,3 +25,13 @@ These are datastores that store documents. They expose a method for passing in a
 ## Chains
 These are pipelines that combine multiple of the above ideas. 
 They vary greatly in complexity and are combination of generic, highly configurable pipelines and more narrow (but usually more complex) pipelines.
+
+## Agents
+As opposed to a chain, whether the steps to be taken are known ahead of time, agents
+use an LLM to determine which tools to call and in what order.
+
+## Memory
+By default, Chains and Agents are stateless, meaning that they treat each incoming query independently.
+In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions,
+both at a short term but also at a long term level. The concept of "Memory" exists to do exactly that.
+

docs/explanation/glossary.md

@@ -29,7 +29,7 @@ This induces the to model to think about what action to take, then take it.
 
 Resources:
 - [Paper](https://arxiv.org/pdf/2210.03629.pdf)
-- [LangChain Example](https://github.com/hwchase17/langchain/blob/master/examples/react.ipynb)
+- [LangChain Example](https://github.com/hwchase17/langchain/blob/master/docs/examples/agents/react.ipynb)
 
 ### Self-ask
 
@@ -38,7 +38,7 @@ In this method, the model explicitly asks itself follow-up questions, which are
 
 Resources:
 - [Paper](https://ofir.io/self-ask.pdf)
-- [LangChain Example](https://github.com/hwchase17/langchain/blob/master/examples/self_ask_with_search.ipynb)
+- [LangChain Example](https://github.com/hwchase17/langchain/blob/master/docs/examples/agents/self_ask_with_search.ipynb)
 
 ### Prompt Chaining
 

docs/getting_started/agents.ipynb

@@ -5,10 +5,12 @@
    "id": "5436020b",
    "metadata": {},
    "source": [
-    "# Routing Chains\n",
-    "Some applications will require not just a predetermined chain of calls to LLMs/other tools, but potentially an unknown chain that depends on the user input. In these types of chains, there is a \"router\" LLM chain which has access to a suite of tools. Depending on the user input, the router can then decide which, if any, of these tools to call.\n",
+    "# Agents\n",
     "\n",
-    "These types of chains are called Routing Chains. When used correctly these can be extremely powerful. The purpose of this notebook is to show you how to easily use routing chains through the simplest, highest level API. If you want more low level control over various components, check out the documentation for custom routing chains."
+    "Agents use an LLM to determine which actions to take and in what order.\n",
+    "An action can either be using a tool and observing its output, or returning to the user.\n",
+    "\n",
+    "When used correctly agents can be extremely powerful. The purpose of this notebook is to show you how to easily use agents through the simplest, highest level API. If you want more low level control over various components, check out the documentation for custom agents (coming soon)."
    ]
   },
   {
@@ -18,10 +20,13 @@
    "source": [
     "## Concepts\n",
     "\n",
-    "In order to understand routing chains, you should understand the following concepts:\n",
+    "In order to load agents, you should understand the following concepts:\n",
+    "\n",
     "- Tool: A function that performs a specific duty. This can be things like: Google Search, Database lookup, Python REPL, other chains. The interface for a tool is currently a function that is expected to have a string as an input, with a string as an output.\n",
-    "- LLM: The language model responsible for doing the router.\n",
-    "- RouterType: The type of the router to use. This should be a string (see more on the allowed router types below). Because this notebook focuses on the simplest, highest level API, this only covers using the standard supported routers. If you want to implement a custom router, see the documentation for custom routing chains."
+    "- LLM: The language model powering the agent.\n",
+    "- Agent: The agent to use. This should be a string that references a support agent class. Because this notebook focuses on the simplest, highest level API, this only covers using the standard supported agents. If you want to implement a custom agent, see the documentation for custom agents (coming soon).\n",
+    "\n",
+    "**For a list of supported agents and their specifications, see [here](../explanation/agents.md)**"
    ]
   },
   {
@@ -30,7 +35,7 @@
    "metadata": {},
    "source": [
     "## Tools\n",
-    "When constructing your own Routing Chain, you will need to provide it with a list of tools that it can use. This is done with a list of Tools. The Tools are used not only to create the Routing Chain, but is also sometimes used to create the router itself (often, the router logic depends on the tools available). \n",
+    "When constructing your own agent, you will need to provide it with a list of Tools that it can use. A Tool is defined as below.\n",
     "\n",
     "```python\n",
     "class Tool(NamedTuple):\n",
@@ -41,7 +46,7 @@
     "    description: Optional[str] = None\n",
     "```\n",
     "\n",
-    "The two required components of a ToolConfig are the name and then the tool itself. A tool description is optional, as it is needed for some routers but not all."
+    "The two required components of a Tool are the name and then the tool itself. A tool description is optional, as it is needed for some agents but not all."
    ]
   },
   {
@@ -49,7 +54,7 @@
    "id": "2558a02d",
    "metadata": {},
    "source": [
-    "## Loading the chains\n"
+    "## Loading an agent\n"
    ]
   },
   {
@@ -60,7 +65,7 @@
    "outputs": [],
    "source": [
     "# Import things that are needed generically\n",
-    "from langchain.routing_chains import load_routing_chain, Tool\n",
+    "from langchain.agents import initialize_agent, Tool\n",
     "from langchain.llms import OpenAI"
    ]
   },
@@ -97,10 +102,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Construct the routing chain. We will use the default router type here.\n",
+    "# Construct the agent. We will use the default agent type here.\n",
     "# See documentation for a full list of options.\n",
-    "router_llm = OpenAI(temperature=0)\n",
-    "chain = load_routing_chain(tools, router_llm, verbose=True)"
+    "llm = OpenAI(temperature=0)\n",
+    "agent = initialize_agent(tools, llm, agent=\"zero-shot-react-description\", verbose=True)"
    ]
   },
   {
@@ -113,9 +118,6 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new chain...\u001b[0m\n",
       "What is the age of Olivia Wilde's boyfriend raised to the 0.23 power?\n",
       "Thought:\u001b[32;1m\u001b[1;3m I need to find the age of Olivia Wilde's boyfriend\n",
       "Action: Search\n",
@@ -143,8 +145,7 @@
       "Observation: \u001b[33;1m\u001b[1;3mAnswer: 2.1520202182226886\n",
       "\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: 2.1520202182226886\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
+      "Final Answer: 2.1520202182226886\u001b[0m"
      ]
     },
     {
@@ -159,7 +160,7 @@
     }
    ],
    "source": [
-    "chain.run(\"What is the age of Olivia Wilde's boyfriend raised to the 0.23 power?\")"
+    "agent.run(\"What is the age of Olivia Wilde's boyfriend raised to the 0.23 power?\")"
    ]
   },
   {

docs/getting_started/llm_chain.md

@@ -1,4 +1,4 @@
-# Using Chains
+# LLM Chains
 
 Calling an LLM is a great first step, but it's just the beginning.
 Normally when you use an LLM in an application, you are not sending user input directly to the LLM.
@@ -33,7 +33,5 @@ Now we can run that can only specifying the product!
 chain.run("colorful socks")
 ```
 
-There we go! There's the first chain.
-
-That is it for the Getting Started example. 
-As a next step, we would suggest checking out the more complex chains in the [Demos section](/examples/demos)
+There we go! There's the first chain - an LLM Chain.
+This is one of the simpler types of chains, but understanding how it works will set you up well for working with more complex chains.

docs/getting_started/memory.ipynb

@@ -0,0 +1,333 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "d31df93e",
+   "metadata": {},
+   "source": [
+    "# Memory\n",
+    "So far, all the chains and agents we've gone through have been stateless. But often, you may want a chain or agent to have some concept of \"memory\" so that it may remember information about its previous interactions. The most clear and simple example of this is when designing a chatbot - you want it to remember previous messages so it can use context from that to have a better conversation. This would be a type of \"short-term memory\". On the more complex side, you could imagine a chain/agent remembering key pieces of information over time - this would be a form of \"long-term memory\".\n",
+    "\n",
+    "LangChain provides several specially created chains just for this purpose. This notebook walk throughs using one of those chains (the `ConversationChain`) with two different types of memory."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d051c1da",
+   "metadata": {},
+   "source": [
+    "### ConversationChain with default memory\n",
+    "By default, the `ConversationChain` has a simple type of memory which remebers all previes inputs/outputs and adds them to the context that is passed. Let's take a look at using this chain (setting `verbose=True` so we can see the prompt)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "ae046bff",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
+      "\n",
+      "Current conversation:\n",
+      "\n",
+      "Human: Hi there!\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "' Hello! How are you today?'"
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain import OpenAI, ConversationChain\n",
+    "\n",
+    "llm = OpenAI(temperature=0)\n",
+    "conversation = ConversationChain(llm=llm, verbose=True)\n",
+    "\n",
+    "conversation.predict(input=\"Hi there!\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "d8e2a6ff",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
+      "\n",
+      "Current conversation:\n",
+      "\n",
+      "Human: Hi there!\n",
+      "AI:  Hello! How are you today?\n",
+      "Human: I'm doing well! Just having a conversation with an AI.\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "\" That's great! What would you like to talk about?\""
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation.predict(input=\"I'm doing well! Just having a conversation with an AI.\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "15eda316",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
+      "\n",
+      "Current conversation:\n",
+      "\n",
+      "Human: Hi there!\n",
+      "AI:  Hello! How are you today?\n",
+      "Human: I'm doing well! Just having a conversation with an AI.\n",
+      "AI:  That's great! What would you like to talk about?\n",
+      "Human: Tell me about yourself.\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'  I am an AI created to provide information and support to humans. I enjoy learning and exploring new things.'"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation.predict(input=\"Tell me about yourself.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4fad9448",
+   "metadata": {},
+   "source": [
+    "### ConversationChain with ConversationSummaryMemory\n",
+    "Now lets take a look at using a slightly more complex type of memory - `ConversationSummaryMemory`. This type of memory creates a summary of the conversation over time. This can be useful for condensing information from the conversation over time.\n",
+    "\n",
+    "Let's walk through an example, again setting `verbose=True` so we can see the prompt."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "f60a2fe8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains.conversation.memory import ConversationSummaryMemory"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "b7274f2c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
+      "\n",
+      "Current conversation:\n",
+      "\n",
+      "Human: Hi, what's up?\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "\"\\n\\nI'm doing well, thank you for asking. I'm currently working on a project that I'm really excited about.\""
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation_with_summary = ConversationChain(llm=llm, memory=ConversationSummaryMemory(llm=OpenAI()), verbose=True)\n",
+    "conversation_with_summary.predict(input=\"Hi, what's up?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "a6b6b88f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
+      "\n",
+      "Current conversation:\n",
+      "\n",
+      "The human and artificial intelligence are talking. The human asked the AI what it is doing, and the AI said that it is working on a project that it is excited about.\n",
+      "Human: Tell me more about it!\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "\"\\n\\nI'm working on a project that I'm really excited about. It's a lot of work, but I think it's going to be really great when it's finished. I can't wait to show it to you!\""
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation_with_summary.predict(input=\"Tell me more about it!\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "dad869fe",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",
+      "\n",
+      "Current conversation:\n",
+      "\n",
+      "\n",
+      "The human and artificial intelligence are talking. The human asked the AI what it is doing, and the AI said that it is working on a project that it is excited about. The AI said that the project is a lot of work, but it is going to be great when it is finished.\n",
+      "Human: Very cool -- what is the scope of the project?\n",
+      "AI:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'\\n\\nThe project is quite large in scope. It involves a lot of data analysis and work with artificial intelligence algorithms.'"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation_with_summary.predict(input=\"Very cool -- what is the scope of the project?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5c8735cc",
+   "metadata": {},
+   "source": [
+    "### More Resources on Memory\n",
+    "\n",
+    "This just scratches the surface of what you can do with memory. For more examples on things like how to implement custom memory classes, how to add memory to a custom LLM chain and how to use memory with and agent, please see the [How-To: Memory](../../examples/memory) section."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "436dda66",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/index.rst

@@ -11,41 +11,82 @@ This library is aimed at assisting in the development of those types of applicat
 
 There are three main areas (with a forth coming soon) that LangChain is designed to help with.
 These are, in increasing order of complexity:
-1. LLM and Prompt usage
-2. Chaining LLMs with other tools in a deterministic manner
-3. Having a router LLM which uses other tools as needed
+
+1. LLM and Prompts
+2. Chains
+3. Agents
 4. (Coming Soon) Memory
 
-**LLMs and Prompts**
+Let's go through these categories and for each one identify key concepts (to clarify terminology) as well as the problems in this area LangChain helps solve.
+
+**🦜 LLMs and Prompts**
 
 Calling out to an LLM once is pretty easy, with most of them being behind well documented APIs.
-However, there are still some challenges going from that to an application running in production that LangChain attempts to address:
-- Easy switching costs: by exposing a standard interface for all the top LLM providers, LangChain makes it easy to switch from one provider to another, whether it be for production use cases or just for testing stuff out.
+However, there are still some challenges going from that to an application running in production that LangChain attempts to address.
+
+*Key Concepts*
+
+- LLM: A large language model, in particular a text-to-text model.
+- Prompt: The input to a language model. Typically this is not simply a hardcoded string but rather a combination of a template, some examples, and user input.
+- Prompt Template: An object responsible for constructing the final prompt to pass to a LLM.
+
+*Problems Solved*
+
+- Switching costs: by exposing a standard interface for all the top LLM providers, LangChain makes it easy to switch from one provider to another, whether it be for production use cases or just for testing stuff out.
 - Prompt management: managing your prompts is easy when you only have one simple one, but can get tricky when you have a bunch or when they start to get more complex. LangChain provides a standard way for storing, constructing, and referencing prompts.
 - Prompt optimization: despite the underlying models getting better and better, there is still currently a need for carefully constructing prompts.
-- More coming soon
 
-**Chains**
+**🔗️ Chains**
+
+Using an LLM in isolation is fine for some simple applications, but many more complex ones require chaining LLMs - either with eachother or with other experts.
+LangChain provides several parts to help with that.
+
+*Key Concepts*
+
+- Tools: APIs designed for assisting with a particular use case (search, databases, Python REPL, etc). Prompt templates, LLMs, and chains can also be considered tools.
+- Chains: A combination of multiple tools in a deterministic manner.
+
+*Problems Solved*
 
-Using an LLM in isolation is fine for some simple applications, but many more complex ones require chaining LLMs - either with eachother or with other tools.
-LangChain provides several parts to help with that:
 - Standard interface for working with Chains
 - Easy way to construct chains of LLMs
-- Lots of integrations with other tools that you may want to use in conjunction with LLMs (search, databases, Python REPL, etc)
+- Lots of integrations with other tools that you may want to use in conjunction with LLMs
 - End-to-end chains for common workflows (database question/answer, recursive summarization, etc)
 
-**Routing Chains**
+**🤖 Agents**
 
 Some applications will require not just a predetermined chain of calls to LLMs/other tools, but potentially an unknown chain that depends on the user input.
-In these types of chains, there is a "router" LLM chain which has access to a suite of tools.
-Depending on the user input, the router can then decide which, if any, of these tools to call.
-To help develop applications like these, LangChain provides:
-- Standard router and router chain interfaces
-- Common router LLM chains from literature
+In these types of chains, there is a “agent” which has access to a suite of tools.
+Depending on the user input, the agent can then decide which, if any, of these tools to call.
+
+*Key Concepts*
+
+- Tools: same as above.
+- Agent: An LLM-powered class responsible for determining which tools to use and in what order.
+
+
+*Problems Solved*
+
+- Standard agent interfaces
+- A selection of powerful agents to choose from
 - Common chains that can be used as tools
 
-**Memory**
-Coming soon.
+**🧠 Memory**
+
+By default, Chains and Agents are stateless, meaning that they treat each incoming query independently.
+In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions,
+both at a short term but also at a long term level. The concept of "Memory" exists to do exactly that.
+
+*Key Concepts*
+
+- Memory: A class that can be added to an Agent or Chain to (1) pull in memory variables before calling that chain/agent, and (2) create new memories after the chain/agent finishes.
+- Memory Variables: Variables returned from a Memory class, to be passed into the chain/agent along with the user input.
+
+*Problems Solved*
+
+- Standard memory interfaces
+- A collection of common memory implementations to choose from
+- Common chains/agents that use memory (e.g. chatbots)
 
 Documentation Structure
 =======================
@@ -60,7 +101,10 @@ The documentation is structured into the following sections:
    getting_started/installation.md
    getting_started/environment.md
    getting_started/llm.md
-   getting_started/chains.md
+   getting_started/llm_chain.md
+   getting_started/sequential_chains.md
+   getting_started/agents.ipynb
+   getting_started/memory.ipynb
 
 Goes over a simple walk through and tutorial for getting started setting up a simple chain that generates a company name based on what the company makes.
 Covers installation, environment set up, calling LLMs, and using prompts.
@@ -72,9 +116,11 @@ Start here if you haven't used LangChain before.
    :caption: How-To Examples
    :name: examples
 
-   examples/demos.rst
-   examples/integrations.rst
    examples/prompts.rst
+   examples/integrations.rst
+   examples/chains.rst
+   examples/agents.rst
+   examples/memory.rst
    examples/model_laboratory.ipynb
 
 More elaborate examples and walk-throughs of particular
@@ -97,7 +143,7 @@ common tasks or cool demos.
    modules/text_splitter
    modules/vectorstore
    modules/chains
-   modules/routing_chains
+   modules/agents
 
 
 Full API documentation. This is the place to look if you want to
@@ -110,6 +156,7 @@ see detailed information about the various classes, methods, and APIs.
    :name: resources
 
    explanation/core_concepts.md
+   explanation/agents.md
    explanation/glossary.md
    Discord <https://discord.gg/6adMQxSpJS>
 

docs/modules/agents.rst

@@ -0,0 +1,7 @@
+:mod:`langchain.agents`
+===============================
+
+.. automodule:: langchain.agents
+   :members:
+   :undoc-members:
+

docs/modules/routing_chains.rst

@@ -1,7 +0,0 @@
-:mod:`langchain.routing_chains`
-===============================
-
-.. automodule:: langchain.routing_chains
-   :members:
-   :undoc-members:
-

langchain/VERSION

@@ -1 +1 @@
-0.0.18
+0.0.22

langchain/__init__.py

@@ -5,7 +5,9 @@ from pathlib import Path
 with open(Path(__file__).absolute().parents[0] / "VERSION") as _f:
     __version__ = _f.read().strip()
 
+from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
 from langchain.chains import (
+    ConversationChain,
     LLMChain,
     LLMMathChain,
     PythonChain,
@@ -13,7 +15,7 @@ from langchain.chains import (
     SQLDatabaseChain,
     VectorDBQA,
 )
-from langchain.docstore import Wikipedia
+from langchain.docstore import InMemoryDocstore, Wikipedia
 from langchain.llms import Cohere, HuggingFaceHub, OpenAI
 from langchain.prompts import (
     BasePromptTemplate,
@@ -21,7 +23,6 @@ from langchain.prompts import (
     Prompt,
     PromptTemplate,
 )
-from langchain.routing_chains import MRKLChain, ReActChain, SelfAskWithSearchChain
 from langchain.sql_database import SQLDatabase
 from langchain.vectorstores import FAISS, ElasticVectorSearch
 
@@ -46,4 +47,6 @@ __all__ = [
     "MRKLChain",
     "VectorDBQA",
     "ElasticVectorSearch",
+    "InMemoryDocstore",
+    "ConversationChain",
 ]

langchain/agents/__init__.py

@@ -0,0 +1,17 @@
+"""Routing chains."""
+from langchain.agents.agent import Agent
+from langchain.agents.loading import initialize_agent
+from langchain.agents.mrkl.base import MRKLChain, ZeroShotAgent
+from langchain.agents.react.base import ReActChain
+from langchain.agents.self_ask_with_search.base import SelfAskWithSearchChain
+from langchain.agents.tools import Tool
+
+__all__ = [
+    "MRKLChain",
+    "SelfAskWithSearchChain",
+    "ReActChain",
+    "Agent",
+    "Tool",
+    "initialize_agent",
+    "ZeroShotAgent",
+]

langchain/agents/agent.py

@@ -0,0 +1,156 @@
+"""Chain that takes in an input and produces an action and action input."""
+from abc import ABC, abstractmethod
+from typing import Any, ClassVar, Dict, List, NamedTuple, Optional, Tuple
+
+from pydantic import BaseModel
+
+from langchain.agents.tools import Tool
+from langchain.chains.base import Chain
+from langchain.chains.llm import LLMChain
+from langchain.input import ChainedInput
+from langchain.printing import get_color_mapping
+from langchain.llms.base import LLM
+from langchain.prompts.base import BasePromptTemplate
+from langchain.logger import CONTEXT_KEY
+
+
+class Action(NamedTuple):
+    """Action to take."""
+
+    tool: str
+    tool_input: str
+    log: str
+
+
+class Agent(Chain, BaseModel, ABC):
+    """Agent that uses an LLM."""
+
+    prompt: ClassVar[BasePromptTemplate]
+    llm_chain: LLMChain
+    tools: List[Tool]
+    input_key: str = "input"  #: :meta private:
+    output_key: str = "output"  #: :meta private:
+
+    @property
+    def input_keys(self) -> List[str]:
+        """Return the singular input key.
+
+        :meta private:
+        """
+        return [self.input_key]
+
+    @property
+    def output_keys(self) -> List[str]:
+        """Return the singular output key.
+
+        :meta private:
+        """
+        return [self.output_key]
+
+    @property
+    @abstractmethod
+    def observation_prefix(self) -> str:
+        """Prefix to append the observation with."""
+
+    @property
+    @abstractmethod
+    def llm_prefix(self) -> str:
+        """Prefix to append the LLM call with."""
+
+    @property
+    def finish_tool_name(self) -> str:
+        """Name of the tool to use to finish the chain."""
+        return "Final Answer"
+
+    @property
+    def starter_string(self) -> str:
+        """Put this string after user input but before first LLM call."""
+        return "\n"
+
+    @abstractmethod
+    def _extract_tool_and_input(self, text: str) -> Optional[Tuple[str, str]]:
+        """Extract tool and tool input from llm output."""
+
+    def _fix_text(self, text: str) -> str:
+        """Fix the text."""
+        raise ValueError("fix_text not implemented for this agent.")
+
+    @property
+    def _stop(self) -> List[str]:
+        return [f"\n{self.observation_prefix}"]
+
+    @classmethod
+    def _validate_tools(cls, tools: List[Tool]) -> None:
+        """Validate that appropriate tools are passed in."""
+        pass
+
+    @classmethod
+    def create_prompt(cls, tools: List[Tool]) -> BasePromptTemplate:
+        """Create a prompt for this class."""
+        return cls.prompt
+
+    @classmethod
+    def from_llm_and_tools(cls, llm: LLM, tools: List[Tool], **kwargs: Any) -> "Agent":
+        """Construct an agent from an LLM and tools."""
+        cls._validate_tools(tools)
+        llm_chain = LLMChain(llm=llm, prompt=cls.create_prompt(tools))
+        return cls(llm_chain=llm_chain, tools=tools, **kwargs)
+
+    def get_action(self, text: str) -> Action:
+        """Given input, decided what to do.
+
+        Args:
+            text: input string
+
+        Returns:
+            Action specifying what tool to use.
+        """
+        input_key = self.llm_chain.input_keys[0]
+        inputs = {input_key: text, "stop": self._stop}
+        full_output = self.llm_chain.predict(**inputs)
+        parsed_output = self._extract_tool_and_input(full_output)
+        while parsed_output is None:
+            full_output = self._fix_text(full_output)
+            inputs = {input_key: text + full_output, "stop": self._stop}
+            output = self.llm_chain.predict(**inputs)
+            full_output += output
+            parsed_output = self._extract_tool_and_input(full_output)
+        tool, tool_input = parsed_output
+        return Action(tool, tool_input, full_output)
+
+    def _call(self, inputs: Dict[str, Any]) -> Dict[str, str]:
+        """Run text through and get agent response."""
+        text = inputs[self.input_key]
+        # Construct a mapping of tool name to tool for easy lookup
+        name_to_tool_map = {tool.name: tool.func for tool in self.tools}
+        # Construct the initial string to pass into the LLM. This is made up
+        # of the user input, the special starter string, and then the LLM prefix.
+        # The starter string is a special string that may be used by a LLM to
+        # immediately follow the user input. The LLM prefix is a string that
+        # prompts the LLM to take an action.
+        starter_string = text + self.starter_string + self.llm_prefix
+        # We use the ChainedInput class to iteratively add to the input over time.
+        chained_input = ChainedInput(starter_string, inputs[CONTEXT_KEY], logger=self.logger)
+        # We construct a mapping from each tool to a color, used for logging.
+        color_mapping = get_color_mapping(
+            [tool.name for tool in self.tools], excluded_colors=["green"]
+        )
+        # We now enter the agent loop (until it returns something).
+        while True:
+            # Call the LLM to see what to do.
+            output = self.get_action(chained_input.input)
+            # Add the log to the Chained Input.
+            chained_input.add(output.log, color="green")
+            # If the tool chosen is the finishing tool, then we end and return.
+            if output.tool == self.finish_tool_name:
+                return {self.output_key: output.tool_input}
+            # Otherwise we lookup the tool
+            chain = name_to_tool_map[output.tool]
+            # We then call the tool on the tool input to get an observation
+            observation = chain(output.tool_input)
+            # We then log the observation
+            chained_input.add(f"\n{self.observation_prefix}")
+            chained_input.add(observation, color=color_mapping[output.tool])
+            # We then add the LLM prefix into the prompt to get the LLM to start
+            # thinking, and start the loop all over.
+            chained_input.add(f"\n{self.llm_prefix}")

langchain/agents/loading.py

@@ -0,0 +1,42 @@
+"""Load agent."""
+from typing import Any, List
+
+from langchain.agents.agent import Agent
+from langchain.agents.mrkl.base import ZeroShotAgent
+from langchain.agents.react.base import ReActDocstoreAgent
+from langchain.agents.self_ask_with_search.base import SelfAskWithSearchAgent
+from langchain.agents.tools import Tool
+from langchain.llms.base import LLM
+
+AGENT_TO_CLASS = {
+    "zero-shot-react-description": ZeroShotAgent,
+    "react-docstore": ReActDocstoreAgent,
+    "self-ask-with-search": SelfAskWithSearchAgent,
+}
+
+
+def initialize_agent(
+    tools: List[Tool],
+    llm: LLM,
+    agent: str = "zero-shot-react-description",
+    **kwargs: Any,
+) -> Agent:
+    """Load agent given tools and LLM.
+
+    Args:
+        tools: List of tools this agent has access to.
+        llm: Language model to use as the agent.
+        agent: The agent to use. Valid options are:
+            `zero-shot-react-description`, `react-docstore`, `self-ask-with-search`.
+        **kwargs: Additional key word arguments to pass to the agent.
+
+    Returns:
+        An agent.
+    """
+    if agent not in AGENT_TO_CLASS:
+        raise ValueError(
+            f"Got unknown agent type: {agent}. "
+            f"Valid types are: {AGENT_TO_CLASS.keys()}."
+        )
+    agent_cls = AGENT_TO_CLASS[agent]
+    return agent_cls.from_llm_and_tools(llm, tools, **kwargs)

langchain/agents/mrkl/base.py

@@ -1,13 +1,11 @@
 """Attempt to implement MRKL systems as described in arxiv.org/pdf/2205.00445.pdf."""
 from typing import Any, Callable, List, NamedTuple, Optional, Tuple
 
-from langchain.chains.llm import LLMChain
+from langchain.agents.agent import Agent
+from langchain.agents.mrkl.prompt import FORMAT_INSTRUCTIONS, PREFIX, SUFFIX
+from langchain.agents.tools import Tool
 from langchain.llms.base import LLM
 from langchain.prompts import PromptTemplate
-from langchain.routing_chains.mrkl.prompt import BASE_TEMPLATE
-from langchain.routing_chains.router import LLMRouter
-from langchain.routing_chains.routing_chain import RoutingChain
-from langchain.routing_chains.tools import Tool
 
 FINAL_ANSWER_ACTION = "Final Answer: "
 
@@ -47,8 +45,8 @@ def get_action_and_input(llm_output: str) -> Tuple[str, str]:
     return action, action_input.strip(" ").strip('"')
 
 
-class ZeroShotRouter(LLMRouter):
-    """Router for the MRKL chain."""
+class ZeroShotAgent(Agent):
+    """Agent for the MRKL chain."""
 
     @property
     def observation_prefix(self) -> str:
@@ -56,25 +54,52 @@ class ZeroShotRouter(LLMRouter):
         return "Observation: "
 
     @property
-    def router_prefix(self) -> str:
-        """Prefix to append the router call with."""
+    def llm_prefix(self) -> str:
+        """Prefix to append the llm call with."""
         return "Thought:"
 
     @classmethod
-    def from_llm_and_tools(cls, llm: LLM, tools: List[Tool]) -> "ZeroShotRouter":
-        """Construct a router from an LLM and tools."""
+    def create_prompt(
+        cls,
+        tools: List[Tool],
+        prefix: str = PREFIX,
+        suffix: str = SUFFIX,
+        input_variables: Optional[List[str]] = None,
+    ) -> PromptTemplate:
+        """Create prompt in the style of the zero shot agent.
+
+        Args:
+            tools: List of tools the agent will have access to, used to format the
+                prompt.
+            prefix: String to put before the list of tools.
+            suffix: String to put after the list of tools.
+            input_variables: List of input variables the final prompt will expect.
+
+        Returns:
+            A PromptTemplate with the template assembled from the pieces here.
+        """
         tool_strings = "\n".join([f"{tool.name}: {tool.description}" for tool in tools])
         tool_names = ", ".join([tool.name for tool in tools])
-        template = BASE_TEMPLATE.format(tools=tool_strings, tool_names=tool_names)
-        prompt = PromptTemplate(template=template, input_variables=["input"])
-        llm_chain = LLMChain(llm=llm, prompt=prompt)
-        return cls(llm_chain=llm_chain)
+        format_instructions = FORMAT_INSTRUCTIONS.format(tool_names=tool_names)
+        template = "\n\n".join([prefix, tool_strings, format_instructions, suffix])
+        if input_variables is None:
+            input_variables = ["input"]
+        return PromptTemplate(template=template, input_variables=input_variables)
+
+    @classmethod
+    def _validate_tools(cls, tools: List[Tool]) -> None:
+        for tool in tools:
+            if tool.description is None:
+                raise ValueError(
+                    f"Got a tool {tool.name} without a description. For this agent, "
+                    f"a description must always be provided."
+                )
 
     def _extract_tool_and_input(self, text: str) -> Optional[Tuple[str, str]]:
         return get_action_and_input(text)
 
 
-class MRKLChain(RoutingChain):
+class MRKLChain(ZeroShotAgent):
     """Chain that implements the MRKL system.
 
     Example:
@@ -89,16 +114,14 @@ class MRKLChain(RoutingChain):
     """
 
     @classmethod
-    def from_chains(
-        cls, llm: LLM, chains: List[ChainConfig], **kwargs: Any
-    ) -> "MRKLChain":
+    def from_chains(cls, llm: LLM, chains: List[ChainConfig], **kwargs: Any) -> "Agent":
         """User friendly way to initialize the MRKL chain.
 
         This is intended to be an easy way to get up and running with the
         MRKL chain.
 
         Args:
-            llm: The LLM to use as the router LLM.
+            llm: The LLM to use as the agent LLM.
             chains: The chains the MRKL system has access to.
             **kwargs: parameters to be passed to initialization.
 
@@ -131,46 +154,4 @@ class MRKLChain(RoutingChain):
             Tool(name=c.action_name, func=c.action, description=c.action_description)
             for c in chains
         ]
-        return cls.from_tools_and_llm(tools, llm, **kwargs)
-
-    @classmethod
-    def from_tools_and_llm(
-        cls, tools: List[Tool], llm: LLM, **kwargs: Any
-    ) -> "MRKLChain":
-        """User friendly way to initialize the MRKL chain.
-
-        This is intended to be an easy way to get up and running with the
-        MRKL chain.
-
-        Args:
-            tools: The tools the MRKL system has access to.
-            llm: The LLM to use as the router LLM.
-            **kwargs: parameters to be passed to initialization.
-
-        Returns:
-            An initialized MRKL chain.
-
-        Example:
-            .. code-block:: python
-
-                from langchain import LLMMathChain, OpenAI, SerpAPIChain, MRKLChain
-                from langchain.routing_chains.tools import ToolConfig
-                llm = OpenAI(temperature=0)
-                search = SerpAPIChain()
-                llm_math_chain = LLMMathChain(llm=llm)
-                tools = [
-                    ToolConfig(
-                        tool_name = "Search",
-                        tool=search.search,
-                        tool_description="useful for searching"
-                    ),
-                    ToolConfig(
-                        tool_name="Calculator",
-                        tool=llm_math_chain.run,
-                        tool_description="useful for doing math"
-                    )
-                ]
-                mrkl = MRKLChain.from_tools_and_llm(llm, tools)
-        """
-        router = ZeroShotRouter.from_llm_and_tools(llm, tools)
-        return cls(router=router, tools=tools, **kwargs)
+        return cls.from_llm_and_tools(llm, tools, **kwargs)

langchain/agents/mrkl/prompt.py

@@ -1,9 +1,6 @@
 # flake8: noqa
-BASE_TEMPLATE = """Answer the following questions as best you can. You have access to the following tools:
-
-{tools}
-
-Use the following format:
+PREFIX = """Answer the following questions as best you can. You have access to the following tools:"""
+FORMAT_INSTRUCTIONS = """Use the following format:
 
 Question: the input question you must answer
 Thought: you should always think about what to do
@@ -12,8 +9,7 @@ Action Input: the input to the action
 Observation: the result of the action
 ... (this Thought/Action/Action Input/Observation can repeat N times)
 Thought: I now know the final answer
-Final Answer: the final answer to the original input question
+Final Answer: the final answer to the original input question"""
+SUFFIX = """Begin!
 
-Begin!
-
-Question: {{input}}"""
+Question: {input}"""

langchain/agents/react/base.py

@@ -1,27 +1,28 @@
 """Chain that implements the ReAct paper from https://arxiv.org/pdf/2210.03629.pdf."""
 import re
-from typing import Any, List, Optional, Tuple
+from typing import Any, ClassVar, List, Optional, Tuple
 
 from pydantic import BaseModel
 
+from langchain.agents.agent import Agent
+from langchain.agents.react.prompt import PROMPT
+from langchain.agents.tools import Tool
 from langchain.chains.llm import LLMChain
 from langchain.docstore.base import Docstore
 from langchain.docstore.document import Document
 from langchain.llms.base import LLM
-from langchain.routing_chains.react.prompt import PROMPT
-from langchain.routing_chains.router import LLMRouter
-from langchain.routing_chains.routing_chain import RoutingChain
-from langchain.routing_chains.tools import Tool
+from langchain.prompts.base import BasePromptTemplate
 
 
-class ReActDocstoreRouter(LLMRouter, BaseModel):
-    """Router for the ReAct chin."""
+class ReActDocstoreAgent(Agent, BaseModel):
+    """Agent for the ReAct chin."""
+
+    prompt: ClassVar[BasePromptTemplate] = PROMPT
 
     i: int = 1
 
     @classmethod
-    def from_llm_and_tools(cls, llm: LLM, tools: List[Tool]) -> "ReActDocstoreRouter":
-        """Construct a router from an LLM and tools."""
+    def _validate_tools(cls, tools: List[Tool]) -> None:
         if len(tools) != 2:
             raise ValueError(f"Exactly two tools must be specified, but got {tools}")
         tool_names = {tool.name for tool in tools}
@@ -30,9 +31,6 @@ class ReActDocstoreRouter(LLMRouter, BaseModel):
                 f"Tool names should be Lookup and Search, got {tool_names}"
             )
 
-        llm_chain = LLMChain(llm=llm, prompt=PROMPT)
-        return cls(llm_chain=llm_chain)
-
     def _fix_text(self, text: str) -> str:
         return text + f"\nAction {self.i}:"
 
@@ -65,8 +63,8 @@ class ReActDocstoreRouter(LLMRouter, BaseModel):
         return [f"\nObservation {self.i}: "]
 
     @property
-    def router_prefix(self) -> str:
-        """Prefix to append the router call with."""
+    def llm_prefix(self) -> str:
+        """Prefix to append the LLM call with."""
         return f"Thought {self.i}:"
 
 
@@ -95,7 +93,7 @@ class DocstoreExplorer:
         return self.document.lookup(term)
 
 
-class ReActChain(RoutingChain):
+class ReActChain(ReActDocstoreAgent):
     """Chain that implements the ReAct paper.
 
     Example:
@@ -112,5 +110,5 @@ class ReActChain(RoutingChain):
             Tool(name="Search", func=docstore_explorer.search),
             Tool(name="Lookup", func=docstore_explorer.lookup),
         ]
-        router = ReActDocstoreRouter.from_llm_and_tools(llm, tools)
-        super().__init__(router=router, tools=tools, **kwargs)
+        llm_chain = LLMChain(llm=llm, prompt=PROMPT)
+        super().__init__(llm_chain=llm_chain, tools=tools, **kwargs)

langchain/agents/self_ask_with_search/base.py

@@ -1,23 +1,22 @@
 """Chain that does self ask with search."""
-from typing import Any, List, Tuple
+from typing import Any, ClassVar, List, Optional, Tuple
 
+from langchain.agents.agent import Agent
+from langchain.agents.self_ask_with_search.prompt import PROMPT
+from langchain.agents.tools import Tool
 from langchain.chains.llm import LLMChain
 from langchain.chains.serpapi import SerpAPIChain
 from langchain.llms.base import LLM
-from langchain.routing_chains.router import LLMRouter
-from langchain.routing_chains.routing_chain import RoutingChain
-from langchain.routing_chains.self_ask_with_search.prompt import PROMPT
-from langchain.routing_chains.tools import Tool
+from langchain.prompts.base import BasePromptTemplate
 
 
-class SelfAskWithSearchRouter(LLMRouter):
-    """Router for the self-ask-with-search paper."""
+class SelfAskWithSearchAgent(Agent):
+    """Agent for the self-ask-with-search paper."""
+
+    prompt: ClassVar[BasePromptTemplate] = PROMPT
 
     @classmethod
-    def from_llm_and_tools(
-        cls, llm: LLM, tools: List[Tool]
-    ) -> "SelfAskWithSearchRouter":
-        """Construct a router from an LLM and tools."""
+    def _validate_tools(cls, tools: List[Tool]) -> None:
         if len(tools) != 1:
             raise ValueError(f"Exactly one tool must be specified, but got {tools}")
         tool_names = {tool.name for tool in tools}
@@ -26,10 +25,7 @@ class SelfAskWithSearchRouter(LLMRouter):
                 f"Tool name should be Intermediate Answer, got {tool_names}"
             )
 
-        llm_chain = LLMChain(llm=llm, prompt=PROMPT)
-        return cls(llm_chain=llm_chain, tools=tools)
-
-    def _extract_tool_and_input(self, text: str) -> Tuple[str, str]:
+    def _extract_tool_and_input(self, text: str) -> Optional[Tuple[str, str]]:
         followup = "Follow up:"
         if "\n" not in text:
             last_line = text
@@ -39,8 +35,8 @@ class SelfAskWithSearchRouter(LLMRouter):
         if followup not in last_line:
             finish_string = "So the final answer is: "
             if finish_string not in last_line:
-                raise ValueError("We should probably never get here")
-            return "Final Answer", text[len(finish_string) :]
+                return None
+            return "Final Answer", last_line[len(finish_string) :]
 
         if ":" not in last_line:
             after_colon = last_line
@@ -54,23 +50,26 @@ class SelfAskWithSearchRouter(LLMRouter):
 
         return "Intermediate Answer", after_colon
 
+    def _fix_text(self, text: str) -> str:
+        return text + "\nSo the final answer is:"
+
     @property
     def observation_prefix(self) -> str:
         """Prefix to append the observation with."""
         return "Intermediate answer: "
 
     @property
-    def router_prefix(self) -> str:
-        """Prefix to append the router call with."""
+    def llm_prefix(self) -> str:
+        """Prefix to append the LLM call with."""
         return ""
 
     @property
     def starter_string(self) -> str:
-        """Put this string after user input but before first router call."""
+        """Put this string after user input but before first LLM call."""
         return "\nAre follow up questions needed here:"
 
 
-class SelfAskWithSearchChain(RoutingChain):
+class SelfAskWithSearchChain(SelfAskWithSearchAgent):
     """Chain that does self ask with search.
 
     Example:
@@ -84,5 +83,5 @@ class SelfAskWithSearchChain(RoutingChain):
     def __init__(self, llm: LLM, search_chain: SerpAPIChain, **kwargs: Any):
         """Initialize with just an LLM and a search chain."""
         search_tool = Tool(name="Intermediate Answer", func=search_chain.run)
-        router = SelfAskWithSearchRouter.from_llm_and_tools(llm, [search_tool])
-        super().__init__(router=router, tools=[search_tool], **kwargs)
+        llm_chain = LLMChain(llm=llm, prompt=PROMPT)
+        super().__init__(llm_chain=llm_chain, tools=[search_tool], **kwargs)

langchain/chains/__init__.py

@@ -1,4 +1,5 @@
 """Chains are easily reusable components which can be linked together."""
+from langchain.chains.conversation.base import ConversationChain
 from langchain.chains.llm import LLMChain
 from langchain.chains.llm_math.base import LLMMathChain
 from langchain.chains.python import PythonChain
@@ -16,4 +17,5 @@ __all__ = [
     "VectorDBQA",
     "SequentialChain",
     "SimpleSequentialChain",
+    "ConversationChain",
 ]

langchain/chains/base.py

@@ -1,15 +1,44 @@
 """Base interface that all chains should implement."""
 from abc import ABC, abstractmethod
-from typing import Any, Dict, List
+from typing import Any, Dict, List, Optional
+from langchain.logger import PrintLogger
+import uuid
 
-from pydantic import BaseModel
+from pydantic import BaseModel, Extra, root_validator
+
+from langchain.logger import Logger, CONTEXT_KEY
+
+class Memory(BaseModel, ABC):
+    """Base interface for memory in chains."""
+
+    class Config:
+        """Configuration for this pydantic object."""
+
+        extra = Extra.forbid
+        arbitrary_types_allowed = True
+
+    @property
+    @abstractmethod
+    def memory_variables(self) -> List[str]:
+        """Input keys this memory class will load dynamically."""
+
+    @abstractmethod
+    def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]:
+        """Return key-value pairs given the text input to the chain."""
+
+    @abstractmethod
+    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
+        """Save the context of this model run to memory."""
 
 
 class Chain(BaseModel, ABC):
     """Base interface that all chains should implement."""
 
+    memory: Optional[Memory] = None
+
     verbose: bool = False
     """Whether to print out response text."""
+    logger: Optional[Logger] = None
 
     @property
     @abstractmethod
@@ -21,6 +50,19 @@ class Chain(BaseModel, ABC):
     def output_keys(self) -> List[str]:
         """Output keys this chain expects."""
 
+    @root_validator()
+    def add_logger(cls, values: Dict) -> Dict:
+        """Add a printing logger if verbose=True and none provided."""
+        if values["verbose"] and values["logger"] is None:
+            values["logger"] = PrintLogger()
+        return values
+
+    class Config:
+        """Configuration for this pydantic object."""
+
+        extra = Extra.forbid
+        arbitrary_types_allowed = True
+
     def _validate_inputs(self, inputs: Dict[str, str]) -> None:
         """Check that all inputs are present."""
         missing_keys = set(self.input_keys).difference(inputs)
@@ -51,13 +93,24 @@ class Chain(BaseModel, ABC):
                 chain will be returned. Defaults to False.
 
         """
+        if CONTEXT_KEY not in inputs:
+            inputs[CONTEXT_KEY] = {}
+        if "id" not in inputs[CONTEXT_KEY]:
+            inputs[CONTEXT_KEY]["id"] = str(uuid.uuid4())
+
+        if self.memory is not None:
+            external_context = self.memory.load_memory_variables(inputs)
+            inputs = dict(inputs, **external_context)
         self._validate_inputs(inputs)
-        if self.verbose:
-            print("\n\n\033[1m> Entering new chain...\033[0m")
+        if self.logger:
+            self.logger.log_start_of_chain(inputs)
         outputs = self._call(inputs)
-        if self.verbose:
-            print("\n\033[1m> Finished chain.\033[0m")
         self._validate_outputs(outputs)
+        outputs[CONTEXT_KEY] = inputs[CONTEXT_KEY]
+        if self.logger:
+            self.logger.log_end_of_chain(outputs)
+        if self.memory is not None:
+            self.memory.save_context(inputs, outputs)
         if return_only_outputs:
             return outputs
         else:

langchain/chains/conversation/__init__.py

@@ -0,0 +1 @@
+"""Chain that carries on a conversation from a prompt plus history."""

langchain/chains/conversation/base.py

@@ -0,0 +1,61 @@
+"""Chain that carries on a conversation and calls an LLM."""
+from typing import Dict, List
+
+from pydantic import BaseModel, Extra, Field, root_validator
+
+from langchain.chains.base import Memory
+from langchain.chains.conversation.memory import ConversationBufferMemory
+from langchain.chains.conversation.prompt import PROMPT
+from langchain.chains.llm import LLMChain
+from langchain.prompts.base import BasePromptTemplate
+
+
+class ConversationChain(LLMChain, BaseModel):
+    """Chain to have a conversation and load context from memory.
+
+    Example:
+        .. code-block:: python
+
+            from langchain import ConversationChain, OpenAI
+            conversation = ConversationChain(llm=OpenAI())
+    """
+
+    memory: Memory = Field(default_factory=ConversationBufferMemory)
+    """Default memory store."""
+    prompt: BasePromptTemplate = PROMPT
+    """Default conversation prompt to use."""
+
+    input_key: str = "input"  #: :meta private:
+    output_key: str = "response"  #: :meta private:
+    buffer: str = ""  #: :meta private:
+
+    class Config:
+        """Configuration for this pydantic object."""
+
+        extra = Extra.forbid
+        arbitrary_types_allowed = True
+
+    @property
+    def input_keys(self) -> List[str]:
+        """Use this since so some prompt vars come from history."""
+        return [self.input_key]
+
+    @root_validator()
+    def validate_prompt_input_variables(cls, values: Dict) -> Dict:
+        """Validate that prompt input variables are consistent."""
+        memory_keys = values["memory"].memory_variables
+        input_key = values["input_key"]
+        if input_key in memory_keys:
+            raise ValueError(
+                f"The input key {input_key} was also found in the memory keys "
+                f"({memory_keys}) - please provide keys that don't overlap."
+            )
+        prompt_variables = values["prompt"].input_variables
+        expected_keys = memory_keys + [input_key]
+        if set(expected_keys) != set(prompt_variables):
+            raise ValueError(
+                "Got unexpected prompt input variables. The prompt expects "
+                f"{prompt_variables}, but got {memory_keys} as inputs from "
+                f"memory, and {input_key} as the normal input key."
+            )
+        return values

langchain/chains/conversation/memory.py

@@ -0,0 +1,91 @@
+"""Memory modules for conversation prompts."""
+from typing import Any, Dict, List
+
+from pydantic import BaseModel, root_validator
+
+from langchain.chains.base import Memory
+from langchain.chains.conversation.prompt import SUMMARY_PROMPT
+from langchain.chains.llm import LLMChain
+from langchain.llms.base import LLM
+from langchain.prompts.base import BasePromptTemplate
+
+
+def _get_prompt_input_key(inputs: Dict[str, Any], memory_variables: List[str]) -> str:
+    # "stop" is a special key that can be passed as input but is not used to
+    # format the prompt.
+    prompt_input_keys = list(set(inputs).difference(memory_variables + ["stop"]))
+    if len(prompt_input_keys) != 1:
+        raise ValueError(f"One input key expected got {prompt_input_keys}")
+    return prompt_input_keys[0]
+
+
+class ConversationBufferMemory(Memory, BaseModel):
+    """Buffer for storing conversation memory."""
+
+    buffer: str = ""
+    memory_key: str = "history"  #: :meta private:
+
+    @property
+    def memory_variables(self) -> List[str]:
+        """Will always return list of memory variables.
+
+        :meta private:
+        """
+        return [self.memory_key]
+
+    def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]:
+        """Return history buffer."""
+        return {self.memory_key: self.buffer}
+
+    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
+        """Save context from this conversation to buffer."""
+        prompt_input_key = _get_prompt_input_key(inputs, self.memory_variables)
+        if len(outputs) != 1:
+            raise ValueError(f"One output key expected, got {outputs.keys()}")
+        human = "Human: " + inputs[prompt_input_key]
+        ai = "AI: " + outputs[list(outputs.keys())[0]]
+        self.buffer += "\n" + "\n".join([human, ai])
+
+
+class ConversationSummaryMemory(Memory, BaseModel):
+    """Conversation summarizer to memory."""
+
+    buffer: str = ""
+    llm: LLM
+    prompt: BasePromptTemplate = SUMMARY_PROMPT
+    memory_key: str = "history"  #: :meta private:
+
+    @property
+    def memory_variables(self) -> List[str]:
+        """Will always return list of memory variables.
+
+        :meta private:
+        """
+        return [self.memory_key]
+
+    def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]:
+        """Return history buffer."""
+        return {self.memory_key: self.buffer}
+
+    @root_validator()
+    def validate_prompt_input_variables(cls, values: Dict) -> Dict:
+        """Validate that prompt input variables are consistent."""
+        prompt_variables = values["prompt"].input_variables
+        expected_keys = {"summary", "new_lines"}
+        if expected_keys != set(prompt_variables):
+            raise ValueError(
+                "Got unexpected prompt input variables. The prompt expects "
+                f"{prompt_variables}, but it should have {expected_keys}."
+            )
+        return values
+
+    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
+        """Save context from this conversation to buffer."""
+        prompt_input_key = _get_prompt_input_key(inputs, self.memory_variables)
+        if len(outputs) != 1:
+            raise ValueError(f"One output key expected, got {outputs.keys()}")
+        human = "Human: " + inputs[prompt_input_key]
+        ai = "AI: " + list(outputs.values())[0]
+        new_lines = "\n".join([human, ai])
+        chain = LLMChain(llm=self.llm, prompt=self.prompt)
+        self.buffer = chain.predict(summary=self.buffer, new_lines=new_lines)

langchain/chains/conversation/prompt.py

@@ -0,0 +1,37 @@
+# flake8: noqa
+from langchain.prompts.prompt import PromptTemplate
+
+_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
+
+Current conversation:
+{history}
+Human: {input}
+AI:"""
+PROMPT = PromptTemplate(
+    input_variables=["history", "input"], template=_DEFAULT_TEMPLATE
+)
+
+_DEFAULT_SUMMARIZER_TEMPLATE = """Progressively summarize the lines of conversation provided, adding onto the previous summary returning a new summary.
+
+EXAMPLE
+Current summary:
+The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good.
+
+New lines of conversation:
+Human: Why do you think artificial intelligence is a force for good?
+AI: Because artificial intelligence will help humans reach their full potential.
+
+New summary:
+The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good because it will help humans reach their full potential.
+END OF EXAMPLE
+
+Current summary:
+{summary}
+
+New lines of conversation:
+{new_lines}
+
+New summary:"""
+SUMMARY_PROMPT = PromptTemplate(
+    input_variables=["summary", "new_lines"], template=_DEFAULT_SUMMARIZER_TEMPLATE
+)

langchain/chains/llm.py

@@ -4,9 +4,10 @@ from typing import Any, Dict, List
 from pydantic import BaseModel, Extra
 
 from langchain.chains.base import Chain
-from langchain.input import print_text
+from langchain.printing import print_text
 from langchain.llms.base import LLM
 from langchain.prompts.base import BasePromptTemplate
+from langchain.logger import CONTEXT_KEY
 
 
 class LLMChain(Chain, BaseModel):
@@ -15,7 +16,7 @@ class LLMChain(Chain, BaseModel):
     Example:
         .. code-block:: python
 
-            from langchain import LLMChain, OpenAI, Prompt
+            from langchain import LLMChain, OpenAI, PromptTemplate
             prompt_template = "Tell me a {adjective} joke"
             prompt = PromptTemplate(
                 input_variables=["adjective"], template=prompt_template
@@ -54,9 +55,9 @@ class LLMChain(Chain, BaseModel):
     def _call(self, inputs: Dict[str, Any]) -> Dict[str, str]:
         selected_inputs = {k: inputs[k] for k in self.prompt.input_variables}
         prompt = self.prompt.format(**selected_inputs)
-        if self.verbose:
-            print("Prompt after formatting:")
-            print_text(prompt, color="green", end="\n")
+        if self.logger:
+            title="Prompt after formatting:"
+            self.logger.log(prompt, inputs[CONTEXT_KEY],title=title, color="green", end="\n")
         kwargs = {}
         if "stop" in inputs:
             kwargs["stop"] = inputs["stop"]

langchain/chains/llm_math/base.py

@@ -1,5 +1,5 @@
 """Chain that interprets a prompt and executes python code to do math."""
-from typing import Dict, List
+from typing import Dict, List, Any
 
 from pydantic import BaseModel, Extra
 
@@ -9,6 +9,7 @@ from langchain.chains.llm_math.prompt import PROMPT
 from langchain.chains.python import PythonChain
 from langchain.input import ChainedInput
 from langchain.llms.base import LLM
+from langchain.logger import CONTEXT_KEY
 
 
 class LLMMathChain(Chain, BaseModel):
@@ -48,10 +49,10 @@ class LLMMathChain(Chain, BaseModel):
         """
         return [self.output_key]
 
-    def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
+    def _call(self, inputs: Dict[str, Any]) -> Dict[str, str]:
         llm_executor = LLMChain(prompt=PROMPT, llm=self.llm)
         python_executor = PythonChain()
-        chained_input = ChainedInput(inputs[self.input_key], verbose=self.verbose)
+        chained_input = ChainedInput(inputs[self.input_key], inputs[CONTEXT_KEY], logger=self.logger)
         t = llm_executor.predict(question=chained_input.input, stop=["```output"])
         chained_input.add(t, color="green")
         t = t.strip()

langchain/chains/natbot/prompt.py

@@ -2,7 +2,7 @@
 from langchain.prompts.prompt import PromptTemplate
 
 _PROMPT_TEMPLATE = """
-You are an agent controlling a browser. You are given:
+You are an agents controlling a browser. You are given:
 
 	(1) an objective that you are trying to achieve
 	(2) the URL of your current web page

langchain/chains/sequential.py

@@ -5,7 +5,6 @@ from typing import Dict, List
 from pydantic import BaseModel, Extra, root_validator
 
 from langchain.chains.base import Chain
-from langchain.input import get_color_mapping, print_text
 
 
 class SequentialChain(Chain, BaseModel):
@@ -127,11 +126,8 @@ class SimpleSequentialChain(Chain, BaseModel):
 
     def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
         _input = inputs[self.input_key]
-        color_mapping = get_color_mapping([str(i) for i in range(len(self.chains))])
         for i, chain in enumerate(self.chains):
             _input = chain.run(_input)
             if self.strip_outputs:
                 _input = _input.strip()
-            if self.verbose:
-                print_text(_input, color=color_mapping[str(i)], end="\n")
         return {self.output_key: _input}

langchain/chains/serpapi.py

@@ -111,5 +111,5 @@ class SerpAPIChain(Chain, BaseModel):
         elif "snippet" in res["organic_results"][0].keys():
             toret = res["organic_results"][0]["snippet"]
         else:
-            toret = None
+            toret = "No good search result found"
         return {self.output_key: toret}

langchain/chains/sql_database/base.py

@@ -1,5 +1,5 @@
 """Chain for interacting with SQL Database."""
-from typing import Dict, List
+from typing import Dict, List, Any
 
 from pydantic import BaseModel, Extra
 
@@ -9,6 +9,7 @@ from langchain.chains.sql_database.prompt import PROMPT
 from langchain.input import ChainedInput
 from langchain.llms.base import LLM
 from langchain.sql_database import SQLDatabase
+from langchain.logger import CONTEXT_KEY
 
 
 class SQLDatabaseChain(Chain, BaseModel):
@@ -51,10 +52,10 @@ class SQLDatabaseChain(Chain, BaseModel):
         """
         return [self.output_key]
 
-    def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
+    def _call(self, inputs: Dict[str, Any]) -> Dict[str, str]:
         llm_chain = LLMChain(llm=self.llm, prompt=PROMPT)
         chained_input = ChainedInput(
-            inputs[self.input_key] + "\nSQLQuery:", verbose=self.verbose
+            inputs[self.input_key] + "\nSQLQuery:", inputs[CONTEXT_KEY], logger=self.logger
         )
         llm_inputs = {
             "input": chained_input.input,

langchain/chains/vector_db_qa/base.py

@@ -5,8 +5,9 @@ from pydantic import BaseModel, Extra
 
 from langchain.chains.base import Chain
 from langchain.chains.llm import LLMChain
-from langchain.chains.vector_db_qa.prompt import prompt
+from langchain.chains.vector_db_qa.prompt import PROMPT
 from langchain.llms.base import LLM
+from langchain.prompts import PromptTemplate
 from langchain.vectorstores.base import VectorStore
 
 
@@ -29,6 +30,8 @@ class VectorDBQA(Chain, BaseModel):
     """Vector Database to connect to."""
     k: int = 4
     """Number of documents to query for."""
+    prompt: PromptTemplate = PROMPT
+    """Prompt to use when questioning the documents."""
     input_key: str = "query"  #: :meta private:
     output_key: str = "result"  #: :meta private:
 
@@ -56,7 +59,7 @@ class VectorDBQA(Chain, BaseModel):
 
     def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
         question = inputs[self.input_key]
-        llm_chain = LLMChain(llm=self.llm, prompt=prompt)
+        llm_chain = LLMChain(llm=self.llm, prompt=self.prompt)
         docs = self.vectorstore.similarity_search(question, k=self.k)
         contexts = []
         for j, doc in enumerate(docs):

langchain/chains/vector_db_qa/prompt.py

@@ -7,6 +7,6 @@ prompt_template = """Use the following pieces of context to answer the question
 
 Question: {question}
 Helpful Answer:"""
-prompt = PromptTemplate(
+PROMPT = PromptTemplate(
     template=prompt_template, input_variables=["context", "question"]
 )

langchain/docstore/__init__.py

@@ -1,4 +1,5 @@
 """Wrappers on top of docstores."""
+from langchain.docstore.in_memory import InMemoryDocstore
 from langchain.docstore.wikipedia import Wikipedia
 
-__all__ = ["Wikipedia"]
+__all__ = ["InMemoryDocstore", "Wikipedia"]

langchain/embeddings/__init__.py

@@ -1,6 +1,12 @@
 """Wrappers around embedding modules."""
 from langchain.embeddings.cohere import CohereEmbeddings
 from langchain.embeddings.huggingface import HuggingFaceEmbeddings
+from langchain.embeddings.huggingface_hub import HuggingFaceHubEmbeddings
 from langchain.embeddings.openai import OpenAIEmbeddings
 
-__all__ = ["OpenAIEmbeddings", "HuggingFaceEmbeddings", "CohereEmbeddings"]
+__all__ = [
+    "OpenAIEmbeddings",
+    "HuggingFaceEmbeddings",
+    "CohereEmbeddings",
+    "HuggingFaceHubEmbeddings",
+]

langchain/embeddings/huggingface.py

@@ -5,6 +5,8 @@ from pydantic import BaseModel, Extra
 
 from langchain.embeddings.base import Embeddings
 
+DEFAULT_MODEL_NAME = "sentence-transformers/all-mpnet-base-v2"
+
 
 class HuggingFaceEmbeddings(BaseModel, Embeddings):
     """Wrapper around sentence_transformers embedding models.
@@ -16,11 +18,11 @@ class HuggingFaceEmbeddings(BaseModel, Embeddings):
 
             from langchain.embeddings import HuggingFaceEmbeddings
             model_name = "sentence-transformers/all-mpnet-base-v2"
-            huggingface = HuggingFaceEmbeddings(model_name=model_name)
+            hf = HuggingFaceEmbeddings(model_name=model_name)
     """
 
     client: Any  #: :meta private:
-    model_name: str = "sentence-transformers/all-mpnet-base-v2"
+    model_name: str = DEFAULT_MODEL_NAME
     """Model name to use."""
 
     def __init__(self, **kwargs: Any):

langchain/embeddings/huggingface_hub.py

@@ -0,0 +1,105 @@
+"""Wrapper around HuggingFace Hub embedding models."""
+from typing import Any, Dict, List, Optional
+
+from pydantic import BaseModel, Extra, root_validator
+
+from langchain.embeddings.base import Embeddings
+from langchain.utils import get_from_dict_or_env
+
+DEFAULT_REPO_ID = "sentence-transformers/all-mpnet-base-v2"
+VALID_TASKS = ("feature-extraction",)
+
+
+class HuggingFaceHubEmbeddings(BaseModel, Embeddings):
+    """Wrapper around HuggingFaceHub embedding models.
+
+    To use, you should have the ``huggingface_hub`` python package installed, and the
+    environment variable ``HUGGINGFACEHUB_API_TOKEN`` set with your API token, or pass
+    it as a named parameter to the constructor.
+
+    Example:
+        .. code-block:: python
+
+            from langchain.embeddings import HuggingFaceHubEmbeddings
+            repo_id = "sentence-transformers/all-mpnet-base-v2"
+            hf = HuggingFaceHubEmbeddings(
+                repo_id=repo_id,
+                task="feature-extraction",
+                huggingfacehub_api_token="my-api-key",
+            )
+    """
+
+    client: Any  #: :meta private:
+    repo_id: str = DEFAULT_REPO_ID
+    """Model name to use."""
+    task: Optional[str] = "feature-extraction"
+    """Task to call the model with."""
+    model_kwargs: Optional[dict] = None
+    """Key word arguments to pass to the model."""
+
+    huggingfacehub_api_token: Optional[str] = None
+
+    class Config:
+        """Configuration for this pydantic object."""
+
+        extra = Extra.forbid
+
+    @root_validator()
+    def validate_environment(cls, values: Dict) -> Dict:
+        """Validate that api key and python package exists in environment."""
+        huggingfacehub_api_token = get_from_dict_or_env(
+            values, "huggingfacehub_api_token", "HUGGINGFACEHUB_API_TOKEN"
+        )
+        try:
+            from huggingface_hub.inference_api import InferenceApi
+
+            repo_id = values["repo_id"]
+            if not repo_id.startswith("sentence-transformers"):
+                raise ValueError(
+                    "Currently only 'sentence-transformers' embedding models "
+                    f"are supported. Got invalid 'repo_id' {repo_id}."
+                )
+            client = InferenceApi(
+                repo_id=repo_id,
+                token=huggingfacehub_api_token,
+                task=values.get("task"),
+            )
+            if client.task not in VALID_TASKS:
+                raise ValueError(
+                    f"Got invalid task {client.task}, "
+                    f"currently only {VALID_TASKS} are supported"
+                )
+            values["client"] = client
+        except ImportError:
+            raise ValueError(
+                "Could not import huggingface_hub python package. "
+                "Please it install it with `pip install huggingface_hub`."
+            )
+        return values
+
+    def embed_documents(self, texts: List[str]) -> List[List[float]]:
+        """Call out to HuggingFaceHub's embedding endpoint for embedding search docs.
+
+        Args:
+            texts: The list of texts to embed.
+
+        Returns:
+            List of embeddings, one for each text.
+        """
+        # replace newlines, which can negatively affect performance.
+        texts = [text.replace("\n", " ") for text in texts]
+        _model_kwargs = self.model_kwargs or {}
+        responses = self.client(inputs=texts, params=_model_kwargs)
+        return responses
+
+    def embed_query(self, text: str) -> List[float]:
+        """Call out to HuggingFaceHub's embedding endpoint for embedding query text.
+
+        Args:
+            text: The text to embed.
+
+        Returns:
+            Embeddings for the text.
+        """
+        response = self.embed_documents([text])[0]
+        return response

langchain/input.py

@@ -1,48 +1,24 @@
 """Handle chained inputs."""
-from typing import Dict, List, Optional
+from typing import Optional
 
-_TEXT_COLOR_MAPPING = {
-    "blue": "36;1",
-    "yellow": "33;1",
-    "pink": "38;5;200",
-    "green": "32;1",
-}
-
-
-def get_color_mapping(
-    items: List[str], excluded_colors: Optional[List] = None
-) -> Dict[str, str]:
-    """Get mapping for items to a support color."""
-    colors = list(_TEXT_COLOR_MAPPING.keys())
-    if excluded_colors is not None:
-        colors = [c for c in colors if c not in excluded_colors]
-    color_mapping = {item: colors[i % len(colors)] for i, item in enumerate(items)}
-    return color_mapping
-
-
-def print_text(text: str, color: Optional[str] = None, end: str = "") -> None:
-    """Print text with highlighting and no end characters."""
-    if color is None:
-        print(text, end=end)
-    else:
-        color_str = _TEXT_COLOR_MAPPING[color]
-        print(f"\u001b[{color_str}m\033[1;3m{text}\u001b[0m", end=end)
+from langchain.logger import Logger
 
 
 class ChainedInput:
     """Class for working with input that is the result of chains."""
 
-    def __init__(self, text: str, verbose: bool = False):
+    def __init__(self, text: str, context: dict, logger: Optional[Logger] = None):
         """Initialize with verbose flag and initial text."""
-        self._verbose = verbose
-        if self._verbose:
-            print_text(text, None)
+        self._logger = logger
+        if self._logger:
+            self._logger.log(text, context)
         self._input = text
+        self._context = context
 
     def add(self, text: str, color: Optional[str] = None) -> None:
         """Add text to input, print if in verbose mode."""
-        if self._verbose:
-            print_text(text, color)
+        if self._logger:
+            self._logger.log(text, self._context, color=color)
         self._input += text
 
     @property

langchain/llms/base.py

@@ -11,9 +11,9 @@ class LLM(ABC):
         """Run the LLM on the given prompt and input."""
 
     @property
-    @abstractmethod
     def _identifying_params(self) -> Mapping[str, Any]:
         """Get the identifying parameters."""
+        return {}
 
     def __str__(self) -> str:
         """Get a string representation of the object for printing."""

langchain/llms/huggingface_hub.py

@@ -51,7 +51,7 @@ class HuggingFaceHub(LLM, BaseModel):
         try:
             from huggingface_hub.inference_api import InferenceApi
 
-            repo_id = values.get("repo_id", DEFAULT_REPO_ID)
+            repo_id = values["repo_id"]
             client = InferenceApi(
                 repo_id=repo_id,
                 token=huggingfacehub_api_token,

langchain/llms/openai.py

@@ -1,7 +1,7 @@
 """Wrapper around OpenAI APIs."""
 from typing import Any, Dict, List, Mapping, Optional
 
-from pydantic import BaseModel, Extra, root_validator
+from pydantic import BaseModel, Extra, Field, root_validator
 
 from langchain.llms.base import LLM
 from langchain.utils import get_from_dict_or_env
@@ -13,6 +13,9 @@ class OpenAI(LLM, BaseModel):
     To use, you should have the ``openai`` python package installed, and the
     environment variable ``OPENAI_API_KEY`` set with your API key.
 
+    Any parameters that are valid to be passed to the openai.create call can be passed
+    in, even if not explicitly saved on this class.
+
     Example:
         .. code-block:: python
 
@@ -37,7 +40,8 @@ class OpenAI(LLM, BaseModel):
     """How many completions to generate for each prompt."""
     best_of: int = 1
     """Generates best_of completions server-side and returns the "best"."""
-
+    model_kwargs: Dict[str, Any] = Field(default_factory=dict)
+    """Holds any model parameters valid for `create` call not explicitly specified."""
     openai_api_key: Optional[str] = None
 
     class Config:
@@ -45,6 +49,20 @@ class OpenAI(LLM, BaseModel):
 
         extra = Extra.forbid
 
+    @root_validator(pre=True)
+    def build_extra(cls, values: Dict[str, Any]) -> Dict[str, Any]:
+        """Build extra kwargs from additional params that were passed in."""
+        all_required_field_names = {field.alias for field in cls.__fields__.values()}
+
+        extra = values.get("model_kwargs", {})
+        for field_name in list(values):
+            if field_name not in all_required_field_names:
+                if field_name in extra:
+                    raise ValueError(f"Found {field_name} supplied twice.")
+                extra[field_name] = values.pop(field_name)
+        values["model_kwargs"] = extra
+        return values
+
     @root_validator()
     def validate_environment(cls, values: Dict) -> Dict:
         """Validate that api key and python package exists in environment."""
@@ -66,7 +84,7 @@ class OpenAI(LLM, BaseModel):
     @property
     def _default_params(self) -> Mapping[str, Any]:
         """Get the default parameters for calling OpenAI API."""
-        return {
+        normal_params = {
             "temperature": self.temperature,
             "max_tokens": self.max_tokens,
             "top_p": self.top_p,
@@ -75,6 +93,7 @@ class OpenAI(LLM, BaseModel):
             "n": self.n,
             "best_of": self.best_of,
         }
+        return {**normal_params, **self.model_kwargs}
 
     @property
     def _identifying_params(self) -> Mapping[str, Any]:

langchain/logger.py

@@ -0,0 +1,70 @@
+from abc import ABC, abstractmethod
+from typing import Optional, Any
+from langchain.printing import print_text
+from pathlib import Path
+
+CONTEXT_KEY = "__context__"
+
+
+class Logger(ABC):
+
+    @abstractmethod
+    def log_start_of_chain(self, inputs):
+        """"""
+
+    @abstractmethod
+    def log_end_of_chain(self, outputs):
+        """"""
+
+    @abstractmethod
+    def log(self, text: str, context: dict, **kwargs):
+        """"""
+
+
+class PrintLogger(Logger):
+    def log_start_of_chain(self, inputs):
+        """"""
+        print("\n\n\033[1m> Entering new chain...\033[0m")
+
+    def log_end_of_chain(self, outputs):
+        """"""
+        print("\n\033[1m> Finished chain.\033[0m")
+
+    def log(self, text: str, context: dict, title: Optional[str ] =None ,**kwargs:Any):
+        """"""
+        if title is not None:
+            print(title)
+        print_text(text, **kwargs)
+
+import json
+class JSONLogger(Logger):
+
+    def __init__(self, log_dir):
+        self.log_dir = Path(log_dir)
+        self.log_dir.mkdir(exist_ok=True)
+
+    def log_start_of_chain(self, inputs):
+        """"""
+        fname = self.log_dir / f"{inputs[CONTEXT_KEY]['id']}.json"
+        if not fname.exists():
+            with open(fname, 'w') as f:
+                json.dump([], f)
+
+    def log_end_of_chain(self, outputs):
+        """"""
+        fname = self.log_dir / f"{outputs[CONTEXT_KEY]['id']}.json"
+        with open(fname) as f:
+            logs = json.load(f)
+        logs.append(outputs)
+        with open(fname, 'w') as f:
+            json.dump(logs, f)
+
+    def log(self, text: str, context: dict, title: Optional[str ] =None ,**kwargs:Any):
+        """"""
+        fname = self.log_dir / f"{context['id']}.json"
+        with open(fname) as f:
+            logs = json.load(f)
+        logs.append({"text": text, "title": title})
+        with open(fname, 'w') as f:
+            json.dump(logs, f)
+

langchain/model_laboratory.py

@@ -1,9 +1,10 @@
 """Experiment with different models."""
-from typing import List, Optional, Sequence
+from typing import List, Optional, Sequence, Union
 
+from langchain.agents.agent import Agent
 from langchain.chains.base import Chain
 from langchain.chains.llm import LLMChain
-from langchain.input import get_color_mapping, print_text
+from langchain.printing import print_text, get_color_mapping
 from langchain.llms.base import LLM
 from langchain.prompts.prompt import PromptTemplate
 
@@ -11,29 +12,32 @@ from langchain.prompts.prompt import PromptTemplate
 class ModelLaboratory:
     """Experiment with different models."""
 
-    def __init__(self, chains: Sequence[Chain], names: Optional[List[str]] = None):
+    def __init__(
+        self, chains: Sequence[Union[Chain, Agent]], names: Optional[List[str]] = None
+    ):
         """Initialize with chains to experiment with.
 
         Args:
             chains: list of chains to experiment with.
         """
-        if not isinstance(chains[0], Chain):
-            raise ValueError(
-                "ModelLaboratory should now be initialized with Chains. "
-                "If you want to initialize with LLMs, use the `from_llms` method "
-                "instead (`ModelLaboratory.from_llms(...)`)"
-            )
         for chain in chains:
-            if len(chain.input_keys) != 1:
+            if not isinstance(chain, (Chain, Agent)):
                 raise ValueError(
-                    "Currently only support chains with one input variable, "
-                    f"got {chain.input_keys}"
-                )
-            if len(chain.output_keys) != 1:
-                raise ValueError(
-                    "Currently only support chains with one output variable, "
-                    f"got {chain.output_keys}"
+                    "ModelLaboratory should now be initialized with Chains or Agents. "
+                    "If you want to initialize with LLMs, use the `from_llms` method "
+                    "instead (`ModelLaboratory.from_llms(...)`)"
                 )
+            if isinstance(chain, Chain):
+                if len(chain.input_keys) != 1:
+                    raise ValueError(
+                        "Currently only support chains with one input variable, "
+                        f"got {chain.input_keys}"
+                    )
+                if len(chain.output_keys) != 1:
+                    raise ValueError(
+                        "Currently only support chains with one output variable, "
+                        f"got {chain.output_keys}"
+                    )
         if names is not None:
             if len(names) != len(chains):
                 raise ValueError("Length of chains does not match length of names.")

langchain/printing.py

@@ -0,0 +1,29 @@
+from typing import Optional, List, Dict
+
+
+def print_text(text: str, color: Optional[str] = None, end: str = "") -> None:
+    """Print text with highlighting and no end characters."""
+    if color is None:
+        print(text, end=end)
+    else:
+        color_str = _TEXT_COLOR_MAPPING[color]
+        print(f"\u001b[{color_str}m\033[1;3m{text}\u001b[0m", end=end)
+
+
+def get_color_mapping(
+    items: List[str], excluded_colors: Optional[List] = None
+) -> Dict[str, str]:
+    """Get mapping for items to a support color."""
+    colors = list(_TEXT_COLOR_MAPPING.keys())
+    if excluded_colors is not None:
+        colors = [c for c in colors if c not in excluded_colors]
+    color_mapping = {item: colors[i % len(colors)] for i, item in enumerate(items)}
+    return color_mapping
+
+
+_TEXT_COLOR_MAPPING = {
+    "blue": "36;1",
+    "yellow": "33;1",
+    "pink": "38;5;200",
+    "green": "32;1",
+}

langchain/prompts/base.py

@@ -1,6 +1,11 @@
 """BasePrompt schema definition."""
+import json
 from abc import ABC, abstractmethod
-from typing import Any, List
+from pathlib import Path
+from typing import Any, Dict, List, Union
+
+import yaml
+from pydantic import BaseModel, root_validator
 
 from langchain.formatting import formatter
 
@@ -27,12 +32,22 @@ def check_valid_template(
         raise ValueError("Invalid prompt schema.")
 
 
-class BasePromptTemplate(ABC):
+class BasePromptTemplate(BaseModel, ABC):
     """Base prompt should expose the format method, returning a prompt."""
 
     input_variables: List[str]
     """A list of the names of the variables the prompt template expects."""
 
+    @root_validator()
+    def validate_variable_names(cls, values: Dict) -> Dict:
+        """Validate variable names do not restricted names."""
+        if "stop" in values["input_variables"]:
+            raise ValueError(
+                "Cannot have an input variable named 'stop', as it is used internally,"
+                " please rename."
+            )
+        return values
+
     @abstractmethod
     def format(self, **kwargs: Any) -> str:
         """Format the prompt with the inputs.
@@ -49,3 +64,39 @@ class BasePromptTemplate(ABC):
 
             prompt.format(variable1="foo")
         """
+
+    def _prompt_dict(self) -> Dict:
+        """Return a dictionary of the prompt."""
+        return self.dict()
+
+    def save(self, file_path: Union[Path, str]) -> None:
+        """Save the prompt.
+
+        Args:
+            file_path: Path to directory to save prompt to.
+
+        Example:
+        .. code-block:: python
+
+            prompt.save(file_path="path/prompt.yaml")
+        """
+        # Convert file to Path object.
+        if isinstance(file_path, str):
+            save_path = Path(file_path)
+        else:
+            save_path = file_path
+
+        directory_path = save_path.parent
+        directory_path.mkdir(parents=True, exist_ok=True)
+
+        # Fetch dictionary to save
+        prompt_dict = self._prompt_dict()
+
+        if save_path.suffix == ".json":
+            with open(file_path, "w") as f:
+                json.dump(prompt_dict, f, indent=4)
+        elif save_path.suffix == ".yaml":
+            with open(file_path, "w") as f:
+                yaml.dump(prompt_dict, f, default_flow_style=False)
+        else:
+            raise ValueError(f"{save_path} must be json or yaml")

langchain/prompts/example_selector/base.py

@@ -6,6 +6,10 @@ from typing import Dict, List
 class BaseExampleSelector(ABC):
     """Interface for selecting examples to include in prompts."""
 
+    @abstractmethod
+    def add_example(self, example: Dict[str, str]) -> None:
+        """Add new example to store for a key."""
+
     @abstractmethod
     def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:
         """Select which examples to use based on the inputs."""

langchain/prompts/example_selector/length_based.py

@@ -25,6 +25,12 @@ class LengthBasedExampleSelector(BaseExampleSelector, BaseModel):
 
     example_text_lengths: List[int] = []  #: :meta private:
 
+    def add_example(self, example: Dict[str, str]) -> None:
+        """Add new example to list."""
+        self.examples.append(example)
+        string_example = self.example_prompt.format(**example)
+        self.example_text_lengths.append(self.get_text_length(string_example))
+
     @validator("example_text_lengths", always=True)
     def calculate_example_text_lengths(cls, v: List[int], values: Dict) -> List[int]:
         """Calculate text lengths if they don't exist."""

langchain/prompts/example_selector/semantic_similarity.py

@@ -24,6 +24,11 @@ class SemanticSimilarityExampleSelector(BaseExampleSelector, BaseModel):
         extra = Extra.forbid
         arbitrary_types_allowed = True
 
+    def add_example(self, example: Dict[str, str]) -> None:
+        """Add new example to vectorstore."""
+        string_example = " ".join(example.values())
+        self.vectorstore.add_texts([string_example], metadatas=[example])
+
     def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:
         """Select which examples to use based on semantic similarity."""
         # Get the docs with the highest similarity.

langchain/prompts/few_shot.py

@@ -108,3 +108,12 @@ class FewShotPromptTemplate(BasePromptTemplate, BaseModel):
         template = self.example_separator.join([piece for piece in pieces if piece])
         # Format the template with the input variables.
         return DEFAULT_FORMATTER_MAPPING[self.template_format](template, **kwargs)
+
+    def _prompt_dict(self) -> Dict:
+        """Return a dictionary of the prompt."""
+        if self.example_selector:
+            raise ValueError("Saving an example selector is not currently supported")
+
+        prompt_dict = self.dict()
+        prompt_dict["_type"] = "few_shot"
+        return prompt_dict

langchain/prompts/prompt.py

@@ -10,7 +10,7 @@ from langchain.prompts.base import (
 )
 
 
-class PromptTemplate(BaseModel, BasePromptTemplate):
+class PromptTemplate(BasePromptTemplate, BaseModel):
     """Schema to represent a prompt for an LLM.
 
     Example:

langchain/routing_chains/__init__.py

@@ -1,18 +0,0 @@
-"""Routing chains."""
-from langchain.routing_chains.loading import load_routing_chain
-from langchain.routing_chains.mrkl.base import MRKLChain
-from langchain.routing_chains.react.base import ReActChain
-from langchain.routing_chains.router import LLMRouter
-from langchain.routing_chains.routing_chain import RoutingChain
-from langchain.routing_chains.self_ask_with_search.base import SelfAskWithSearchChain
-from langchain.routing_chains.tools import Tool
-
-__all__ = [
-    "MRKLChain",
-    "SelfAskWithSearchChain",
-    "ReActChain",
-    "LLMRouter",
-    "RoutingChain",
-    "Tool",
-    "load_routing_chain",
-]

langchain/routing_chains/loading.py

@@ -1,43 +0,0 @@
-"""Load routing chains."""
-from typing import Any, List
-
-from langchain.llms.base import LLM
-from langchain.routing_chains.mrkl.base import ZeroShotRouter
-from langchain.routing_chains.react.base import ReActDocstoreRouter
-from langchain.routing_chains.routing_chain import RoutingChain
-from langchain.routing_chains.self_ask_with_search.base import SelfAskWithSearchRouter
-from langchain.routing_chains.tools import Tool
-
-ROUTER_TYPE_TO_CLASS = {
-    "zero-shot-react-description": ZeroShotRouter,
-    "react-docstore": ReActDocstoreRouter,
-    "self-ask-with-search": SelfAskWithSearchRouter,
-}
-
-
-def load_routing_chain(
-    tools: List[Tool],
-    llm: LLM,
-    router_type: str = "zero-shot-react-description",
-    **kwargs: Any,
-) -> RoutingChain:
-    """Load routing chain given tools and LLM.
-
-    Args:
-        tools: List of tools this routing chain has access to.
-        llm: Language model to use as the router.
-        router_type: The router to use. Valid options are:
-            `zero-shot-react-description`.
-        **kwargs: Additional key word arguments to pass to the routing chain.
-
-    Returns:
-        A routing chain.
-    """
-    if router_type not in ROUTER_TYPE_TO_CLASS:
-        raise ValueError(
-            f"Got unknown router type: {router_type}. "
-            f"Valid types are: {ROUTER_TYPE_TO_CLASS.keys()}."
-        )
-    router_cls = ROUTER_TYPE_TO_CLASS[router_type]
-    router = router_cls.from_llm_and_tools(llm, tools)
-    return RoutingChain(router=router, tools=tools, **kwargs)

langchain/routing_chains/router.py

@@ -1,97 +0,0 @@
-"""Chain that takes in an input and produces an action and action input."""
-from abc import ABC, abstractmethod
-from typing import List, NamedTuple, Optional, Tuple
-
-from pydantic import BaseModel
-
-from langchain.chains.llm import LLMChain
-from langchain.llms.base import LLM
-from langchain.routing_chains.tools import Tool
-
-
-class RouterOutput(NamedTuple):
-    """Output of a router."""
-
-    tool: str
-    tool_input: str
-    log: str
-
-
-class Router(ABC):
-    """Chain responsible for deciding the action to take."""
-
-    @abstractmethod
-    def route(self, text: str) -> RouterOutput:
-        """Given input, decided how to route it.
-
-        Args:
-            text: input string
-
-        Returns:
-            RouterOutput specifying what tool to use.
-        """
-
-    @property
-    @abstractmethod
-    def observation_prefix(self) -> str:
-        """Prefix to append the observation with."""
-
-    @property
-    @abstractmethod
-    def router_prefix(self) -> str:
-        """Prefix to append the router call with."""
-
-    @property
-    def finish_tool_name(self) -> str:
-        """Name of the tool to use to finish the chain."""
-        return "Final Answer"
-
-    @property
-    def starter_string(self) -> str:
-        """Put this string after user input but before first router call."""
-        return "\n"
-
-
-class LLMRouter(Router, BaseModel, ABC):
-    """Router that uses an LLM."""
-
-    llm_chain: LLMChain
-
-    @abstractmethod
-    def _extract_tool_and_input(self, text: str) -> Optional[Tuple[str, str]]:
-        """Extract tool and tool input from llm output."""
-
-    def _fix_text(self, text: str) -> str:
-        """Fix the text."""
-        raise ValueError("fix_text not implemented for this router.")
-
-    @property
-    def _stop(self) -> List[str]:
-        return [f"\n{self.observation_prefix}"]
-
-    @classmethod
-    @abstractmethod
-    def from_llm_and_tools(cls, llm: LLM, tools: List[Tool]) -> "Router":
-        """Construct a router from an LLM and tools."""
-
-    def route(self, text: str) -> RouterOutput:
-        """Given input, decided how to route it.
-
-        Args:
-            text: input string
-
-        Returns:
-            RouterOutput specifying what tool to use.
-        """
-        input_key = self.llm_chain.input_keys[0]
-        inputs = {input_key: text, "stop": self._stop}
-        full_output = self.llm_chain.predict(**inputs)
-        parsed_output = self._extract_tool_and_input(full_output)
-        while parsed_output is None:
-            full_output = self._fix_text(full_output)
-            inputs = {input_key: text + full_output, "stop": self._stop}
-            output = self.llm_chain.predict(**inputs)
-            full_output += output
-            parsed_output = self._extract_tool_and_input(full_output)
-        tool, tool_input = parsed_output
-        return RouterOutput(tool, tool_input, full_output)

langchain/routing_chains/routing_chain.py

@@ -1,81 +0,0 @@
-"""Router-Expert framework."""
-from typing import Dict, List
-
-from pydantic import BaseModel, Extra
-
-from langchain.chains.base import Chain
-from langchain.input import ChainedInput, get_color_mapping
-from langchain.routing_chains.router import Router
-from langchain.routing_chains.tools import Tool
-
-
-class RoutingChain(Chain, BaseModel):
-    """Chain that uses a router to use tools."""
-
-    router: Router
-    """Router to use."""
-    tools: List[Tool]
-    """Tools this chain has access to."""
-    input_key: str = "question"  #: :meta private:
-    output_key: str = "answer"  #: :meta private:
-
-    class Config:
-        """Configuration for this pydantic object."""
-
-        extra = Extra.forbid
-        arbitrary_types_allowed = True
-
-    @property
-    def input_keys(self) -> List[str]:
-        """Expect input key.
-
-        :meta private:
-        """
-        return [self.input_key]
-
-    @property
-    def output_keys(self) -> List[str]:
-        """Expect output key.
-
-        :meta private:
-        """
-        return [self.output_key]
-
-    def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
-        # Construct a mapping of tool name to tool for easy lookup
-        name_to_tool_map = {tool.name: tool.func for tool in self.tools}
-        # Construct the initial string to pass into the router. This is made up
-        # of the user input, the special starter string, and then the router prefix.
-        # The starter string is a special string that may be used by a router to
-        # immediately follow the user input. The router prefix is a string that
-        # prompts the router to start routing.
-        starter_string = (
-            inputs[self.input_key]
-            + self.router.starter_string
-            + self.router.router_prefix
-        )
-        # We use the ChainedInput class to iteratively add to the input over time.
-        chained_input = ChainedInput(starter_string, verbose=self.verbose)
-        # We construct a mapping from each tool to a color, used for logging.
-        color_mapping = get_color_mapping(
-            [tool.name for tool in self.tools], excluded_colors=["green"]
-        )
-        # We now enter the router loop (until it returns something).
-        while True:
-            # Call the router to see what to do.
-            output = self.router.route(chained_input.input)
-            # Add the log to the Chained Input.
-            chained_input.add(output.log, color="green")
-            # If the tool chosen is the finishing tool, then we end and return.
-            if output.tool == self.router.finish_tool_name:
-                return {self.output_key: output.tool_input}
-            # Otherwise we lookup the tool
-            chain = name_to_tool_map[output.tool]
-            # We then call the tool on the tool input to get an observation
-            observation = chain(output.tool_input)
-            # We then log the observation
-            chained_input.add(f"\n{self.router.observation_prefix}")
-            chained_input.add(observation, color=color_mapping[output.tool])
-            # We then add the router prefix into the prompt to get the router to start
-            # thinking, and start the loop all over.
-            chained_input.add(f"\n{self.router.router_prefix}")

langchain/vectorstores/base.py

@@ -10,7 +10,9 @@ class VectorStore(ABC):
     """Interface for vector stores."""
 
     @abstractmethod
-    def add_texts(self, texts: Iterable[str]) -> None:
+    def add_texts(
+        self, texts: Iterable[str], metadatas: Optional[List[dict]] = None
+    ) -> None:
         """Run more texts through the embeddings and add to the vectorstore."""
 
     @abstractmethod

langchain/vectorstores/elastic_vector_search.py

@@ -65,7 +65,9 @@ class ElasticVectorSearch(VectorStore):
             )
         self.client = es_client
 
-    def add_texts(self, texts: Iterable[str]) -> None:
+    def add_texts(
+        self, texts: Iterable[str], metadatas: Optional[List[dict]] = None
+    ) -> None:
         """Run more texts through the embeddings and add to the vectorstore."""
         try:
             from elasticsearch.helpers import bulk
@@ -76,11 +78,13 @@ class ElasticVectorSearch(VectorStore):
             )
         requests = []
         for i, text in enumerate(texts):
+            metadata = metadatas[i] if metadatas else {}
             request = {
                 "_op_type": "index",
                 "_index": self.index_name,
                 "vector": self.embedding_function(text),
                 "text": text,
+                "metadata": metadata,
             }
             requests.append(request)
         bulk(self.client, requests)

langchain/vectorstores/faiss.py

@@ -37,7 +37,9 @@ class FAISS(VectorStore):
         self.docstore = docstore
         self.index_to_docstore_id = index_to_docstore_id
 
-    def add_texts(self, texts: Iterable[str]) -> None:
+    def add_texts(
+        self, texts: Iterable[str], metadatas: Optional[List[dict]] = None
+    ) -> None:
         """Run more texts through the embeddings and add to the vectorstore."""
         if not isinstance(self.docstore, AddableMixin):
             raise ValueError(
@@ -46,7 +48,10 @@ class FAISS(VectorStore):
             )
         # Embed and create the documents.
         embeddings = [self.embedding_function(text) for text in texts]
-        documents = [Document(page_content=text) for text in texts]
+        documents = []
+        for i, text in enumerate(texts):
+            metadata = metadatas[i] if metadatas else {}
+            documents.append(Document(page_content=text, metadata=metadata))
         # Add to the index, the index_to_id mapping, and the docstore.
         starting_len = len(self.index_to_docstore_id)
         self.index.add(np.array(embeddings, dtype=np.float32))

setup.py

@@ -6,7 +6,7 @@ from setuptools import find_packages, setup
 with open(Path(__file__).absolute().parents[0] / "langchain" / "VERSION") as _f:
     __version__ = _f.read().strip()
 
-with open("README.md", "r") as f:
+with open("README.md", "r", encoding="utf-8") as f:
     long_description = f.read()
 
 LLM_DEPENDENCIES = ["cohere", "openai", "nlpcloud", "huggingface_hub"]

tests/integration_tests/chains/test_react.py

@@ -1,8 +1,8 @@
 """Integration test for self ask with search."""
 
+from langchain.agents.react.base import ReActChain
 from langchain.docstore.wikipedia import Wikipedia
 from langchain.llms.openai import OpenAI
-from langchain.routing_chains.react.base import ReActChain
 
 
 def test_react() -> None:

tests/integration_tests/chains/test_self_ask_with_search.py

@@ -1,7 +1,7 @@
 """Integration test for self ask with search."""
+from langchain.agents.self_ask_with_search.base import SelfAskWithSearchChain
 from langchain.chains.serpapi import SerpAPIChain
 from langchain.llms.openai import OpenAI
-from langchain.routing_chains.self_ask_with_search.base import SelfAskWithSearchChain
 
 
 def test_self_ask_with_search() -> None:

tests/integration_tests/embeddings/test_huggingface_hub.py

@@ -0,0 +1,28 @@
+"""Test HuggingFaceHub embeddings."""
+import pytest
+
+from langchain.embeddings import HuggingFaceHubEmbeddings
+
+
+def test_huggingfacehub_embedding_documents() -> None:
+    """Test huggingfacehub embeddings."""
+    documents = ["foo bar"]
+    embedding = HuggingFaceHubEmbeddings()
+    output = embedding.embed_documents(documents)
+    assert len(output) == 1
+    assert len(output[0]) == 768
+
+
+def test_huggingfacehub_embedding_query() -> None:
+    """Test huggingfacehub embeddings."""
+    document = "foo bar"
+    embedding = HuggingFaceHubEmbeddings()
+    output = embedding.embed_query(document)
+    assert len(output) == 768
+
+
+def test_huggingfacehub_embedding_invalid_repo() -> None:
+    """Test huggingfacehub embedding repo id validation."""
+    # Only sentence-transformers models are currently supported.
+    with pytest.raises(ValueError):
+        HuggingFaceHubEmbeddings(repo_id="allenai/specter")

tests/integration_tests/llms/test_openai.py

@@ -1,5 +1,7 @@
 """Test OpenAI API wrapper."""
 
+import pytest
+
 from langchain.llms.openai import OpenAI
 
 
@@ -8,3 +10,19 @@ def test_openai_call() -> None:
     llm = OpenAI(max_tokens=10)
     output = llm("Say foo:")
     assert isinstance(output, str)
+
+
+def test_openai_extra_kwargs() -> None:
+    """Test extra kwargs to openai."""
+    # Check that foo is saved in extra_kwargs.
+    llm = OpenAI(foo=3, max_tokens=10)
+    assert llm.max_tokens == 10
+    assert llm.model_kwargs == {"foo": 3}
+
+    # Test that if extra_kwargs are provided, they are added to it.
+    llm = OpenAI(foo=3, model_kwargs={"bar": 2})
+    assert llm.model_kwargs == {"foo": 3, "bar": 2}
+
+    # Test that if provided twice it errors
+    with pytest.raises(ValueError):
+        OpenAI(foo=3, model_kwargs={"foo": 2})

tests/unit_tests/agents/__init__.py

@@ -0,0 +1 @@
+"""Test agent functionality."""

tests/unit_tests/agents/test_mrkl.py

@@ -2,10 +2,10 @@
 
 import pytest
 
+from langchain.agents.mrkl.base import ZeroShotAgent, get_action_and_input
+from langchain.agents.mrkl.prompt import FORMAT_INSTRUCTIONS, PREFIX, SUFFIX
+from langchain.agents.tools import Tool
 from langchain.prompts import PromptTemplate
-from langchain.routing_chains.mrkl.base import ZeroShotRouter, get_action_and_input
-from langchain.routing_chains.mrkl.prompt import BASE_TEMPLATE
-from langchain.routing_chains.tools import Tool
 from tests.unit_tests.llms.fake_llm import FakeLLM
 
 
@@ -56,12 +56,17 @@ def test_from_chains() -> None:
         Tool(name="foo", func=lambda x: "foo", description="foobar1"),
         Tool(name="bar", func=lambda x: "bar", description="foobar2"),
     ]
-    router_chain = ZeroShotRouter.from_llm_and_tools(FakeLLM(), chain_configs)
+    agent = ZeroShotAgent.from_llm_and_tools(FakeLLM(), chain_configs)
     expected_tools_prompt = "foo: foobar1\nbar: foobar2"
     expected_tool_names = "foo, bar"
-    expected_template = BASE_TEMPLATE.format(
-        tools=expected_tools_prompt, tool_names=expected_tool_names
+    expected_template = "\n\n".join(
+        [
+            PREFIX,
+            expected_tools_prompt,
+            FORMAT_INSTRUCTIONS.format(tool_names=expected_tool_names),
+            SUFFIX,
+        ]
     )
-    prompt = router_chain.llm_chain.prompt
+    prompt = agent.llm_chain.prompt
     assert isinstance(prompt, PromptTemplate)
     assert prompt.template == expected_template

tests/unit_tests/agents/test_react.py

@@ -4,12 +4,12 @@ from typing import Any, List, Mapping, Optional, Union
 
 import pytest
 
+from langchain.agents.react.base import ReActChain, ReActDocstoreAgent
+from langchain.agents.tools import Tool
 from langchain.docstore.base import Docstore
 from langchain.docstore.document import Document
 from langchain.llms.base import LLM
 from langchain.prompts.prompt import PromptTemplate
-from langchain.routing_chains.react.base import ReActChain, ReActDocstoreRouter
-from langchain.routing_chains.tools import Tool
 
 _PAGE_CONTENT = """This is a page about LangChain.
 
@@ -57,8 +57,8 @@ def test_predict_until_observation_normal() -> None:
         Tool("Search", lambda x: x),
         Tool("Lookup", lambda x: x),
     ]
-    router_chain = ReActDocstoreRouter.from_llm_and_tools(fake_llm, tools)
-    output = router_chain.route("")
+    agent = ReActDocstoreAgent.from_llm_and_tools(fake_llm, tools)
+    output = agent.get_action("")
     assert output.log == outputs[0]
     assert output.tool == "Search"
     assert output.tool_input == "foo"
@@ -72,8 +72,8 @@ def test_predict_until_observation_repeat() -> None:
         Tool("Search", lambda x: x),
         Tool("Lookup", lambda x: x),
     ]
-    router_chain = ReActDocstoreRouter.from_llm_and_tools(fake_llm, tools)
-    output = router_chain.route("")
+    agent = ReActDocstoreAgent.from_llm_and_tools(fake_llm, tools)
+    output = agent.get_action("")
     assert output.log == "foo\nAction 1: Search[foo]"
     assert output.tool == "Search"
     assert output.tool_input == "foo"
@@ -88,9 +88,8 @@ def test_react_chain() -> None:
     ]
     fake_llm = FakeListLLM(responses)
     react_chain = ReActChain(llm=fake_llm, docstore=FakeDocstore())
-    inputs = {"question": "when was langchain made"}
-    output = react_chain(inputs)
-    assert output["answer"] == "2022"
+    output = react_chain.run("when was langchain made")
+    assert output == "2022"
 
 
 def test_react_chain_bad_action() -> None:

tests/unit_tests/chains/test_conversation.py

@@ -0,0 +1,68 @@
+"""Test conversation chain and memory."""
+import pytest
+
+from langchain.chains.base import Memory
+from langchain.chains.conversation.base import ConversationChain
+from langchain.chains.conversation.memory import (
+    ConversationBufferMemory,
+    ConversationSummaryMemory,
+)
+from langchain.prompts.prompt import PromptTemplate
+from tests.unit_tests.llms.fake_llm import FakeLLM
+
+
+def test_conversation_chain_works() -> None:
+    """Test that conversation chain works in basic setting."""
+    llm = FakeLLM()
+    prompt = PromptTemplate(input_variables=["foo", "bar"], template="{foo} {bar}")
+    memory = ConversationBufferMemory(memory_key="foo")
+    chain = ConversationChain(llm=llm, prompt=prompt, memory=memory, input_key="bar")
+    chain.run("foo")
+
+
+def test_conversation_chain_errors_bad_prompt() -> None:
+    """Test that conversation chain works in basic setting."""
+    llm = FakeLLM()
+    prompt = PromptTemplate(input_variables=[], template="nothing here")
+    with pytest.raises(ValueError):
+        ConversationChain(llm=llm, prompt=prompt)
+
+
+def test_conversation_chain_errors_bad_variable() -> None:
+    """Test that conversation chain works in basic setting."""
+    llm = FakeLLM()
+    prompt = PromptTemplate(input_variables=["foo"], template="{foo}")
+    memory = ConversationBufferMemory(memory_key="foo")
+    with pytest.raises(ValueError):
+        ConversationChain(llm=llm, prompt=prompt, memory=memory, input_key="foo")
+
+
+@pytest.mark.parametrize(
+    "memory",
+    [
+        ConversationBufferMemory(memory_key="baz"),
+        ConversationSummaryMemory(llm=FakeLLM(), memory_key="baz"),
+    ],
+)
+def test_conversation_memory(memory: Memory) -> None:
+    """Test basic conversation memory functionality."""
+    # This is a good input because the input is not the same as baz.
+    good_inputs = {"foo": "bar", "baz": "foo"}
+    # This is a good output because these is one variable.
+    good_outputs = {"bar": "foo"}
+    memory.save_context(good_inputs, good_outputs)
+    # This is a bad input because there are two variables that aren't the same as baz.
+    bad_inputs = {"foo": "bar", "foo1": "bar"}
+    with pytest.raises(ValueError):
+        memory.save_context(bad_inputs, good_outputs)
+    # This is a bad input because the only variable is the same as baz.
+    bad_inputs = {"baz": "bar"}
+    with pytest.raises(ValueError):
+        memory.save_context(bad_inputs, good_outputs)
+    # This is a bad output because it is empty.
+    with pytest.raises(ValueError):
+        memory.save_context(good_inputs, {})
+    # This is a bad output because there are two keys.
+    bad_outputs = {"foo": "bar", "foo1": "bar"}
+    with pytest.raises(ValueError):
+        memory.save_context(good_inputs, bad_outputs)

tests/unit_tests/prompts/test_length_based_example_selector.py

@@ -1,4 +1,4 @@
-"""Test functionality related to dynamic prompts."""
+"""Test functionality related to length based selector."""
 import pytest
 
 from langchain.prompts.example_selector.length_based import LengthBasedExampleSelector
@@ -22,25 +22,34 @@ def selector() -> LengthBasedExampleSelector:
     return selector
 
 
-def test_dynamic_prompt_valid(selector: LengthBasedExampleSelector) -> None:
-    """Test dynamic prompt can be successfully constructed from examples."""
+def test_selector_valid(selector: LengthBasedExampleSelector) -> None:
+    """Test LengthBasedExampleSelector can select examples.."""
     short_question = "Short question?"
     output = selector.select_examples({"question": short_question})
     assert output == EXAMPLES
 
 
-def test_dynamic_prompt_trims_one_example(selector: LengthBasedExampleSelector) -> None:
-    """Test dynamic prompt can trim one example."""
+def test_selector_add_example(selector: LengthBasedExampleSelector) -> None:
+    """Test LengthBasedExampleSelector can add an example."""
+    new_example = {"question": "Question: what are you?\nAnswer: bar"}
+    selector.add_example(new_example)
+    short_question = "Short question?"
+    output = selector.select_examples({"question": short_question})
+    assert output == EXAMPLES + [new_example]
+
+
+def test_selector_trims_one_example(selector: LengthBasedExampleSelector) -> None:
+    """Test LengthBasedExampleSelector can trim one example."""
     long_question = """I am writing a really long question,
     this probably is going to affect the example right?"""
     output = selector.select_examples({"question": long_question})
     assert output == EXAMPLES[:1]
 
 
-def test_dynamic_prompt_trims_all_examples(
+def test_selector_trims_all_examples(
     selector: LengthBasedExampleSelector,
 ) -> None:
-    """Test dynamic prompt can trim all examples."""
+    """Test LengthBasedExampleSelector can trim all examples."""
     longest_question = """This question is super super super,
     super super super super super super super super super super super,
     super super super super long, this will affect the example right?"""

tests/unit_tests/prompts/test_loading.py

@@ -43,6 +43,34 @@ def test_loading_from_JSON() -> None:
         assert prompt == expected_prompt
 
 
+def test_saving_loading_round_trip(tmp_path: Path) -> None:
+    """Test equality when saving and loading a prompt."""
+    simple_prompt = PromptTemplate(
+        input_variables=["adjective", "content"],
+        template="Tell me a {adjective} joke about {content}.",
+    )
+    simple_prompt.save(file_path=tmp_path / "prompt.yaml")
+    loaded_prompt = load_prompt(tmp_path / "prompt.yaml")
+    assert loaded_prompt == simple_prompt
+
+    few_shot_prompt = FewShotPromptTemplate(
+        input_variables=["adjective"],
+        prefix="Write antonyms for the following words.",
+        example_prompt=PromptTemplate(
+            input_variables=["input", "output"],
+            template="Input: {input}\nOutput: {output}",
+        ),
+        examples=[
+            {"input": "happy", "output": "sad"},
+            {"input": "tall", "output": "short"},
+        ],
+        suffix="Input: {adjective}\nOutput:",
+    )
+    few_shot_prompt.save(file_path=tmp_path / "few_shot.yaml")
+    loaded_prompt = load_prompt(tmp_path / "few_shot.yaml")
+    assert loaded_prompt == few_shot_prompt
+
+
 def test_loading_with_template_as_file() -> None:
     """Test loading when the template is a file."""
     with change_directory():

tests/unit_tests/routing_chains/__init__.py

@@ -1 +0,0 @@
-"""Test routing chain functionality."""

tests/unit_tests/test_input.py

@@ -3,7 +3,8 @@
 import sys
 from io import StringIO
 
-from langchain.input import ChainedInput, get_color_mapping
+from langchain.input import ChainedInput
+from langchain.printing import get_color_mapping
 
 
 def test_chained_input_not_verbose() -> None: