bump 256 (#8870)

This adds support for [Xata](https://xata.io) (data platform based on
Postgres) as a vector store. We have recently added [Xata to
Langchain.js](https://github.com/hwchase17/langchainjs/pull/2125) and
would love to have the equivalent in the Python project as well.

The PR includes integration tests and a Jupyter notebook as docs. Please
let me know if anything else would be needed or helpful.

I have added the xata python SDK as an optional dependency.

## To run the integration tests

You will need to create a DB in xata (see the docs), then run something
like:

```
OPENAI_API_KEY=sk-... XATA_API_KEY=xau_... XATA_DB_URL='https://....xata.sh/db/langchain'  poetry run pytest tests/integration_tests/vectorstores/test_xata.py
```

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
- Tag maintainer: for a quicker response, tag the relevant maintainer
(see below),
- Twitter handle: we announce bigger features on Twitter. If your PR
gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before
submitting. Run `make format`, `make lint` and `make test` to check this
locally.

If you're adding a new integration, please include:
1. a test for the integration, preferably unit tests that do not rely on
network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the
same people again.

See contribution guidelines for more information on how to write/run
tests, lint, etc:
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

---------

Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>
Co-authored-by: Philip Krauss <35487337+philkra@users.noreply.github.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>

Makefile

@@ -43,7 +43,12 @@ spell_fix:
 
 help:
 	@echo '----'
-	@echo 'coverage                     - run unit tests and generate coverage report'
+	@echo 'clean                        - run docs_clean and api_docs_clean'
 	@echo 'docs_build                   - build the documentation'
 	@echo 'docs_clean                   - clean the documentation build artifacts'
 	@echo 'docs_linkcheck               - run linkchecker on the documentation'
+	@echo 'api_docs_build               - build the API Reference documentation'
+	@echo 'api_docs_clean               - clean the API Reference documentation build artifacts'
+	@echo 'api_docs_linkcheck           - run linkchecker on the API Reference documentation'
+	@echo 'spell_check               	- run codespell on the project'
+	@echo 'spell_fix               		- run codespell on the project and fix the errors'

README.md

@@ -18,8 +18,8 @@
 
 Looking for the JS/TS version? Check out [LangChain.js](https://github.com/hwchase17/langchainjs).
 
-**Production Support:** As you move your LangChains into production, we'd love to offer more comprehensive support.
-Please fill out [this form](https://6w1pwbss0py.typeform.com/to/rrbrdTH2) and we'll set up a dedicated support Slack channel.
+**Production Support:** As you move your LangChains into production, we'd love to offer more hands-on support.
+Fill out [this form](https://airtable.com/appwQzlErAS2qiP0L/shrGtGaVBVAz7NcV2) to share more about what you're building, and our team will get in touch.
 
 ## 🚨Breaking Changes for select chains (SQLDatabase) on 7/28
 

docs/api_reference/conf.py

@@ -100,6 +100,9 @@ extensions = [
 ]
 source_suffix = [".rst"]
 
+# some autodoc pydantic options are repeated in the actual template.
+# potentially user error, but there may be bugs in the sphinx extension
+# with options not being passed through correctly (from either the location in the code)
 autodoc_pydantic_model_show_json = False
 autodoc_pydantic_field_list_validators = False
 autodoc_pydantic_config_members = False
@@ -112,13 +115,6 @@ autodoc_member_order = "groupwise"
 autoclass_content = "both"
 autodoc_typehints_format = "short"
 
-autodoc_default_options = {
-    "members": True,
-    "show-inheritance": True,
-    "inherited-members": "BaseModel",
-    "undoc-members": True,
-    "special-members": "__call__",
-}
 # autodoc_typehints = "description"
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["templates"]

docs/api_reference/create_api_rst.py

@@ -1,49 +1,209 @@
-"""Script for auto-generating api_reference.rst"""
-import glob
-import re
+"""Script for auto-generating api_reference.rst."""
+import importlib
+import inspect
+import typing
 from pathlib import Path
+from typing import TypedDict, Sequence, List, Dict, Literal, Union
+from enum import Enum
+
+from pydantic import BaseModel
 
 ROOT_DIR = Path(__file__).parents[2].absolute()
+HERE = Path(__file__).parent
+
 PKG_DIR = ROOT_DIR / "libs" / "langchain" / "langchain"
 EXP_DIR = ROOT_DIR / "libs" / "experimental" / "langchain_experimental"
-WRITE_FILE = Path(__file__).parent / "api_reference.rst"
-EXP_WRITE_FILE = Path(__file__).parent / "experimental_api_reference.rst"
+WRITE_FILE = HERE / "api_reference.rst"
+EXP_WRITE_FILE = HERE / "experimental_api_reference.rst"
 
 
-def load_members(dir: Path) -> dict:
-    members: dict = {}
-    for py in glob.glob(str(dir) + "/**/*.py", recursive=True):
-        module = py[len(str(dir)) + 1 :].replace(".py", "").replace("/", ".")
-        top_level = module.split(".")[0]
-        if top_level not in members:
-            members[top_level] = {"classes": [], "functions": []}
-        with open(py, "r") as f:
-            for line in f.readlines():
-                cls = re.findall(r"^class ([^_].*)\(", line)
-                members[top_level]["classes"].extend([module + "." + c for c in cls])
-                func = re.findall(r"^def ([^_].*)\(", line)
-                afunc = re.findall(r"^async def ([^_].*)\(", line)
-                func_strings = [module + "." + f for f in func + afunc]
-                members[top_level]["functions"].extend(func_strings)
-    return members
+ClassKind = Literal["TypedDict", "Regular", "Pydantic", "enum"]
 
 
-def construct_doc(pkg: str, members: dict) -> str:
+class ClassInfo(TypedDict):
+    """Information about a class."""
+
+    name: str
+    """The name of the class."""
+    qualified_name: str
+    """The fully qualified name of the class."""
+    kind: ClassKind
+    """The kind of the class."""
+    is_public: bool
+    """Whether the class is public or not."""
+
+
+class FunctionInfo(TypedDict):
+    """Information about a function."""
+
+    name: str
+    """The name of the function."""
+    qualified_name: str
+    """The fully qualified name of the function."""
+    is_public: bool
+    """Whether the function is public or not."""
+
+
+class ModuleMembers(TypedDict):
+    """A dictionary of module members."""
+
+    classes_: Sequence[ClassInfo]
+    functions: Sequence[FunctionInfo]
+
+
+def _load_module_members(module_path: str, namespace: str) -> ModuleMembers:
+    """Load all members of a module.
+
+    Args:
+        module_path: Path to the module.
+        namespace: the namespace of the module.
+
+    Returns:
+        list: A list of loaded module objects.
+    """
+    classes_: List[ClassInfo] = []
+    functions: List[FunctionInfo] = []
+    module = importlib.import_module(module_path)
+    for name, type_ in inspect.getmembers(module):
+        if not hasattr(type_, "__module__"):
+            continue
+        if type_.__module__ != module_path:
+            continue
+
+        if inspect.isclass(type_):
+            if type(type_) == typing._TypedDictMeta:  # type: ignore
+                kind: ClassKind = "TypedDict"
+            elif issubclass(type_, Enum):
+                kind = "enum"
+            elif issubclass(type_, BaseModel):
+                kind = "Pydantic"
+            else:
+                kind = "Regular"
+
+            classes_.append(
+                ClassInfo(
+                    name=name,
+                    qualified_name=f"{namespace}.{name}",
+                    kind=kind,
+                    is_public=not name.startswith("_"),
+                )
+            )
+        elif inspect.isfunction(type_):
+            functions.append(
+                FunctionInfo(
+                    name=name,
+                    qualified_name=f"{namespace}.{name}",
+                    is_public=not name.startswith("_"),
+                )
+            )
+        else:
+            continue
+
+    return ModuleMembers(
+        classes_=classes_,
+        functions=functions,
+    )
+
+
+def _merge_module_members(
+    module_members: Sequence[ModuleMembers],
+) -> ModuleMembers:
+    """Merge module members."""
+    classes_: List[ClassInfo] = []
+    functions: List[FunctionInfo] = []
+    for module in module_members:
+        classes_.extend(module["classes_"])
+        functions.extend(module["functions"])
+
+    return ModuleMembers(
+        classes_=classes_,
+        functions=functions,
+    )
+
+
+def _load_package_modules(
+    package_directory: Union[str, Path]
+) -> Dict[str, ModuleMembers]:
+    """Recursively load modules of a package based on the file system.
+
+    Traversal based on the file system makes it easy to determine which
+    of the modules/packages are part of the package vs. 3rd party or built-in.
+
+    Parameters:
+        package_directory: Path to the package directory.
+
+    Returns:
+        list: A list of loaded module objects.
+    """
+    package_path = (
+        Path(package_directory)
+        if isinstance(package_directory, str)
+        else package_directory
+    )
+    modules_by_namespace = {}
+
+    package_name = package_path.name
+
+    for file_path in package_path.rglob("*.py"):
+        if not file_path.name.startswith("__"):
+            relative_module_name = file_path.relative_to(package_path)
+            # Get the full namespace of the module
+            namespace = str(relative_module_name).replace(".py", "").replace("/", ".")
+            # Keep only the top level namespace
+            top_namespace = namespace.split(".")[0]
+
+            try:
+                module_members = _load_module_members(
+                    f"{package_name}.{namespace}", namespace
+                )
+                # Merge module members if the namespace already exists
+                if top_namespace in modules_by_namespace:
+                    existing_module_members = modules_by_namespace[top_namespace]
+                    _module_members = _merge_module_members(
+                        [existing_module_members, module_members]
+                    )
+                else:
+                    _module_members = module_members
+
+                modules_by_namespace[top_namespace] = _module_members
+
+            except ImportError as e:
+                print(f"Error: Unable to import module '{namespace}' with error: {e}")
+
+    return modules_by_namespace
+
+
+def _construct_doc(pkg: str, members_by_namespace: Dict[str, ModuleMembers]) -> str:
+    """Construct the contents of the reference.rst file for the given package.
+
+    Args:
+        pkg: The package name
+        members_by_namespace: The members of the package, dict organized by top level
+                              module contains a list of classes and functions
+                              inside of the top level namespace.
+
+    Returns:
+        The contents of the reference.rst file.
+    """
     full_doc = f"""\
-=============
+=======================
 ``{pkg}`` API Reference
-=============
+=======================
 
 """
-    for module, _members in sorted(members.items(), key=lambda kv: kv[0]):
-        classes = _members["classes"]
+    namespaces = sorted(members_by_namespace)
+
+    for module in namespaces:
+        _members = members_by_namespace[module]
+        classes = _members["classes_"]
         functions = _members["functions"]
         if not (classes or functions):
             continue
         section = f":mod:`{pkg}.{module}`"
+        underline = "=" * (len(section) + 1)
         full_doc += f"""\
 {section}
-{'=' * (len(section) + 1)}
+{underline}
 
 .. automodule:: {pkg}.{module}
     :no-members:
@@ -52,7 +212,6 @@ def construct_doc(pkg: str, members: dict) -> str:
 """
 
         if classes:
-            cstring = "\n    ".join(sorted(classes))
             full_doc += f"""\
 Classes
 --------------
@@ -60,13 +219,31 @@ Classes
 
 .. autosummary::
     :toctree: {module}
-    :template: class.rst
-
-    {cstring}
-
 """
+
+            for class_ in classes:
+                if not class_['is_public']:
+                    continue
+                    
+                if class_["kind"] == "TypedDict":
+                    template = "typeddict.rst"
+                elif class_["kind"] == "enum":
+                    template = "enum.rst"
+                elif class_["kind"] == "Pydantic":
+                    template = "pydantic.rst"
+                else:
+                    template = "class.rst"
+
+                full_doc += f"""\
+    :template: {template}
+    
+    {class_["qualified_name"]}
+    
+"""
+
         if functions:
-            fstring = "\n    ".join(sorted(functions))
+            _functions = [f["qualified_name"] for f in functions if f["is_public"]]
+            fstring = "\n    ".join(sorted(_functions))
             full_doc += f"""\
 Functions
 --------------
@@ -83,12 +260,15 @@ Functions
 
 
 def main() -> None:
-    lc_members = load_members(PKG_DIR)
-    lc_doc = ".. _api_reference:\n\n" + construct_doc("langchain", lc_members)
+    """Generate the reference.rst file for each package."""
+    lc_members = _load_package_modules(PKG_DIR)
+    lc_doc = ".. _api_reference:\n\n" + _construct_doc("langchain", lc_members)
     with open(WRITE_FILE, "w") as f:
         f.write(lc_doc)
-    exp_members = load_members(EXP_DIR)
-    exp_doc = ".. _experimental_api_reference:\n\n" + construct_doc("langchain_experimental", exp_members)
+    exp_members = _load_package_modules(EXP_DIR)
+    exp_doc = ".. _experimental_api_reference:\n\n" + _construct_doc(
+        "langchain_experimental", exp_members
+    )
     with open(EXP_WRITE_FILE, "w") as f:
         f.write(exp_doc)
 

docs/api_reference/requirements.txt

@@ -1,4 +1,5 @@
 -e libs/langchain
+-e libs/experimental
 autodoc_pydantic==1.8.0
 myst_parser
 nbsphinx==0.8.9
@@ -10,4 +11,4 @@ sphinx-panels
 toml
 myst_nb
 sphinx_copybutton
-pydata-sphinx-theme==0.13.1
+pydata-sphinx-theme==0.13.1

docs/api_reference/templates/class.rst

@@ -5,17 +5,6 @@
 
 .. autoclass:: {{ objname }}
 
-   {% block methods %}
-   {% if methods %}
-   .. rubric:: {{ _('Methods') }}
-
-   .. autosummary::
-   {% for item in methods %}
-      ~{{ name }}.{{ item }}
-   {%- endfor %}
-   {% endif %}
-   {% endblock %}
-
    {% block attributes %}
    {% if attributes %}
    .. rubric:: {{ _('Attributes') }}
@@ -27,4 +16,21 @@
    {% endif %}
    {% endblock %}
 
+   {% block methods %}
+   {% if methods %}
+   .. rubric:: {{ _('Methods') }}
+
+   .. autosummary::
+   {% for item in methods %}
+      ~{{ name }}.{{ item }}
+   {%- endfor %}
+
+   {% for item in methods %}
+   .. automethod:: {{ name }}.{{ item }}
+   {%- endfor %}
+
+   {% endif %}
+   {% endblock %}
+
+
 .. example_links:: {{ objname }}

docs/api_reference/templates/enum.rst

@@ -0,0 +1,14 @@
+:mod:`{{module}}`.{{objname}}
+{{ underline }}==============
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+
+    {% block attributes %}
+    {% for item in attributes %}
+    .. autoattribute:: {{ item }}
+    {% endfor %}
+    {% endblock %}
+
+.. example_links:: {{ objname }}

docs/api_reference/templates/pydantic.rst

@@ -0,0 +1,22 @@
+:mod:`{{module}}`.{{objname}}
+{{ underline }}==============
+
+.. currentmodule:: {{ module }}
+
+.. autopydantic_model:: {{ objname }}
+    :model-show-json: False
+    :model-show-config-summary: False
+    :model-show-validator-members: False
+    :model-show-field-summary: False
+    :field-signature-prefix: param
+    :members:
+    :undoc-members:
+    :inherited-members:
+    :member-order: groupwise
+    :show-inheritance: True
+    :special-members: __call__
+
+    {% block attributes %}
+    {% endblock %}
+
+.. example_links:: {{ objname }}

docs/api_reference/templates/typeddict.rst

@@ -0,0 +1,14 @@
+:mod:`{{module}}`.{{objname}}
+{{ underline }}==============
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+
+    {% block attributes %}
+   {% for item in attributes %}
+  .. autoattribute:: {{ item }}
+   {% endfor %}
+   {% endblock %}
+
+.. example_links:: {{ objname }}

docs/api_reference/themes/scikit-learn-modern/layout.html

@@ -19,7 +19,7 @@
   {% block htmltitle %}
   <title>{{ title|striptags|e }}{{ titlesuffix }}</title>
   {% endblock %}
-  <link rel="canonical" href="http://scikit-learn.org/stable/{{pagename}}.html" />
+  <link rel="canonical" href="https://api.python.langchain.com/en/latest/{{pagename}}.html" />
 
   {% if favicon_url %}
   <link rel="shortcut icon" href="{{ favicon_url|e }}"/>

docs/api_reference/themes/scikit-learn-modern/nav.html

@@ -6,17 +6,6 @@
   {%- set top_container_cls = "sk-landing-container" %}
 {%- endif %}
 
-{% if theme_link_to_live_contributing_page|tobool %}
-{# Link to development page for live builds #}
-  {%- set development_link = "https://scikit-learn.org/dev/developers/index.html" %}
-{# Open on a new development page in new window/tab for live builds #}
-  {%- set development_attrs = 'target="_blank" rel="noopener noreferrer"' %}
-{%- else %}
-  {%- set development_link = pathto('developers/index') %}
-  {%- set development_attrs = '' %}
-{%- endif %}
-
-
 <nav id="navbar" class="{{ nav_bar_class }} navbar navbar-expand-md navbar-light bg-light py-0">
   <div class="container-fluid {{ top_container_cls }} px-0">
     {%- if logo_url %}

docs/docs_skeleton/docs/modules/agents/agent_types/index.mdx

@@ -28,7 +28,7 @@ navigating around a browser.
 ### [OpenAI Functions](/docs/modules/agents/agent_types/openai_functions_agent.html)
 
 Certain OpenAI models (like gpt-3.5-turbo-0613 and gpt-4-0613) have been explicitly fine-tuned to detect when a
-function should to be called and respond with the inputs that should be passed to the function.
+function should be called and respond with the inputs that should be passed to the function.
 The OpenAI Functions Agent is designed to work with these models.
 
 ### [Conversational](/docs/modules/agents/agent_types/chat_conversation_agent.html)

docs/docs_skeleton/docs/modules/agents/agent_types/openai_functions_agent.mdx

@@ -1,6 +1,6 @@
 # OpenAI functions
 
-Certain OpenAI models (like gpt-3.5-turbo-0613 and gpt-4-0613) have been fine-tuned to detect when a function should to be called and respond with the inputs that should be passed to the function.
+Certain OpenAI models (like gpt-3.5-turbo-0613 and gpt-4-0613) have been fine-tuned to detect when a function should be called and respond with the inputs that should be passed to the function.
 In an API call, you can describe functions and have the model intelligently choose to output a JSON object containing arguments to call those functions.
 The goal of the OpenAI Function APIs is to more reliably return valid and useful function calls than a generic text completion or chat API.
 

docs/docs_skeleton/docs/modules/model_io/prompts/index.mdx

@@ -3,10 +3,12 @@ sidebar_position: 0
 ---
 # Prompts
 
-The new way of programming models is through prompts.
-A **prompt** refers to the input to the model.
-This input is often constructed from multiple components.
-LangChain provides several classes and functions to make constructing and working with prompts easy.
+A prompt for a language model is a set of instructions or input provided by a user to
+guide the model's response, helping it understand the context and generate relevant
+and coherent language-based output, such as answering questions, completing sentences,
+or engaging in a conversation.
 
-- [Prompt templates](/docs/modules/model_io/prompts/prompt_templates/): Parametrize model inputs
+LangChain provides several classes and functions to help construct and work with prompts.
+
+- [Prompt templates](/docs/modules/model_io/prompts/prompt_templates/): Parametrized model inputs
 - [Example selectors](/docs/modules/model_io/prompts/example_selectors/): Dynamically select examples to include in prompts

docs/docs_skeleton/docs/modules/model_io/prompts/prompt_templates/index.mdx

@@ -4,18 +4,15 @@ sidebar_position: 0
 
 # Prompt templates
 
-Language models take text as input - that text is commonly referred to as a prompt.
-Typically this is not simply a hardcoded string but rather a combination of a template, some examples, and user input.
-LangChain provides several classes and functions to make constructing and working with prompts easy.
+Prompt templates are pre-defined recipes for generating prompts for language models.
 
-## What is a prompt template?
+A template may include instructions, few shot examples, and specific context and
+questions appropriate for a given task.
 
-A prompt template refers to a reproducible way to generate a prompt. It contains a text string ("the template"), that can take in a set of parameters from the end user and generates a prompt.
+LangChain provides tooling to create and work with prompt templates.
 
-A prompt template can contain:
-- instructions to the language model,
-- a set of few shot examples to help the language model generate a better response,
-- a question to the language model.
+LangChain strives to create model agnostic templates to make it easy to reuse
+existing templates across different language models.
 
 import GetStarted from "@snippets/modules/model_io/prompts/prompt_templates/get_started.mdx"
 

docs/docs_skeleton/docs/use_cases/summarization/summarize.mdx

@@ -1,8 +0,0 @@
-# Summarization
-
-A summarization chain can be used to summarize multiple documents. One way is to input multiple smaller documents, after they have been divided into chunks, and operate over them with a MapReduceDocumentsChain. You can also choose instead for the chain that does summarization to be a StuffDocumentsChain, or a RefineDocumentsChain.
-
-import Example from "@snippets/modules/chains/popular/summarize.mdx"
-
-<Example/>
-

docs/docs_skeleton/package-lock.json

@@ -12,7 +12,7 @@
         "@docusaurus/preset-classic": "2.4.0",
         "@docusaurus/remark-plugin-npm2yarn": "^2.4.0",
         "@mdx-js/react": "^1.6.22",
-        "@mendable/search": "^0.0.125",
+        "@mendable/search": "^0.0.137",
         "clsx": "^1.2.1",
         "json-loader": "^0.5.7",
         "process": "^0.11.10",
@@ -3212,10 +3212,11 @@
       }
     },
     "node_modules/@mendable/search": {
-      "version": "0.0.125",
-      "resolved": "https://registry.npmjs.org/@mendable/search/-/search-0.0.125.tgz",
-      "integrity": "sha512-Mb1J3zDhOyBZV9cXqJocSOBNYGpe8+LQDqd9n9laPWxosSJcSTUewqtlIbMerrYsScBsxskoSiWgRsc7xF5z0Q==",
+      "version": "0.0.137",
+      "resolved": "https://registry.npmjs.org/@mendable/search/-/search-0.0.137.tgz",
+      "integrity": "sha512-2J2fd5eqToK+mLzrSDA6NAr4F1kfql7QRiHpD7AUJJX0nqpvInhr/mMJKBCUSCv2z76UKCmF5wLuPSw+C90Qdg==",
       "dependencies": {
+        "html-react-parser": "^4.2.0",
         "posthog-js": "^1.45.1"
       },
       "peerDependencies": {
@@ -8332,6 +8333,33 @@
         "safe-buffer": "~5.1.0"
       }
     },
+    "node_modules/html-dom-parser": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/html-dom-parser/-/html-dom-parser-4.0.0.tgz",
+      "integrity": "sha512-TUa3wIwi80f5NF8CVWzkopBVqVAtlawUzJoLwVLHns0XSJGynss4jiY0mTWpiDOsuyw+afP+ujjMgRh9CoZcXw==",
+      "dependencies": {
+        "domhandler": "5.0.3",
+        "htmlparser2": "9.0.0"
+      }
+    },
+    "node_modules/html-dom-parser/node_modules/htmlparser2": {
+      "version": "9.0.0",
+      "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-9.0.0.tgz",
+      "integrity": "sha512-uxbSI98wmFT/G4P2zXx4OVx04qWUmyFPrD2/CNepa2Zo3GPNaCaaxElDgwUrwYWkK1nr9fft0Ya8dws8coDLLQ==",
+      "funding": [
+        "https://github.com/fb55/htmlparser2?sponsor=1",
+        {
+          "type": "github",
+          "url": "https://github.com/sponsors/fb55"
+        }
+      ],
+      "dependencies": {
+        "domelementtype": "^2.3.0",
+        "domhandler": "^5.0.3",
+        "domutils": "^3.1.0",
+        "entities": "^4.5.0"
+      }
+    },
     "node_modules/html-entities": {
       "version": "2.4.0",
       "resolved": "https://registry.npmjs.org/html-entities/-/html-entities-2.4.0.tgz",
@@ -8375,6 +8403,20 @@
         "node": ">= 12"
       }
     },
+    "node_modules/html-react-parser": {
+      "version": "4.2.0",
+      "resolved": "https://registry.npmjs.org/html-react-parser/-/html-react-parser-4.2.0.tgz",
+      "integrity": "sha512-gzU55AS+FI6qD7XaKe5BLuLFM2Xw0/LodfMWZlxV9uOHe7LCD5Lukx/EgYuBI3c0kLu0XlgFXnSzO0qUUn3Vrg==",
+      "dependencies": {
+        "domhandler": "5.0.3",
+        "html-dom-parser": "4.0.0",
+        "react-property": "2.0.0",
+        "style-to-js": "1.1.3"
+      },
+      "peerDependencies": {
+        "react": "0.14 || 15 || 16 || 17 || 18"
+      }
+    },
     "node_modules/html-tags": {
       "version": "3.3.1",
       "resolved": "https://registry.npmjs.org/html-tags/-/html-tags-3.3.1.tgz",
@@ -11762,6 +11804,11 @@
         "webpack": ">=4.41.1 || 5.x"
       }
     },
+    "node_modules/react-property": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/react-property/-/react-property-2.0.0.tgz",
+      "integrity": "sha512-kzmNjIgU32mO4mmH5+iUyrqlpFQhF8K2k7eZ4fdLSOPFrD1XgEuSBv9LDEgxRXTMBqMd8ppT0x6TIzqE5pdGdw=="
+    },
     "node_modules/react-router": {
       "version": "5.3.4",
       "resolved": "https://registry.npmjs.org/react-router/-/react-router-5.3.4.tgz",
@@ -13127,6 +13174,22 @@
         "url": "https://github.com/sponsors/sindresorhus"
       }
     },
+    "node_modules/style-to-js": {
+      "version": "1.1.3",
+      "resolved": "https://registry.npmjs.org/style-to-js/-/style-to-js-1.1.3.tgz",
+      "integrity": "sha512-zKI5gN/zb7LS/Vm0eUwjmjrXWw8IMtyA8aPBJZdYiQTXj4+wQ3IucOLIOnF7zCHxvW8UhIGh/uZh/t9zEHXNTQ==",
+      "dependencies": {
+        "style-to-object": "0.4.1"
+      }
+    },
+    "node_modules/style-to-js/node_modules/style-to-object": {
+      "version": "0.4.1",
+      "resolved": "https://registry.npmjs.org/style-to-object/-/style-to-object-0.4.1.tgz",
+      "integrity": "sha512-HFpbb5gr2ypci7Qw+IOhnP2zOU7e77b+rzM+wTzXzfi1PrtBCX0E7Pk4wL4iTLnhzZ+JgEGAhX81ebTg/aYjQw==",
+      "dependencies": {
+        "inline-style-parser": "0.1.1"
+      }
+    },
     "node_modules/style-to-object": {
       "version": "0.3.0",
       "resolved": "https://registry.npmjs.org/style-to-object/-/style-to-object-0.3.0.tgz",

docs/docs_skeleton/package.json

@@ -23,7 +23,7 @@
     "@docusaurus/preset-classic": "2.4.0",
     "@docusaurus/remark-plugin-npm2yarn": "^2.4.0",
     "@mdx-js/react": "^1.6.22",
-    "@mendable/search": "^0.0.125",
+    "@mendable/search": "^0.0.137",
     "clsx": "^1.2.1",
     "json-loader": "^0.5.7",
     "process": "^0.11.10",

docs/docs_skeleton/vercel.json

@@ -3478,7 +3478,7 @@
     },
     {
       "source": "/en/latest/modules/prompts/example_selectors.html",
-      "destination": "/docs/modules/model_io/example_selectors"
+      "destination": "/docs/modules/model_io/prompts/example_selectors"
     },
     {
       "source": "/en/latest/modules/prompts/example_selectors/examples/custom_example_selector.html",
@@ -3494,7 +3494,7 @@
     },
     {
       "source": "/en/latest/modules/prompts/prompt_templates.html",
-      "destination": "/docs/modules/model_io/prompt_templates"
+      "destination": "/docs/modules/model_io/prompts/prompt_templates"
     },
     {
       "source": "/en/latest/modules/prompts/prompt_templates/examples/connecting_to_a_feature_store.html",
@@ -3951,6 +3951,10 @@
     {
       "source": "/docs/modules/chains/additional/tagging",
       "destination": "/docs/use_cases/tagging"
+    },
+    {
+      "source": "docs/integrations/providers/agent_with_wandb_tracing",
+      "destination": "docs/integrations/providers/wandb_tracing"
     }
   ]
 }

docs/extras/guides/expression_language/cookbook.ipynb

@@ -25,16 +25,7 @@
    "execution_count": 1,
    "id": "466b65b3",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.14) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.\n",
-      "  warnings.warn(\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from langchain.prompts import ChatPromptTemplate\n",
     "from langchain.chat_models import ChatOpenAI"
@@ -42,7 +33,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 11,
    "id": "3c634ef0",
    "metadata": {},
    "outputs": [],
@@ -180,9 +171,7 @@
    "cell_type": "code",
    "execution_count": 9,
    "id": "decf7710",
-   "metadata": {
-    "scrolled": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -211,7 +200,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 10,
    "id": "f799664d",
    "metadata": {},
    "outputs": [],
@@ -356,7 +345,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 12,
    "id": "5d3d8ffe",
    "metadata": {},
    "outputs": [],
@@ -377,7 +366,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 2,
    "id": "33be32af",
    "metadata": {},
    "outputs": [],
@@ -389,7 +378,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 3,
    "id": "df3f3fa2",
    "metadata": {},
    "outputs": [],
@@ -401,7 +390,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
+   "execution_count": 16,
    "id": "bfc47ec1",
    "metadata": {},
    "outputs": [],
@@ -416,7 +405,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 24,
+   "execution_count": 17,
    "id": "eae31755",
    "metadata": {},
    "outputs": [],
@@ -431,11 +420,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 25,
+   "execution_count": 18,
    "id": "f3040b0c",
-   "metadata": {
-    "scrolled": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "name": "stderr",
@@ -450,7 +437,7 @@
        "'Harrison worked at Kensho.'"
       ]
      },
-     "execution_count": 25,
+     "execution_count": 18,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -461,7 +448,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 27,
+   "execution_count": 19,
    "id": "e1d20c7c",
    "metadata": {},
    "outputs": [],
@@ -484,11 +471,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 28,
+   "execution_count": 20,
    "id": "7ee8b2d4",
-   "metadata": {
-    "scrolled": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "name": "stderr",
@@ -503,7 +488,7 @@
        "'Harrison ha lavorato a Kensho.'"
       ]
      },
-     "execution_count": 28,
+     "execution_count": 20,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -512,6 +497,317 @@
     "chain.invoke({\"question\": \"where did harrison work\", \"language\": \"italian\"})"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "f007669c",
+   "metadata": {},
+   "source": [
+    "## Conversational Retrieval Chain\n",
+    "\n",
+    "We can easily add in conversation history. This primarily means adding in chat_message_history"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "3f30c348",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.schema.runnable import RunnableMap\n",
+    "from langchain.schema import format_document"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "64ab1dbf",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts.prompt import PromptTemplate\n",
+    "\n",
+    "_template = \"\"\"Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, in its original language.\n",
+    "\n",
+    "Chat History:\n",
+    "{chat_history}\n",
+    "Follow Up Input: {question}\n",
+    "Standalone question:\"\"\"\n",
+    "CONDENSE_QUESTION_PROMPT = PromptTemplate.from_template(_template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "7d628c97",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "template = \"\"\"Answer the question based only on the following context:\n",
+    "{context}\n",
+    "\n",
+    "Question: {question}\n",
+    "\"\"\"\n",
+    "ANSWER_PROMPT = ChatPromptTemplate.from_template(template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "f60a5d0f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "DEFAULT_DOCUMENT_PROMPT = PromptTemplate.from_template(template=\"{page_content}\")\n",
+    "def _combine_documents(docs, document_prompt = DEFAULT_DOCUMENT_PROMPT, document_separator=\"\\n\\n\"):\n",
+    "    doc_strings = [format_document(doc, document_prompt) for doc in docs]\n",
+    "    return document_separator.join(doc_strings)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "7d007db6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import Tuple, List\n",
+    "def _format_chat_history(chat_history: List[Tuple]) -> str:\n",
+    "    buffer = \"\"\n",
+    "    for dialogue_turn in chat_history:\n",
+    "        human = \"Human: \" + dialogue_turn[0]\n",
+    "        ai = \"Assistant: \" + dialogue_turn[1]\n",
+    "        buffer += \"\\n\" + \"\\n\".join([human, ai])\n",
+    "    return buffer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "5c32cc89",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "_inputs = RunnableMap(\n",
+    "    {\n",
+    "        \"standalone_question\": {\n",
+    "            \"question\": lambda x: x[\"question\"],\n",
+    "            \"chat_history\": lambda x: _format_chat_history(x['chat_history'])\n",
+    "        } | CONDENSE_QUESTION_PROMPT | ChatOpenAI(temperature=0) | StrOutputParser(),\n",
+    "    }\n",
+    ")\n",
+    "_context = {\n",
+    "    \"context\": itemgetter(\"standalone_question\") | retriever | _combine_documents,\n",
+    "    \"question\": lambda x: x[\"standalone_question\"]\n",
+    "}\n",
+    "conversational_qa_chain = _inputs | _context | ANSWER_PROMPT | ChatOpenAI()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "135c8205",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Number of requested results 4 is greater than number of elements in index 1, updating n_results = 1\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='Harrison was employed at Kensho.', additional_kwargs={}, example=False)"
+      ]
+     },
+     "execution_count": 17,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversational_qa_chain.invoke({\n",
+    "    \"question\": \"where did harrison work?\",\n",
+    "    \"chat_history\": [],\n",
+    "})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "424e7e7a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Number of requested results 4 is greater than number of elements in index 1, updating n_results = 1\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='Harrison worked at Kensho.', additional_kwargs={}, example=False)"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversational_qa_chain.invoke({\n",
+    "    \"question\": \"where did he work?\",\n",
+    "    \"chat_history\": [(\"Who wrote this notebook?\", \"Harrison\")],\n",
+    "})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c5543183",
+   "metadata": {},
+   "source": [
+    "### With Memory and returning source documents\n",
+    "\n",
+    "This shows how to use memory with the above. For memory, we need to manage that outside at the memory. For returning the retrieved documents, we just need to pass them through all the way."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "e31dd17c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.memory import ConversationBufferMemory"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 44,
+   "id": "d4bffe94",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "memory = ConversationBufferMemory(return_messages=True, output_key=\"answer\", input_key=\"question\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 45,
+   "id": "733be985",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# First we add a step to load memory\n",
+    "# This needs to be a RunnableMap because its the first input\n",
+    "loaded_memory = RunnableMap(\n",
+    "    {\n",
+    "        \"question\": itemgetter(\"question\"),\n",
+    "        \"memory\": memory.load_memory_variables,\n",
+    "    }\n",
+    ")\n",
+    "# Next we add a step to expand memory into the variables\n",
+    "expanded_memory = {\n",
+    "    \"question\": itemgetter(\"question\"),\n",
+    "    \"chat_history\": lambda x: x[\"memory\"][\"history\"]\n",
+    "}\n",
+    "\n",
+    "# Now we calculate the standalone question\n",
+    "standalone_question = {\n",
+    "    \"standalone_question\": {\n",
+    "        \"question\": lambda x: x[\"question\"],\n",
+    "        \"chat_history\": lambda x: _format_chat_history(x['chat_history'])\n",
+    "    } | CONDENSE_QUESTION_PROMPT | ChatOpenAI(temperature=0) | StrOutputParser(),\n",
+    "}\n",
+    "# Now we retrieve the documents\n",
+    "retrieved_documents = {\n",
+    "    \"docs\": itemgetter(\"standalone_question\") | retriever,\n",
+    "    \"question\": lambda x: x[\"standalone_question\"]\n",
+    "}\n",
+    "# Now we construct the inputs for the final prompt\n",
+    "final_inputs = {\n",
+    "    \"context\": lambda x: _combine_documents(x[\"docs\"]),\n",
+    "    \"question\": itemgetter(\"question\")\n",
+    "}\n",
+    "# And finally, we do the part that returns the answers\n",
+    "answer = {\n",
+    "    \"answer\": final_inputs | ANSWER_PROMPT | ChatOpenAI(),\n",
+    "    \"docs\": itemgetter(\"docs\"),\n",
+    "}\n",
+    "# And now we put it all together!\n",
+    "final_chain = loaded_memory | expanded_memory | standalone_question | retrieved_documents | answer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 46,
+   "id": "806e390c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Number of requested results 4 is greater than number of elements in index 1, updating n_results = 1\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "{'answer': AIMessage(content='Harrison was employed at Kensho.', additional_kwargs={}, example=False),\n",
+       " 'docs': [Document(page_content='harrison worked at kensho', metadata={})]}"
+      ]
+     },
+     "execution_count": 46,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "inputs = {\"question\": \"where did harrison work?\"}\n",
+    "result = final_chain.invoke(inputs)\n",
+    "result"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 47,
+   "id": "977399fd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Note that the memory does not save automatically\n",
+    "# This will be improved in the future\n",
+    "# For now you need to save it yourself\n",
+    "memory.save_context(inputs, {\"answer\": result[\"answer\"].content})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 48,
+   "id": "f94f7de4",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'history': [HumanMessage(content='where did harrison work?', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Harrison was employed at Kensho.', additional_kwargs={}, example=False)]}"
+      ]
+     },
+     "execution_count": 48,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "memory.load_memory_variables({})"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "0f2bf8d3",
@@ -1083,13 +1379,275 @@
     "chain.invoke({\"input\": \"whats 2 plus 2\"})"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "5062941a",
+   "metadata": {},
+   "source": [
+    "## Memory\n",
+    "\n",
+    "This shows how to add memory to an arbitrary chain. Right now, you can use the memory classes but need to hook it up manually"
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "9be88499",
+   "execution_count": 99,
+   "id": "7998efd8",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "from langchain.memory import ConversationBufferMemory\n",
+    "from langchain.schema.runnable import RunnableMap\n",
+    "from langchain.prompts import MessagesPlaceholder\n",
+    "model = ChatOpenAI()\n",
+    "prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"You are a helpful chatbot\"),\n",
+    "    MessagesPlaceholder(variable_name=\"history\"),\n",
+    "    (\"human\", \"{input}\")\n",
+    "])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 100,
+   "id": "fa0087f3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "memory = ConversationBufferMemory(return_messages=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 101,
+   "id": "06b531ae",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'history': []}"
+      ]
+     },
+     "execution_count": 101,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "memory.load_memory_variables({})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 102,
+   "id": "d9437af6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = RunnableMap({\n",
+    "    \"input\": lambda x: x[\"input\"],\n",
+    "    \"memory\": memory.load_memory_variables\n",
+    "}) | {\n",
+    "    \"input\": lambda x: x[\"input\"],\n",
+    "    \"history\": lambda x: x[\"memory\"][\"history\"]\n",
+    "} | prompt | model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 103,
+   "id": "bed1e260",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='Hello Bob! How can I assist you today?', additional_kwargs={}, example=False)"
+      ]
+     },
+     "execution_count": 103,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "inputs = {\"input\": \"hi im bob\"}\n",
+    "response = chain.invoke(inputs)\n",
+    "response"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 104,
+   "id": "890475b4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "memory.save_context(inputs, {\"output\": response.content})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 105,
+   "id": "e8fcb77f",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'history': [HumanMessage(content='hi im bob', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Hello Bob! How can I assist you today?', additional_kwargs={}, example=False)]}"
+      ]
+     },
+     "execution_count": 105,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "memory.load_memory_variables({})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 106,
+   "id": "d837d5c3",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='Your name is Bob. You mentioned it in your previous message. Is there anything else I can help you with, Bob?', additional_kwargs={}, example=False)"
+      ]
+     },
+     "execution_count": 106,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "inputs = {\"input\": \"whats my name\"}\n",
+    "response = chain.invoke(inputs)\n",
+    "response"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4927a727-b4c8-453c-8c83-bd87b4fcac14",
+   "metadata": {},
+   "source": [
+    "## Moderation\n",
+    "\n",
+    "This shows how to add in moderation (or other safeguards) around your LLM application."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "4f5f6449-940a-4f5c-97c0-39b71c3e2a68",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains import OpenAIModerationChain\n",
+    "from langchain.llms import OpenAI"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 35,
+   "id": "fcb8312b-7e7a-424f-a3ec-76738c9a9d21",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "moderate = OpenAIModerationChain()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "b24b9148-f6b0-4091-8ea8-d3fb281bd950",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model = OpenAI()\n",
+    "prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"repeat after me: {input}\")\n",
+    "])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "1c8ed87c-9ca6-4559-bf60-d40e94a0af08",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = prompt | model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "id": "5256b9bd-381a-42b0-bfa8-7e6d18f853cb",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'\\n\\nYou are stupid.'"
+      ]
+     },
+     "execution_count": 34,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke({\"input\": \"you are stupid\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "id": "fe6e3b33-dc9a-49d5-b194-ba750c58a628",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "moderated_chain = chain | moderate"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "id": "d8ba0cbd-c739-4d23-be9f-6ae092bd5ffb",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'input': '\\n\\nYou are stupid.',\n",
+       " 'output': \"Text was found that violates OpenAI's content policy.\"}"
+      ]
+     },
+     "execution_count": 37,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "moderated_chain.invoke({\"input\": \"you are stupid\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a0a85ba4-f782-47b8-b16f-8b7a61d6dab7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "## Conversational Retrieval With Memory"
+   ]
   }
  ],
  "metadata": {
@@ -1108,7 +1666,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.1"
   }
  },
  "nbformat": 4,

docs/extras/guides/expression_language/interface.ipynb

@@ -19,9 +19,22 @@
     "- `ainvoke`: call the chain on an input async\n",
     "- `abatch`: call the chain on a list of inputs async\n",
     "\n",
-    "The type of the input varies by component. For a prompt it is a dictionary, for a retriever it is a single string, for a model either a single string, a list of chat messages, or a PromptValue.\n",
+    "The type of the input varies by component:\n",
     "\n",
-    "The output type also varies by component. For an LLM it is a string, for a ChatModel it's a ChatMessage, for a prompt it's a PromptValue, for a retriever it's a list of documents.\n",
+    "| Component | Input Type |\n",
+    "| --- | --- |\n",
+    "|Prompt|Dictionary|\n",
+    "|Retriever|Single string|\n",
+    "|Model| Single string, list of chat messages or a PromptValue|\n",
+    "\n",
+    "The output type also varies by component:\n",
+    "\n",
+    "| Component | Output Type |\n",
+    "| --- | --- |\n",
+    "| LLM | String |\n",
+    "| ChatModel | ChatMessage |\n",
+    "| Prompt | PromptValue |\n",
+    "| Retriever | List of documents |\n",
     "\n",
     "Let's take a look at these methods! To do so, we'll create a super simple PromptTemplate + ChatModel chain."
    ]

docs/extras/integrations/callbacks/streamlit.md

@@ -71,3 +71,6 @@ or any other local ENV management tool.
 
 Currently `StreamlitCallbackHandler` is geared towards use with a LangChain Agent Executor. Support for additional agent types,
 use directly with Chains, etc will be added in the future.
+
+You may also be interested in using
+[StreamlitChatMessageHistory](/docs/integrations/memory/streamlit_chat_message_history) for LangChain.

docs/extras/integrations/chat/azureml_chat_endpoint.ipynb

@@ -0,0 +1,95 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# AzureML Chat Online Endpoint\n",
+    "\n",
+    "[AzureML](https://azure.microsoft.com/en-us/products/machine-learning/) is a platform used to build, train, and deploy machine learning models. Users can explore the types of models to deploy in the Model Catalog, which provides Azure Foundation Models and OpenAI Models. Azure Foundation Models include various open-source models and popular Hugging Face models. Users can also import models of their liking into AzureML.\n",
+    "\n",
+    "This notebook goes over how to use a chat model hosted on an `AzureML online endpoint`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models.azureml_endpoint import AzureMLChatOnlineEndpoint"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Set up\n",
+    "\n",
+    "To use the wrapper, you must [deploy a model on AzureML](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-foundation-models?view=azureml-api-2#deploying-foundation-models-to-endpoints-for-inferencing) and obtain the following parameters:\n",
+    "\n",
+    "* `endpoint_api_key`: The API key provided by the endpoint\n",
+    "* `endpoint_url`: The REST endpoint url provided by the endpoint"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Content Formatter\n",
+    "\n",
+    "The `content_formatter` parameter is a handler class for transforming the request and response of an AzureML endpoint to match with required schema. Since there are a wide range of models in the model catalog, each of which may process data differently from one another, a `ContentFormatterBase` class is provided to allow users to transform data to their liking. The following content formatters are provided:\n",
+    "\n",
+    "* `LLamaContentFormatter`: Formats request and response data for LLaMa2-chat"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content='  The Collatz Conjecture is one of the most famous unsolved problems in mathematics, and it has been the subject of much study and research for many years. While it is impossible to predict with certainty whether the conjecture will ever be solved, there are several reasons why it is considered a challenging and important problem:\\n\\n1. Simple yet elusive: The Collatz Conjecture is a deceptively simple statement that has proven to be extraordinarily difficult to prove or disprove. Despite its simplicity, the conjecture has eluded some of the brightest minds in mathematics, and it remains one of the most famous open problems in the field.\\n2. Wide-ranging implications: The Collatz Conjecture has far-reaching implications for many areas of mathematics, including number theory, algebra, and analysis. A solution to the conjecture could have significant impacts on these fields and potentially lead to new insights and discoveries.\\n3. Computational evidence: While the conjecture remains unproven, extensive computational evidence supports its validity. In fact, no counterexample to the conjecture has been found for any starting value up to 2^64 (a number', additional_kwargs={}, example=False)"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.chat_models.azureml_endpoint import LlamaContentFormatter\n",
+    "from langchain.schema import HumanMessage\n",
+    "\n",
+    "chat = AzureMLChatOnlineEndpoint(content_formatter=LlamaContentFormatter())\n",
+    "response = chat(messages=[\n",
+    "    HumanMessage(content=\"Will the Collatz conjecture ever be solved?\")\n",
+    "])\n",
+    "response"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/extras/integrations/chat/google_vertex_ai_palm.ipynb

@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -8,11 +9,7 @@
     "\n",
     "Note: This is seperate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
     "\n",
-    "PaLM API on Vertex AI is a Preview offering, subject to the Pre-GA Offerings Terms of the [GCP Service Specific Terms](https://cloud.google.com/terms/service-terms). \n",
-    "\n",
-    "Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages). Further, by using PaLM API on Vertex AI, you agree to the Generative AI Preview [terms and conditions](https://cloud.google.com/trustedtester/aitos) (Preview Terms).\n",
-    "\n",
-    "For PaLM API on Vertex AI, you can process personal data as outlined in the Cloud Data Processing Addendum, subject to applicable restrictions and obligations in the Agreement (as defined in the Preview Terms).\n",
+    "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) Customer Data to train its foundation models as part of Google Cloud`s AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
     "\n",
     "To use Vertex AI PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
     "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",
@@ -90,6 +87,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -142,6 +140,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {
     "execution": {

docs/extras/integrations/document_loaders/concurrent.ipynb

@@ -0,0 +1,94 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "23c6e167",
+   "metadata": {},
+   "source": [
+    "# Concurrent Loader\n",
+    "\n",
+    "Works just like the GenericLoader but concurrently for those who choose to optimize their workflow.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "6ff3fb1f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import ConcurrentLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "ce96fa20",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = ConcurrentLoader.from_filesystem('example_data/', glob=\"**/*.txt\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "06a6cf5d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "files = loader.load()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "b87d3e58",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "2"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(files)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "668f1ee5",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/document_loaders/example_data/sample_rss_feeds.opml

@@ -0,0 +1,13 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<opml version="1.0">
+    <head>
+        <title>Sample RSS feed subscriptions</title>
+    </head>
+    <body>
+        <outline text="Tech" title="Tech">
+            <outline type="rss" text="Engadget" title="Engadget" xmlUrl="http://www.engadget.com/rss-full.xml" htmlUrl="http://www.engadget.com"/>
+            <outline type="rss" text="Ars Technica - All content" title="Ars Technica - All content" xmlUrl="http://feeds.arstechnica.com/arstechnica/index/" htmlUrl="https://arstechnica.com"/>
+        </outline>
+    </body>
+</opml>

docs/extras/integrations/document_loaders/huawei_obs_directory.ipynb

@@ -0,0 +1,178 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "c83b6a4c",
+   "metadata": {},
+   "source": [
+    "# Huawei OBS Directory\n",
+    "The following code demonstrates how to load objects from the Huawei OBS (Object Storage Service) as documents."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c2191935",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install the required package\n",
+    "# pip install esdk-obs-python"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "55fca3b4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import OBSDirectoryLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "c3ed419f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "endpoint = \"your-endpoint\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "3428fd4e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Configure your access credentials\\n\n",
+    "config = {\n",
+    "    \"ak\": \"your-access-key\",\n",
+    "    \"sk\": \"your-secret-key\"\n",
+    "}\n",
+    "loader = OBSDirectoryLoader(\"your-bucket-name\", endpoint=endpoint, config=config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9beede9f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1e20a839",
+   "metadata": {},
+   "source": [
+    "## Specify a Prefix for Loading\n",
+    "If you want to load objects with a specific prefix from the bucket, you can use the following code:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "125f311d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = OBSDirectoryLoader(\"your-bucket-name\", endpoint=endpoint, config=config, prefix=\"test_prefix\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b3488037",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "84c82c0a",
+   "metadata": {},
+   "source": [
+    "## Get Authentication Information from ECS\n",
+    "If your langchain is deployed on Huawei Cloud ECS and [Agency is set up](https://support.huaweicloud.com/intl/en-us/usermanual-ecs/ecs_03_0166.html#section7), the loader can directly get the security token from ECS without needing access key and secret key. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "1db99969",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "config = {\"get_token_from_ecs\": True}\n",
+    "loader = OBSDirectoryLoader(\"your-bucket-name\", endpoint=endpoint, config=config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "57dd9f35",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "30205d25",
+   "metadata": {},
+   "source": [
+    "## Use a Public Bucket\n",
+    "If your bucket's bucket policy allows anonymous access (anonymous users have `listBucket` and `GetObject` permissions), you can directly load the objects without configuring the `config` parameter."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "4dfa2ef0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = OBSDirectoryLoader(\"your-bucket-name\", endpoint=endpoint)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "67d4c1d0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/document_loaders/huawei_obs_file.ipynb

@@ -0,0 +1,180 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "4394a872",
+   "metadata": {},
+   "source": [
+    "# Huawei OBS File\n",
+    "The following code demonstrates how to load an object from the Huawei OBS (Object Storage Service) as document."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c43d811b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install the required package\n",
+    "# pip install esdk-obs-python"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "5e16bae6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders.obs_file import OBSFileLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "75cc7e7c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "endpoint = \"your-endpoint\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "f9816984",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from obs import ObsClient\n",
+    "obs_client = ObsClient(access_key_id=\"your-access-key\", secret_access_key=\"your-secret-key\", server=endpoint)\n",
+    "loader = OBSFileLoader(\"your-bucket-name\", \"your-object-key\", client=obs_client)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6143b39b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "633e05ca",
+   "metadata": {},
+   "source": [
+    "## Each Loader with Separate Authentication Information\n",
+    "If you don't need to reuse OBS connections between different loaders, you can directly configure the `config`. The loader will use the config information to initialize its own OBS client."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "a5dd6a5d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Configure your access credentials\\n\n",
+    "config = {\n",
+    "    \"ak\": \"your-access-key\",\n",
+    "    \"sk\": \"your-secret-key\"\n",
+    "}\n",
+    "loader = OBSFileLoader(\"your-bucket-name\", \"your-object-key\",endpoint=endpoint, config=config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9a741f1c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1e2e611c",
+   "metadata": {},
+   "source": [
+    "## Get Authentication Information from ECS\n",
+    "If your langchain is deployed on Huawei Cloud ECS and [Agency is set up](https://support.huaweicloud.com/intl/en-us/usermanual-ecs/ecs_03_0166.html#section7), the loader can directly get the security token from ECS without needing access key and secret key. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "338fafef",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "config = {\"get_token_from_ecs\": True}\n",
+    "loader = OBSFileLoader(\"your-bucket-name\", \"your-object-key\", endpoint=endpoint, config=config)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "73976c55",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b77aa18c",
+   "metadata": {},
+   "source": [
+    "## Access a Publicly Accessible Object\n",
+    "If the object you want to access allows anonymous user access (anonymous users have `GetObject` permission), you can directly load the object without configuring the `config` parameter."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "df83d121",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = OBSFileLoader(\"your-bucket-name\", \"your-object-key\", endpoint=endpoint)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "82a844ba",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader.load()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/document_loaders/news.ipynb

@@ -0,0 +1,192 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "2dfc4698",
+   "metadata": {},
+   "source": [
+    "# News URL\n",
+    "\n",
+    "This covers how to load HTML news articles from a list of URLs into a document format that we can use downstream."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "16c3699e",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-08-02T21:18:18.886031400Z",
+     "start_time": "2023-08-02T21:18:17.682345Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import NewsURLLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "836fbac1",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-08-02T21:18:18.895539800Z",
+     "start_time": "2023-08-02T21:18:18.895539800Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "urls = [\n",
+    "    \"https://www.bbc.com/news/world-us-canada-66388172\",\n",
+    "    \"https://www.bbc.com/news/entertainment-arts-66384971\",\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "33089aba-ff74-4d00-8f40-9449c29587cc",
+   "metadata": {},
+   "source": [
+    "Pass in urls to load them into Documents"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "00f46fda",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-08-02T21:18:19.227074500Z",
+     "start_time": "2023-08-02T21:18:18.895539800Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "First article:  page_content='In testimony to the congressional committee examining the 6 January riot, Mrs Powell said she did not review all of the many claims of election fraud she made, telling them that \"no reasonable person\" would view her claims as fact. Neither she nor her representatives have commented.' metadata={'title': 'Donald Trump indictment: What do we know about the six co-conspirators?', 'link': 'https://www.bbc.com/news/world-us-canada-66388172', 'authors': [], 'language': 'en', 'description': 'Six people accused of helping Mr Trump undermine the election have been described by prosecutors.', 'publish_date': None}\n",
+      "\n",
+      "Second article:  page_content='Ms Williams added: \"If there\\'s anything that I can do in my power to ensure that dancers or singers or whoever decides to work with her don\\'t have to go through that same experience, I\\'m going to do that.\"' metadata={'title': \"Lizzo dancers Arianna Davis and Crystal Williams: 'No one speaks out, they are scared'\", 'link': 'https://www.bbc.com/news/entertainment-arts-66384971', 'authors': [], 'language': 'en', 'description': 'The US pop star is being sued for sexual harassment and fat-shaming but has yet to comment.', 'publish_date': None}\n"
+     ]
+    }
+   ],
+   "source": [
+    "loader = NewsURLLoader(urls=urls)\n",
+    "data = loader.load()\n",
+    "print(\"First article: \", data[0])\n",
+    "print(\"\\nSecond article: \", data[1])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Use nlp=True to run nlp analysis and generate keywords + summary"
+   ],
+   "metadata": {
+    "collapsed": false
+   },
+   "id": "98ac26c488315bff"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "b68a26b3",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-08-02T21:18:19.585758200Z",
+     "start_time": "2023-08-02T21:18:19.227074500Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "First article:  page_content='In testimony to the congressional committee examining the 6 January riot, Mrs Powell said she did not review all of the many claims of election fraud she made, telling them that \"no reasonable person\" would view her claims as fact. Neither she nor her representatives have commented.' metadata={'title': 'Donald Trump indictment: What do we know about the six co-conspirators?', 'link': 'https://www.bbc.com/news/world-us-canada-66388172', 'authors': [], 'language': 'en', 'description': 'Six people accused of helping Mr Trump undermine the election have been described by prosecutors.', 'publish_date': None, 'keywords': ['powell', 'know', 'donald', 'trump', 'review', 'indictment', 'telling', 'view', 'reasonable', 'person', 'testimony', 'coconspirators', 'riot', 'representatives', 'claims'], 'summary': 'In testimony to the congressional committee examining the 6 January riot, Mrs Powell said she did not review all of the many claims of election fraud she made, telling them that \"no reasonable person\" would view her claims as fact.\\nNeither she nor her representatives have commented.'}\n",
+      "\n",
+      "Second article:  page_content='Ms Williams added: \"If there\\'s anything that I can do in my power to ensure that dancers or singers or whoever decides to work with her don\\'t have to go through that same experience, I\\'m going to do that.\"' metadata={'title': \"Lizzo dancers Arianna Davis and Crystal Williams: 'No one speaks out, they are scared'\", 'link': 'https://www.bbc.com/news/entertainment-arts-66384971', 'authors': [], 'language': 'en', 'description': 'The US pop star is being sued for sexual harassment and fat-shaming but has yet to comment.', 'publish_date': None, 'keywords': ['davis', 'lizzo', 'singers', 'experience', 'crystal', 'ensure', 'arianna', 'theres', 'williams', 'power', 'going', 'dancers', 'im', 'speaks', 'work', 'ms', 'scared'], 'summary': 'Ms Williams added: \"If there\\'s anything that I can do in my power to ensure that dancers or singers or whoever decides to work with her don\\'t have to go through that same experience, I\\'m going to do that.\"'}\n"
+     ]
+    }
+   ],
+   "source": [
+    "loader = NewsURLLoader(urls=urls, nlp=True)\n",
+    "data = loader.load()\n",
+    "print(\"First article: \", data[0])\n",
+    "print(\"\\nSecond article: \", data[1])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "outputs": [
+    {
+     "data": {
+      "text/plain": "['powell',\n 'know',\n 'donald',\n 'trump',\n 'review',\n 'indictment',\n 'telling',\n 'view',\n 'reasonable',\n 'person',\n 'testimony',\n 'coconspirators',\n 'riot',\n 'representatives',\n 'claims']"
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data[0].metadata['keywords']"
+   ],
+   "metadata": {
+    "collapsed": false,
+    "ExecuteTime": {
+     "end_time": "2023-08-02T21:18:19.585758200Z",
+     "start_time": "2023-08-02T21:18:19.585758200Z"
+    }
+   },
+   "id": "ae37e004e0284b1d"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "outputs": [
+    {
+     "data": {
+      "text/plain": "'In testimony to the congressional committee examining the 6 January riot, Mrs Powell said she did not review all of the many claims of election fraud she made, telling them that \"no reasonable person\" would view her claims as fact.\\nNeither she nor her representatives have commented.'"
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data[0].metadata['summary']"
+   ],
+   "metadata": {
+    "collapsed": false,
+    "ExecuteTime": {
+     "end_time": "2023-08-02T21:18:19.598966800Z",
+     "start_time": "2023-08-02T21:18:19.594950200Z"
+    }
+   },
+   "id": "7676155fb175e53e"
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/document_loaders/nuclia.ipynb

@@ -0,0 +1,144 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Nuclia Understanding API document loader\n",
+    "\n",
+    "[Nuclia](https://nuclia.com) automatically indexes your unstructured data from any internal and external source, providing optimized search results and generative answers. It can handle video and audio transcription, image content extraction, and document parsing.\n",
+    "\n",
+    "The Nuclia Understanding API supports the processing of unstructured data, including text, web pages, documents, and audio/video contents. It extracts all texts wherever they are (using speech-to-text or OCR when needed), it also extracts metadata, embedded files (like images in a PDF), and web links. If machine learning is enabled, it identifies entities, provides a summary of the content and generates embeddings for all the sentences.\n",
+    "\n",
+    "To use the Nuclia Understanding API, you need to have a Nuclia account. You can create one for free at [https://nuclia.cloud](https://nuclia.cloud), and then [create a NUA key](https://docs.nuclia.dev/docs/docs/using/understanding/intro)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install --upgrade protobuf\n",
+    "#!pip install nucliadb-protos"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"NUCLIA_ZONE\"] = \"<YOUR_ZONE>\"  # e.g. europe-1\n",
+    "os.environ[\"NUCLIA_NUA_KEY\"] = \"<YOUR_API_KEY>\""
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To use the Nuclia document loader, you need to instantiate a `NucliaUnderstandingAPI` tool:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.tools.nuclia import NucliaUnderstandingAPI\n",
+    "\n",
+    "nua = NucliaUnderstandingAPI(enable_ml=False)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders.nuclia import NucliaLoader\n",
+    "\n",
+    "loader = NucliaLoader(\"./interview.mp4\", nua)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can now call the `load` the document in a loop until you get the document."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import time\n",
+    "\n",
+    "pending = True\n",
+    "while pending:\n",
+    "    time.sleep(15)\n",
+    "    docs = loader.load()\n",
+    "    if len(docs) > 0:\n",
+    "        print(docs[0].page_content)\n",
+    "        print(docs[0].metadata)\n",
+    "        pending = False\n",
+    "    else:\n",
+    "        print(\"waiting...\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Retrieved information\n",
+    "\n",
+    "Nuclia returns the following information:\n",
+    "\n",
+    "- file metadata\n",
+    "- extracted text\n",
+    "- nested text (like text in an embedded image)\n",
+    "- paragraphs and sentences splitting (defined by the position of their first and last characters, plus start time and end time for a video or audio file)\n",
+    "- links\n",
+    "- a thumbnail\n",
+    "- embedded files\n",
+    "\n",
+    "Note:\n",
+    "\n",
+    "  Generated files (thumbnail, extracted embedded files, etc.) are provided as a token. You can download them with the [`/processing/download` endpoint](https://docs.nuclia.dev/docs/api#operation/Download_binary_file_processing_download_get).\n",
+    "\n",
+    "  Also at any level, if an attribute exceeds a certain size, it will be put in a downloadable file and will be replaced in the document by a file pointer. This will consist of `{\"file\": {\"uri\": \"JWT_TOKEN\"}}`. The rule is that if the size of the message is greater than 1000000 characters, the biggest parts will be moved to downloadable files. First, the compression process will target vectors. If that is not enough, it will target large field metadata, and finally it will target extracted text.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "langchain",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.5"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/extras/integrations/document_loaders/recursive_url_loader.ipynb

@@ -9,7 +9,7 @@
     "\n",
     "We may want to process load all URLs under a root directory.\n",
     "\n",
-    "For example, let's look at the [LangChain JS documentation](https://js.langchain.com/docs/).\n",
+    "For example, let's look at the [Python 3.9 Document](https://docs.python.org/3.9/).\n",
     "\n",
     "This has many interesting child pages that we may want to read in bulk.\n",
     "\n",
@@ -19,13 +19,28 @@
     " \n",
     "We do this using the `RecursiveUrlLoader`.\n",
     "\n",
-    "This also gives us the flexibility to exclude some children (e.g., the `api` directory with > 800 child pages)."
+    "This also gives us the flexibility to exclude some children, customize the extractor, and more."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1be8094f",
+   "metadata": {},
+   "source": [
+    "# Parameters\n",
+    "- url: str, the target url to crawl.\n",
+    "- exclude_dirs: Optional[str], webpage directories to exclude.\n",
+    "- use_async: Optional[bool], wether to use async requests, using async requests is usually faster in large tasks. However, async will disable the lazy loading feature(the function still works, but it is not lazy). By default, it is set to False.\n",
+    "- extractor: Optional[Callable[[str], str]], a function to extract the text of the document from the webpage, by default it returns the page as it is. It is recommended to use tools like goose3 and beautifulsoup to extract the text. By default, it just returns the page as it is.\n",
+    "- max_depth: Optional[int] = None, the maximum depth to crawl. By default, it is set to 2. If you need to crawl the whole website, set it to a number that is large enough would simply do the job.\n",
+    "- timeout: Optional[int] = None, the timeout for each request, in the unit of seconds. By default, it is set to 10.\n",
+    "- prevent_outside: Optional[bool] = None, whether to prevent crawling outside the root url. By default, it is set to True."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
-   "id": "2e3532b2",
+   "execution_count": null,
+   "id": "23c18539",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -42,13 +57,15 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
-   "id": "d69e5620",
+   "execution_count": null,
+   "id": "55394afe",
    "metadata": {},
    "outputs": [],
    "source": [
-    "url = \"https://js.langchain.com/docs/modules/memory/examples/\"\n",
-    "loader = RecursiveUrlLoader(url=url)\n",
+    "from bs4 import BeautifulSoup as Soup\n",
+    "\n",
+    "url = \"https://docs.python.org/3.9/\"\n",
+    "loader = RecursiveUrlLoader(url=url, max_depth=2, extractor=lambda x: Soup(x, \"html.parser\").text)\n",
     "docs = loader.load()"
    ]
   },
@@ -61,7 +78,7 @@
     {
      "data": {
       "text/plain": [
-       "12"
+       "'\\n\\n\\n\\n\\nPython Frequently Asked Questions — Python 3.'"
       ]
      },
      "execution_count": 3,
@@ -70,19 +87,21 @@
     }
    ],
    "source": [
-    "len(docs)"
+    "docs[0].page_content[:50]"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 4,
-   "id": "89355b7c",
+   "id": "13bd7e16",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "'\\n\\n\\n\\n\\nBuffer Window Memory | 🦜️🔗 Langchain\\n\\n\\n\\n\\n\\nSki'"
+       "{'source': 'https://docs.python.org/3.9/library/index.html',\n",
+       " 'title': 'The Python Standard Library — Python 3.9.17 documentation',\n",
+       " 'language': None}"
       ]
      },
      "execution_count": 4,
@@ -91,137 +110,48 @@
     }
    ],
    "source": [
-    "docs[0].page_content[:50]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "13bd7e16",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "{'source': 'https://js.langchain.com/docs/modules/memory/examples/buffer_window_memory',\n",
-       " 'title': 'Buffer Window Memory | 🦜️🔗 Langchain',\n",
-       " 'description': 'BufferWindowMemory keeps track of the back-and-forths in conversation, and then uses a window of size k to surface the last k back-and-forths to use as memory.',\n",
-       " 'language': 'en'}"
-      ]
-     },
-     "execution_count": 5,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "docs[0].metadata"
+    "docs[-1].metadata"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "40fc13ef",
+   "id": "5866e5a6",
    "metadata": {},
    "source": [
-    "Now, let's try a more extensive example, the `docs` root dir.\n",
-    "\n",
-    "We will skip everything under `api`.\n",
-    "\n",
-    "For this, we can `lazy_load`  each page as we crawl the tree, using `WebBaseLoader` to load each as we go."
+    "However, since it's hard to perform a perfect filter, you may still see some irrelevant results in the results. You can perform a filter on the returned documents by yourself, if it's needed. Most of the time, the returned results are good enough."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4ec8ecef",
+   "metadata": {},
+   "source": [
+    "Testing on LangChain docs."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "5c938b9f",
+   "execution_count": 2,
+   "id": "349b5598",
    "metadata": {},
-   "outputs": [],
-   "source": [
-    "url = \"https://js.langchain.com/docs/\"\n",
-    "exclude_dirs = [\"https://js.langchain.com/docs/api/\"]\n",
-    "loader = RecursiveUrlLoader(url=url, exclude_dirs=exclude_dirs)\n",
-    "# Lazy load each\n",
-    "docs = [print(doc) or doc for doc in loader.lazy_load()]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "30ff61d3",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Load all pages\n",
-    "docs = loader.load()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "457e30f3",
-   "metadata": {
-    "scrolled": true
-   },
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "188"
+       "8"
       ]
      },
-     "execution_count": 8,
+     "execution_count": 2,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
+    "url = \"https://js.langchain.com/docs/modules/memory/integrations/\"\n",
+    "loader = RecursiveUrlLoader(url=url)\n",
+    "docs = loader.load()\n",
     "len(docs)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "bca80b4a",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'\\n\\n\\n\\n\\nAgent Simulations | 🦜️🔗 Langchain\\n\\n\\n\\n\\n\\nSkip t'"
-      ]
-     },
-     "execution_count": 9,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "docs[0].page_content[:50]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "df97cf22",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "{'source': 'https://js.langchain.com/docs/use_cases/agent_simulations/',\n",
-       " 'title': 'Agent Simulations | 🦜️🔗 Langchain',\n",
-       " 'description': 'Agent simulations involve taking multiple agents and having them interact with each other.',\n",
-       " 'language': 'en'}"
-      ]
-     },
-     "execution_count": 10,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "docs[0].metadata"
-   ]
   }
  ],
  "metadata": {

docs/extras/integrations/document_loaders/rss.ipynb

@@ -0,0 +1,311 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "2dfc4698",
+   "metadata": {},
+   "source": [
+    "# RSS Feeds\n",
+    "\n",
+    "This covers how to load HTML news articles from a list of RSS feed URLs into a document format that we can use downstream."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e7c2cd52-c1f7-4a06-8539-b0117da91fba",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "!pip install feedparser newspaper3k listparser"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "16c3699e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import RSSFeedLoader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "836fbac1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "urls = [\"https://news.ycombinator.com/rss\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "33089aba-ff74-4d00-8f40-9449c29587cc",
+   "metadata": {},
+   "source": [
+    "Pass in urls to load them into Documents"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "00f46fda",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = RSSFeedLoader(urls=urls)\n",
+    "data = loader.load()\n",
+    "print(len(data))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 35,
+   "id": "b447468cc42266d0",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(next Rich)\n",
+      "\n",
+      "04 August 2023\n",
+      "\n",
+      "Rich Hickey\n",
+      "\n",
+      "It is with a mixture of heartache and optimism that I announce today my (long planned) retirement from commercial software development, and my employment at Nubank. It’s been thrilling to see Clojure and Datomic successfully applied at scale.\n",
+      "\n",
+      "I look forward to continuing to lead ongoing work maintaining and enhancing Clojure with Alex, Stu, Fogus and many others, as an independent developer once again. We have many useful things planned for 1.12 and beyond. The community remains friendly, mature and productive, and is taking Clojure into many interesting new domains.\n",
+      "\n",
+      "I want to highlight and thank Nubank for their ongoing sponsorship of Alex, Fogus and the core team, as well as the Clojure community at large.\n",
+      "\n",
+      "Stu will continue to lead the development of Datomic at Nubank, where the Datomic team grows and thrives. I’m particularly excited to see where the new free availability of Datomic will lead.\n",
+      "\n",
+      "My time with Cognitect remains the highlight of my career. I have learned from absolutely everyone on our team, and am forever grateful to all for our interactions. There are too many people to thank here, but I must extend my sincerest appreciation and love to Stu and Justin for (repeatedly) taking a risk on me and my ideas, and for being the best of partners and friends, at all times fully embodying the notion of integrity. And of course to Alex Miller - who possesses in abundance many skills I lack, and without whose indomitable spirit, positivity and friendship Clojure would not have become what it did.\n",
+      "\n",
+      "I have made many friends through Clojure and Cognitect, and I hope to nurture those friendships moving forward.\n",
+      "\n",
+      "Retirement returns me to the freedom and independence I had when originally developing Clojure. The journey continues!\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(data[0].page_content)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c36d3b0d329faf2a",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "source": [
+    "You can pass arguments to the NewsURLLoader which it uses to load articles."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "id": "5fdada62470d3019",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Error fetching or processing https://twitter.com/andrewmccalip/status/1687405505604734978, exception: You must `parse()` an article first!\n",
+      "Error processing entry https://twitter.com/andrewmccalip/status/1687405505604734978, exception: list index out of range\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "13\n"
+     ]
+    }
+   ],
+   "source": [
+    "loader = RSSFeedLoader(urls=urls, nlp=True)\n",
+    "data = loader.load()\n",
+    "print(len(data))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "id": "11d71963f7735c1d",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['nubank',\n",
+       " 'alex',\n",
+       " 'stu',\n",
+       " 'taking',\n",
+       " 'team',\n",
+       " 'remains',\n",
+       " 'rich',\n",
+       " 'clojure',\n",
+       " 'thank',\n",
+       " 'planned',\n",
+       " 'datomic']"
+      ]
+     },
+     "execution_count": 37,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data[0].metadata['keywords']"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "id": "9fb64ba0e8780966",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'It’s been thrilling to see Clojure and Datomic successfully applied at scale.\\nI look forward to continuing to lead ongoing work maintaining and enhancing Clojure with Alex, Stu, Fogus and many others, as an independent developer once again.\\nThe community remains friendly, mature and productive, and is taking Clojure into many interesting new domains.\\nI want to highlight and thank Nubank for their ongoing sponsorship of Alex, Fogus and the core team, as well as the Clojure community at large.\\nStu will continue to lead the development of Datomic at Nubank, where the Datomic team grows and thrives.'"
+      ]
+     },
+     "execution_count": 38,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data[0].metadata['summary']"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "98ac26c488315bff",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "source": [
+    "You can also use an OPML file such as a Feedly export.  Pass in either a URL or the OPML contents."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "id": "8b6f07ae526a897c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Error fetching http://www.engadget.com/rss-full.xml, exception: Error fetching http://www.engadget.com/rss-full.xml, exception: document declared as us-ascii, but parsed as utf-8\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "20\n"
+     ]
+    }
+   ],
+   "source": [
+    "with open(\"example_data/sample_rss_feeds.opml\", \"r\") as f:\n",
+    "    loader = RSSFeedLoader(opml=f.read())\n",
+    "data = loader.load()\n",
+    "print(len(data))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 40,
+   "id": "b68a26b3",
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'The electric vehicle startup Fisker made a splash in Huntington Beach last night, showing off a range of new EVs it plans to build alongside the Fisker Ocean, which is slowly beginning deliveries in Europe and the US. With shades of Lotus circa 2010, it seems there\\'s something for most tastes, with a powerful four-door GT, a versatile pickup truck, and an affordable electric city car.\\n\\n\"We want the world to know that we have big plans and intend to move into several different segments, redefining each with our unique blend of design, innovation, and sustainability,\" said CEO Henrik Fisker.\\n\\nStarting with the cheapest, the Fisker PEAR—a cutesy acronym for \"Personal Electric Automotive Revolution\"—is said to use 35 percent fewer parts than other small EVs. Although it\\'s a smaller car, the PEAR seats six thanks to front and rear bench seats. Oh, and it has a frunk, which the company is calling the \"froot,\" something that will satisfy some British English speakers like Ars\\' friend and motoring journalist Jonny Smith.\\n\\nBut most exciting is the price—starting at $29,900 and scheduled for 2025. Fisker plans to contract with Foxconn to build the PEAR in Lordstown, Ohio, meaning it would be eligible for federal tax incentives.\\n\\nAdvertisement\\n\\nThe Fisker Alaska is the company\\'s pickup truck, built on a modified version of the platform used by the Ocean. It has an extendable cargo bed, which can be as little as 4.5 feet (1,371 mm) or as much as 9.2 feet (2,804 mm) long. Fisker claims it will be both the lightest EV pickup on sale and the most sustainable pickup truck in the world. Range will be an estimated 230–240 miles (370–386 km).\\n\\nThis, too, is slated for 2025, and also at a relatively affordable price, starting at $45,400. Fisker hopes to build this car in North America as well, although it isn\\'t saying where that might take place.\\n\\nFinally, there\\'s the Ronin, a four-door GT that bears more than a passing resemblance to the Fisker Karma, Henrik Fisker\\'s 2012 creation. There\\'s no price for this one, but Fisker says its all-wheel drive powertrain will boast 1,000 hp (745 kW) and will hit 60 mph from a standing start in two seconds—just about as fast as modern tires will allow. Expect a massive battery in this one, as Fisker says it\\'s targeting a 600-mile (956 km) range.\\n\\n\"Innovation and sustainability, along with design, are our three brand values. By 2027, we intend to produce the world’s first climate-neutral vehicle, and as our customers reinvent their relationships with mobility, we want to be a leader in software-defined transportation,\" Fisker said.'"
+      ]
+     },
+     "execution_count": 40,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data[0].page_content"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d5a0cbe8-18a6-4af2-b447-7abb8b734451",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "poetry-venv",
+   "language": "python",
+   "name": "poetry-venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/document_loaders/unstructured_file.ipynb

@@ -18,8 +18,7 @@
    "outputs": [],
    "source": [
     "# # Install package\n",
-    "!pip install \"unstructured[local-inference]\"\n",
-    "!pip install layoutparser[layoutmodels,tesseract]"
+    "!pip install \"unstructured[all-docs]\"\n"
    ]
   },
   {

docs/extras/integrations/document_loaders/youtube_audio.ipynb

@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "e48afb8d",
    "metadata": {},
@@ -11,7 +12,8 @@
     "\n",
     "Below we show how to easily go from a YouTube url to text to chat!\n",
     "\n",
-    "We wil use the `OpenAIWhisperParser`, which will use the OpenAI Whisper API to transcribe audio to text.\n",
+    "We wil use the `OpenAIWhisperParser`, which will use the OpenAI Whisper API to transcribe audio to text, \n",
+    "and the  `OpenAIWhisperParserLocal` for local support and running on private clouds or on premise.\n",
     "\n",
     "Note: You will need to have an `OPENAI_API_KEY` supplied."
    ]
@@ -24,7 +26,7 @@
    "outputs": [],
    "source": [
     "from langchain.document_loaders.generic import GenericLoader\n",
-    "from langchain.document_loaders.parsers import OpenAIWhisperParser\n",
+    "from langchain.document_loaders.parsers import OpenAIWhisperParser, OpenAIWhisperParserLocal\n",
     "from langchain.document_loaders.blob_loaders.youtube_audio import YoutubeAudioLoader"
    ]
   },
@@ -46,7 +48,8 @@
    "outputs": [],
    "source": [
     "! pip install yt_dlp\n",
-    "! pip install pydub"
+    "! pip install pydub\n",
+    "! pip install librosa"
    ]
   },
   {
@@ -63,6 +66,18 @@
     "Let's take the first lecture of Andrej Karpathy's YouTube course as an example! "
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8682f256",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# set a flag to switch between local and remote parsing\n",
+    "# change this to True if you want to use local parsing\n",
+    "local = False"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 2,
@@ -102,7 +117,10 @@
     "save_dir = \"~/Downloads/YouTube\"\n",
     "\n",
     "# Transcribe the videos to text\n",
-    "loader = GenericLoader(YoutubeAudioLoader(urls, save_dir), OpenAIWhisperParser())\n",
+    "if local:\n",
+    "    loader = GenericLoader(YoutubeAudioLoader(urls, save_dir), OpenAIWhisperParserLocal())\n",
+    "else:\n",
+    "    loader = GenericLoader(YoutubeAudioLoader(urls, save_dir), OpenAIWhisperParser())\n",
     "docs = loader.load()"
    ]
   },
@@ -275,7 +293,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -289,7 +307,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.10.11"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "97cc609b13305c559618ec78a438abc56230b9381f827f22d070313b9a1f3777"
+   }
   }
  },
  "nbformat": 4,

docs/extras/integrations/document_transformers/nuclia_transformer.ipynb

@@ -0,0 +1,103 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Nuclia Understanding API document transformer\n",
+    "\n",
+    "[Nuclia](https://nuclia.com) automatically indexes your unstructured data from any internal and external source, providing optimized search results and generative answers. It can handle video and audio transcription, image content extraction, and document parsing.\n",
+    "\n",
+    "The Nuclia Understanding API document transformer splits text into paragraphs and sentences, identifies entities, provides a summary of the text and generates embeddings for all the sentences.\n",
+    "\n",
+    "To use the Nuclia Understanding API, you need to have a Nuclia account. You can create one for free at [https://nuclia.cloud](https://nuclia.cloud), and then [create a NUA key](https://docs.nuclia.dev/docs/docs/using/understanding/intro).\n",
+    "\n",
+    "from langchain.document_transformers.nuclia_text_transform import NucliaTextTransformer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install --upgrade protobuf\n",
+    "#!pip install nucliadb-protos"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"NUCLIA_ZONE\"] = \"<YOUR_ZONE>\"  # e.g. europe-1\n",
+    "os.environ[\"NUCLIA_NUA_KEY\"] = \"<YOUR_API_KEY>\""
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To use the Nuclia document transformer, you need to instantiate a `NucliaUnderstandingAPI` tool with `enable_ml` set to `True`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.tools.nuclia import NucliaUnderstandingAPI\n",
+    "\n",
+    "nua = NucliaUnderstandingAPI(enable_ml=True)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The Nuclia document transformer must be called in async mode, so you need to use the `atransform_documents` method:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import asyncio\n",
+    "\n",
+    "from langchain.document_transformers.nuclia_text_transform import NucliaTextTransformer\n",
+    "from langchain.schema.document import Document\n",
+    "\n",
+    "\n",
+    "async def process():\n",
+    "    documents = [\n",
+    "        Document(page_content=\"<TEXT 1>\", metadata={}),\n",
+    "        Document(page_content=\"<TEXT 2>\", metadata={}),\n",
+    "        Document(page_content=\"<TEXT 3>\", metadata={}),\n",
+    "    ]\n",
+    "    nuclia_transformer = NucliaTextTransformer(nua)\n",
+    "    transformed_documents = await nuclia_transformer.atransform_documents(documents)\n",
+    "    print(transformed_documents)\n",
+    "\n",
+    "\n",
+    "asyncio.run(process())"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/extras/integrations/llms/Fireworks.ipynb

@@ -0,0 +1,230 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "cc6caafa",
+   "metadata": {},
+   "source": [
+    "# Fireworks\n",
+    "\n",
+    ">[Fireworks](https://app.fireworks.ai/) accelerates product development on generative AI by creating an innovative AI experiment and production platform. \n",
+    "\n",
+    "This example goes over how to use LangChain to interact with `Fireworks` models."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "60b6dbb2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms.fireworks import Fireworks, FireworksChat\n",
+    "from langchain import PromptTemplate, LLMChain\n",
+    "from langchain.prompts.chat import (\n",
+    "    ChatPromptTemplate,\n",
+    "    HumanMessagePromptTemplate,\n",
+    ")\n",
+    "import os"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ccff689e",
+   "metadata": {},
+   "source": [
+    "# Setup\n",
+    "\n",
+    "Contact Fireworks AI for the an API Key to access our models\n",
+    "\n",
+    "Set up your model using a model id. If the model is not set, the default model is fireworks-llama-v2-7b-chat."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "9ca87a2e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Initialize a Fireworks LLM\n",
+    "os.environ['FIREWORKS_API_KEY'] = \"\" #change this to your own API KEY\n",
+    "llm = Fireworks(model_id=\"accounts/fireworks/models/fireworks-llama-v2-13b-chat\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "43a11ba8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create LLM chain\n",
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "acc24d0c",
+   "metadata": {},
+   "source": [
+    "# Calling the Model\n",
+    "\n",
+    "You can use the LLMs to call the model for specified prompt(s). \n",
+    "\n",
+    "Current Specified Models: \n",
+    "* accounts/fireworks/models/fireworks-falcon-7b, accounts/fireworks/models/fireworks-falcon-40b-w8a16\n",
+    "* accounts/fireworks/models/fireworks-starcoder-1b-w8a16-1gpu, accounts/fireworks/models/fireworks-starcoder-3b-w8a16-1gpu, accounts/fireworks/models/fireworks-starcoder-7b-w8a16-1gpu, accounts/fireworks/models/fireworks-starcoder-16b-w8a16 \n",
+    "* accounts/fireworks/models/fireworks-llama-v2-13b, accounts/fireworks/models/fireworks-llama-v2-13b-chat, accounts/fireworks/models/fireworks-llama-v2-13b-w8a16, accounts/fireworks/models/fireworks-llama-v2-13b-chat-w8a16\n",
+    "* accounts/fireworks/models/fireworks-llama-v2-7b, accounts/fireworks/models/fireworks-llama-v2-7b-chat, accounts/fireworks/models/fireworks-llama-v2-7b-w8a16, accounts/fireworks/models/fireworks-llama-v2-7b-chat-w8a16, accounts/fireworks/models/fireworks-llama-v2-70b-chat-4gpu"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "bf0a425c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "It's a question that has been debated for years, with different analysts and fans making their cases for various signal-callers. Here are some of the top contenders for the title of best quarterback in the NFL:\n",
+      "\n",
+      "1. Tom Brady: The New England Patriots legend has won six Super Bowls and has been named Super Bowl MVP four times. He's known for his precision passing, pocket presence, and ability to read defenses.\n",
+      "2. Aaron Rodgers: The Green Bay Packers quarterback has won two Super Bowls and has been named NFL MVP twice. He's known for his quick release, accuracy, and ability to extend plays with his feet.\n",
+      "3. Drew Brees: The New Orleans Saints quarterback has won a Super Bowl and has been named NFL MVP once. He's known for his accuracy, pocket presence, and ability to read defenses.\n",
+      "4. Patrick Mahomes: The Kansas City Chiefs quarterback has won a Super Bowl and has been named NFL MVP twice. He's known for his arm strength, athleticism, and ability to make plays outside of the pocket.\n",
+      "5. Russell Wilson: The Seattle Seahawks quarterback has won a Super Bowl and has been named NFL MVP once. He's known for his mobility, accuracy, and ability to extend plays with his feet.\n",
+      "\n",
+      "Of course, there are other talented quarterbacks in the league, such as Lamar Jackson, Deshaun Watson, and Carson Wentz, who could also be considered among the best. Ultimately, the answer to the question of who's the best quarterback in the NFL is subjective and can vary depending on individual perspectives and criteria.\n"
+     ]
+    }
+   ],
+   "source": [
+    "#single prompt\n",
+    "output = llm(\"Who's the best quarterback in the NFL?\")\n",
+    "print(output)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "afc7de6f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "generations=[[Generation(text=\"\\nWho is the best cricket player in the world in 2016?\\nThe best cricket player in the world in 2016 is Virat Kohli. The Indian captain has had a fabulous year, scoring heavily in all formats of the game, leading India to several victories, and breaking several records. In Test cricket, Kohli has scored 1215 runs at an average of 75.33 with 6 centuries and 4 fifties, which is the highest number of runs scored by any player in a calendar year. In ODI cricket, he has scored 1143 runs at an average of 83.42 with 7 centuries and 6 fifties, which is also the highest number of runs scored by any player in a calendar year. Additionally, Kohli has led India to the number one ranking in Test cricket, and has been named the ICC Test Player of the Year and the ICC ODI Player of the Year.\\nVirat Kohli has been in incredible form in 2016, and his performances have made him the standout player of the year. Other players who have had a great year include Steve Smith, Joe Root, and Kane Williamson, but Kohli's consistency and dominance in all formats of the game make him the best cricket player in the world in 2016.\", generation_info=None)], [Generation(text=\"\\n\\nA: LeBron James.\\n\\nB: Kevin Durant.\\n\\nC: Steph Curry.\\n\\nD: James Harden.\\n\\nE: Other (please specify).\\n\\nWhat's your answer?\", generation_info=None)]] llm_output={'token_usage': {}, 'model_id': 'fireworks-llama-v2-13b-chat'} run=[RunInfo(run_id=UUID('d14b6bee-7692-46ad-8798-acb6f72fc7fb')), RunInfo(run_id=UUID('b9f5b3b5-9e62-4eaf-b269-ecf0cbbcfb82'))]\n"
+     ]
+    }
+   ],
+   "source": [
+    "#calling multiple prompts\n",
+    "output = llm.generate([\"Who's the best cricket player in 2016?\", \"Who's the best basketball player in the league?\"])\n",
+    "print(output)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "b801c20d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Kansas City in December can be quite chilly, with average high\n"
+     ]
+    }
+   ],
+   "source": [
+    "#setting parameters: model_id, temperature, max_tokens, top_p\n",
+    "llm = Fireworks(model_id=\"accounts/fireworks/models/fireworks-llama-v2-13b-chat\", temperature=0.7, max_tokens=15, top_p=1.0)\n",
+    "print(llm(\"What's the weather like in Kansas City in December?\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "137662a6",
+   "metadata": {},
+   "source": [
+    "# Create and Run Chain\n",
+    "\n",
+    "Create a prompt template to be used with the LLM Chain. Once this prompt template is created, initialize the chain with the LLM and prompt template, and run the chain with the specified prompts."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "fd2c6bc1",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "(Note: I'm just an AI and not a branding expert, so take this as a starting point for your own research and brainstorming.)\n",
+      "A good name for a company that makes football helmets could be:\n",
+      "\n",
+      "1. Helix Pro: This name plays off the idea of a helix, or spiral, shape that is commonly associated with football helmets. \"Pro\" implies a professional-level product.\n",
+      "2. Gridiron Gear: \"Gridiron\" is a term used to describe a football field, and \"gear\" highlights the company's focus on producing high-quality football helmets.\n",
+      "3. Linebacker Lab: \"Linebacker\" is a position on the football field, and \"Lab\" suggests a focus on research and development.\n",
+      "4. Helmet Hut: This name is simple and easy to remember, and it immediately conveys the company's focus on football helmets.\n",
+      "5. Tackle Tech: \"Tackle\" is a term used in football to describe a hit or collision, and \"Tech\" implies a focus on advanced technology and innovation.\n",
+      "6. Victory Vest: \"Victory\" implies a focus on winning and success, and \"Vest\" could suggest a protective or armored design.\n",
+      "7. Pigskin Pro: \"Pigskin\" is a term used to describe a football, and \"Pro\" implies a professional-level product.\n",
+      "8. Football Fusion: This name could suggest a combination of different materials or technologies to create a high-quality football helmet.\n",
+      "9. Endzone Edge: \"Endzone\" is the area of the football field where a team scores a touchdown, and \"Edge\" implies a competitive advantage.\n",
+      "10. MVP Masks: \"MVP\" stands for \"Most Valuable Player,\" and \"Masks\" highlights the protective nature of the company's football helmets.\n",
+      "\n",
+      "Remember, the name you choose for your company should be memorable, easy to pronounce and spell, and convey a sense of quality and professionalism. It's also important to check that the name isn't already in use by another company, and to consider any potential trademark issues.\n"
+     ]
+    }
+   ],
+   "source": [
+    "human_message_prompt = HumanMessagePromptTemplate(\n",
+    "    prompt=PromptTemplate(\n",
+    "        template=\"What is a good name for a company that makes {product}?\",\n",
+    "        input_variables=[\"product\"],\n",
+    "    )\n",
+    ")\n",
+    "\n",
+    "chat_prompt_template = ChatPromptTemplate.from_messages([human_message_prompt])\n",
+    "chat = Fireworks()\n",
+    "chain = LLMChain(llm=chat, prompt=chat_prompt_template)\n",
+    "output = chain.run(\"football helmets\")\n",
+    "\n",
+    "print(output)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/llms/azureml_endpoint_example.ipynb

@@ -28,9 +28,9 @@
     "\n",
     "To use the wrapper, you must [deploy a model on AzureML](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-foundation-models?view=azureml-api-2#deploying-foundation-models-to-endpoints-for-inferencing) and obtain the following parameters:\n",
     "\n",
-    "* `endpoint_api_key`: The API key provided by the endpoint\n",
-    "* `endpoint_url`: The REST endpoint url provided by the endpoint\n",
-    "* `deployment_name`: The deployment name of the endpoint"
+    "* `endpoint_api_key`: Required - The API key provided by the endpoint\n",
+    "* `endpoint_url`: Required - The REST endpoint url provided by the endpoint\n",
+    "* `deployment_name`: Not required - The deployment name of the model using the endpoint"
    ]
   },
   {
@@ -39,11 +39,14 @@
    "source": [
     "## Content Formatter\n",
     "\n",
-    "The `content_formatter` parameter is a handler class for transforming the request and response of an AzureML endpoint to match with required schema. Since there are a wide range of models in the model catalog, each of which may process data differently from one another, a `ContentFormatterBase` class is provided to allow users to transform data to their liking. Additionally, there are three content formatters already provided:\n",
+    "The `content_formatter` parameter is a handler class for transforming the request and response of an AzureML endpoint to match with required schema. Since there are a wide range of models in the model catalog, each of which may process data differently from one another, a `ContentFormatterBase` class is provided to allow users to transform data to their liking. The following content formatters are provided:\n",
     "\n",
-    "* `OSSContentFormatter`: Formats request and response data for models from the Open Source category in the Model Catalog. Note, that not all models in the Open Source category may follow the same schema\n",
-    "* `DollyContentFormatter`: Formats request and response data for the `dolly-v2-12b` model\n",
+    "* `GPT2ContentFormatter`: Formats request and response data for GPT2\n",
+    "* `DollyContentFormatter`: Formats request and response data for the Dolly-v2\n",
     "* `HFContentFormatter`: Formats request and response data for text-generation Hugging Face models\n",
+    "* `LLamaContentFormatter`: Formats request and response data for LLaMa2\n",
+    "\n",
+    "*Note: `OSSContentFormatter` is being deprecated and replaced with `GPT2ContentFormatter`. The logic is the same but `GPT2ContentFormatter` is a more suitable name. You can still continue to use `OSSContentFormatter` as the changes are backwards compatibile.*\n",
     "\n",
     "Below is an example using a summarization model from Hugging Face."
    ]
@@ -100,7 +103,6 @@
     "llm = AzureMLOnlineEndpoint(\n",
     "    endpoint_api_key=os.getenv(\"BART_ENDPOINT_API_KEY\"),\n",
     "    endpoint_url=os.getenv(\"BART_ENDPOINT_URL\"),\n",
-    "    deployment_name=\"linydub-bart-large-samsum-3\",\n",
     "    model_kwargs={\"temperature\": 0.8, \"max_new_tokens\": 400},\n",
     "    content_formatter=content_formatter,\n",
     ")\n",
@@ -167,7 +169,6 @@
     "llm = AzureMLOnlineEndpoint(\n",
     "    endpoint_api_key=os.getenv(\"DOLLY_ENDPOINT_API_KEY\"),\n",
     "    endpoint_url=os.getenv(\"DOLLY_ENDPOINT_URL\"),\n",
-    "    deployment_name=\"databricks-dolly-v2-12b-4\",\n",
     "    model_kwargs={\"temperature\": 0.8, \"max_tokens\": 300},\n",
     "    content_formatter=content_formatter,\n",
     ")\n",

docs/extras/integrations/llms/google_vertex_ai_palm.ipynb

@@ -1,18 +1,15 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "# Google Cloud Platform Vertex AI PaLM \n",
     "\n",
-    "Note: This is seperate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
+    "Note: This is seperate from the Google PaLM integration, it exposes [Vertex AI PaLM API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on Google Cloud. \n",
     "\n",
-    "PaLM API on Vertex AI is a Preview offering, subject to the Pre-GA Offerings Terms of the [GCP Service Specific Terms](https://cloud.google.com/terms/service-terms). \n",
-    "\n",
-    "Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages). Further, by using PaLM API on Vertex AI, you agree to the Generative AI Preview [terms and conditions](https://cloud.google.com/trustedtester/aitos) (Preview Terms).\n",
-    "\n",
-    "For PaLM API on Vertex AI, you can process personal data as outlined in the Cloud Data Processing Addendum, subject to applicable restrictions and obligations in the Agreement (as defined in the Preview Terms).\n",
+    "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) Customer Data to train its foundation models as part of Google Cloud`s AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
     "\n",
     "To use Vertex AI PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
     "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",
@@ -101,6 +98,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [

docs/extras/integrations/llms/llamacpp.ipynb

@@ -4,12 +4,12 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Llama-cpp\n",
+    "# Llama.cpp\n",
     "\n",
-    "[llama-cpp](https://github.com/abetlen/llama-cpp-python) is a Python binding for [llama.cpp](https://github.com/ggerganov/llama.cpp). \n",
+    "[llama-cpp-python](https://github.com/abetlen/llama-cpp-python) is a Python binding for [llama.cpp](https://github.com/ggerganov/llama.cpp). \n",
     "It supports [several LLMs](https://github.com/ggerganov/llama.cpp).\n",
     "\n",
-    "This notebook goes over how to run `llama-cpp` within LangChain."
+    "This notebook goes over how to run `llama-cpp-python` within LangChain."
    ]
   },
   {
@@ -18,7 +18,7 @@
    "source": [
     "## Installation\n",
     "\n",
-    "There is a bunch of options how to install the llama-cpp package: \n",
+    "There are different options on how to install the llama-cpp package: \n",
     "- only CPU usage\n",
     "- CPU + GPU (using one of many BLAS backends)\n",
     "- Metal GPU (MacOS with Apple Silicon Chip) \n",
@@ -61,7 +61,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "**IMPORTANT**: If you have already installed a cpu only version of the package, you need to reinstall it from scratch: consider the following command: "
+    "**IMPORTANT**: If you have already installed the CPU only version of the package, you need to reinstall it from scratch. Consider the following command: "
    ]
   },
   {
@@ -79,7 +79,7 @@
    "source": [
     "### Installation with Metal\n",
     "\n",
-    "`lama.cpp` supports Apple silicon first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks. Use the `FORCE_CMAKE=1` environment variable to force the use of cmake and install the pip package for the Metal support ([source](https://github.com/abetlen/llama-cpp-python/blob/main/docs/install/macos.md)).\n",
+    "`llama.cpp` supports Apple silicon first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks. Use the `FORCE_CMAKE=1` environment variable to force the use of cmake and install the pip package for the Metal support ([source](https://github.com/abetlen/llama-cpp-python/blob/main/docs/install/macos.md)).\n",
     "\n",
     "Example installation with Metal Support:"
    ]
@@ -143,7 +143,7 @@
     "\n",
     "#### Compiling and installing\n",
     "\n",
-    "In the same command prompt (anaconda prompt) you set the variables, you can cd into `llama-cpp-python` directory and run the following commands.\n",
+    "In the same command prompt (anaconda prompt) you set the variables, you can `cd` into `llama-cpp-python` directory and run the following commands.\n",
     "\n",
     "```\n",
     "python setup.py clean\n",
@@ -164,7 +164,9 @@
    "source": [
     "Make sure you are following all instructions to [install all necessary model files](https://github.com/ggerganov/llama.cpp).\n",
     "\n",
-    "You don't need an `API_TOKEN`!"
+    "You don't need an `API_TOKEN` as you will run the LLM locally.\n",
+    "\n",
+    "It is worth understanding which models are suitable to be used on the desired machine."
    ]
   },
   {
@@ -227,7 +229,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "`Llama-v2`"
+    "Example using a LLaMA 2 7B model"
    ]
   },
   {
@@ -304,7 +306,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "`Llama-v1`"
+    "Example using a LLaMA v1 model"
    ]
   },
   {
@@ -381,7 +383,7 @@
    "source": [
     "### GPU\n",
     "\n",
-    "If the installation with BLAS backend was correct, you will see an `BLAS = 1` indicator in model properties.\n",
+    "If the installation with BLAS backend was correct, you will see a `BLAS = 1` indicator in model properties.\n",
     "\n",
     "Two of the most important parameters for use with GPU are:\n",
     "\n",
@@ -473,22 +475,15 @@
     "llm_chain.run(question)"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "### Metal\n",
     "\n",
-    "If the installation with Metal was correct, you will see an `NEON = 1` indicator in model properties.\n",
+    "If the installation with Metal was correct, you will see a `NEON = 1` indicator in model properties.\n",
     "\n",
-    "Two of the most important parameters for use with GPU are:\n",
+    "Two of the most important GPU parameters are:\n",
     "\n",
     "- `n_gpu_layers` - determines how many layers of the model are offloaded to your Metal GPU, in the most case, set it to `1` is enough for Metal\n",
     "- `n_batch` - how many tokens are processed in parallel, default is 8, set to bigger number.\n",
@@ -522,7 +517,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The rest are almost same as GPU, the console log will show the following log to indicate the Metal was enable properly.\n",
+    "The console log will show the following log to indicate Metal was enable properly.\n",
     "\n",
     "```\n",
     "ggml_metal_init: allocating\n",
@@ -530,7 +525,9 @@
     "...\n",
     "```\n",
     "\n",
-    "You also could check the `Activity Monitor` by watching the % GPU of the process, the % CPU will drop dramatically after turn on `n_gpu_layers=1`. Also for the first time call LLM, the performance might be slow due to the model compilation in Metal GPU."
+    "You also could check `Activity Monitor` by watching the GPU usage of the process, the CPU usage will drop dramatically after turn on `n_gpu_layers=1`. \n",
+    "\n",
+    "For the first call to the LLM, the performance may be slow due to the model compilation in Metal GPU."
    ]
   }
  ],

docs/extras/integrations/llms/vllm.ipynb

@@ -0,0 +1,196 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "499c3142-2033-437d-a60a-731988ac6074",
+   "metadata": {},
+   "source": [
+    "# vLLM\n",
+    "\n",
+    "[vLLM](https://vllm.readthedocs.io/en/latest/index.html) is a fast and easy-to-use library for LLM inference and serving, offering:\n",
+    "* State-of-the-art serving throughput \n",
+    "* Efficient management of attention key and value memory with PagedAttention\n",
+    "* Continuous batching of incoming requests\n",
+    "* Optimized CUDA kernels\n",
+    "\n",
+    "This notebooks goes over how to use a LLM with langchain and vLLM.\n",
+    "\n",
+    "To use, you should have the `vllm` python package installed."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "8a3f2666-5c75-4797-967a-7915a247bf33",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "#!pip install vllm -q"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "84e350f7-21f6-455b-b1f0-8b0116a2fd49",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "INFO 08-06 11:37:33 llm_engine.py:70] Initializing an LLM engine with config: model='mosaicml/mpt-7b', tokenizer='mosaicml/mpt-7b', tokenizer_mode=auto, trust_remote_code=True, dtype=torch.bfloat16, use_dummy_weights=False, download_dir=None, use_np_weights=False, tensor_parallel_size=1, seed=0)\n",
+      "INFO 08-06 11:37:41 llm_engine.py:196] # GPU blocks: 861, # CPU blocks: 512\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.00it/s]"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "What is the capital of France ? The capital of France is Paris.\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain.llms import VLLM\n",
+    "\n",
+    "llm = VLLM(model=\"mosaicml/mpt-7b\",\n",
+    "           trust_remote_code=True,  # mandatory for hf models\n",
+    "           max_new_tokens=128,\n",
+    "           top_k=10,\n",
+    "           top_p=0.95,\n",
+    "           temperature=0.8,\n",
+    ")\n",
+    "\n",
+    "print(llm(\"What is the capital of France ?\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "94a3b41d-8329-4f8f-94f9-453d7f132214",
+   "metadata": {},
+   "source": [
+    "## Integrate the model in an LLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "5605b7a1-fa63-49c1-934d-8b4ef8d71dd5",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.34s/it]"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "1. The first Pokemon game was released in 1996.\n",
+      "2. The president was Bill Clinton.\n",
+      "3. Clinton was president from 1993 to 2001.\n",
+      "4. The answer is Clinton.\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain import PromptTemplate, LLMChain\n",
+    "\n",
+    "template = \"\"\"Question: {question}\n",
+    "\n",
+    "Answer: Let's think step by step.\"\"\"\n",
+    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])\n",
+    "\n",
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
+    "\n",
+    "question = \"Who was the US president in the year the first Pokemon game was released?\"\n",
+    "\n",
+    "print(llm_chain.run(question))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "56826aba-d08b-4838-8bfa-ca96e463b25d",
+   "metadata": {},
+   "source": [
+    "## Distributed Inference\n",
+    "\n",
+    "vLLM supports distributed tensor-parallel inference and serving. \n",
+    "\n",
+    "To run multi-GPU inference with the LLM class, set the `tensor_parallel_size` argument to the number of GPUs you want to use. For example, to run inference on 4 GPUs"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f8c25c35-47b5-459d-9985-3cf546e9ac16",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import VLLM\n",
+    "\n",
+    "llm = VLLM(model=\"mosaicml/mpt-30b\",\n",
+    "           tensor_parallel_size=4,\n",
+    "           trust_remote_code=True,  # mandatory for hf models\n",
+    ")\n",
+    "\n",
+    "llm(\"What is the future of AI?\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "conda_pytorch_p310",
+   "language": "python",
+   "name": "conda_pytorch_p310"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/memory/streamlit_chat_message_history.ipynb

@@ -0,0 +1,154 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "91c6a7ef",
+   "metadata": {},
+   "source": [
+    "# Streamlit Chat Message History\n",
+    "\n",
+    "This notebook goes over how to store and use chat message history in a Streamlit app. StreamlitChatMessageHistory will store messages in\n",
+    "[Streamlit session state](https://docs.streamlit.io/library/api-reference/session-state)\n",
+    "at the specified `key=`. The default key is `\"langchain_messages\"`.\n",
+    "\n",
+    "- Note, StreamlitChatMessageHistory only works when run in a Streamlit app.\n",
+    "- You may also be interested in [StreamlitCallbackHandler](/docs/integrations/callbacks/streamlit) for LangChain.\n",
+    "- For more on Streamlit check out their\n",
+    "[getting started documentation](https://docs.streamlit.io/library/get-started).\n",
+    "\n",
+    "You can see the [full app example running here](https://langchain-st-memory.streamlit.app/), and more examples in\n",
+    "[github.com/langchain-ai/streamlit-agent](https://github.com/langchain-ai/streamlit-agent)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d15e3302",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.memory import StreamlitChatMessageHistory\n",
+    "\n",
+    "history = StreamlitChatMessageHistory(key=\"chat_messages\")\n",
+    "\n",
+    "history.add_user_message(\"hi!\")\n",
+    "history.add_ai_message(\"whats up?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "64fc465e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "history.messages"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b60dc735",
+   "metadata": {},
+   "source": [
+    "You can integrate StreamlitChatMessageHistory into ConversationBufferMemory and chains or agents as usual. The history will be persisted across re-runs of the Streamlit app within a given user session. A given StreamlitChatMessageHistory will NOT be persisted or shared across user sessions."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "42ab5bf3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n",
+    "from langchain.memory import ConversationBufferMemory\n",
+    "from langchain.memory.chat_message_histories import StreamlitChatMessageHistory\n",
+    "\n",
+    "# Optionally, specify your own session_state key for storing messages\n",
+    "msgs = StreamlitChatMessageHistory(key=\"special_app_key\")\n",
+    "\n",
+    "memory = ConversationBufferMemory(memory_key=\"history\", chat_memory=msgs)\n",
+    "if len(msgs.messages) == 0:\n",
+    "    msgs.add_ai_message(\"How can I help you?\")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a29252de",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains import LLMChain\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "template = \"\"\"You are an AI chatbot having a conversation with a human.\n",
+    "\n",
+    "{history}\n",
+    "Human: {human_input}\n",
+    "AI: \"\"\"\n",
+    "prompt = PromptTemplate(input_variables=[\"history\", \"human_input\"], template=template)\n",
+    "\n",
+    "# Add the memory to an LLMChain as usual\n",
+    "llm_chain = LLMChain(llm=OpenAI(), prompt=prompt, memory=memory)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7cd99b4b",
+   "metadata": {},
+   "source": [
+    "Conversational Streamlit apps will often re-draw each previous chat message on every re-run. This is easy to do by iterating through `StreamlitChatMessageHistory.messages`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3bdb637b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import streamlit as st\n",
+    "\n",
+    "for msg in msgs.messages:\n",
+    "    st.chat_message(msg.type).write(msg.content)\n",
+    "\n",
+    "if prompt := st.chat_input():\n",
+    "    st.chat_message(\"human\").write(prompt)\n",
+    "\n",
+    "    # As usual, new messages are added to StreamlitChatMessageHistory when the Chain is called.\n",
+    "    response = llm_chain.run(prompt)\n",
+    "    st.chat_message(\"ai\").write(response)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7adaf3d6",
+   "metadata": {},
+   "source": [
+    "**[View the final app](https://langchain-st-memory.streamlit.app/).**"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "poetry-venv",
+   "language": "python",
+   "name": "poetry-venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/providers/fireworks.md

@@ -0,0 +1,22 @@
+# Fireworks
+
+This page covers how to use the Fireworks models within Langchain.
+
+## Installation and Setup
+
+- To use the Fireworks model, you need to have a Fireworks API key. To generate one, sign up at platform.fireworks.ai
+- Authenticate by setting the FIREWORKS_API_KEY environment variable.
+
+## LLM
+
+Fireworks integrates with Langchain through the LLM module, which allows for standardized usage of any models deployed on the Fireworks models.
+
+In this example, we'll work the llama-v2-13b. 
+```python
+from langchain.llms.fireworks import Fireworks 
+
+llm = Fireworks(model="fireworks-llama-v2-13b-chat", max_tokens=256, temperature=0.4)
+llm("Name 3 sports.")
+```
+
+For a more detailed walkthrough, see [here](/docs/integrations/llms/Fireworks).

docs/extras/integrations/providers/mlflow_ai_gateway.mdx

@@ -1,6 +1,6 @@
 # MLflow AI Gateway
 
-The MLflow AI Gateway service is a powerful tool designed to streamline the usage and management of various large language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests. See [the MLflow AI Gateway documentation](https://mlflow.org/docs/latest/gateway/index.html) for more details.
+>`The MLflow AI Gateway` service is a powerful tool designed to streamline the usage and management of various large language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests. See [the MLflow AI Gateway documentation](https://mlflow.org/docs/latest/gateway/index.html) for more details.
 
 ## Installation and Setup
 

docs/extras/integrations/providers/mlflow_tracking.ipynb

@@ -1,19 +1,49 @@
 {
  "cells": [
   {
-   "attachments": {},
    "cell_type": "markdown",
+   "id": "5d184f91",
    "metadata": {},
    "source": [
     "# MLflow\n",
     "\n",
-    "This notebook goes over how to track your LangChain experiments into your MLflow Server"
-   ],
-   "id": "5d184f91"
+    ">[MLflow](https://www.mlflow.org/docs/latest/what-is-mlflow.html) is a versatile, expandable, open-source platform for managing workflows and artifacts across the machine learning lifecycle. It has built-in integrations with many popular ML libraries, but can be used with any library, algorithm, or deployment tool. It is designed to be extensible, so you can write plugins to support new workflows, libraries, and tools.\n",
+    "\n",
+    "This notebook goes over how to track your LangChain experiments into your `MLflow Server`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ea73efae-7182-4a89-a492-c865b1fcf981",
+   "metadata": {},
+   "source": [
+    "## External examples"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "97361a84-4e8f-45ba-b291-814cf73cd8f2",
+   "metadata": {},
+   "source": [
+    "`MLflow` provides [several examples](https://github.com/mlflow/mlflow/tree/master/examples/langchain) for the `LangChain` integration:\n",
+    "- [simple_chain](https://github.com/mlflow/mlflow/blob/master/examples/langchain/simple_chain.py)\n",
+    "- [simple_agent](https://github.com/mlflow/mlflow/blob/master/examples/langchain/simple_agent.py)\n",
+    "- [retriever_chain](https://github.com/mlflow/mlflow/blob/master/examples/langchain/retriever_chain.py)\n",
+    "- [retrieval_qa_chain](https://github.com/mlflow/mlflow/blob/master/examples/langchain/retrieval_qa_chain.py)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e0cbd74b-1542-45a4-a72b-b2eedeffd2e0",
+   "metadata": {},
+   "source": [
+    "## Example"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "ca7bd72f",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -24,12 +54,12 @@
     "!pip install openai\n",
     "!pip install google-search-results\n",
     "!python -m spacy download en_core_web_sm"
-   ],
-   "id": "ca7bd72f"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "bf8e1f5c",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -38,23 +68,23 @@
     "os.environ[\"MLFLOW_TRACKING_URI\"] = \"\"\n",
     "os.environ[\"OPENAI_API_KEY\"] = \"\"\n",
     "os.environ[\"SERPAPI_API_KEY\"] = \"\""
-   ],
-   "id": "bf8e1f5c"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "fd49fd45",
    "metadata": {},
    "outputs": [],
    "source": [
     "from langchain.callbacks import MlflowCallbackHandler\n",
     "from langchain.llms import OpenAI"
-   ],
-   "id": "fd49fd45"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "578cac8c",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -70,12 +100,12 @@
     "llm = OpenAI(\n",
     "    model_name=\"gpt-3.5-turbo\", temperature=0, callbacks=[mlflow_callback], verbose=True\n",
     ")"
-   ],
-   "id": "578cac8c"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "9b20acae",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -83,23 +113,23 @@
     "llm_result = llm.generate([\"Tell me a joke\"])\n",
     "\n",
     "mlflow_callback.flush_tracker(llm)"
-   ],
-   "id": "9b20acae"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "8b872046",
    "metadata": {},
    "outputs": [],
    "source": [
     "from langchain.prompts import PromptTemplate\n",
     "from langchain.chains import LLMChain"
-   ],
-   "id": "8b872046"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "1b2627ef",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -117,12 +147,12 @@
     "]\n",
     "synopsis_chain.apply(test_prompts)\n",
     "mlflow_callback.flush_tracker(synopsis_chain)"
-   ],
-   "id": "1b2627ef"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "e002823a",
    "metadata": {
     "id": "_jN73xcPVEpI"
    },
@@ -130,12 +160,12 @@
    "source": [
     "from langchain.agents import initialize_agent, load_tools\n",
     "from langchain.agents import AgentType"
-   ],
-   "id": "e002823a"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "655bd47e",
    "metadata": {
     "id": "Gpq4rk6VT9cu"
    },
@@ -154,8 +184,7 @@
     "    \"Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?\"\n",
     ")\n",
     "mlflow_callback.flush_tracker(agent, finish=True)"
-   ],
-   "id": "655bd47e"
+   ]
   }
  ],
  "metadata": {
@@ -177,9 +206,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}

docs/extras/integrations/providers/sagemaker_tracking.ipynb

@@ -0,0 +1,916 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "ef3909cf-72ca-4841-85c6-ef4e0eae3aaf",
+   "metadata": {},
+   "source": [
+    "# SageMaker Tracking\n",
+    "\n",
+    "This notebook shows how LangChain Callback can be used to log and track prompts and other LLM hyperparameters into SageMaker Experiments. Here, we use different scenarios to showcase the capability:\n",
+    "* **Scenario 1**: *Single LLM* - A case where a single LLM model is used to generate output based on a given prompt.\n",
+    "* **Scenario 2**: *Sequential Chain* - A case where a sequential chain of two LLM models is used.\n",
+    "* **Scenario 3**: *Agent with Tools (Chain of Thought)* - A case where multiple tools (search and math) are used in addition to an LLM.\n",
+    "\n",
+    "[Amazon SageMaker](https://aws.amazon.com/sagemaker/) is a fully managed service that is used to quickly and easily build, train and deploy machine learning (ML) models. \n",
+    "\n",
+    "[Amazon SageMaker Experiments](https://docs.aws.amazon.com/sagemaker/latest/dg/experiments.html) is a capability of Amazon SageMaker that lets you organize, track, compare and evaluate ML experiments and model versions.\n",
+    "\n",
+    "In this notebook, we will create a single experiment to log the prompts from each scenario."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "94c22cb4-3b1c-432b-b3be-0235eec79c5c",
+   "metadata": {},
+   "source": [
+    "## Installation and Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2353436d-17fe-4f58-a2f9-c299d56393fd",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!pip install sagemaker\n",
+    "!pip install openai\n",
+    "!pip install google-search-results"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "65dcf62e-7a38-4119-adb9-d9e884e82499",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "First, setup the required API keys\n",
+    "* OpenAI: https://platform.openai.com/account/api-keys (For OpenAI LLM model)\n",
+    "* Google SERP API: https://serpapi.com/manage-api-key (For Google Search Tool)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5ec2b898-0cfc-4308-8e86-569cd7b7cf41",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "## Add your API keys below\n",
+    "os.environ[\"OPENAI_API_KEY\"] = \"<ADD-KEY-HERE>\"\n",
+    "os.environ[\"SERPAPI_API_KEY\"] = \"<ADD-KEY-HERE>\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "80968ebf-519f-46de-8703-97532ac39e3e",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.llms import OpenAI\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains import LLMChain, SimpleSequentialChain\n",
+    "from langchain.agents import initialize_agent, load_tools\n",
+    "from langchain.agents import Tool\n",
+    "from langchain.callbacks import SageMakerCallbackHandler\n",
+    "\n",
+    "from sagemaker.analytics import ExperimentAnalytics\n",
+    "from sagemaker.session import Session\n",
+    "from sagemaker.experiments.run import Run"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b67d031f-a01f-4009-ad29-c80ab8ad50ea",
+   "metadata": {},
+   "source": [
+    "## LLM Prompt Tracking"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "da2d70ee-173b-469d-a718-54c33d862844",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "#LLM Hyperparameters\n",
+    "HPARAMS = {\n",
+    "    \"temperature\": 0.1,\n",
+    "    \"model_name\": \"text-davinci-003\",\n",
+    "}\n",
+    "\n",
+    "#Bucket used to save prompt logs (Use `None` is used to save the default bucket or otherwise change it)\n",
+    "BUCKET_NAME = None\n",
+    "\n",
+    "#Experiment name\n",
+    "EXPERIMENT_NAME = \"langchain-sagemaker-tracker\"\n",
+    "\n",
+    "#Create SageMaker Session with the given bucket\n",
+    "session = Session(default_bucket=BUCKET_NAME)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7239a39a-08d8-43cb-8922-81abdd5d9ebf",
+   "metadata": {},
+   "source": [
+    "### Scenario 1 - LLM"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "abc00335-50c8-4119-adb8-4c4ab8522e23",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "RUN_NAME = \"run-scenario-1\"\n",
+    "PROMPT_TEMPLATE = \"tell me a joke about {topic}\"\n",
+    "INPUT_VARIABLES = {\"topic\": \"fish\"}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4a3a3cbe-db85-4255-8d8b-eaafdca8c6e2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:\n",
+    "\n",
+    "    # Create SageMaker Callback\n",
+    "    sagemaker_callback = SageMakerCallbackHandler(run)\n",
+    "\n",
+    "    # Define LLM model with callback\n",
+    "    llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)\n",
+    "\n",
+    "    # Create prompt template\n",
+    "    prompt = PromptTemplate.from_template(template=PROMPT_TEMPLATE)\n",
+    "\n",
+    "    # Create LLM Chain\n",
+    "    chain = LLMChain(llm=llm, prompt=prompt, callbacks=[sagemaker_callback])\n",
+    "\n",
+    "    # Run chain\n",
+    "    chain.run(**INPUT_VARIABLES)\n",
+    "\n",
+    "    # Reset the callback\n",
+    "    sagemaker_callback.flush_tracker()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7dc69934-9f42-40b7-9931-36a3371a38da",
+   "metadata": {},
+   "source": [
+    "### Scenario 2 - Sequential Chain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "50b75ef9-9825-4ccc-8414-4cd7525a1b68",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "RUN_NAME = \"run-scenario-2\"\n",
+    "\n",
+    "PROMPT_TEMPLATE_1 = \"\"\"You are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
+    "Title: {title}\n",
+    "Playwright: This is a synopsis for the above play:\"\"\"\n",
+    "PROMPT_TEMPLATE_2 = \"\"\"You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a review for that play.\n",
+    "Play Synopsis: {synopsis}\n",
+    "Review from a New York Times play critic of the above play:\"\"\"\n",
+    "\n",
+    "INPUT_VARIABLES = {\n",
+    "    \"input\": \"documentary about good video games that push the boundary of game design\"\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fb7fff5f-e89f-40e2-96b4-3641a0b6e9b4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:\n",
+    "\n",
+    "    # Create SageMaker Callback\n",
+    "    sagemaker_callback = SageMakerCallbackHandler(run)\n",
+    "\n",
+    "    # Create prompt templates for the chain\n",
+    "    prompt_template1 = PromptTemplate.from_template(template=PROMPT_TEMPLATE_1)\n",
+    "    prompt_template2 = PromptTemplate.from_template(template=PROMPT_TEMPLATE_2)\n",
+    "\n",
+    "    # Define LLM model with callback\n",
+    "    llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)\n",
+    "\n",
+    "    # Create chain1\n",
+    "    chain1 = LLMChain(llm=llm, prompt=prompt_template1, callbacks=[sagemaker_callback])\n",
+    "\n",
+    "    # Create chain2\n",
+    "    chain2 = LLMChain(llm=llm, prompt=prompt_template2, callbacks=[sagemaker_callback])\n",
+    "\n",
+    "    # Create Sequential chain\n",
+    "    overall_chain = SimpleSequentialChain(chains=[chain1, chain2], callbacks=[sagemaker_callback])\n",
+    "\n",
+    "    # Run overall sequential chain\n",
+    "    overall_chain.run(**INPUT_VARIABLES)\n",
+    "\n",
+    "    # Reset the callback\n",
+    "    sagemaker_callback.flush_tracker()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6b82bd0e-c626-4797-bb06-c1983f176315",
+   "metadata": {},
+   "source": [
+    "### Scenario 3 - Agent with Tools"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b5066f03-49dc-4868-be8e-d21ce22063fe",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "RUN_NAME = \"run-scenario-3\"\n",
+    "PROMPT_TEMPLATE = \"Who is the oldest person alive? And what is their current age raised to the power of 1.51?\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "98385c42-9e44-4b03-b76d-007cb4797864",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "with Run(experiment_name=EXPERIMENT_NAME, run_name=RUN_NAME, sagemaker_session=session) as run:\n",
+    "\n",
+    "    # Create SageMaker Callback\n",
+    "    sagemaker_callback = SageMakerCallbackHandler(run)\n",
+    "\n",
+    "    # Define LLM model with callback\n",
+    "    llm = OpenAI(callbacks=[sagemaker_callback], **HPARAMS)\n",
+    "\n",
+    "    # Define tools\n",
+    "    tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callbacks=[sagemaker_callback])\n",
+    "\n",
+    "    # Initialize agent with all the tools\n",
+    "    agent = initialize_agent(tools, llm, agent=\"zero-shot-react-description\", callbacks=[sagemaker_callback])\n",
+    "\n",
+    "    # Run agent\n",
+    "    agent.run(input=PROMPT_TEMPLATE)\n",
+    "\n",
+    "    # Reset the callback\n",
+    "    sagemaker_callback.flush_tracker()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c306a1c9-99f8-476d-96db-347746f5cfe0",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Load Log Data\n",
+    "\n",
+    "Once the prompts are logged, we can easily load and convert them to Pandas DataFrame as follows."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ec7b4af2-e01d-4f6c-9de5-70d2b4acb9e6",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "#Load\n",
+    "logs = ExperimentAnalytics(experiment_name=EXPERIMENT_NAME)\n",
+    "\n",
+    "#Convert as pandas dataframe\n",
+    "df = logs.dataframe(force_refresh=True)\n",
+    "\n",
+    "print(df.shape)\n",
+    "df.head()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "29991c75-f9cf-4c36-abfd-903c09fb170d",
+   "metadata": {},
+   "source": [
+    "As can be seen above, there are three runs (rows) in the experiment corresponding to each scenario. Each run logs the prompts and related LLM settings/hyperparameters as json and are saved in s3 bucket. Feel free to load and explore the log data from each json path."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "61a695d6-0aef-4284-9e12-eea8bc143dbd",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "availableInstances": [
+   {
+    "_defaultOrder": 0,
+    "_isFastLaunch": true,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 4,
+    "name": "ml.t3.medium",
+    "vcpuNum": 2
+   },
+   {
+    "_defaultOrder": 1,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 8,
+    "name": "ml.t3.large",
+    "vcpuNum": 2
+   },
+   {
+    "_defaultOrder": 2,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 16,
+    "name": "ml.t3.xlarge",
+    "vcpuNum": 4
+   },
+   {
+    "_defaultOrder": 3,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 32,
+    "name": "ml.t3.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 4,
+    "_isFastLaunch": true,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 8,
+    "name": "ml.m5.large",
+    "vcpuNum": 2
+   },
+   {
+    "_defaultOrder": 5,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 16,
+    "name": "ml.m5.xlarge",
+    "vcpuNum": 4
+   },
+   {
+    "_defaultOrder": 6,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 32,
+    "name": "ml.m5.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 7,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 64,
+    "name": "ml.m5.4xlarge",
+    "vcpuNum": 16
+   },
+   {
+    "_defaultOrder": 8,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 128,
+    "name": "ml.m5.8xlarge",
+    "vcpuNum": 32
+   },
+   {
+    "_defaultOrder": 9,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 192,
+    "name": "ml.m5.12xlarge",
+    "vcpuNum": 48
+   },
+   {
+    "_defaultOrder": 10,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 256,
+    "name": "ml.m5.16xlarge",
+    "vcpuNum": 64
+   },
+   {
+    "_defaultOrder": 11,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 384,
+    "name": "ml.m5.24xlarge",
+    "vcpuNum": 96
+   },
+   {
+    "_defaultOrder": 12,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 8,
+    "name": "ml.m5d.large",
+    "vcpuNum": 2
+   },
+   {
+    "_defaultOrder": 13,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 16,
+    "name": "ml.m5d.xlarge",
+    "vcpuNum": 4
+   },
+   {
+    "_defaultOrder": 14,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 32,
+    "name": "ml.m5d.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 15,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 64,
+    "name": "ml.m5d.4xlarge",
+    "vcpuNum": 16
+   },
+   {
+    "_defaultOrder": 16,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 128,
+    "name": "ml.m5d.8xlarge",
+    "vcpuNum": 32
+   },
+   {
+    "_defaultOrder": 17,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 192,
+    "name": "ml.m5d.12xlarge",
+    "vcpuNum": 48
+   },
+   {
+    "_defaultOrder": 18,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 256,
+    "name": "ml.m5d.16xlarge",
+    "vcpuNum": 64
+   },
+   {
+    "_defaultOrder": 19,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 384,
+    "name": "ml.m5d.24xlarge",
+    "vcpuNum": 96
+   },
+   {
+    "_defaultOrder": 20,
+    "_isFastLaunch": false,
+    "category": "General purpose",
+    "gpuNum": 0,
+    "hideHardwareSpecs": true,
+    "memoryGiB": 0,
+    "name": "ml.geospatial.interactive",
+    "supportedImageNames": [
+     "sagemaker-geospatial-v1-0"
+    ],
+    "vcpuNum": 0
+   },
+   {
+    "_defaultOrder": 21,
+    "_isFastLaunch": true,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 4,
+    "name": "ml.c5.large",
+    "vcpuNum": 2
+   },
+   {
+    "_defaultOrder": 22,
+    "_isFastLaunch": false,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 8,
+    "name": "ml.c5.xlarge",
+    "vcpuNum": 4
+   },
+   {
+    "_defaultOrder": 23,
+    "_isFastLaunch": false,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 16,
+    "name": "ml.c5.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 24,
+    "_isFastLaunch": false,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 32,
+    "name": "ml.c5.4xlarge",
+    "vcpuNum": 16
+   },
+   {
+    "_defaultOrder": 25,
+    "_isFastLaunch": false,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 72,
+    "name": "ml.c5.9xlarge",
+    "vcpuNum": 36
+   },
+   {
+    "_defaultOrder": 26,
+    "_isFastLaunch": false,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 96,
+    "name": "ml.c5.12xlarge",
+    "vcpuNum": 48
+   },
+   {
+    "_defaultOrder": 27,
+    "_isFastLaunch": false,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 144,
+    "name": "ml.c5.18xlarge",
+    "vcpuNum": 72
+   },
+   {
+    "_defaultOrder": 28,
+    "_isFastLaunch": false,
+    "category": "Compute optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 192,
+    "name": "ml.c5.24xlarge",
+    "vcpuNum": 96
+   },
+   {
+    "_defaultOrder": 29,
+    "_isFastLaunch": true,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 16,
+    "name": "ml.g4dn.xlarge",
+    "vcpuNum": 4
+   },
+   {
+    "_defaultOrder": 30,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 32,
+    "name": "ml.g4dn.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 31,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 64,
+    "name": "ml.g4dn.4xlarge",
+    "vcpuNum": 16
+   },
+   {
+    "_defaultOrder": 32,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 128,
+    "name": "ml.g4dn.8xlarge",
+    "vcpuNum": 32
+   },
+   {
+    "_defaultOrder": 33,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 4,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 192,
+    "name": "ml.g4dn.12xlarge",
+    "vcpuNum": 48
+   },
+   {
+    "_defaultOrder": 34,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 256,
+    "name": "ml.g4dn.16xlarge",
+    "vcpuNum": 64
+   },
+   {
+    "_defaultOrder": 35,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 61,
+    "name": "ml.p3.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 36,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 4,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 244,
+    "name": "ml.p3.8xlarge",
+    "vcpuNum": 32
+   },
+   {
+    "_defaultOrder": 37,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 8,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 488,
+    "name": "ml.p3.16xlarge",
+    "vcpuNum": 64
+   },
+   {
+    "_defaultOrder": 38,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 8,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 768,
+    "name": "ml.p3dn.24xlarge",
+    "vcpuNum": 96
+   },
+   {
+    "_defaultOrder": 39,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 16,
+    "name": "ml.r5.large",
+    "vcpuNum": 2
+   },
+   {
+    "_defaultOrder": 40,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 32,
+    "name": "ml.r5.xlarge",
+    "vcpuNum": 4
+   },
+   {
+    "_defaultOrder": 41,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 64,
+    "name": "ml.r5.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 42,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 128,
+    "name": "ml.r5.4xlarge",
+    "vcpuNum": 16
+   },
+   {
+    "_defaultOrder": 43,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 256,
+    "name": "ml.r5.8xlarge",
+    "vcpuNum": 32
+   },
+   {
+    "_defaultOrder": 44,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 384,
+    "name": "ml.r5.12xlarge",
+    "vcpuNum": 48
+   },
+   {
+    "_defaultOrder": 45,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 512,
+    "name": "ml.r5.16xlarge",
+    "vcpuNum": 64
+   },
+   {
+    "_defaultOrder": 46,
+    "_isFastLaunch": false,
+    "category": "Memory Optimized",
+    "gpuNum": 0,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 768,
+    "name": "ml.r5.24xlarge",
+    "vcpuNum": 96
+   },
+   {
+    "_defaultOrder": 47,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 16,
+    "name": "ml.g5.xlarge",
+    "vcpuNum": 4
+   },
+   {
+    "_defaultOrder": 48,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 32,
+    "name": "ml.g5.2xlarge",
+    "vcpuNum": 8
+   },
+   {
+    "_defaultOrder": 49,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 64,
+    "name": "ml.g5.4xlarge",
+    "vcpuNum": 16
+   },
+   {
+    "_defaultOrder": 50,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 128,
+    "name": "ml.g5.8xlarge",
+    "vcpuNum": 32
+   },
+   {
+    "_defaultOrder": 51,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 1,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 256,
+    "name": "ml.g5.16xlarge",
+    "vcpuNum": 64
+   },
+   {
+    "_defaultOrder": 52,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 4,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 192,
+    "name": "ml.g5.12xlarge",
+    "vcpuNum": 48
+   },
+   {
+    "_defaultOrder": 53,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 4,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 384,
+    "name": "ml.g5.24xlarge",
+    "vcpuNum": 96
+   },
+   {
+    "_defaultOrder": 54,
+    "_isFastLaunch": false,
+    "category": "Accelerated computing",
+    "gpuNum": 8,
+    "hideHardwareSpecs": false,
+    "memoryGiB": 768,
+    "name": "ml.g5.48xlarge",
+    "vcpuNum": 192
+   }
+  ],
+  "instance_type": "ml.t3.large",
+  "kernelspec": {
+   "display_name": "conda_pytorch_p310",
+   "language": "python",
+   "name": "conda_pytorch_p310"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/providers/unstructured.mdx

@@ -11,7 +11,9 @@ ecosystem within LangChain.
 If you are using a loader that runs locally, use the following steps to get `unstructured` and
 its dependencies running locally.
 
-- Install the Python SDK with `pip install "unstructured[local-inference]"`
+- Install the Python SDK with `pip install unstructured`.
+    - You can install document specific dependencies with extras, i.e. `pip install "unstructured[docx]"`.
+    - To install the dependencies for all document types, use `pip install "unstructured[all-docs]"`.
 - Install the following system dependencies if they are not already available on your system.
   Depending on what document types you're parsing, you may not need all of these.
     - `libmagic-dev` (filetype detection)

docs/extras/integrations/providers/wandb_tracing.ipynb

@@ -177,7 +177,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,

docs/extras/integrations/retrievers/re_phrase.ipynb

@@ -0,0 +1,222 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "e8624be2",
+   "metadata": {},
+   "source": [
+    "# RePhraseQueryRetriever\n",
+    "\n",
+    "Simple retriever that applies an LLM between the user input and the query pass the to retriever.\n",
+    "\n",
+    "It can be used to pre-process the user input in any way.\n",
+    "\n",
+    "The default prompt used in the `from_llm` classmethod:\n",
+    "\n",
+    "```\n",
+    "DEFAULT_TEMPLATE = \"\"\"You are an assistant tasked with taking a natural language \\\n",
+    "query from a user and converting it into a query for a vectorstore. \\\n",
+    "In this process, you strip out information that is not relevant for \\\n",
+    "the retrieval task. Here is the user query: {question}\"\"\"\n",
+    "```\n",
+    "\n",
+    "Create a vectorstore."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "1bfa6834",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import WebBaseLoader\n",
+    "\n",
+    "loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
+    "data = loader.load()\n",
+    "\n",
+    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
+    "\n",
+    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)\n",
+    "all_splits = text_splitter.split_documents(data)\n",
+    "\n",
+    "from langchain.vectorstores import Chroma\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "\n",
+    "vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "d0b51556",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import logging\n",
+    "\n",
+    "logging.basicConfig()\n",
+    "logging.getLogger(\"langchain.retrievers.re_phraser\").setLevel(logging.INFO)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "20e1e787",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.retrievers import RePhraseQueryRetriever"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "88c0a972",
+   "metadata": {},
+   "source": [
+    "## Using the default prompt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "503994bd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = ChatOpenAI(temperature=0)\n",
+    "retriever_from_llm = RePhraseQueryRetriever.from_llm(\n",
+    "    retriever=vectorstore.as_retriever(), llm=llm\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "8d17ecc9",
+   "metadata": {
+    "scrolled": false
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:langchain.retrievers.re_phraser:Re-phrased question: The user query can be converted into a query for a vectorstore as follows:\n",
+      "\n",
+      "\"approaches to Task Decomposition\"\n"
+     ]
+    }
+   ],
+   "source": [
+    "docs = retriever_from_llm.get_relevant_documents(\n",
+    "    \"Hi I'm Lance. What are the approaches to Task Decomposition?\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "76d54f1a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:langchain.retrievers.re_phraser:Re-phrased question: Query for vectorstore: \"Types of Memory\"\n"
+     ]
+    }
+   ],
+   "source": [
+    "docs = retriever_from_llm.get_relevant_documents(\n",
+    "    \"I live in San Francisco. What are the Types of Memory?\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0513a6e2",
+   "metadata": {},
+   "source": [
+    "## Supply a prompt"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "410d6a64",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import LLMChain\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "\n",
+    "QUERY_PROMPT = PromptTemplate(\n",
+    "    input_variables=[\"question\"],\n",
+    "    template=\"\"\"You are an assistant tasked with taking a natural languge query from a user\n",
+    "    and converting it into a query for a vectorstore. In the process, strip out all \n",
+    "    information that is not relevant for the retrieval task and return a new, simplified\n",
+    "    question for vectorstore retrieval. The new user query should be in pirate speech.\n",
+    "    Here is the user query: {question} \"\"\",\n",
+    ")\n",
+    "llm = ChatOpenAI(temperature=0)\n",
+    "llm_chain = LLMChain(llm=llm, prompt=QUERY_PROMPT)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "2dbffdd3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "retriever_from_llm_chain = RePhraseQueryRetriever(\n",
+    "    retriever=vectorstore.as_retriever(), llm_chain=llm_chain\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "103b4be3",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:langchain.retrievers.re_phraser:Re-phrased question: Ahoy matey! What be Maximum Inner Product Search, ye scurvy dog?\n"
+     ]
+    }
+   ],
+   "source": [
+    "docs = retriever_from_llm_chain.get_relevant_documents(\n",
+    "    \"Hi I'm Lance. What is Maximum Inner Product Search?\"\n",
+    ")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/retrievers/tf_idf.ipynb

@@ -16,7 +16,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "id": "a801b57c",
    "metadata": {},
    "outputs": [],
@@ -26,7 +26,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "id": "393ac030",
    "metadata": {
     "tags": []
@@ -46,7 +46,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "id": "98b1c017",
    "metadata": {
     "tags": []
@@ -133,6 +133,68 @@
    "source": [
     "result"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "363f3c04",
+   "metadata": {},
+   "source": [
+    "## Save and load\n",
+    "\n",
+    "You can easily save and load this retriever, making it handy for local development!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "10c90d03",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "retriever.save_local(\"testing.pkl\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "fb3b153c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "retriever_copy = TFIDFRetriever.load_local(\"testing.pkl\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "c03ff3c7",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='foo', metadata={}),\n",
+       " Document(page_content='foo bar', metadata={}),\n",
+       " Document(page_content='hello', metadata={}),\n",
+       " Document(page_content='world', metadata={})]"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "retriever_copy.get_relevant_documents(\"foo\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2d7c5728",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -151,7 +213,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.10.1"
   }
  },
  "nbformat": 4,

docs/extras/integrations/text_embedding/aleph_alpha.ipynb

@@ -20,7 +20,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 1,
    "id": "8a920a89",
    "metadata": {},
    "outputs": [],
@@ -30,7 +30,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 2,
    "id": "f2d04da3",
    "metadata": {},
    "outputs": [],
@@ -41,17 +41,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "id": "e6ecde96",
    "metadata": {},
    "outputs": [],
    "source": [
-    "embeddings = AlephAlphaAsymmetricSemanticEmbedding()"
+    "embeddings = AlephAlphaAsymmetricSemanticEmbedding(normalize=True, compress_to_size=128)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 4,
    "id": "90e68411",
    "metadata": {},
    "outputs": [],
@@ -61,7 +61,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 5,
    "id": "55903233",
    "metadata": {},
    "outputs": [],
@@ -79,7 +79,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 6,
    "id": "eabb763a",
    "metadata": {},
    "outputs": [],
@@ -89,7 +89,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 7,
    "id": "0ad799f7",
    "metadata": {},
    "outputs": [],
@@ -99,17 +99,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 8,
    "id": "af86dc10",
    "metadata": {},
    "outputs": [],
    "source": [
-    "embeddings = AlephAlphaSymmetricSemanticEmbedding()"
+    "embeddings = AlephAlphaSymmetricSemanticEmbedding(normalize=True, compress_to_size=128)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 9,
    "id": "d292536f",
    "metadata": {},
    "outputs": [],
@@ -119,7 +119,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 10,
    "id": "c704a7cf",
    "metadata": {},
    "outputs": [],
@@ -130,7 +130,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "33492471",
+   "id": "5d999f8f",
    "metadata": {},
    "outputs": [],
    "source": []
@@ -152,7 +152,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.9.13"
   },
   "vscode": {
    "interpreter": {

docs/extras/integrations/text_embedding/edenai.ipynb

@@ -0,0 +1,169 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# EDEN AI"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Eden AI is an AI consulting company that was founded to use its resources to empower people and create impactful products that use AI to improve the quality of life for individuals, businesses and societies at large."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This example goes over how to use LangChain to interact with Eden AI embedding models\n",
+    "\n",
+    "-----------------------------------------------------------------------------------\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Accessing the EDENAI's API requires an API key, \n",
+    "\n",
+    "which you can get by creating an account https://app.edenai.run/user/register  and heading here https://app.edenai.run/admin/account/settings\n",
+    "\n",
+    "Once we have a key we'll want to set it as an environment variable by running:\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "export EDENAI_API_KEY=\"...\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If you'd prefer not to set an environment variable you can pass the key in directly via the edenai_api_key named parameter\n",
+    "\n",
+    " when initiating the EdenAI embedding class:\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings.edenai import EdenAiEmbeddings"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "embeddings = EdenAiEmbeddings(edenai_api_key=\"...\",provider=\"...\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Calling a model\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The EdenAI API brings together various providers.\n",
+    "\n",
+    "To access a specific model, you can simply use the \"provider\" when calling.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "embeddings = EdenAiEmbeddings(provider=\"openai\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "docs = [\"It's raining right now\", \"cats are cute\"]\n",
+    "document_result = embeddings.embed_documents(docs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query = \"my umbrella is broken\"\n",
+    "query_result = embeddings.embed_query(query)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Cosine similarity between \"It's raining right now\" and query: 0.849261496107252\n",
+      "Cosine similarity between \"cats are cute\" and query: 0.7525900655705218\n"
+     ]
+    }
+   ],
+   "source": [
+    "import numpy as np\n",
+    "\n",
+    "query_numpy = np.array(query_result)\n",
+    "for doc_res, doc in zip(document_result, docs):\n",
+    "    document_numpy = np.array(doc_res)\n",
+    "    similarity = np.dot(query_numpy, document_numpy) / (\n",
+    "        np.linalg.norm(query_numpy) * np.linalg.norm(document_numpy)\n",
+    "    )\n",
+    "    print(f'Cosine similarity between \"{doc}\" and query: {similarity}')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.11"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/extras/integrations/text_embedding/google_vertex_ai_palm.ipynb

@@ -1,18 +1,15 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "# Google Cloud Platform Vertex AI PaLM \n",
     "\n",
-    "Note: This is seperate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
+    "Note: This is seperate from the Google PaLM integration, it exposes [Vertex AI PaLM API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on Google Cloud. \n",
     "\n",
-    "PaLM API on Vertex AI is a Preview offering, subject to the Pre-GA Offerings Terms of the [GCP Service Specific Terms](https://cloud.google.com/terms/service-terms). \n",
-    "\n",
-    "Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages). Further, by using PaLM API on Vertex AI, you agree to the Generative AI Preview [terms and conditions](https://cloud.google.com/trustedtester/aitos) (Preview Terms).\n",
-    "\n",
-    "For PaLM API on Vertex AI, you can process personal data as outlined in the Cloud Data Processing Addendum, subject to applicable restrictions and obligations in the Agreement (as defined in the Preview Terms).\n",
+    "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) Customer Data to train its foundation models as part of Google Cloud`s AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
     "\n",
     "To use Vertex AI PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
     "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",

docs/extras/integrations/toolkits/multion.ipynb

@@ -5,7 +5,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Multion Toolkit\n",
+    "# MultiOn Toolkit\n",
     "\n",
     "This notebook walks you through connecting LangChain to the MultiOn Client in your browser\n",
     "\n",
@@ -18,7 +18,32 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install --upgrade multion > /dev/null"
+    "!pip install --upgrade multion langchain -q"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents.agent_toolkits import MultionToolkit\n",
+    "import os\n",
+    "\n",
+    "\n",
+    "toolkit = MultionToolkit()\n",
+    "\n",
+    "toolkit"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tools = toolkit.get_tools()\n",
+    "tools"
    ]
   },
   {
@@ -38,8 +63,9 @@
    "outputs": [],
    "source": [
     "# Authorize connection to your Browser extention\n",
-    "import multion \n",
-    "multion.login()\n"
+    "import multion\n",
+    "multion.login()\n",
+    "\n"
    ]
   },
   {
@@ -57,38 +83,18 @@
    },
    "outputs": [],
    "source": [
-    "from langchain.agents.agent_toolkits import create_multion_agent\n",
-    "from langchain.tools.multion.tool import MultionClientTool\n",
-    "from langchain.agents.agent_types import AgentType\n",
-    "from langchain.chat_models import ChatOpenAI"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "\n",
-    "agent_executor = create_multion_agent(\n",
-    "    llm=ChatOpenAI(temperature=0),\n",
-    "    tool=MultionClientTool(),\n",
-    "    agent_type=AgentType.OPENAI_FUNCTIONS,\n",
-    "    verbose=True\n",
-    ")\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "agent.run(\"show me the weather today\")"
+    "from langchain import OpenAI\n",
+    "from langchain.agents import initialize_agent, AgentType\n",
+    "llm = OpenAI(temperature=0)\n",
+    "from langchain.agents.agent_toolkits import MultionToolkit\n",
+    "toolkit = MultionToolkit()\n",
+    "tools=toolkit.get_tools()\n",
+    "agent = initialize_agent(\n",
+    "    tools=toolkit.get_tools(),\n",
+    "    llm=llm,\n",
+    "    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
+    "    verbose = True\n",
+    ")"
    ]
   },
   {
@@ -100,7 +106,7 @@
    "outputs": [],
    "source": [
     "agent.run(\n",
-    "    \"Tweet about Elon Musk\"\n",
+    "    \"Tweet 'Hi from MultiOn'\"\n",
     ")"
    ]
   }

docs/extras/integrations/tools/nuclia.ipynb

@@ -0,0 +1,173 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Nuclia Understanding API tool\n",
+    "\n",
+    "[Nuclia](https://nuclia.com) automatically indexes your unstructured data from any internal and external source, providing optimized search results and generative answers. It can handle video and audio transcription, image content extraction, and document parsing.\n",
+    "\n",
+    "The Nuclia Understanding API supports the processing of unstructured data, including text, web pages, documents, and audio/video contents. It extracts all texts wherever it is (using speech-to-text or OCR when needed), it identifies entities, it aslo extracts metadata, embedded files (like images in a PDF), and web links. It also provides a summary of the content.\n",
+    "\n",
+    "To use the Nuclia Understanding API, you need to have a Nuclia account. You can create one for free at [https://nuclia.cloud](https://nuclia.cloud), and then [create a NUA key](https://docs.nuclia.dev/docs/docs/using/understanding/intro)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install --upgrade protobuf\n",
+    "#!pip install nucliadb-protos"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"NUCLIA_ZONE\"] = \"<YOUR_ZONE>\"  # e.g. europe-1\n",
+    "os.environ[\"NUCLIA_NUA_KEY\"] = \"<YOUR_API_KEY>\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.tools.nuclia import NucliaUnderstandingAPI\n",
+    "\n",
+    "nua = NucliaUnderstandingAPI(enable_ml=False)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can push files to the Nuclia Understanding API using the `push` action. As the processing is done asynchronously, the results might be returned in a different order than the files were pushed. That is why you need to provide an `id` to match the results with the corresponding file."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "nua.run({\"action\": \"push\", \"id\": \"1\", \"path\": \"./report.docx\"})\n",
+    "nua.run({\"action\": \"push\", \"id\": \"2\", \"path\": \"./interview.mp4\"})"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can now call the `pull` action in a loop until you get the JSON-formatted result."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import time\n",
+    "\n",
+    "pending = True\n",
+    "data = None\n",
+    "while pending:\n",
+    "    time.sleep(15)\n",
+    "    data = nua.run({\"action\": \"pull\", \"id\": \"1\", \"path\": None})\n",
+    "    if data:\n",
+    "        print(data)\n",
+    "        pending = False\n",
+    "    else:\n",
+    "        print(\"waiting...\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can also do it in one step in `async` mode, you only need to do a push, and it will wait until the results are pulled:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import asyncio\n",
+    "\n",
+    "\n",
+    "async def process():\n",
+    "    data = await nua.arun(\n",
+    "        {\"action\": \"push\", \"id\": \"1\", \"path\": \"./talk.mp4\", \"text\": None}\n",
+    "    )\n",
+    "    print(data)\n",
+    "\n",
+    "\n",
+    "asyncio.run(process())"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Retrieved information\n",
+    "\n",
+    "Nuclia returns the following information:\n",
+    "\n",
+    "- file metadata\n",
+    "- extracted text\n",
+    "- nested text (like text in an embedded image)\n",
+    "- a summary (only when `enable_ml` is set to `True`)\n",
+    "- paragraphs and sentences splitting (defined by the position of their first and last characters, plus start time and end time for a video or audio file)\n",
+    "- named entities: people, dates, places, organizations, etc. (only when `enable_ml` is set to `True`)\n",
+    "- links\n",
+    "- a thumbnail\n",
+    "- embedded files\n",
+    "- the vector representations of the text (only when `enable_ml` is set to `True`)\n",
+    "\n",
+    "Note:\n",
+    "\n",
+    "  Generated files (thumbnail, extracted embedded files, etc.) are provided as a token. You can download them with the [`/processing/download` endpoint](https://docs.nuclia.dev/docs/api#operation/Download_binary_file_processing_download_get).\n",
+    "\n",
+    "  Also at any level, if an attribute exceeds a certain size, it will be put in a downloadable file and will be replaced in the document by a file pointer. This will consist of `{\"file\": {\"uri\": \"JWT_TOKEN\"}}`. The rule is that if the size of the message is greater than 1000000 characters, the biggest parts will be moved to downloadable files. First, the compression process will target vectors. If that is not enough, it will target large field metadata, and finally it will target extracted text.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "langchain",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.5"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/extras/integrations/vectorstores/chroma.ipynb

@@ -269,8 +269,9 @@
     "    )\n",
     "\n",
     "# tell LangChain to use our client and collection name\n",
-    "db4 = Chroma(client=client, collection_name=\"my_collection\")\n",
-    "docs = db.similarity_search(query)\n",
+    "db4 = Chroma(client=client, collection_name=\"my_collection\", embedding_function=embedding_function)\n",
+    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+    "docs = db4.similarity_search(query)\n",
     "print(docs[0].page_content)"
    ]
   },

docs/extras/integrations/vectorstores/faiss.ipynb

@@ -80,7 +80,7 @@
    "source": [
     "from langchain.document_loaders import TextLoader\n",
     "\n",
-    "loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
+    "loader = TextLoader(\"../../../extras/modules/state_of_the_union.txt\")\n",
     "documents = loader.load()\n",
     "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
     "docs = text_splitter.split_documents(documents)\n",
@@ -90,7 +90,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 3,
    "id": "5eabdb75",
    "metadata": {
     "tags": []
@@ -249,6 +249,50 @@
     "docs[0]"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "30c8f57b",
+   "metadata": {},
+   "source": [
+    "# Serializing and De-Serializing to bytes\n",
+    "\n",
+    "you can pickle the FAISS Index by these functions. If you use embeddings model which is of 90 mb (sentence-transformers/all-MiniLM-L6-v2 or any other model), the resultant pickle size would be more than 90 mb. the size of the model is also included in the overall size. To overcome this, use the below functions. These functions only serializes FAISS index and size would be much lesser. this can be helpful if you wish to store the index in database like sql."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d8faead5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pkl = db.serialize_to_bytes() # serializes the faiss index"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "eb083247",
+   "metadata": {
+    "vscode": {
+     "languageId": "r"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "embeddings = HuggingFaceEmbeddings(model_name=\"all-MiniLM-L6-v2\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e36e220b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "db = FAISS.deserialize_from_bytes(embeddings = embeddings, serialized = pkl) # Load the index"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "57da60d4",
@@ -473,6 +517,67 @@
     "for doc in results:\n",
     "    print(f\"Content: {doc.page_content}, Metadata: {doc.metadata}\")"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1becca53",
+   "metadata": {},
+   "source": [
+    "## Delete\n",
+    "\n",
+    "You can also delete ids. Note that the ids to delete should be the ids in the docstore."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "1408b870",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "db.delete([db.index_to_docstore_id[0]])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "d13daf33",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "False"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Is now missing\n",
+    "0 in db.index_to_docstore_id"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "30ace43e",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -491,7 +596,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.9.17"
   }
  },
  "nbformat": 4,

docs/extras/integrations/vectorstores/opensearch.ipynb

@@ -1,436 +1,440 @@
 {
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "683953b3",
-   "metadata": {},
-   "source": [
-    "# OpenSearch\n",
-    "\n",
-    "> [OpenSearch](https://opensearch.org/) is a scalable, flexible, and extensible open-source software suite for search, analytics, and observability applications licensed under Apache 2.0. `OpenSearch` is a distributed search and analytics engine based on `Apache Lucene`.\n",
-    "\n",
-    "\n",
-    "This notebook shows how to use functionality related to the `OpenSearch` database.\n",
-    "\n",
-    "To run, you should have an OpenSearch instance up and running: [see here for an easy Docker installation](https://hub.docker.com/r/opensearchproject/opensearch).\n",
-    "\n",
-    "`similarity_search` by default performs the Approximate k-NN Search which uses one of the several algorithms like lucene, nmslib, faiss recommended for\n",
-    "large datasets. To perform brute force search we have other search methods known as Script Scoring and Painless Scripting.\n",
-    "Check [this](https://opensearch.org/docs/latest/search-plugins/knn/index/) for more details."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "94963977-9dfc-48b7-872a-53f2947f46c6",
-   "metadata": {},
-   "source": [
-    "## Installation\n",
-    "Install the Python client."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6e606066-9386-4427-8a87-1b93f435c57e",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "!pip install opensearch-py"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b1fa637e-4fbf-4d5a-9188-2cad826a193e",
-   "metadata": {},
-   "source": [
-    "We want to use OpenAIEmbeddings so we have to get the OpenAI API Key."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "28e5455e-322d-4010-9e3b-491d522ef5db",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "import getpass\n",
-    "\n",
-    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "aac9563e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
-    "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain.vectorstores import OpenSearchVectorSearch\n",
-    "from langchain.document_loaders import TextLoader"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a3c3999a",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.document_loaders import TextLoader\n",
-    "\n",
-    "loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
-    "documents = loader.load()\n",
-    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
-    "docs = text_splitter.split_documents(documents)\n",
-    "\n",
-    "embeddings = OpenAIEmbeddings()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "01a9a035",
-   "metadata": {},
-   "source": [
-    "## similarity_search using Approximate k-NN\n",
-    "\n",
-    "`similarity_search` using `Approximate k-NN` Search with Custom Parameters"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "803fe12b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "    docs, embeddings, opensearch_url=\"http://localhost:9200\"\n",
-    ")\n",
-    "\n",
-    "# If using the default Docker installation, use this instantiation instead:\n",
-    "# docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "#     docs,\n",
-    "#     embeddings,\n",
-    "#     opensearch_url=\"https://localhost:9200\",\n",
-    "#     http_auth=(\"admin\", \"admin\"),\n",
-    "#     use_ssl = False,\n",
-    "#     verify_certs = False,\n",
-    "#     ssl_assert_hostname = False,\n",
-    "#     ssl_show_warn = False,\n",
-    "# )"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "db3fa309",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "docs = docsearch.similarity_search(query, k=10)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "c160d5bb",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(docs[0].page_content)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "96215c90",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "    docs,\n",
-    "    embeddings,\n",
-    "    opensearch_url=\"http://localhost:9200\",\n",
-    "    engine=\"faiss\",\n",
-    "    space_type=\"innerproduct\",\n",
-    "    ef_construction=256,\n",
-    "    m=48,\n",
-    ")\n",
-    "\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "docs = docsearch.similarity_search(query)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "62a7cea0",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(docs[0].page_content)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0d0cd877",
-   "metadata": {},
-   "source": [
-    "## similarity_search using Script Scoring\n",
-    "\n",
-    "`similarity_search` using `Script Scoring` with Custom Parameters"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "0a8e3c0e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "    docs, embeddings, opensearch_url=\"http://localhost:9200\", is_appx_search=False\n",
-    ")\n",
-    "\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "docs = docsearch.similarity_search(\n",
-    "    \"What did the president say about Ketanji Brown Jackson\",\n",
-    "    k=1,\n",
-    "    search_type=\"script_scoring\",\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "92bc40db",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(docs[0].page_content)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a4af96cc",
-   "metadata": {},
-   "source": [
-    "## similarity_search using Painless Scripting\n",
-    "\n",
-    "`similarity_search` using `Painless Scripting` with Custom Parameters"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6d9f436e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "    docs, embeddings, opensearch_url=\"http://localhost:9200\", is_appx_search=False\n",
-    ")\n",
-    "filter = {\"bool\": {\"filter\": {\"term\": {\"text\": \"smuggling\"}}}}\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "docs = docsearch.similarity_search(\n",
-    "    \"What did the president say about Ketanji Brown Jackson\",\n",
-    "    search_type=\"painless_scripting\",\n",
-    "    space_type=\"cosineSimilarity\",\n",
-    "    pre_filter=filter,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "8ca50bce",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(docs[0].page_content)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4f8fb0d0",
-   "metadata": {},
-   "source": [
-    "## Maximum marginal relevance search (MMR)\n",
-    "If you’d like to look up for some similar documents, but you’d also like to receive diverse results, MMR is method you should consider. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ba85e092",
-   "metadata": {
-    "collapsed": false,
-    "jupyter": {
-     "outputs_hidden": false
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "id": "683953b3",
+      "metadata": {},
+      "source": [
+        "# OpenSearch\n",
+        "\n",
+        "> [OpenSearch](https://opensearch.org/) is a scalable, flexible, and extensible open-source software suite for search, analytics, and observability applications licensed under Apache 2.0. `OpenSearch` is a distributed search and analytics engine based on `Apache Lucene`.\n",
+        "\n",
+        "\n",
+        "This notebook shows how to use functionality related to the `OpenSearch` database.\n",
+        "\n",
+        "To run, you should have an OpenSearch instance up and running: [see here for an easy Docker installation](https://hub.docker.com/r/opensearchproject/opensearch).\n",
+        "\n",
+        "`similarity_search` by default performs the Approximate k-NN Search which uses one of the several algorithms like lucene, nmslib, faiss recommended for\n",
+        "large datasets. To perform brute force search we have other search methods known as Script Scoring and Painless Scripting.\n",
+        "Check [this](https://opensearch.org/docs/latest/search-plugins/knn/index/) for more details."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "94963977-9dfc-48b7-872a-53f2947f46c6",
+      "metadata": {},
+      "source": [
+        "## Installation\n",
+        "Install the Python client."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "6e606066-9386-4427-8a87-1b93f435c57e",
+      "metadata": {
+        "tags": []
+      },
+      "outputs": [],
+      "source": [
+        "!pip install opensearch-py"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "b1fa637e-4fbf-4d5a-9188-2cad826a193e",
+      "metadata": {},
+      "source": [
+        "We want to use OpenAIEmbeddings so we have to get the OpenAI API Key."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "28e5455e-322d-4010-9e3b-491d522ef5db",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "import getpass\n",
+        "\n",
+        "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "aac9563e",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+        "from langchain.text_splitter import CharacterTextSplitter\n",
+        "from langchain.vectorstores import OpenSearchVectorSearch\n",
+        "from langchain.document_loaders import TextLoader"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "a3c3999a",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from langchain.document_loaders import TextLoader\n",
+        "\n",
+        "loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
+        "documents = loader.load()\n",
+        "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
+        "docs = text_splitter.split_documents(documents)\n",
+        "\n",
+        "embeddings = OpenAIEmbeddings()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "01a9a035",
+      "metadata": {},
+      "source": [
+        "## similarity_search using Approximate k-NN\n",
+        "\n",
+        "`similarity_search` using `Approximate k-NN` Search with Custom Parameters"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "803fe12b",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "docsearch = OpenSearchVectorSearch.from_documents(\n",
+        "    docs, embeddings, opensearch_url=\"http://localhost:9200\"\n",
+        ")\n",
+        "\n",
+        "# If using the default Docker installation, use this instantiation instead:\n",
+        "# docsearch = OpenSearchVectorSearch.from_documents(\n",
+        "#     docs,\n",
+        "#     embeddings,\n",
+        "#     opensearch_url=\"https://localhost:9200\",\n",
+        "#     http_auth=(\"admin\", \"admin\"),\n",
+        "#     use_ssl = False,\n",
+        "#     verify_certs = False,\n",
+        "#     ssl_assert_hostname = False,\n",
+        "#     ssl_show_warn = False,\n",
+        "# )"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "db3fa309",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+        "docs = docsearch.similarity_search(query, k=10)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "c160d5bb",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "print(docs[0].page_content)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "96215c90",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "docsearch = OpenSearchVectorSearch.from_documents(\n",
+        "    docs,\n",
+        "    embeddings,\n",
+        "    opensearch_url=\"http://localhost:9200\",\n",
+        "    engine=\"faiss\",\n",
+        "    space_type=\"innerproduct\",\n",
+        "    ef_construction=256,\n",
+        "    m=48,\n",
+        ")\n",
+        "\n",
+        "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+        "docs = docsearch.similarity_search(query)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "62a7cea0",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "print(docs[0].page_content)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "0d0cd877",
+      "metadata": {},
+      "source": [
+        "## similarity_search using Script Scoring\n",
+        "\n",
+        "`similarity_search` using `Script Scoring` with Custom Parameters"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "0a8e3c0e",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "docsearch = OpenSearchVectorSearch.from_documents(\n",
+        "    docs, embeddings, opensearch_url=\"http://localhost:9200\", is_appx_search=False\n",
+        ")\n",
+        "\n",
+        "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+        "docs = docsearch.similarity_search(\n",
+        "    \"What did the president say about Ketanji Brown Jackson\",\n",
+        "    k=1,\n",
+        "    search_type=\"script_scoring\",\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "92bc40db",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "print(docs[0].page_content)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "a4af96cc",
+      "metadata": {},
+      "source": [
+        "## similarity_search using Painless Scripting\n",
+        "\n",
+        "`similarity_search` using `Painless Scripting` with Custom Parameters"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "6d9f436e",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "docsearch = OpenSearchVectorSearch.from_documents(\n",
+        "    docs, embeddings, opensearch_url=\"http://localhost:9200\", is_appx_search=False\n",
+        ")\n",
+        "filter = {\"bool\": {\"filter\": {\"term\": {\"text\": \"smuggling\"}}}}\n",
+        "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+        "docs = docsearch.similarity_search(\n",
+        "    \"What did the president say about Ketanji Brown Jackson\",\n",
+        "    search_type=\"painless_scripting\",\n",
+        "    space_type=\"cosineSimilarity\",\n",
+        "    pre_filter=filter,\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "8ca50bce",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "print(docs[0].page_content)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "4f8fb0d0",
+      "metadata": {},
+      "source": [
+        "## Maximum marginal relevance search (MMR)\n",
+        "If you\u2019d like to look up for some similar documents, but you\u2019d also like to receive diverse results, MMR is method you should consider. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "ba85e092",
+      "metadata": {
+        "collapsed": false,
+        "jupyter": {
+          "outputs_hidden": false
+        }
+      },
+      "outputs": [],
+      "source": [
+        "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+        "docs = docsearch.max_marginal_relevance_search(query, k=2, fetch_k=10, lambda_param=0.5)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "73264864",
+      "metadata": {},
+      "source": [
+        "## Using a preexisting OpenSearch instance\n",
+        "\n",
+        "It's also possible to use a preexisting OpenSearch instance with documents that already have vectors present."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "82a23440",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# this is just an example, you would need to change these values to point to another opensearch instance\n",
+        "docsearch = OpenSearchVectorSearch(\n",
+        "    index_name=\"index-*\",\n",
+        "    embedding_function=embeddings,\n",
+        "    opensearch_url=\"http://localhost:9200\",\n",
+        ")\n",
+        "\n",
+        "# you can specify custom field names to match the fields you're using to store your embedding, document text value, and metadata\n",
+        "docs = docsearch.similarity_search(\n",
+        "    \"Who was asking about getting lunch today?\",\n",
+        "    search_type=\"script_scoring\",\n",
+        "    space_type=\"cosinesimil\",\n",
+        "    vector_field=\"message_embedding\",\n",
+        "    text_field=\"message\",\n",
+        "    metadata_field=\"message_metadata\",\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## Using AOSS (Amazon OpenSearch Service Serverless)"
+      ],
+      "metadata": {
+        "collapsed": false,
+        "pycharm": {
+          "name": "#%% md\n"
+        }
+      },
+      "id": "5f590d35"
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "outputs": [],
+      "source": [
+        "# This is just an example to show how to use AOSS with faiss engine and efficient_filter, you need to set proper values.\n",
+        "\n",
+        "service = 'aoss' # must set the service as 'aoss'\n",
+        "region = 'us-east-2'\n",
+        "credentials = boto3.Session(aws_access_key_id='xxxxxx',aws_secret_access_key='xxxxx').get_credentials()\n",
+        "awsauth = AWS4Auth('xxxxx', 'xxxxxx', region,service, session_token=credentials.token)\n",
+        "\n",
+        "docsearch = OpenSearchVectorSearch.from_documents(\n",
+        "    docs,\n",
+        "    embeddings,\n",
+        "    opensearch_url=\"host url\",\n",
+        "    http_auth=awsauth,\n",
+        "    timeout = 300,\n",
+        "    use_ssl = True,\n",
+        "    verify_certs = True,\n",
+        "    connection_class = RequestsHttpConnection,\n",
+        "    index_name=\"test-index-using-aoss\",\n",
+        "    engine=\"faiss\",\n",
+        ")\n",
+        "\n",
+        "docs = docsearch.similarity_search(\n",
+        "    \"What is feature selection\",\n",
+        "     efficient_filter=filter,\n",
+        "     k=200,\n",
+        ")"
+      ],
+      "metadata": {
+        "collapsed": false,
+        "pycharm": {
+          "name": "#%%\n"
+        }
+      },
+      "id": "de397be7"
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## Using AOS (Amazon OpenSearch Service)"
+      ],
+      "metadata": {
+        "collapsed": false
+      },
+      "id": "0aa012c8"
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "outputs": [],
+      "source": [
+        "# This is just an example to show how to use AOS , you need to set proper values.\n",
+        "\n",
+        "service = 'es' # must set the service as 'es'\n",
+        "region = 'us-east-2'\n",
+        "credentials = boto3.Session(aws_access_key_id='xxxxxx',aws_secret_access_key='xxxxx').get_credentials()\n",
+        "awsauth = AWS4Auth('xxxxx', 'xxxxxx', region,service, session_token=credentials.token)\n",
+        "\n",
+        "docsearch = OpenSearchVectorSearch.from_documents(\n",
+        "    docs,\n",
+        "    embeddings,\n",
+        "    opensearch_url=\"host url\",\n",
+        "    http_auth=awsauth,\n",
+        "    timeout = 300,\n",
+        "    use_ssl = True,\n",
+        "    verify_certs = True,\n",
+        "    connection_class = RequestsHttpConnection,\n",
+        "    index_name=\"test-index\",\n",
+        ")\n",
+        "\n",
+        "docs = docsearch.similarity_search(\n",
+        "    \"What is feature selection\",\n",
+        "     k=200,\n",
+        ")"
+      ],
+      "metadata": {
+        "collapsed": false,
+        "pycharm": {
+          "name": "#%%\n"
+        }
+      },
+      "id": "2c47e408"
     }
-   },
-   "outputs": [],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "docs = docsearch.max_marginal_relevance_search(query, k=2, fetch_k=10, lambda_param=0.5)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "73264864",
-   "metadata": {},
-   "source": [
-    "## Using a preexisting OpenSearch instance\n",
-    "\n",
-    "It's also possible to use a preexisting OpenSearch instance with documents that already have vectors present."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "82a23440",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# this is just an example, you would need to change these values to point to another opensearch instance\n",
-    "docsearch = OpenSearchVectorSearch(\n",
-    "    index_name=\"index-*\",\n",
-    "    embedding_function=embeddings,\n",
-    "    opensearch_url=\"http://localhost:9200\",\n",
-    ")\n",
-    "\n",
-    "# you can specify custom field names to match the fields you're using to store your embedding, document text value, and metadata\n",
-    "docs = docsearch.similarity_search(\n",
-    "    \"Who was asking about getting lunch today?\",\n",
-    "    search_type=\"script_scoring\",\n",
-    "    space_type=\"cosinesimil\",\n",
-    "    vector_field=\"message_embedding\",\n",
-    "    text_field=\"message\",\n",
-    "    metadata_field=\"message_metadata\",\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "## Using AOSS (Amazon OpenSearch Service Serverless)"
-   ],
-   "metadata": {
-    "collapsed": false,
-    "pycharm": {
-     "name": "#%% md\n"
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3 (ipykernel)",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.10.6"
     }
-   }
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "# This is just an example to show how to use AOSS with faiss engine and efficient_filter, you need to set proper values.\n",
-    "\n",
-    "service = 'aoss' # must set the service as 'aoss'\n",
-    "region = 'us-east-2'\n",
-    "credentials = boto3.Session(aws_access_key_id='xxxxxx',aws_secret_access_key='xxxxx').get_credentials()\n",
-    "awsauth = AWS4Auth('xxxxx', 'xxxxxx', region,service, session_token=credentials.token)\n",
-    "\n",
-    "docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "    docs,\n",
-    "    embeddings,\n",
-    "    opensearch_url=\"host url\",\n",
-    "    http_auth=awsauth,\n",
-    "    timeout = 300,\n",
-    "    use_ssl = True,\n",
-    "    verify_certs = True,\n",
-    "    connection_class = RequestsHttpConnection,\n",
-    "    index_name=\"test-index-using-aoss\",\n",
-    "    engine=\"faiss\",\n",
-    ")\n",
-    "\n",
-    "docs = docsearch.similarity_search(\n",
-    "    \"What is feature selection\",\n",
-    "     efficient_filter=filter,\n",
-    "     k=200,\n",
-    ")"
-   ],
-   "metadata": {
-    "collapsed": false,
-    "pycharm": {
-     "name": "#%%\n"
-    }
-   }
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "## Using AOS (Amazon OpenSearch Service)"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "# This is just an example to show how to use AOS , you need to set proper values.\n",
-    "\n",
-    "service = 'es' # must set the service as 'es'\n",
-    "region = 'us-east-2'\n",
-    "credentials = boto3.Session(aws_access_key_id='xxxxxx',aws_secret_access_key='xxxxx').get_credentials()\n",
-    "awsauth = AWS4Auth('xxxxx', 'xxxxxx', region,service, session_token=credentials.token)\n",
-    "\n",
-    "docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "    docs,\n",
-    "    embeddings,\n",
-    "    opensearch_url=\"host url\",\n",
-    "    http_auth=awsauth,\n",
-    "    timeout = 300,\n",
-    "    use_ssl = True,\n",
-    "    verify_certs = True,\n",
-    "    connection_class = RequestsHttpConnection,\n",
-    "    index_name=\"test-index\",\n",
-    ")\n",
-    "\n",
-    "docs = docsearch.similarity_search(\n",
-    "    \"What is feature selection\",\n",
-    "     k=200,\n",
-    ")"
-   ],
-   "metadata": {
-    "collapsed": false,
-    "pycharm": {
-     "name": "#%%\n"
-    }
-   }
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.6"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
+  "nbformat": 4,
+  "nbformat_minor": 5
 }

docs/extras/integrations/vectorstores/rockset.ipynb

@@ -21,7 +21,10 @@
     "\n",
     "1. Leverage the `Rockset` console to create a [collection](https://rockset.com/docs/collections/) with the Write API as your source. In this walkthrough, we create a collection named `langchain_demo`. \n",
     "    \n",
-    "    Configure the following [ingest transformation](https://rockset.com/docs/ingest-transformation/) to mark your embeddings field and take advantage of performance and storage optimizations:"
+    "    Configure the following [ingest transformation](https://rockset.com/docs/ingest-transformation/) to mark your embeddings field and take advantage of performance and storage optimizations:\n",
+    "\n",
+    "\n",
+    "   (We used OpenAI `text-embedding-ada-002` for this examples, where #length_of_vector_embedding = 1536)"
    ]
   },
   {
@@ -75,23 +78,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "id": "29505c1e",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "InitializationException",
-     "evalue": "The rockset client was initialized incorrectly: An api key must be provided as a parameter to the RocksetClient or the Configuration object.",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mInitializationException\u001b[0m                   Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[5], line 6\u001b[0m\n\u001b[1;32m      4\u001b[0m ROCKSET_API_KEY \u001b[39m=\u001b[39m os\u001b[39m.\u001b[39menviron\u001b[39m.\u001b[39mget(\u001b[39m\"\u001b[39m\u001b[39mROCKSET_API_KEY\u001b[39m\u001b[39m\"\u001b[39m) \u001b[39m# Verify ROCKSET_API_KEY environment variable\u001b[39;00m\n\u001b[1;32m      5\u001b[0m ROCKSET_API_SERVER \u001b[39m=\u001b[39m rockset\u001b[39m.\u001b[39mRegions\u001b[39m.\u001b[39musw2a1 \u001b[39m# Verify Rockset region\u001b[39;00m\n\u001b[0;32m----> 6\u001b[0m rockset_client \u001b[39m=\u001b[39m rockset\u001b[39m.\u001b[39;49mRocksetClient(ROCKSET_API_SERVER, ROCKSET_API_KEY)\n\u001b[1;32m      8\u001b[0m COLLECTION_NAME\u001b[39m=\u001b[39m\u001b[39m'\u001b[39m\u001b[39mlangchain_demo\u001b[39m\u001b[39m'\u001b[39m\n\u001b[1;32m      9\u001b[0m TEXT_KEY\u001b[39m=\u001b[39m\u001b[39m'\u001b[39m\u001b[39mdescription\u001b[39m\u001b[39m'\u001b[39m\n",
-      "File \u001b[0;32m~/Library/Python/3.9/lib/python/site-packages/rockset/rockset_client.py:242\u001b[0m, in \u001b[0;36mRocksetClient.__init__\u001b[0;34m(self, host, api_key, max_workers, config)\u001b[0m\n\u001b[1;32m    239\u001b[0m     config\u001b[39m.\u001b[39mhost \u001b[39m=\u001b[39m host\n\u001b[1;32m    241\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m config\u001b[39m.\u001b[39mapi_key:\n\u001b[0;32m--> 242\u001b[0m     \u001b[39mraise\u001b[39;00m InitializationException(\n\u001b[1;32m    243\u001b[0m         \u001b[39m\"\u001b[39m\u001b[39mAn api key must be provided as a parameter to the RocksetClient or the Configuration object.\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[1;32m    244\u001b[0m     )\n\u001b[1;32m    246\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mapi_client \u001b[39m=\u001b[39m ApiClient(config, max_workers\u001b[39m=\u001b[39mmax_workers)\n\u001b[1;32m    248\u001b[0m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mAliases \u001b[39m=\u001b[39m AliasesApiWrapper(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mapi_client)\n",
-      "\u001b[0;31mInitializationException\u001b[0m: The rockset client was initialized incorrectly: An api key must be provided as a parameter to the RocksetClient or the Configuration object."
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "import os\n",
     "import rockset\n",
@@ -118,18 +108,7 @@
    "execution_count": null,
    "id": "9740d8c4",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "",
-     "evalue": "",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[1;31mRunning cells with '/opt/local/bin/python3.11' requires the ipykernel package.\n",
-      "\u001b[1;31mRun the following command to install 'ipykernel' into the Python environment. \n",
-      "\u001b[1;31mCommand: '/opt/local/bin/python3.11 -m pip install ipykernel -U --user --force-reinstall'"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from langchain.embeddings.openai import OpenAIEmbeddings\n",
     "from langchain.text_splitter import CharacterTextSplitter\n",
@@ -155,20 +134,9 @@
    "execution_count": null,
    "id": "85b6a6c5",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "",
-     "evalue": "",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[1;31mRunning cells with '/opt/local/bin/python3.11' requires the ipykernel package.\n",
-      "\u001b[1;31mRun the following command to install 'ipykernel' into the Python environment. \n",
-      "\u001b[1;31mCommand: '/opt/local/bin/python3.11 -m pip install ipykernel -U --user --force-reinstall'"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
-    "embeddings = OpenAIEmbeddings() # Verify OPENAI_KEY environment variable\n",
+    "embeddings = OpenAIEmbeddings() # Verify OPENAI_API_KEY environment variable\n",
     "\n",
     "docsearch = Rockset(\n",
     "    client=rockset_client,\n",
@@ -194,22 +162,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "0bbf3df0",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "NameError",
-     "evalue": "name 'docsearch' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[1], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m query \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39mWhat did the president say about Ketanji Brown Jackson?\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[0;32m----> 2\u001b[0m output \u001b[39m=\u001b[39m docsearch\u001b[39m.\u001b[39msimilarity_search_with_relevance_scores(query, \u001b[39m4\u001b[39m, Rockset\u001b[39m.\u001b[39mDistanceFunction\u001b[39m.\u001b[39mCOSINE_SIM)\n\u001b[1;32m      4\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39moutput length:\u001b[39m\u001b[39m\"\u001b[39m, \u001b[39mlen\u001b[39m(output))\n\u001b[1;32m      5\u001b[0m \u001b[39mfor\u001b[39;00m d, dist \u001b[39min\u001b[39;00m output:\n",
-      "\u001b[0;31mNameError\u001b[0m: name 'docsearch' is not defined"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "query = \"What did the president say about Ketanji Brown Jackson\"\n",
     "output = docsearch.similarity_search_with_relevance_scores(\n",
@@ -313,7 +269,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.6"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/extras/integrations/vectorstores/scann.ipynb

@@ -0,0 +1,190 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "e4afbbb6",
+   "metadata": {},
+   "source": [
+    "# ScaNN\n",
+    "\n",
+    "ScaNN (Scalable Nearest Neighbors) is a method for efficient vector similarity search at scale.\n",
+    "\n",
+    "ScaNN includes search space pruning and quantization for Maximum Inner Product Search and also supports other distance functions such as Euclidean distance. The implementation is optimized for x86 processors with AVX2 support. See its [Google Research github](https://github.com/google-research/google-research/tree/master/scann) for more details."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "082f593e",
+   "metadata": {},
+   "source": [
+    "## Installation\n",
+    "Install ScaNN through pip. Alternatively, you can follow instructions on the [ScaNN Website](https://github.com/google-research/google-research/tree/master/scann#building-from-source) to install from source."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a35e4f09",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install scann"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "44bf38a8",
+   "metadata": {},
+   "source": [
+    "## Retrieval Demo\n",
+    "\n",
+    "Below we show how to use ScaNN in conjunction with Huggingface Embeddings."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "377bc723",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \\n\\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \\n\\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': 'state_of_the_union.txt'})"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.embeddings import HuggingFaceEmbeddings\n",
+    "from langchain.text_splitter import CharacterTextSplitter\n",
+    "from langchain.vectorstores import ScaNN\n",
+    "from langchain.document_loaders import TextLoader\n",
+    "\n",
+    "loader = TextLoader(\"state_of_the_union.txt\")\n",
+    "documents = loader.load()\n",
+    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
+    "docs = text_splitter.split_documents(documents)\n",
+    "\n",
+    "from langchain.embeddings import TensorflowHubEmbeddings\n",
+    "embeddings = HuggingFaceEmbeddings()\n",
+    "\n",
+    "db = ScaNN.from_documents(docs, embeddings)\n",
+    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+    "docs = db.similarity_search(query)\n",
+    "\n",
+    "docs[0]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9ad5b151",
+   "metadata": {},
+   "source": [
+    "## RetrievalQA Demo\n",
+    "\n",
+    "Next, we demonstrate using ScaNN in conjunction with Google PaLM API.\n",
+    "\n",
+    "You can obtain an API key from https://developers.generativeai.google/tutorials/setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "fc27ad51",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains import RetrievalQA\n",
+    "from langchain.chat_models import google_palm\n",
+    "\n",
+    "palm_client = google_palm.ChatGooglePalm(google_api_key='YOUR_GOOGLE_PALM_API_KEY')\n",
+    "\n",
+    "qa = RetrievalQA.from_chain_type(\n",
+    "    llm=palm_client,\n",
+    "    chain_type=\"stuff\",\n",
+    "    retriever=db.as_retriever(search_kwargs={'k': 10})\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "5b77f919",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The president said that Ketanji Brown Jackson is one of our nation's top legal minds, who will continue Justice Breyer's legacy of excellence.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(qa.run('What did the president say about Ketanji Brown Jackson?'))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "0c6deec6",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The president did not mention Michael Phelps in his speech.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(qa.run('What did the president say about Michael Phelps?'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8a49f4a6",
+   "metadata": {},
+   "source": [
+    "## Save and loading local retrieval index"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "6b7496b9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "db.save_local('/tmp/db', 'state_of_union')\n",
+    "restored_db = ScaNN.load_local('/tmp/db', embeddings, index_name='state_of_union')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/integrations/vectorstores/vectara.ipynb

@@ -8,32 +8,55 @@
    "source": [
     "# Vectara\n",
     "\n",
-    ">[Vectara](https://vectara.com/) is a API platform for building LLM-powered applications. It provides a simple to use API for document indexing and query that is managed by Vectara and is optimized for performance and accuracy. \n",
+    ">[Vectara](https://vectara.com/) is a API platform for building GenAI applications. It provides an easy-to-use API for document indexing and query that is managed by Vectara and is optimized for performance and accuracy. \n",
+    "See the [Vectara API documentation ](https://docs.vectara.com/docs/) for more information on how to use the API.\n",
     "\n",
+    "This notebook shows how to use functionality related to the `Vectara`'s integration with langchain.\n",
+    "Note that unlike many other integrations in this category, Vectara provides an end-to-end managed service for [Grounded Generation](https://vectara.com/grounded-generation/) (aka retrieval agumented generation), which includes:\n",
+    "1. A way to extract text from document files and chunk them into sentences.\n",
+    "2. Its own embeddings model and vector store - each text segment is encoded into a vector embedding and stored in the Vectara internal vector store\n",
+    "3. A query service that automatically encodes the query into embedding, and retrieves the most relevant text segments (including support for [Hybrid Search](https://docs.vectara.com/docs/api-reference/search-apis/lexical-matching))\n",
     "\n",
-    "This notebook shows how to use functionality related to the `Vectara` vector database or the `Vectara` retriever. \n",
-    "\n",
-    "See the [Vectara API documentation ](https://docs.vectara.com/docs/) for more information on how to use the API."
+    "All of these are supported in this LangChain integration."
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "aac9563e",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:22.282884Z",
-     "start_time": "2023-04-04T10:51:21.408077Z"
-    },
-    "tags": []
-   },
-   "outputs": [],
+   "cell_type": "markdown",
+   "id": "dc0f4344",
+   "metadata": {},
    "source": [
+    "# Setup\n",
+    "\n",
+    "You will need a Vectara account to use Vectara with LangChain. To get started, use the following steps:\n",
+    "1. [Sign up](https://console.vectara.com/signup) for a Vectara account if you don't already have one. Once you have completed your sign up you will have a Vectara customer ID. You can find your customer ID by clicking on your name, on the top-right of the Vectara console window.\n",
+    "2. Within your account you can create one or more corpora. Each corpus represents an area that stores text data upon ingest from input documents. To create a corpus, use the **\"Create Corpus\"** button. You then provide a name to your corpus as well as a description. Optionally you can define filtering attributes and apply some advanced options. If you click on your created corpus, you can see its name and corpus ID right on the top.\n",
+    "3. Next you'll need to create API keys to access the corpus. Click on the **\"Authorization\"** tab in the corpus view and then the **\"Create API Key\"** button. Give your key a name, and choose whether you want query only or query+index for your key. Click \"Create\" and you now have an active API key. Keep this key confidential. \n",
+    "\n",
+    "To use LangChain with Vectara, you'll need to have these three values: customer ID, corpus ID and api_key.\n",
+    "You can provide those to LangChain in two ways:\n",
+    "\n",
+    "1. Include in your environment these three variables: `VECTARA_CUSTOMER_ID`, `VECTARA_CORPUS_ID` and `VECTARA_API_KEY`.\n",
+    "\n",
+    "> For example, you can set these variables using os.environ and getpass as follows:\n",
+    "\n",
+    "```python\n",
     "import os\n",
-    "from langchain.embeddings import FakeEmbeddings\n",
-    "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain.vectorstores import Vectara\n",
-    "from langchain.document_loaders import TextLoader"
+    "import getpass\n",
+    "\n",
+    "os.environ[\"VECTARA_CUSTOMER_ID\"] = getpass.getpass(\"Vectara Customer ID:\")\n",
+    "os.environ[\"VECTARA_CORPUS_ID\"] = getpass.getpass(\"Vectara Corpus ID:\")\n",
+    "os.environ[\"VECTARA_API_KEY\"] = getpass.getpass(\"Vectara API Key:\")\n",
+    "```\n",
+    "\n",
+    "2. Add them to the Vectara vectorstore constructor:\n",
+    "\n",
+    "```python\n",
+    "vectorstore = Vectara(\n",
+    "                vectara_customer_id=vectara_customer_id,\n",
+    "                vectara_corpus_id=vectara_corpus_id,\n",
+    "                vectara_api_key=vectara_api_key\n",
+    "            )\n",
+    "```"
    ]
   },
   {
@@ -44,8 +67,21 @@
    "source": [
     "## Connecting to Vectara from LangChain\n",
     "\n",
-    "The Vectara API provides simple API endpoints for indexing and querying, which is encapsulated in the Vectara integration.\n",
-    "First let's ingest the documents using the from_documents() method:"
+    "To get started, let's ingest the documents using the from_documents() method.\n",
+    "We assume here that you've added your VECTARA_CUSTOMER_ID, VECTARA_CORPUS_ID and query+indexing VECTARA_API_KEY as environment variables."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "04a1f1a0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings import FakeEmbeddings\n",
+    "from langchain.text_splitter import CharacterTextSplitter\n",
+    "from langchain.vectorstores import Vectara\n",
+    "from langchain.document_loaders import TextLoader"
    ]
   },
   {
@@ -88,7 +124,7 @@
    "metadata": {},
    "source": [
     "Vectara's indexing API provides a file upload API where the file is handled directly by Vectara - pre-processed, chunked optimally and added to the Vectara vector store.\n",
-    "To use this, we added the add_files() method (and from_files()). \n",
+    "To use this, we added the add_files() method (as well as from_files()). \n",
     "\n",
     "Let's see this in action. We pick two PDF documents to upload: \n",
     "1. The \"I have a dream\" speech by Dr. King\n",
@@ -296,7 +332,7 @@
    "source": [
     "## Vectara as a Retriever\n",
     "\n",
-    "Vectara, as all the other vector stores, can be used also as a LangChain Retriever:"
+    "Vectara, as all the other LangChain vectorstores, is most often used as a LangChain Retriever:"
    ]
   },
   {
@@ -378,7 +414,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,

docs/extras/integrations/vectorstores/xata.ipynb

@@ -0,0 +1,240 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Xata\n",
+    "\n",
+    "> [Xata](https://xata.io) is a serverless data platform, based on PostgreSQL. It provides a Python SDK for interacting with your database, and a UI for managing your data.\n",
+    "> Xata has a native vector type, which can be added to any table, and supports similarity search. LangChain inserts vectors directly to Xata, and queries it for the nearest neighbors of a given vector, so that you can use all the LangChain Embeddings integrations with Xata."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This notebook guides you how to use Xata as a VectorStore."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup\n",
+    "\n",
+    "### Create a database to use as a vector store\n",
+    "\n",
+    "In the [Xata UI](https://app.xata.io) create a new database. You can name it whatever you want, in this notepad we'll use `langchain`.\n",
+    "Create a table, again you can name it anything, but we will use `vectors`. Add the following columns via the UI:\n",
+    "\n",
+    "* `content` of type \"Text\". This is used to store the `Document.pageContent` values.\n",
+    "* `embedding` of type \"Vector\". Use the dimension used by the model you plan to use. In this notebook we use OpenAI embeddings, which have 1536 dimensions.\n",
+    "* `search` of type \"Text\". This is used as a metadata column by this example.\n",
+    "* any other columns you want to use as metadata. They are populated from the `Document.metadata` object. For example, if in the `Document.metadata` object you have a `title` property, you can create a `title` column in the table and it will be populated.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's first install our dependencies:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "!pip install xata==1.0.0a7 openai tiktoken langchain"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's load the OpenAI key to the environemnt. If you don't have one you can create an OpenAI account and create a key on this [page](https://platform.openai.com/account/api-keys)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import getpass\n",
+    "\n",
+    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Similarly, we need to get the environment variables for Xata. You can create a new API key by visiting your [account settings](https://app.xata.io/settings). To find the database URL, go to the Settings page of the database that you have created. The database URL should look something like this: `https://demo-uni3q8.eu-west-1.xata.sh/db/langchain`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "api_key = getpass.getpass(\"Xata API key: \")\n",
+    "db_url = input(\"Xata database URL (copy it from your DB settings):\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+    "from langchain.text_splitter import CharacterTextSplitter\n",
+    "from langchain.document_loaders import TextLoader\n",
+    "from langchain.vectorstores.xata import XataVectorStore\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Create the Xata vector store\n",
+    "Let's import our test dataset:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
+    "documents = loader.load()\n",
+    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
+    "docs = text_splitter.split_documents(documents)\n",
+    "\n",
+    "embeddings = OpenAIEmbeddings()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now create the actual vector store, backed by the Xata table."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "vector_store = XataVectorStore.from_documents(docs, embeddings, api_key=api_key, db_url=db_url, table_name=\"vectors\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After running the above command, if you go to the Xata UI, you should see the documents loaded together with their embeddings."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Similarity Search"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+    "found_docs = vector_store.similarity_search(query)\n",
+    "print(found_docs)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Similarity Search with score (vector distance)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
+    "result = vector_store.similarity_search_with_score(query)\n",
+    "for doc, score in result:\n",
+    "    print(f\"document={doc}, score={score}\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/extras/modules/data_connection/document_transformers/post_retrieval/long_context_reorder.ipynb

@@ -109,11 +109,11 @@
    "source": [
     "# Reorder the documents:\n",
     "# Less relevant document will be at the middle of the list and more\n",
-    "# relevant elements at begining / end.\n",
+    "# relevant elements at beginning / end.\n",
     "reordering = LongContextReorder()\n",
     "reordered_docs = reordering.transform_documents(docs)\n",
     "\n",
-    "# Confirm that the 4 relevant documents are at begining and end.\n",
+    "# Confirm that the 4 relevant documents are at beginning and end.\n",
     "reordered_docs"
    ]
   },

docs/extras/modules/model_io/prompts/prompt_templates/connecting_to_a_feature_store.ipynb

@@ -25,7 +25,7 @@
     "\n",
     "To start, we will use the popular open source feature store framework [Feast](https://github.com/feast-dev/feast).\n",
     "\n",
-    "This assumes you have already run the steps in the README around getting started. We will build of off that example in getting started, and create and LLMChain to write a note to a specific driver regarding their up-to-date statistics."
+    "This assumes you have already run the steps in the README around getting started. We will build off of that example in getting started, and create and LLMChain to write a note to a specific driver regarding their up-to-date statistics."
    ]
   },
   {

docs/extras/use_cases/agents/baby_agi.ipynb

@@ -343,7 +343,7 @@
     "                self.print_task_result(result)\n",
     "\n",
     "                # Step 3: Store the result in Pinecone\n",
-    "                result_id = f\"result_{task['task_id']}\"\n",
+    "                result_id = f\"result_{task['task_id']}_{num_iters}\"\n",
     "                self.vectorstore.add_texts(\n",
     "                    texts=[result],\n",
     "                    metadatas=[{\"task\": task[\"task_name\"]}],\n",

docs/extras/use_cases/agents/baby_agi_with_agent.ipynb

@@ -358,7 +358,7 @@
     "                self.print_task_result(result)\n",
     "\n",
     "                # Step 3: Store the result in Pinecone\n",
-    "                result_id = f\"result_{task['task_id']}\"\n",
+    "                result_id = f\"result_{task['task_id']}_{num_iters}\"\n",
     "                self.vectorstore.add_texts(\n",
     "                    texts=[result],\n",
     "                    metadatas=[{\"task\": task[\"task_name\"]}],\n",

docs/extras/use_cases/chatbots.ipynb

@@ -0,0 +1,727 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "ee7f95e4",
+   "metadata": {},
+   "source": [
+    "# Chatbots\n",
+    "\n",
+    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/extras/use_cases/chatbots/chatbots.ipynb)\n",
+    "\n",
+    "## Use case\n",
+    "\n",
+    "Chatbots are one of the central LLM use-cases. The core features of chatbots are that they can have long-running conversations and have access to information that users want to know about.\n",
+    "\n",
+    "Aside from basic prompting and LLMs, memory and retrieval are the core components of a chatbot. Memory allows a chatbot to remember past interactions, and retrieval provides a chatbot with up-to-date, domain-specific information."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "56615b45",
+   "metadata": {},
+   "source": [
+    "![Image description](/img/chat_use_case.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ff48f490",
+   "metadata": {},
+   "source": [
+    "## Overview\n",
+    "\n",
+    "The chat model interface is based around messages rather than raw text. Several components are important to consider for chat:\n",
+    "\n",
+    "* `chat model`: See [here](/docs/integrations/chat) for a list of chat model integrations and [here](/docs/modules/model_io/models/chat) for documentation on the chat model interface in LangChain. You can use `LLMs` (see [here](/docs/modules/model_io/models/llms)) for chatbots as well, but chat models have a more conversational tone and natively support a message interface.\n",
+    "* `prompt template`: Prompt templates make it easy to assemble prompts that combine default messages, user input, chat history, and (optionally) additional retrieved context.\n",
+    "* `memory`: [See here](/docs/modules/memory/) for in-depth documentation on memory types\n",
+    "* `retriever` (optional): [See here](/docs/modules/data_connection/retrievers) for in-depth documentation on retrieval systems. These are useful if you want to build a chatbot with domain-specific knowledge.\n",
+    "\n",
+    "## Quickstart\n",
+    "\n",
+    "Here's a quick preview of how we can create chatbot interfaces. First let's install some dependencies and set the required credentials:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5070a1fd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install langchain openai \n",
+    "\n",
+    "# Set env var OPENAI_API_KEY or load from a .env file:\n",
+    "# import dotenv\n",
+    "# dotenv.load_env()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "88197b95",
+   "metadata": {},
+   "source": [
+    "With a plain chat model, we can get chat completions by [passing one or more messages](/docs/modules/model_io/models/chat) to the model.\n",
+    "\n",
+    "The chat model will respond with a message."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "5b0d84ae",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=\"J'adore la programmation.\", additional_kwargs={}, example=False)"
+      ]
+     },
+     "execution_count": 17,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.schema import (\n",
+    "    AIMessage,\n",
+    "    HumanMessage,\n",
+    "    SystemMessage\n",
+    ")\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "\n",
+    "chat = ChatOpenAI()\n",
+    "chat([HumanMessage(content=\"Translate this sentence from English to French: I love programming.\")])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7935d9a5",
+   "metadata": {},
+   "source": [
+    "And if we pass in a list of messages:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "afd27a9f",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=\"J'adore la programmation.\", additional_kwargs={}, example=False)"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "messages = [\n",
+    "    SystemMessage(content=\"You are a helpful assistant that translates English to French.\"),\n",
+    "    HumanMessage(content=\"I love programming.\")\n",
+    "]\n",
+    "chat(messages)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c7a1d169",
+   "metadata": {},
+   "source": [
+    "We can then wrap our chat model in a `ConversationChain`, which has built-in memory for remembering past user inputs and model outputs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "fdb05d74",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Je adore la programmation.'"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.chains import ConversationChain  \n",
+    "  \n",
+    "conversation = ConversationChain(llm=chat)  \n",
+    "conversation.run(\"Translate this sentence from English to French: I love programming.\")  "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "d801a173",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Ich liebe Programmieren.'"
+      ]
+     },
+     "execution_count": 21,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation.run(\"Translate it to German.\")  "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9e86788c",
+   "metadata": {},
+   "source": [
+    "## Memory \n",
+    "\n",
+    "As we mentioned above, the core component of chatbots is the memory system. One of the simplest and most commonly used forms of memory is `ConversationBufferMemory`:\n",
+    "* This memory allows for storing of messages in a `buffer`\n",
+    "* When called in a chain, it returns all of the messages it has stored\n",
+    "\n",
+    "LangChain comes with many other types of memory, too. [See here](/docs/modules/memory/) for in-depth documentation on memory types.\n",
+    "\n",
+    "For now let's take a quick look at ConversationBufferMemory. We can manually add a few chat messages to the memory like so:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "1380a4ea",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.memory import ConversationBufferMemory\n",
+    "\n",
+    "memory = ConversationBufferMemory()\n",
+    "memory.chat_memory.add_user_message(\"hi!\")\n",
+    "memory.chat_memory.add_ai_message(\"whats up?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a3d5d1f8",
+   "metadata": {},
+   "source": [
+    "And now we can load from our memory. The key method exposed by all `Memory` classes is `load_memory_variables`. This takes in any initial chain input and returns a list of memory variables which are added to the chain input. \n",
+    "\n",
+    "Since this simple memory type doesn't actually take into account the chain input when loading memory, we can pass in an empty input for now:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "982467e7",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'history': 'Human: hi!\\nAI: whats up?'}"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "memory.load_memory_variables({})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7c1b20d4",
+   "metadata": {},
+   "source": [
+    "We can also keep a sliding window of the most recent `k` interactions using `ConversationBufferWindowMemory`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "f72b9ff7",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'history': 'Human: not much you\\nAI: not much'}"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.memory import ConversationBufferWindowMemory\n",
+    "\n",
+    "memory = ConversationBufferWindowMemory(k=1)\n",
+    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})\n",
+    "memory.save_context({\"input\": \"not much you\"}, {\"output\": \"not much\"})\n",
+    "memory.load_memory_variables({})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7b84f90a",
+   "metadata": {},
+   "source": [
+    "`ConversationSummaryMemory` is an extension of this theme.\n",
+    "\n",
+    "It creates a summary of the conversation over time. \n",
+    "\n",
+    "This memory is most useful for longer conversations where the full message history would consume many tokens."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "id": "ca2596ed",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import OpenAI\n",
+    "from langchain.memory import ConversationSummaryMemory\n",
+    "\n",
+    "llm = OpenAI(temperature=0)\n",
+    "memory = ConversationSummaryMemory(llm=llm)\n",
+    "memory.save_context({\"input\": \"hi\"},{\"output\": \"whats up\"})\n",
+    "memory.save_context({\"input\": \"im working on better docs for chatbots\"},{\"output\": \"oh, that sounds like a lot of work\"})\n",
+    "memory.save_context({\"input\": \"yes, but it's worth the effort\"},{\"output\": \"agreed, good docs are important!\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "060f69b7",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'history': '\\nThe human greets the AI, to which the AI responds. The human then mentions they are working on better docs for chatbots, to which the AI responds that it sounds like a lot of work. The human agrees that it is worth the effort, and the AI agrees that good docs are important.'}"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "memory.load_memory_variables({})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4bf036f6",
+   "metadata": {},
+   "source": [
+    "`ConversationSummaryBufferMemory` extends this a bit further:\n",
+    "\n",
+    "It uses token length rather than number of interactions to determine when to flush interactions."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "38b42728",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.memory import ConversationSummaryBufferMemory\n",
+    "memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=10)\n",
+    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})\n",
+    "memory.save_context({\"input\": \"not much you\"}, {\"output\": \"not much\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ff0db09f",
+   "metadata": {},
+   "source": [
+    "## Conversation \n",
+    "\n",
+    "We can unpack what goes under the hood with `ConversationChain`. \n",
+    "\n",
+    "We can specify our memory, `ConversationSummaryMemory` and we can specify the prompt. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "id": "fccd6995",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mSystem: You are a nice chatbot having a conversation with a human.\n",
+      "Human: hi\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "{'question': 'hi',\n",
+       " 'chat_history': [HumanMessage(content='hi', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Hello! How can I assist you today?', additional_kwargs={}, example=False)],\n",
+       " 'text': 'Hello! How can I assist you today?'}"
+      ]
+     },
+     "execution_count": 24,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.prompts import (\n",
+    "    ChatPromptTemplate,\n",
+    "    MessagesPlaceholder,\n",
+    "    SystemMessagePromptTemplate,\n",
+    "    HumanMessagePromptTemplate,\n",
+    ")\n",
+    "from langchain.chains import LLMChain\n",
+    "\n",
+    "# LLM\n",
+    "llm = ChatOpenAI()\n",
+    "\n",
+    "# Prompt \n",
+    "prompt = ChatPromptTemplate(\n",
+    "    messages=[\n",
+    "        SystemMessagePromptTemplate.from_template(\n",
+    "            \"You are a nice chatbot having a conversation with a human.\"\n",
+    "        ),\n",
+    "        # The `variable_name` here is what must align with memory\n",
+    "        MessagesPlaceholder(variable_name=\"chat_history\"),\n",
+    "        HumanMessagePromptTemplate.from_template(\"{question}\")\n",
+    "    ]\n",
+    ")\n",
+    "\n",
+    "# Notice that we `return_messages=True` to fit into the MessagesPlaceholder\n",
+    "# Notice that `\"chat_history\"` aligns with the MessagesPlaceholder name\n",
+    "memory = ConversationBufferMemory(memory_key=\"chat_history\",return_messages=True)\n",
+    "conversation = LLMChain(\n",
+    "    llm=llm,\n",
+    "    prompt=prompt,\n",
+    "    verbose=True,\n",
+    "    memory=memory\n",
+    ")\n",
+    "\n",
+    "# Notice that we just pass in the `question` variables - `chat_history` gets populated by memory\n",
+    "conversation({\"question\": \"hi\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "eb0cadfd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mSystem: You are a nice chatbot having a conversation with a human.\n",
+      "Human: hi\n",
+      "AI: Hello! How can I assist you today?\n",
+      "Human: Translate this sentence from English to French: I love programming.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "{'question': 'Translate this sentence from English to French: I love programming.',\n",
+       " 'chat_history': [HumanMessage(content='hi', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Hello! How can I assist you today?', additional_kwargs={}, example=False),\n",
+       "  HumanMessage(content='Translate this sentence from English to French: I love programming.', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Sure! The translation of \"I love programming\" from English to French is \"J\\'adore programmer.\"', additional_kwargs={}, example=False)],\n",
+       " 'text': 'Sure! The translation of \"I love programming\" from English to French is \"J\\'adore programmer.\"'}"
+      ]
+     },
+     "execution_count": 25,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation({\"question\": \"Translate this sentence from English to French: I love programming.\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "c56d6219",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mSystem: You are a nice chatbot having a conversation with a human.\n",
+      "Human: hi\n",
+      "AI: Hello! How can I assist you today?\n",
+      "Human: Translate this sentence from English to French: I love programming.\n",
+      "AI: Sure! The translation of \"I love programming\" from English to French is \"J'adore programmer.\"\n",
+      "Human: Now translate the sentence to German.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "{'question': 'Now translate the sentence to German.',\n",
+       " 'chat_history': [HumanMessage(content='hi', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Hello! How can I assist you today?', additional_kwargs={}, example=False),\n",
+       "  HumanMessage(content='Translate this sentence from English to French: I love programming.', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Sure! The translation of \"I love programming\" from English to French is \"J\\'adore programmer.\"', additional_kwargs={}, example=False),\n",
+       "  HumanMessage(content='Now translate the sentence to German.', additional_kwargs={}, example=False),\n",
+       "  AIMessage(content='Certainly! The translation of \"I love programming\" from English to German is \"Ich liebe das Programmieren.\"', additional_kwargs={}, example=False)],\n",
+       " 'text': 'Certainly! The translation of \"I love programming\" from English to German is \"Ich liebe das Programmieren.\"'}"
+      ]
+     },
+     "execution_count": 26,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "conversation({\"question\": \"Now translate the sentence to German.\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "43858489",
+   "metadata": {},
+   "source": [
+    "We can see the chat history preserved in the prompt using the [LangSmith trace](https://smith.langchain.com/public/dce34c57-21ca-4283-9020-a8e0d78a59de/r).\n",
+    "\n",
+    "![Image description](/img/chat_use_case_2.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3f35cc16",
+   "metadata": {},
+   "source": [
+    "## Chat Retrieval\n",
+    "\n",
+    "Now, suppose we want to [chat with documents](https://twitter.com/mayowaoshin/status/1640385062708424708?s=20) or some other source of knowledge.\n",
+    "\n",
+    "This is popular use case, combining chat with [document retrieval](/docs/use_cases/question_answering).\n",
+    "\n",
+    "It allows us to chat with specific information that the model was not trained on."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1a01e7b5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install tiktoken chromadb"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "88e220de",
+   "metadata": {},
+   "source": [
+    "Load a blog post."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "id": "1b99b36c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import WebBaseLoader\n",
+    "\n",
+    "loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
+    "data = loader.load()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3662ce79",
+   "metadata": {},
+   "source": [
+    "Split and store this in a vector."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "058f1541",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
+    "\n",
+    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)\n",
+    "all_splits = text_splitter.split_documents(data)\n",
+    "\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.vectorstores import Chroma\n",
+    "\n",
+    "vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "603d9441",
+   "metadata": {},
+   "source": [
+    "Create our memory, as before, but's let's use `ConversationSummaryMemory`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "id": "f89fd3f5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "memory = ConversationSummaryMemory(llm=llm,memory_key=\"chat_history\",return_messages=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "id": "28503423",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.chains import ConversationalRetrievalChain\n",
+    "\n",
+    "llm = ChatOpenAI()\n",
+    "retriever = vectorstore.as_retriever()\n",
+    "qa = ConversationalRetrievalChain.from_llm(llm, retriever=retriever, memory=memory)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "id": "a9c3bd5e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'question': 'How do agents use Task decomposition?',\n",
+       " 'chat_history': [SystemMessage(content='', additional_kwargs={})],\n",
+       " 'answer': 'Agents can use task decomposition in several ways:\\n\\n1. Simple prompting: Agents can use Language Model based prompting to break down tasks into subgoals. For example, by providing prompts like \"Steps for XYZ\" or \"What are the subgoals for achieving XYZ?\", the agent can generate a sequence of smaller steps that lead to the completion of the overall task.\\n\\n2. Task-specific instructions: Agents can be given task-specific instructions to guide their planning process. For example, if the task is to write a novel, the agent can be instructed to \"Write a story outline.\" This provides a high-level structure for the task and helps in breaking it down into smaller components.\\n\\n3. Human inputs: Agents can also take inputs from humans to decompose tasks. This can be done through direct communication or by leveraging human expertise. Humans can provide guidance and insights to help the agent break down complex tasks into manageable subgoals.\\n\\nOverall, task decomposition allows agents to break down large tasks into smaller, more manageable subgoals, enabling them to plan and execute complex tasks efficiently.'}"
+      ]
+     },
+     "execution_count": 39,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "qa(\"How do agents use Task decomposition?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 40,
+   "id": "a29a7713",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'question': 'What are the various ways to implemet memory to support it?',\n",
+       " 'chat_history': [SystemMessage(content='The human asks how agents use task decomposition. The AI explains that agents can use task decomposition in several ways, including simple prompting, task-specific instructions, and human inputs. Task decomposition allows agents to break down large tasks into smaller, more manageable subgoals, enabling them to plan and execute complex tasks efficiently.', additional_kwargs={})],\n",
+       " 'answer': 'There are several ways to implement memory to support task decomposition:\\n\\n1. Long-Term Memory Management: This involves storing and organizing information in a long-term memory system. The agent can retrieve past experiences, knowledge, and learned strategies to guide the task decomposition process.\\n\\n2. Internet Access: The agent can use internet access to search for relevant information and gather resources to aid in task decomposition. This allows the agent to access a vast amount of information and utilize it in the decomposition process.\\n\\n3. GPT-3.5 Powered Agents: The agent can delegate simple tasks to GPT-3.5 powered agents. These agents can perform specific tasks or provide assistance in task decomposition, allowing the main agent to focus on higher-level planning and decision-making.\\n\\n4. File Output: The agent can store the results of task decomposition in files or documents. This allows for easy retrieval and reference during the execution of the task.\\n\\nThese memory resources help the agent in organizing and managing information, making informed decisions, and effectively decomposing complex tasks into smaller, manageable subgoals.'}"
+      ]
+     },
+     "execution_count": 40,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "qa(\"What are the various ways to implemet memory to support it?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d5e8d5f4",
+   "metadata": {},
+   "source": [
+    "Again, we can use the [LangSmith trace](https://smith.langchain.com/public/18460363-0c70-4c72-81c7-3b57253bb58c/r) to explore the prompt structure.\n",
+    "\n",
+    "### Going deeper \n",
+    "\n",
+    "* Agents, such as the [conversational retrieval agent](/docs/use_cases/question_answering/how_to/conversational_retrieval_agents), can be used for retrieval when necessary while also holding a conversation.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1ff8925f-4c21-4680-a9cd-3670ad4852b3",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/use_cases/chatbots/index.mdx

@@ -1,20 +0,0 @@
----
-sidebar_position: 4
----
-
-# Chatbots
-
-Since language models are good at producing text, that makes them ideal for creating chatbots.
-Aside from the base prompts/LLMs, an important concept to know for Chatbots is `memory`.
-Most chat based applications rely on remembering what happened in previous interactions, which `memory` is designed to help with.
-
-The following resources exist:
-- [ChatGPT Clone](/docs/modules/agents/how_to/chatgpt_clone.html): A notebook walking through how to recreate a ChatGPT-like experience with LangChain.
-- [Conversation Agent](/docs/modules/agents/agent_types/chat_conversation_agent.html): A notebook walking through how to create an agent optimized for conversation.
-
-
-Additional related resources include:
-- [Memory concepts and examples](/docs/modules/memory/): Explanation of key concepts related to memory along with how-to's and examples.
-
-More end-to-end examples include:
-- [Voice Assistant](./voice_assistant.html): A notebook walking through how to create a voice assistant using LangChain.

docs/extras/use_cases/chatbots/voice_assistant.ipynb

@@ -1,482 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Voice Assistant\n",
-    "\n",
-    "This chain creates a clone of ChatGPT with a few modifications to make it a voice assistant. \n",
-    "It uses the `pyttsx3` and `speech_recognition` libraries to convert text to speech and speech to text respectively. The prompt template is also changed to make it more suitable for voice assistant use."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain import OpenAI, LLMChain, PromptTemplate\n",
-    "from langchain.memory import ConversationBufferWindowMemory\n",
-    "\n",
-    "\n",
-    "template = \"\"\"Assistant is a large language model trained by OpenAI.\n",
-    "\n",
-    "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-    "\n",
-    "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-    "\n",
-    "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-    "\n",
-    "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-    "\n",
-    "{history}\n",
-    "Human: {human_input}\n",
-    "Assistant:\"\"\"\n",
-    "\n",
-    "prompt = PromptTemplate(input_variables=[\"history\", \"human_input\"], template=template)\n",
-    "\n",
-    "\n",
-    "chatgpt_chain = LLMChain(\n",
-    "    llm=OpenAI(temperature=0),\n",
-    "    prompt=prompt,\n",
-    "    verbose=True,\n",
-    "    memory=ConversationBufferWindowMemory(k=2),\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import speech_recognition as sr\n",
-    "import pyttsx3\n",
-    "\n",
-    "engine = pyttsx3.init()\n",
-    "\n",
-    "\n",
-    "def listen():\n",
-    "    r = sr.Recognizer()\n",
-    "    with sr.Microphone() as source:\n",
-    "        print(\"Calibrating...\")\n",
-    "        r.adjust_for_ambient_noise(source, duration=5)\n",
-    "        # optional parameters to adjust microphone sensitivity\n",
-    "        # r.energy_threshold = 200\n",
-    "        # r.pause_threshold=0.5\n",
-    "\n",
-    "        print(\"Okay, go!\")\n",
-    "        while 1:\n",
-    "            text = \"\"\n",
-    "            print(\"listening now...\")\n",
-    "            try:\n",
-    "                audio = r.listen(source, timeout=5, phrase_time_limit=30)\n",
-    "                print(\"Recognizing...\")\n",
-    "                # whisper model options are found here: https://github.com/openai/whisper#available-models-and-languages\n",
-    "                # other speech recognition models are also available.\n",
-    "                text = r.recognize_whisper(\n",
-    "                    audio,\n",
-    "                    model=\"medium.en\",\n",
-    "                    show_dict=True,\n",
-    "                )[\"text\"]\n",
-    "            except Exception as e:\n",
-    "                unrecognized_speech_text = (\n",
-    "                    f\"Sorry, I didn't catch that. Exception was: {e}s\"\n",
-    "                )\n",
-    "                text = unrecognized_speech_text\n",
-    "            print(text)\n",
-    "\n",
-    "            response_text = chatgpt_chain.predict(human_input=text)\n",
-    "            print(response_text)\n",
-    "            engine.say(response_text)\n",
-    "            engine.runAndWait()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Calibrating...\n",
-      "Okay, go!\n",
-      "listening now...\n",
-      "Recognizing...\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "C:\\Users\\jaden\\AppData\\Roaming\\Python\\Python310\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
-      "  from .autonotebook import tqdm as notebook_tqdm\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      " Hello, Assistant. What's going on?\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "\n",
-      "Human:  Hello, Assistant. What's going on?\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " Hi there! It's great to hear from you. I'm doing well. How can I help you today?\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " That's cool. Isn't that neat? Yeah, I'm doing great.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Hello, Assistant. What's going on?\n",
-      "AI:  Hi there! It's great to hear from you. I'm doing well. How can I help you today?\n",
-      "Human:  That's cool. Isn't that neat? Yeah, I'm doing great.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      "  That's great to hear! What can I do for you today?\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Thank you.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Hello, Assistant. What's going on?\n",
-      "AI:  Hi there! It's great to hear from you. I'm doing well. How can I help you today?\n",
-      "Human:  That's cool. Isn't that neat? Yeah, I'm doing great.\n",
-      "AI:   That's great to hear! What can I do for you today?\n",
-      "Human:  Thank you.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " You're welcome! Is there anything else I can help you with?\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " I'd like to learn more about neural networks.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  That's cool. Isn't that neat? Yeah, I'm doing great.\n",
-      "AI:   That's great to hear! What can I do for you today?\n",
-      "Human:  Thank you.\n",
-      "AI:  You're welcome! Is there anything else I can help you with?\n",
-      "Human:  I'd like to learn more about neural networks.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " Sure! Neural networks are a type of artificial intelligence that use a network of interconnected nodes to process data and make decisions. They are used in a variety of applications, from image recognition to natural language processing. Neural networks are often used to solve complex problems that are too difficult for traditional algorithms.\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Tell me a fun fact about neural networks.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Thank you.\n",
-      "AI:  You're welcome! Is there anything else I can help you with?\n",
-      "Human:  I'd like to learn more about neural networks.\n",
-      "AI:  Sure! Neural networks are a type of artificial intelligence that use a network of interconnected nodes to process data and make decisions. They are used in a variety of applications, from image recognition to natural language processing. Neural networks are often used to solve complex problems that are too difficult for traditional algorithms.\n",
-      "Human:  Tell me a fun fact about neural networks.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " Neural networks are inspired by the way the human brain works. They are composed of interconnected nodes that process data and make decisions, just like neurons in the brain. Neural networks can learn from their mistakes and improve their performance over time, just like humans do.\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Tell me about a brand new discovered bird species.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  I'd like to learn more about neural networks.\n",
-      "AI:  Sure! Neural networks are a type of artificial intelligence that use a network of interconnected nodes to process data and make decisions. They are used in a variety of applications, from image recognition to natural language processing. Neural networks are often used to solve complex problems that are too difficult for traditional algorithms.\n",
-      "Human:  Tell me a fun fact about neural networks.\n",
-      "AI:  Neural networks are inspired by the way the human brain works. They are composed of interconnected nodes that process data and make decisions, just like neurons in the brain. Neural networks can learn from their mistakes and improve their performance over time, just like humans do.\n",
-      "Human:  Tell me about a brand new discovered bird species.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " A new species of bird was recently discovered in the Amazon rainforest. The species, called the Spix's Macaw, is a small, blue parrot that is believed to be extinct in the wild. It is the first new species of bird to be discovered in the Amazon in over 100 years.\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Tell me a children's story about the importance of honesty and trust.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Tell me a fun fact about neural networks.\n",
-      "AI:  Neural networks are inspired by the way the human brain works. They are composed of interconnected nodes that process data and make decisions, just like neurons in the brain. Neural networks can learn from their mistakes and improve their performance over time, just like humans do.\n",
-      "Human:  Tell me about a brand new discovered bird species.\n",
-      "AI:  A new species of bird was recently discovered in the Amazon rainforest. The species, called the Spix's Macaw, is a small, blue parrot that is believed to be extinct in the wild. It is the first new species of bird to be discovered in the Amazon in over 100 years.\n",
-      "Human:  Tell me a children's story about the importance of honesty and trust.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " Once upon a time, there was a young boy named Jack who lived in a small village. Jack was always honest and trustworthy, and his friends and family knew they could always count on him. One day, Jack was walking through the forest when he stumbled upon a magical tree. The tree told Jack that if he was honest and trustworthy, he would be rewarded with a special gift. Jack was so excited, and he promised to always be honest and trustworthy. Sure enough, the tree rewarded Jack with a beautiful golden apple. From that day forward, Jack was always honest and trustworthy, and he was rewarded with many more magical gifts. The moral of the story is that honesty and trust are the most important things in life.\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Wow, Assistant, that was a really good story. Congratulations!\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Tell me about a brand new discovered bird species.\n",
-      "AI:  A new species of bird was recently discovered in the Amazon rainforest. The species, called the Spix's Macaw, is a small, blue parrot that is believed to be extinct in the wild. It is the first new species of bird to be discovered in the Amazon in over 100 years.\n",
-      "Human:  Tell me a children's story about the importance of honesty and trust.\n",
-      "AI:  Once upon a time, there was a young boy named Jack who lived in a small village. Jack was always honest and trustworthy, and his friends and family knew they could always count on him. One day, Jack was walking through the forest when he stumbled upon a magical tree. The tree told Jack that if he was honest and trustworthy, he would be rewarded with a special gift. Jack was so excited, and he promised to always be honest and trustworthy. Sure enough, the tree rewarded Jack with a beautiful golden apple. From that day forward, Jack was always honest and trustworthy, and he was rewarded with many more magical gifts. The moral of the story is that honesty and trust are the most important things in life.\n",
-      "Human:  Wow, Assistant, that was a really good story. Congratulations!\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " Thank you! I'm glad you enjoyed it.\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Thank you.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Tell me a children's story about the importance of honesty and trust.\n",
-      "AI:  Once upon a time, there was a young boy named Jack who lived in a small village. Jack was always honest and trustworthy, and his friends and family knew they could always count on him. One day, Jack was walking through the forest when he stumbled upon a magical tree. The tree told Jack that if he was honest and trustworthy, he would be rewarded with a special gift. Jack was so excited, and he promised to always be honest and trustworthy. Sure enough, the tree rewarded Jack with a beautiful golden apple. From that day forward, Jack was always honest and trustworthy, and he was rewarded with many more magical gifts. The moral of the story is that honesty and trust are the most important things in life.\n",
-      "Human:  Wow, Assistant, that was a really good story. Congratulations!\n",
-      "AI:  Thank you! I'm glad you enjoyed it.\n",
-      "Human:  Thank you.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " You're welcome!\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way?\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Wow, Assistant, that was a really good story. Congratulations!\n",
-      "AI:  Thank you! I'm glad you enjoyed it.\n",
-      "Human:  Thank you.\n",
-      "AI:  You're welcome!\n",
-      "Human:  Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way?\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " Yes, there are several online brands that offer photo editing and other creative tools without the need to download any software. Adobe Photoshop Express, Pixlr, and Fotor are some of the most popular online photo editing tools. Freq is an online music production platform that allows users to create and share music without downloading any software.\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " Our whole process of awesome is free.\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Thank you.\n",
-      "AI:  You're welcome!\n",
-      "Human:  Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way?\n",
-      "AI:  Yes, there are several online brands that offer photo editing and other creative tools without the need to download any software. Adobe Photoshop Express, Pixlr, and Fotor are some of the most popular online photo editing tools. Freq is an online music production platform that allows users to create and share music without downloading any software.\n",
-      "Human:  Our whole process of awesome is free.\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " That's great! It's always nice to have access to free tools and resources.\n",
-      "listening now...\n",
-      "Recognizing...\n",
-      " No, I meant to ask, are those options that you mentioned free? No, I meant to ask, are those options that you mentioned free?\n",
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
-      "Prompt after formatting:\n",
-      "\u001b[32;1m\u001b[1;3mAssistant is a large language model trained by OpenAI.\n",
-      "\n",
-      "Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\n",
-      "\n",
-      "Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\n",
-      "\n",
-      "Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.\n",
-      "\n",
-      "Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.\n",
-      "\n",
-      "Human:  Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way?\n",
-      "AI:  Yes, there are several online brands that offer photo editing and other creative tools without the need to download any software. Adobe Photoshop Express, Pixlr, and Fotor are some of the most popular online photo editing tools. Freq is an online music production platform that allows users to create and share music without downloading any software.\n",
-      "Human:  Our whole process of awesome is free.\n",
-      "AI:  That's great! It's always nice to have access to free tools and resources.\n",
-      "Human:  No, I meant to ask, are those options that you mentioned free? No, I meant to ask, are those options that you mentioned free?\n",
-      "Assistant:\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n",
-      " Yes, the online brands I mentioned are all free to use. Adobe Photoshop Express, Pixlr, and Fotor are all free to use, and Freq is a free music production platform.\n",
-      "listening now...\n"
-     ]
-    },
-    {
-     "ename": "KeyboardInterrupt",
-     "evalue": "",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
-      "Cell \u001b[1;32mIn[6], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m listen(\u001b[39mNone\u001b[39;49;00m)\n",
-      "Cell \u001b[1;32mIn[5], line 20\u001b[0m, in \u001b[0;36mlisten\u001b[1;34m(command_queue)\u001b[0m\n\u001b[0;32m     18\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m'\u001b[39m\u001b[39mlistening now...\u001b[39m\u001b[39m'\u001b[39m)\n\u001b[0;32m     19\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m---> 20\u001b[0m     audio \u001b[39m=\u001b[39m r\u001b[39m.\u001b[39;49mlisten(source, timeout\u001b[39m=\u001b[39;49m\u001b[39m5\u001b[39;49m, phrase_time_limit\u001b[39m=\u001b[39;49m\u001b[39m30\u001b[39;49m)\n\u001b[0;32m     21\u001b[0m     \u001b[39m# audio = r.record(source,duration = 5)\u001b[39;00m\n\u001b[0;32m     22\u001b[0m     \u001b[39mprint\u001b[39m(\u001b[39m'\u001b[39m\u001b[39mRecognizing...\u001b[39m\u001b[39m'\u001b[39m)\n",
-      "File \u001b[1;32mc:\\ProgramData\\miniconda3\\envs\\lang\\lib\\site-packages\\speech_recognition\\__init__.py:523\u001b[0m, in \u001b[0;36mRecognizer.listen\u001b[1;34m(self, source, timeout, phrase_time_limit, snowboy_configuration)\u001b[0m\n\u001b[0;32m    520\u001b[0m \u001b[39mif\u001b[39;00m phrase_time_limit \u001b[39mand\u001b[39;00m elapsed_time \u001b[39m-\u001b[39m phrase_start_time \u001b[39m>\u001b[39m phrase_time_limit:\n\u001b[0;32m    521\u001b[0m     \u001b[39mbreak\u001b[39;00m\n\u001b[1;32m--> 523\u001b[0m buffer \u001b[39m=\u001b[39m source\u001b[39m.\u001b[39;49mstream\u001b[39m.\u001b[39;49mread(source\u001b[39m.\u001b[39;49mCHUNK)\n\u001b[0;32m    524\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mlen\u001b[39m(buffer) \u001b[39m==\u001b[39m \u001b[39m0\u001b[39m: \u001b[39mbreak\u001b[39;00m  \u001b[39m# reached end of the stream\u001b[39;00m\n\u001b[0;32m    525\u001b[0m frames\u001b[39m.\u001b[39mappend(buffer)\n",
-      "File \u001b[1;32mc:\\ProgramData\\miniconda3\\envs\\lang\\lib\\site-packages\\speech_recognition\\__init__.py:199\u001b[0m, in \u001b[0;36mMicrophone.MicrophoneStream.read\u001b[1;34m(self, size)\u001b[0m\n\u001b[0;32m    198\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mread\u001b[39m(\u001b[39mself\u001b[39m, size):\n\u001b[1;32m--> 199\u001b[0m     \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mpyaudio_stream\u001b[39m.\u001b[39;49mread(size, exception_on_overflow\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m)\n",
-      "File \u001b[1;32mc:\\ProgramData\\miniconda3\\envs\\lang\\lib\\site-packages\\pyaudio\\__init__.py:570\u001b[0m, in \u001b[0;36mPyAudio.Stream.read\u001b[1;34m(self, num_frames, exception_on_overflow)\u001b[0m\n\u001b[0;32m    567\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_is_input:\n\u001b[0;32m    568\u001b[0m     \u001b[39mraise\u001b[39;00m \u001b[39mIOError\u001b[39;00m(\u001b[39m\"\u001b[39m\u001b[39mNot input stream\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[0;32m    569\u001b[0m                   paCanNotReadFromAnOutputOnlyStream)\n\u001b[1;32m--> 570\u001b[0m \u001b[39mreturn\u001b[39;00m pa\u001b[39m.\u001b[39;49mread_stream(\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_stream, num_frames,\n\u001b[0;32m    571\u001b[0m                       exception_on_overflow)\n",
-      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
-     ]
-    }
-   ],
-   "source": [
-    "listen(None)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "lang",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.10"
-  },
-  "orig_nbformat": 4
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

docs/extras/use_cases/extraction.ipynb

@@ -0,0 +1,597 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b84edb4e",
+   "metadata": {},
+   "source": [
+    "# Extraction\n",
+    "\n",
+    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/extras/use_cases/extraction/extraction.ipynb)\n",
+    "\n",
+    "## Use case\n",
+    "\n",
+    "Getting structured output from raw LLM generations is hard.\n",
+    "\n",
+    "For example, suppose you need the model output formatted with a specific schema for:\n",
+    "\n",
+    "- Extracting a structured row to insert into a database \n",
+    "- Extracting API parameters\n",
+    "- Extracting different parts of a user query (e.g., for semantic vs keyword search)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "178dbc59",
+   "metadata": {},
+   "source": [
+    "![Image description](/img/extraction.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "97f474d4",
+   "metadata": {},
+   "source": [
+    "## Overview \n",
+    "\n",
+    "There are two primary approaches for this:\n",
+    "\n",
+    "- `Functions`: Some LLMs can call [functions](https://openai.com/blog/function-calling-and-other-api-updates) to extract arbitrary entities from LLM responses.\n",
+    "\n",
+    "- `Parsing`: [Output parsers](/docs/modules/model_io/output_parsers/) are classes that structure LLM responses. \n",
+    "\n",
+    "Only some LLMs support functions (e.g., OpenAI), and they are more general than parsers. \n",
+    "\n",
+    "Parsers extract precisely what is enumerated in a provided schema (e.g., specific attributes of a person).\n",
+    "\n",
+    "Functions can infer things beyond of a provided schema (e.g., attributes about a person that you did not ask for)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "25d89f21",
+   "metadata": {},
+   "source": [
+    "## Quickstart\n",
+    "\n",
+    "OpenAI funtions are one way to get started with extraction.\n",
+    "\n",
+    "Define a schema that specifies the properties we want to extract from the LLM output.\n",
+    "\n",
+    "Then, we can use `create_extraction_chain` to extract our desired schema using an OpenAI function call."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3f5ec7a3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pip install langchain openai \n",
+    "\n",
+    "# Set env var OPENAI_API_KEY or load from a .env file:\n",
+    "# import dotenv\n",
+    "# dotenv.load_env()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "3e017ba0",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'name': 'Alex', 'height': 5, 'hair_color': 'blonde'},\n",
+       " {'name': 'Claudia', 'height': 6, 'hair_color': 'brunette'}]"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.chains import create_extraction_chain\n",
+    "\n",
+    "# Schema\n",
+    "schema = {\n",
+    "    \"properties\": {\n",
+    "        \"name\": {\"type\": \"string\"},\n",
+    "        \"height\": {\"type\": \"integer\"},\n",
+    "        \"hair_color\": {\"type\": \"string\"},\n",
+    "    },\n",
+    "    \"required\": [\"name\", \"height\"],\n",
+    "}\n",
+    "\n",
+    "# Input \n",
+    "inp = \"\"\"Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\"\"\"\n",
+    "\n",
+    "# Run chain\n",
+    "llm = ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo\")\n",
+    "chain = create_extraction_chain(schema, llm)\n",
+    "chain.run(inp)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6f7eb826",
+   "metadata": {},
+   "source": [
+    "## Option 1: OpenAI funtions\n",
+    "\n",
+    "### Looking under the hood\n",
+    "\n",
+    "Let's dig into what is happening when we call `create_extraction_chain`.\n",
+    "\n",
+    "The [LangSmith trace](https://smith.langchain.com/public/72bc3205-7743-4ca6-929a-966a9d4c2a77/r) shows that we call the function `information_extraction` on the input string, `inp`.\n",
+    "\n",
+    "![Image description](/img/extraction_trace_function.png)\n",
+    "\n",
+    "This `information_extraction` function is defined [here](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chains/openai_functions/extraction.py) and returns a dict.\n",
+    "\n",
+    "We can see the `dict` in the model output:\n",
+    "```\n",
+    " {\n",
+    "      \"info\": [\n",
+    "        {\n",
+    "          \"name\": \"Alex\",\n",
+    "          \"height\": 5,\n",
+    "          \"hair_color\": \"blonde\"\n",
+    "        },\n",
+    "        {\n",
+    "          \"name\": \"Claudia\",\n",
+    "          \"height\": 6,\n",
+    "          \"hair_color\": \"brunette\"\n",
+    "        }\n",
+    "      ]\n",
+    "    }\n",
+    "```\n",
+    "\n",
+    "The `create_extraction_chain` then parses the raw LLM output for us using [`JsonKeyOutputFunctionsParser`](https://github.com/langchain-ai/langchain/blob/f81e613086d211327b67b0fb591fd4d5f9a85860/libs/langchain/langchain/chains/openai_functions/extraction.py#L62).\n",
+    "\n",
+    "This results in the list of JSON objects returned by the chain above:\n",
+    "```\n",
+    "[{'name': 'Alex', 'height': 5, 'hair_color': 'blonde'},\n",
+    " {'name': 'Claudia', 'height': 6, 'hair_color': 'brunette'}]\n",
+    " ```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dcb03138",
+   "metadata": {},
+   "source": [
+    "### Multiple entity types\n",
+    "\n",
+    "We can extend this further.\n",
+    "\n",
+    "Let's say we want to differentiate between dogs and people.\n",
+    "\n",
+    "We can add `person_` and `dog_` prefixes for each property"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "01eae733",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'person_name': 'Alex',\n",
+       "  'person_height': 5,\n",
+       "  'person_hair_color': 'blonde',\n",
+       "  'dog_name': 'Frosty',\n",
+       "  'dog_breed': 'labrador'},\n",
+       " {'person_name': 'Claudia',\n",
+       "  'person_height': 6,\n",
+       "  'person_hair_color': 'brunette'}]"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "schema = {\n",
+    "    \"properties\": {\n",
+    "        \"person_name\": {\"type\": \"string\"},\n",
+    "        \"person_height\": {\"type\": \"integer\"},\n",
+    "        \"person_hair_color\": {\"type\": \"string\"},\n",
+    "        \"dog_name\": {\"type\": \"string\"},\n",
+    "        \"dog_breed\": {\"type\": \"string\"},\n",
+    "    },\n",
+    "    \"required\": [\"person_name\", \"person_height\"],\n",
+    "}\n",
+    "\n",
+    "chain = create_extraction_chain(schema, llm)\n",
+    "\n",
+    "inp = \"\"\"Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
+    "Alex's dog Frosty is a labrador and likes to play hide and seek.\"\"\"\n",
+    "\n",
+    "chain.run(inp)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f205905c",
+   "metadata": {},
+   "source": [
+    "### Unrelated entities\n",
+    "\n",
+    "If we use `required: []`, we allow the model to return **only** person attributes or **only** dog attributes for a single entity (person or dog)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "6ff4ac7e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'person_name': 'Alex', 'person_height': 5, 'person_hair_color': 'blonde'},\n",
+       " {'person_name': 'Claudia',\n",
+       "  'person_height': 6,\n",
+       "  'person_hair_color': 'brunette'},\n",
+       " {'dog_name': 'Willow', 'dog_breed': 'German Shepherd'},\n",
+       " {'dog_name': 'Milo', 'dog_breed': 'border collie'}]"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "schema = {\n",
+    "    \"properties\": {\n",
+    "        \"person_name\": {\"type\": \"string\"},\n",
+    "        \"person_height\": {\"type\": \"integer\"},\n",
+    "        \"person_hair_color\": {\"type\": \"string\"},\n",
+    "        \"dog_name\": {\"type\": \"string\"},\n",
+    "        \"dog_breed\": {\"type\": \"string\"},\n",
+    "    },\n",
+    "    \"required\": [],\n",
+    "}\n",
+    "\n",
+    "chain = create_extraction_chain(schema, llm)\n",
+    "\n",
+    "inp = \"\"\"Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
+    "Willow is a German Shepherd that likes to play with other dogs and can always be found playing with Milo, a border collie that lives close by.\"\"\"\n",
+    "\n",
+    "chain.run(inp)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "34f3b958",
+   "metadata": {},
+   "source": [
+    "### Extra information\n",
+    "\n",
+    "The power of functions (relative to using parsers alone) lies in the ability to perform sematic extraction.\n",
+    "\n",
+    "In particular, `we can ask for things that are not explictly enumerated in the schema`.\n",
+    "\n",
+    "Suppose we want unspecified additional information about dogs. \n",
+    "\n",
+    "We can use add a placeholder for unstructured extraction, `dog_extra_info`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "40c7b26f",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'person_name': 'Alex', 'person_height': 5, 'person_hair_color': 'blonde'},\n",
+       " {'person_name': 'Claudia',\n",
+       "  'person_height': 6,\n",
+       "  'person_hair_color': 'brunette'},\n",
+       " {'dog_name': 'Willow',\n",
+       "  'dog_breed': 'German Shepherd',\n",
+       "  'dog_extra_info': 'likes to play with other dogs'},\n",
+       " {'dog_name': 'Milo',\n",
+       "  'dog_breed': 'border collie',\n",
+       "  'dog_extra_info': 'lives close by'}]"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "schema = {\n",
+    "    \"properties\": {\n",
+    "        \"person_name\": {\"type\": \"string\"},\n",
+    "        \"person_height\": {\"type\": \"integer\"},\n",
+    "        \"person_hair_color\": {\"type\": \"string\"},\n",
+    "        \"dog_name\": {\"type\": \"string\"},\n",
+    "        \"dog_breed\": {\"type\": \"string\"},\n",
+    "        \"dog_extra_info\": {\"type\": \"string\"},\n",
+    "    },\n",
+    "}\n",
+    "\n",
+    "chain = create_extraction_chain(schema, llm)\n",
+    "chain.run(inp)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a949c60",
+   "metadata": {},
+   "source": [
+    "This gives us additional information about the dogs."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bf71ddce",
+   "metadata": {},
+   "source": [
+    "### Pydantic \n",
+    "\n",
+    "Pydantic is a data validation and settings management library for Python. \n",
+    "\n",
+    "It allows you to create data classes with attributes that are automatically validated when you instantiate an object.\n",
+    "\n",
+    "Lets define a class with attributes annotated with types."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "d36a743b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Properties(person_name='Alex', person_height=5, person_hair_color='blonde', dog_breed=None, dog_name=None),\n",
+       " Properties(person_name='Claudia', person_height=6, person_hair_color='brunette', dog_breed=None, dog_name=None)]"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from typing import Optional, List\n",
+    "from pydantic import BaseModel, Field\n",
+    "from langchain.chains import create_extraction_chain_pydantic\n",
+    "\n",
+    "# Pydantic data class\n",
+    "class Properties(BaseModel):\n",
+    "    person_name: str\n",
+    "    person_height: int\n",
+    "    person_hair_color: str\n",
+    "    dog_breed: Optional[str]\n",
+    "    dog_name: Optional[str]\n",
+    "        \n",
+    "# Extraction\n",
+    "chain = create_extraction_chain_pydantic(pydantic_schema=Properties, llm=llm)\n",
+    "\n",
+    "# Run \n",
+    "inp = \"\"\"Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\"\"\"\n",
+    "chain.run(inp)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "07a0351a",
+   "metadata": {},
+   "source": [
+    "As we can see from the [trace](https://smith.langchain.com/public/fed50ae6-26bb-4235-a254-e0b7a229d10f/r), we use the function `information_extraction`, as above, with the Pydantic schema. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cbd9f121",
+   "metadata": {},
+   "source": [
+    "## Option 2: Parsing\n",
+    "\n",
+    "[Output parsers](/docs/modules/model_io/output_parsers/) are classes that help structure language model responses. \n",
+    "\n",
+    "As shown above, they are used to parse the output of the OpenAI function calls in `create_extraction_chain`.\n",
+    "\n",
+    "But, they can be used independent of functions.\n",
+    "\n",
+    "### Pydantic\n",
+    "\n",
+    "Just as a above, let's parse a generation based on a Pydantic data class."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "64650362",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "People(people=[Person(person_name='Alex', person_height=5, person_hair_color='blonde', dog_breed=None, dog_name=None), Person(person_name='Claudia', person_height=6, person_hair_color='brunette', dog_breed=None, dog_name=None)])"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from typing import Sequence\n",
+    "from langchain.prompts import (\n",
+    "    PromptTemplate,\n",
+    "    ChatPromptTemplate,\n",
+    "    HumanMessagePromptTemplate,\n",
+    ")\n",
+    "from langchain.llms import OpenAI\n",
+    "from pydantic import BaseModel, Field, validator\n",
+    "from langchain.output_parsers import PydanticOutputParser\n",
+    "\n",
+    "class Person(BaseModel):\n",
+    "    person_name: str\n",
+    "    person_height: int\n",
+    "    person_hair_color: str\n",
+    "    dog_breed: Optional[str]\n",
+    "    dog_name: Optional[str]\n",
+    "\n",
+    "class People(BaseModel):\n",
+    "    \"\"\"Identifying information about all people in a text.\"\"\"\n",
+    "    people: Sequence[Person]\n",
+    "\n",
+    "        \n",
+    "# Run \n",
+    "query = \"\"\"Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\"\"\"\n",
+    "\n",
+    "# Set up a parser + inject instructions into the prompt template.\n",
+    "parser = PydanticOutputParser(pydantic_object=People)\n",
+    "\n",
+    "# Prompt\n",
+    "prompt = PromptTemplate(\n",
+    "    template=\"Answer the user query.\\n{format_instructions}\\n{query}\\n\",\n",
+    "    input_variables=[\"query\"],\n",
+    "    partial_variables={\"format_instructions\": parser.get_format_instructions()},\n",
+    ")\n",
+    "\n",
+    "# Run\n",
+    "_input = prompt.format_prompt(query=query)\n",
+    "model = OpenAI(temperature=0)\n",
+    "output = model(_input.to_string())\n",
+    "parser.parse(output)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "826899df",
+   "metadata": {},
+   "source": [
+    "We can see from the [LangSmith trace](https://smith.langchain.com/public/8e3aa858-467e-46a5-aa49-5db65f0a2b9a/r) that we get the same output as above.\n",
+    "\n",
+    "![Image description](/img/extraction_trace_function_2.png)\n",
+    "\n",
+    "We can see that we provide a two-shot prompt in order to instruct the LLM to output in our desired format.\n",
+    "\n",
+    "And, we need to do a bit more work:\n",
+    "\n",
+    "* Define a class that holds multiple instances of `Person`\n",
+    "* Explicty parse the output of the LLM to the Pydantic class\n",
+    "\n",
+    "We can see this for other cases, too."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "837c350e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Joke(setup='Why did the chicken cross the road?', punchline='To get to the other side!')"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.prompts import (\n",
+    "    PromptTemplate,\n",
+    "    ChatPromptTemplate,\n",
+    "    HumanMessagePromptTemplate,\n",
+    ")\n",
+    "from langchain.llms import OpenAI\n",
+    "from pydantic import BaseModel, Field, validator\n",
+    "from langchain.output_parsers import PydanticOutputParser\n",
+    "\n",
+    "# Define your desired data structure.\n",
+    "class Joke(BaseModel):\n",
+    "    setup: str = Field(description=\"question to set up a joke\")\n",
+    "    punchline: str = Field(description=\"answer to resolve the joke\")\n",
+    "\n",
+    "    # You can add custom validation logic easily with Pydantic.\n",
+    "    @validator(\"setup\")\n",
+    "    def question_ends_with_question_mark(cls, field):\n",
+    "        if field[-1] != \"?\":\n",
+    "            raise ValueError(\"Badly formed question!\")\n",
+    "        return field\n",
+    "\n",
+    "# And a query intented to prompt a language model to populate the data structure.\n",
+    "joke_query = \"Tell me a joke.\"\n",
+    "\n",
+    "# Set up a parser + inject instructions into the prompt template.\n",
+    "parser = PydanticOutputParser(pydantic_object=Joke)\n",
+    "\n",
+    "# Prompt\n",
+    "prompt = PromptTemplate(\n",
+    "    template=\"Answer the user query.\\n{format_instructions}\\n{query}\\n\",\n",
+    "    input_variables=[\"query\"],\n",
+    "    partial_variables={\"format_instructions\": parser.get_format_instructions()},\n",
+    ")\n",
+    "\n",
+    "# Run\n",
+    "_input = prompt.format_prompt(query=joke_query)\n",
+    "model = OpenAI(temperature=0)\n",
+    "output = model(_input.to_string())\n",
+    "parser.parse(output)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d3601bde",
+   "metadata": {},
+   "source": [
+    "As we can see, we get an output of the `Joke` class, which respects our originally desired schema: 'setup' and 'punchline'.\n",
+    "\n",
+    "We can look at the [LangSmith trace](https://smith.langchain.com/public/69f11d41-41be-4319-93b0-6d0eda66e969/r) to see exactly what is going on under the hood.\n",
+    "\n",
+    "![Image description](/img/extraction_trace_joke.png)\n",
+    "\n",
+    "### Going deeper\n",
+    "\n",
+    "* The [output parser](/docs/modules/model_io/output_parsers/) documentation includes various parser examples for specific types (e.g., lists, datetimne, enum, etc).  \n",
+    "* [JSONFormer](/docs/integrations/llms/jsonformer_experimental) offers another way for structured decoding of a subset of the JSON Schema.\n",
+    "* [Kor](https://eyurtsev.github.io/kor/) is another library for extraction where schema and examples can be provided to the LLM."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/use_cases/extraction/index.mdx

@@ -1,24 +0,0 @@
----
-sidebar_position: 2
----
-
-# Extraction
-
-Most APIs and databases still deal with structured information.
-Therefore, in order to better work with those, it can be useful to extract structured information from text.
-Examples of this include:
-
-- Extracting a structured row to insert into a database from a sentence
-- Extracting multiple rows to insert into a database from a long document
-- Extracting the correct API parameters from a user query
-
-This work is extremely related to [output parsing](/docs/modules/model_io/output_parsers/).
-Output parsers are responsible for instructing the LLM to respond in a specific format.
-In this case, the output parsers specify the format of the data you would like to extract from the document.
-Then, in addition to the output format instructions, the prompt should also contain the data you would like to extract information from.
-
-While normal output parsers are good enough for basic structuring of response data,
-when doing extraction you often want to extract more complicated or nested structures.
-For a deep dive on extraction, we recommend checking out [`kor`](https://eyurtsev.github.io/kor/),
-a library that uses the existing LangChain chain and OutputParser abstractions
-but deep dives on allowing extraction of more complicated schemas.

docs/extras/use_cases/extraction/openai_extraction.ipynb

@@ -1,566 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "6605e7f7",
-   "metadata": {},
-   "source": [
-    "# Extraction with OpenAI Functions\n",
-    "\n",
-    "The extraction chain uses the OpenAI `functions` parameter to specify a schema to extract entities from a document. This helps us make sure that the model outputs exactly the schema of entities and properties that we want, with their appropriate types.\n",
-    "\n",
-    "The extraction chain is to be used when we want to extract several entities with their properties from the same passage (i.e. what people were mentioned in this passage?)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "34f04daf",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.4) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.\n",
-      "  warnings.warn(\n"
-     ]
-    }
-   ],
-   "source": [
-    "from langchain.chat_models import ChatOpenAI\n",
-    "from langchain.chains import create_extraction_chain, create_extraction_chain_pydantic\n",
-    "from langchain.prompts import ChatPromptTemplate"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "a2648974",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm = ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5ef034ce",
-   "metadata": {},
-   "source": [
-    "## Extracting entities"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "78ff9df9",
-   "metadata": {},
-   "source": [
-    "To extract entities, we need to create a schema where we specify all the properties we want to find and the type we expect them to have. We can also specify which of these properties are required and which are optional."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "4ac43eba",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "schema = {\n",
-    "    \"properties\": {\n",
-    "        \"name\": {\"type\": \"string\"},\n",
-    "        \"height\": {\"type\": \"integer\"},\n",
-    "        \"hair_color\": {\"type\": \"string\"},\n",
-    "    },\n",
-    "    \"required\": [\"name\", \"height\"],\n",
-    "}"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "640bd005",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inp = \"\"\"\n",
-    "Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
-    "        \"\"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "64313214",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = create_extraction_chain(schema, llm)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "17c48adb",
-   "metadata": {},
-   "source": [
-    "As we can see, we extracted the required entities and their properties in the required format (it even calculated Claudia's height before returning!)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "cc5436ed",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[{'name': 'Alex', 'height': 5, 'hair_color': 'blonde'},\n",
-       " {'name': 'Claudia', 'height': 6, 'hair_color': 'brunette'}]"
-      ]
-     },
-     "execution_count": 7,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.run(inp)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8d51fcdc",
-   "metadata": {},
-   "source": [
-    "## Several entity types"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5813affe",
-   "metadata": {},
-   "source": [
-    "Notice that we are using OpenAI functions under the hood and thus the model can only call one function per request (with one, unique schema)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "511b9838",
-   "metadata": {},
-   "source": [
-    "If we want to extract more than one entity type, we need to introduce a little hack - we will define our properties with an included entity type. \n",
-    "\n",
-    "Following we have an example where we also want to extract dog attributes from the passage. Notice the 'person_' and 'dog_' prefixes we use for each property; this tells the model which entity type the property refers to. In this way, the model can return properties from several entity types in one single call."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "cf243a26",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "schema = {\n",
-    "    \"properties\": {\n",
-    "        \"person_name\": {\"type\": \"string\"},\n",
-    "        \"person_height\": {\"type\": \"integer\"},\n",
-    "        \"person_hair_color\": {\"type\": \"string\"},\n",
-    "        \"dog_name\": {\"type\": \"string\"},\n",
-    "        \"dog_breed\": {\"type\": \"string\"},\n",
-    "    },\n",
-    "    \"required\": [\"person_name\", \"person_height\"],\n",
-    "}"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "52841fb3",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inp = \"\"\"\n",
-    "Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
-    "Alex's dog Frosty is a labrador and likes to play hide and seek.\n",
-    "        \"\"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "93f904ab",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = create_extraction_chain(schema, llm)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "eb074f7b",
-   "metadata": {},
-   "source": [
-    "People attributes and dog attributes were correctly extracted from the text in the same call"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "db3e9e17",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[{'person_name': 'Alex',\n",
-       "  'person_height': 5,\n",
-       "  'person_hair_color': 'blonde',\n",
-       "  'dog_name': 'Frosty',\n",
-       "  'dog_breed': 'labrador'},\n",
-       " {'person_name': 'Claudia',\n",
-       "  'person_height': 6,\n",
-       "  'person_hair_color': 'brunette'}]"
-      ]
-     },
-     "execution_count": 11,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.run(inp)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0273e0e2",
-   "metadata": {},
-   "source": [
-    "## Unrelated entities"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c07b3480",
-   "metadata": {},
-   "source": [
-    "What if our entities are unrelated? In that case, the model will return the unrelated entities in different dictionaries, allowing us to successfully extract several unrelated entity types in the same call."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "01d98af0",
-   "metadata": {},
-   "source": [
-    "Notice that we use `required: []`: we need to allow the model to return **only** person attributes or **only** dog attributes for a single entity (person or dog)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 48,
-   "id": "e584c993",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "schema = {\n",
-    "    \"properties\": {\n",
-    "        \"person_name\": {\"type\": \"string\"},\n",
-    "        \"person_height\": {\"type\": \"integer\"},\n",
-    "        \"person_hair_color\": {\"type\": \"string\"},\n",
-    "        \"dog_name\": {\"type\": \"string\"},\n",
-    "        \"dog_breed\": {\"type\": \"string\"},\n",
-    "    },\n",
-    "    \"required\": [],\n",
-    "}"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 49,
-   "id": "ad6b105f",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inp = \"\"\"\n",
-    "Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
-    "\n",
-    "Willow is a German Shepherd that likes to play with other dogs and can always be found playing with Milo, a border collie that lives close by.\n",
-    "\"\"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 50,
-   "id": "6bfe5a33",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = create_extraction_chain(schema, llm)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "24fe09af",
-   "metadata": {},
-   "source": [
-    "We have each entity in its own separate dictionary, with only the appropriate attributes being returned"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 51,
-   "id": "f6e1fd89",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[{'person_name': 'Alex', 'person_height': 5, 'person_hair_color': 'blonde'},\n",
-       " {'person_name': 'Claudia',\n",
-       "  'person_height': 6,\n",
-       "  'person_hair_color': 'brunette'},\n",
-       " {'dog_name': 'Willow', 'dog_breed': 'German Shepherd'},\n",
-       " {'dog_name': 'Milo', 'dog_breed': 'border collie'}]"
-      ]
-     },
-     "execution_count": 51,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.run(inp)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0ac466d1",
-   "metadata": {},
-   "source": [
-    "## Extra info for an entity"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "d240ffc1",
-   "metadata": {},
-   "source": [
-    "What if.. _we don't know what we want?_ More specifically, say we know a few properties we want to extract for a given entity but we also want to know if there's any extra information in the passage. Fortunately, we don't need to structure everything - we can have unstructured extraction as well. \n",
-    "\n",
-    "We can do this by introducing another hack, namely the *extra_info* attribute - let's see an example."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 68,
-   "id": "f19685f6",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "schema = {\n",
-    "    \"properties\": {\n",
-    "        \"person_name\": {\"type\": \"string\"},\n",
-    "        \"person_height\": {\"type\": \"integer\"},\n",
-    "        \"person_hair_color\": {\"type\": \"string\"},\n",
-    "        \"dog_name\": {\"type\": \"string\"},\n",
-    "        \"dog_breed\": {\"type\": \"string\"},\n",
-    "        \"dog_extra_info\": {\"type\": \"string\"},\n",
-    "    },\n",
-    "}"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 81,
-   "id": "200c3477",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inp = \"\"\"\n",
-    "Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
-    "\n",
-    "Willow is a German Shepherd that likes to play with other dogs and can always be found playing with Milo, a border collie that lives close by.\n",
-    "\"\"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 82,
-   "id": "ddad7dc6",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = create_extraction_chain(schema, llm)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e5c0dbbc",
-   "metadata": {},
-   "source": [
-    "It is nice to know more about Willow and Milo!"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 83,
-   "id": "c22cfd30",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[{'person_name': 'Alex', 'person_height': 5, 'person_hair_color': 'blonde'},\n",
-       " {'person_name': 'Claudia',\n",
-       "  'person_height': 6,\n",
-       "  'person_hair_color': 'brunette'},\n",
-       " {'dog_name': 'Willow',\n",
-       "  'dog_breed': 'German Shepherd',\n",
-       "  'dog_extra_information': 'likes to play with other dogs'},\n",
-       " {'dog_name': 'Milo',\n",
-       "  'dog_breed': 'border collie',\n",
-       "  'dog_extra_information': 'lives close by'}]"
-      ]
-     },
-     "execution_count": 83,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.run(inp)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "698b4c4d",
-   "metadata": {},
-   "source": [
-    "## Pydantic example"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6504a6d9",
-   "metadata": {},
-   "source": [
-    "We can also use a Pydantic schema to choose the required properties and types and we will set as 'Optional' those that are not strictly required.\n",
-    "\n",
-    "By using the `create_extraction_chain_pydantic` function, we can send a Pydantic schema as input and the output will be an instantiated object that respects our desired schema. \n",
-    "\n",
-    "In this way, we can specify our schema in the same manner that we would a new class or function in Python - with purely Pythonic types."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "6792866b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from typing import Optional, List\n",
-    "from pydantic import BaseModel, Field"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "36a63761",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class Properties(BaseModel):\n",
-    "    person_name: str\n",
-    "    person_height: int\n",
-    "    person_hair_color: str\n",
-    "    dog_breed: Optional[str]\n",
-    "    dog_name: Optional[str]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "8ffd1e57",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = create_extraction_chain_pydantic(pydantic_schema=Properties, llm=llm)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "24baa954",
-   "metadata": {
-    "scrolled": false
-   },
-   "outputs": [],
-   "source": [
-    "inp = \"\"\"\n",
-    "Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.\n",
-    "Alex's dog Frosty is a labrador and likes to play hide and seek.\n",
-    "        \"\"\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "84e0a241",
-   "metadata": {},
-   "source": [
-    "As we can see, we extracted the required entities and their properties in the required format:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "id": "f771df58",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[Properties(person_name='Alex', person_height=5, person_hair_color='blonde', dog_breed='labrador', dog_name='Frosty'),\n",
-       " Properties(person_name='Claudia', person_height=6, person_hair_color='brunette', dog_breed=None, dog_name=None)]"
-      ]
-     },
-     "execution_count": 11,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.run(inp)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "0df61283",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/extras/use_cases/question_answering/how_to/conversational_retrieval_agents.ipynb

@@ -0,0 +1,583 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "839f3c76",
+   "metadata": {},
+   "source": [
+    "# Conversational Retrieval Agent\n",
+    "\n",
+    "This is an agent specifically optimized for doing retrieval when necessary and also holding a conversation.\n",
+    "\n",
+    "To start, we will set up the retriever we want to use, and then turn it into a retriever tool. Next, we will use the high level constructor for this type of agent. Finally, we will walk through how to construct a conversational retrieval agent from components."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dc66a262",
+   "metadata": {},
+   "source": [
+    "## The Retriever\n",
+    "\n",
+    "To start, we need a retriever to use! The code here is mostly just example code. Feel free to use your own retriever and skip to the section on creating a retriever tool."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "22533f46",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.document_loaders import TextLoader\n",
+    "loader = TextLoader('../../../../../docs/extras/modules/state_of_the_union.txt')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "c252c7e9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.text_splitter import CharacterTextSplitter\n",
+    "from langchain.vectorstores import FAISS\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "\n",
+    "documents = loader.load()\n",
+    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
+    "texts = text_splitter.split_documents(documents)\n",
+    "embeddings = OpenAIEmbeddings()\n",
+    "db = FAISS.from_documents(texts, embeddings)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "5b8a404f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "retriever = db.as_retriever()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9cd528f5",
+   "metadata": {},
+   "source": [
+    "## Retriever Tool\n",
+    "\n",
+    "Now we need to create a tool for our retriever. The main things we need to pass in are a name for the retriever as well as a description. These will both be used by the language model, so they should be informative."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "9a82f72a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents.agent_toolkits import create_retriever_tool"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "dfbd92a2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool = create_retriever_tool(\n",
+    "    retriever, \n",
+    "    \"search_state_of_union\",\n",
+    "    \"Searches and returns documents regarding the state-of-the-union.\"\n",
+    ")\n",
+    "tools = [tool]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6d9b7210",
+   "metadata": {},
+   "source": [
+    "## Agent Constructor\n",
+    "\n",
+    "Here, we will use the high level `create_conversational_retrieval_agent` API to construct the agent.\n",
+    "\n",
+    "Notice that beside the list of tools, the only thing we need to pass in is a language model to use.\n",
+    "Under the hood, this agent is using the OpenAIFunctionsAgent, so we need to use an ChatOpenAI model."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "0cd147eb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents.agent_toolkits import create_conversational_retrieval_agent "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "9fa4661b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "llm = ChatOpenAI(temperature = 0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "da2ad764",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent_executor = create_conversational_retrieval_agent(llm, tools, verbose=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "85ee7677",
+   "metadata": {},
+   "source": [
+    "We can now try it out!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "03059322",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mHello Bob! How can I assist you today?\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "result = agent_executor({\"input\": \"hi, im bob\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "33073ff0",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Hello Bob! How can I assist you today?'"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "result[\"output\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5f1f0b79",
+   "metadata": {},
+   "source": [
+    "Notice that it remembers your name"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "4ad92bc7",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mYour name is Bob.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "result = agent_executor({\"input\": \"whats my name?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "7ae62ecd",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Your name is Bob.'"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "result[\"output\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8e258ade",
+   "metadata": {},
+   "source": [
+    "Notice that it now does retrieval"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "6cd17d67",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3m\n",
+      "Invoking: `search_state_of_union` with `{'query': 'Kentaji Brown Jackson'}`\n",
+      "\n",
+      "\n",
+      "\u001b[0m\u001b[36;1m\u001b[1;3m[Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \\n\\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \\n\\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': '../../../../../docs/extras/modules/state_of_the_union.txt'}), Document(page_content='One was stationed at bases and breathing in toxic smoke from “burn pits” that incinerated wastes of war—medical and hazard material, jet fuel, and more. \\n\\nWhen they came home, many of the world’s fittest and best trained warriors were never the same. \\n\\nHeadaches. Numbness. Dizziness. \\n\\nA cancer that would put them in a flag-draped coffin. \\n\\nI know. \\n\\nOne of those soldiers was my son Major Beau Biden. \\n\\nWe don’t know for sure if a burn pit was the cause of his brain cancer, or the diseases of so many of our troops. \\n\\nBut I’m committed to finding out everything we can. \\n\\nCommitted to military families like Danielle Robinson from Ohio. \\n\\nThe widow of Sergeant First Class Heath Robinson.  \\n\\nHe was born a soldier. Army National Guard. Combat medic in Kosovo and Iraq. \\n\\nStationed near Baghdad, just yards from burn pits the size of football fields. \\n\\nHeath’s widow Danielle is here with us tonight. They loved going to Ohio State football games. He loved building Legos with their daughter.', metadata={'source': '../../../../../docs/extras/modules/state_of_the_union.txt'}), Document(page_content='A former top litigator in private practice. A former federal public defender. And from a family of public school educators and police officers. A consensus builder. Since she’s been nominated, she’s received a broad range of support—from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. \\n\\nAnd if we are to advance liberty and justice, we need to secure the Border and fix the immigration system. \\n\\nWe can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.  \\n\\nWe’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.  \\n\\nWe’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster. \\n\\nWe’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.', metadata={'source': '../../../../../docs/extras/modules/state_of_the_union.txt'}), Document(page_content='We can’t change how divided we’ve been. But we can change how we move forward—on COVID-19 and other issues we must face together. \\n\\nI recently visited the New York City Police Department days after the funerals of Officer Wilbert Mora and his partner, Officer Jason Rivera. \\n\\nThey were responding to a 9-1-1 call when a man shot and killed them with a stolen gun. \\n\\nOfficer Mora was 27 years old. \\n\\nOfficer Rivera was 22. \\n\\nBoth Dominican Americans who’d grown up on the same streets they later chose to patrol as police officers. \\n\\nI spoke with their families and told them that we are forever in debt for their sacrifice, and we will carry on their mission to restore the trust and safety every community deserves. \\n\\nI’ve worked on these issues a long time. \\n\\nI know what works: Investing in crime preventionand community police officers who’ll walk the beat, who’ll know the neighborhood, and who can restore trust and safety.', metadata={'source': '../../../../../docs/extras/modules/state_of_the_union.txt'})]\u001b[0m\u001b[32;1m\u001b[1;3mIn the most recent state of the union, the President mentioned Kentaji Brown Jackson. The President nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to serve on the United States Supreme Court. The President described Judge Ketanji Brown Jackson as one of our nation's top legal minds who will continue Justice Breyer's legacy of excellence.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "result = agent_executor({\"input\": \"what did the president say about kentaji brown jackson in the most recent state of the union?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "c51de037",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"In the most recent state of the union, the President mentioned Kentaji Brown Jackson. The President nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to serve on the United States Supreme Court. The President described Judge Ketanji Brown Jackson as one of our nation's top legal minds who will continue Justice Breyer's legacy of excellence.\""
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "result[\"output\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d3bad540",
+   "metadata": {},
+   "source": [
+    "Notice that the follow up question asks about information previously retrieved, so no need to do another retrieval"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "527ea5fd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mThe President nominated Judge Ketanji Brown Jackson four days ago.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "result = agent_executor({\"input\": \"how long ago did he nominate her?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "a34d20fd",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'The President nominated Judge Ketanji Brown Jackson four days ago.'"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "result[\"output\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e599dbd3",
+   "metadata": {},
+   "source": [
+    "## Creating from components\n",
+    "\n",
+    "What actually is going on underneath the hood? Let's take a look so we can understand how to modify going forward.\n",
+    "\n",
+    "There are a few components:\n",
+    "\n",
+    "- The memory\n",
+    "- The prompt template\n",
+    "- The agent\n",
+    "- The agent executor"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "1b21be1d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This is needed for both the memory and the prompt\n",
+    "memory_key = \"history\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f827f95f",
+   "metadata": {},
+   "source": [
+    "### The Memory\n",
+    "\n",
+    "In this example, we want the agent to remember not only previous conversations, but also previous intermediate steps. For that, we can use `AgentTokenBufferMemory`. Note that if you want to change whether the agent remembers intermediate steps, or how the long the buffer is, or anything like that you should change this part."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "138b0675",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents.openai_functions_agent.agent_token_buffer_memory import AgentTokenBufferMemory\n",
+    "\n",
+    "memory = AgentTokenBufferMemory(memory_key=memory_key, llm=llm)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4827993f",
+   "metadata": {},
+   "source": [
+    "## The Prompt Template\n",
+    "\n",
+    "For the prompt template, we will use the `OpenAIFunctionsAgent` default way of creating one, but pass in a system prompt and a placeholder for memory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "779272dd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents.openai_functions_agent.base import OpenAIFunctionsAgent\n",
+    "from langchain.schema.messages import SystemMessage\n",
+    "from langchain.prompts import MessagesPlaceholder"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "824bc74e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "system_message = SystemMessage(\n",
+    "        content=(\n",
+    "            \"Do your best to answer the questions. \"\n",
+    "            \"Feel free to use any tools available to look up \"\n",
+    "            \"relevant information, only if neccessary\"\n",
+    "        )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "07e41722",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt = OpenAIFunctionsAgent.create_prompt(\n",
+    "        system_message=system_message,\n",
+    "        extra_prompt_messages=[MessagesPlaceholder(variable_name=memory_key)]\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "001e455e",
+   "metadata": {},
+   "source": [
+    "## The Agent\n",
+    "\n",
+    "We will use the OpenAIFunctionsAgent"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "adf4b5b3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=prompt)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2c5c321e",
+   "metadata": {},
+   "source": [
+    "## The Agent Executor\n",
+    "\n",
+    "Importantly, we pass in `return_intermediate_steps=True` since we are recording that with our memory object"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "2e7ffe96",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import AgentExecutor"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "id": "e39a095f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True,\n",
+    "                                   return_intermediate_steps=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "96136958",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mHello Bob! How can I assist you today?\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "result = agent_executor({\"input\": \"hi, im bob\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "8de674cb",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mYour name is Bob.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "result = agent_executor({\"input\": \"whats my name\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bf655a48",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/use_cases/question_answering/index.mdx

@@ -138,9 +138,9 @@ len(docs_svm)
     4
 
 Some common ways to improve on vector similarity search include:
-- `MultiQueryRetriever` [generates variants of the input question](/docs/modules/data_connection/retrievers/how_to/MultiQueryRetriever) to improve retrieval.
+- `MultiQueryRetriever` [generates variants of the input question](/docs/modules/data_connection/retrievers/MultiQueryRetriever) to improve retrieval.
 - `Max marginal relevance` selects for [relevance and diversity](https://www.cs.cmu.edu/~jgc/publication/The_Use_MMR_Diversity_Based_LTMIR_1998.pdf) among the retrieved documents.
-- Documents can be filtered during retrieval using [`metadata` filters](/docs/use_cases/question_answering/document-context-aware-QA).
+- Documents can be filtered during retrieval using [`metadata` filters](/docs/use_cases/question_answering/how_to/document-context-aware-QA).
 
 
 ```python

docs/extras/use_cases/summarization.ipynb

@@ -0,0 +1,511 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "cf13f702",
+   "metadata": {},
+   "source": [
+    "# Summarization\n",
+    "\n",
+    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/extras/use_cases/summarization.ipynb)\n",
+    "\n",
+    "## Use case\n",
+    "\n",
+    "Suppose you have a set of documents (PDFs, Notion pages, customer questions, etc.) and you want to summarize the content. \n",
+    "\n",
+    "LLMs are a great tool for this given their proficiency in understanding and synthesizing text.\n",
+    "\n",
+    "In this walkthrough we'll go over how to perform document summarization using LLMs."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8e233997",
+   "metadata": {},
+   "source": [
+    "![Image description](/img/summarization_use_case_1.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4715b4ff",
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
+   "source": [
+    "## Overview\n",
+    "\n",
+    "A central question for building a summarizer is how to pass your documents into the LLM's context window. Two common approaches for this are:\n",
+    "\n",
+    "1. `Stuff`: Simply \"stuff\" all your documents into a single prompt. This is the simplest approach (see [here](/docs/modules/chains/document/stuff) for more on the `StuffDocumentsChains`, which is used for this method).\n",
+    "\n",
+    "2. `Map-reduce`: Summarize each document on it's own in a \"map\" step and then \"reduce\" the summaries into a final summary (see [here](/docs/modules/chains/document/map_reduce) for more on the `MapReduceDocumentsChain`, which is used for this method)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "08ec66bc",
+   "metadata": {},
+   "source": [
+    "![Image description](/img/summarization_use_case_2.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bea785ac",
+   "metadata": {},
+   "source": [
+    "## Quickstart\n",
+    "\n",
+    "To give you a sneak preview, either pipeline can be wrapped in a single object: `load_summarize_chain`. \n",
+    "\n",
+    "Suppose we want to summarize a blog post. We can create this in a few lines of code.\n",
+    "\n",
+    "First set environment variables and install packages:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "578d6a90",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install openai tiktoken chromadb langchain\n",
+    "\n",
+    "# Set env var OPENAI_API_KEY or load from a .env file\n",
+    "# import dotenv\n",
+    "\n",
+    "# dotenv.load_env()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "36138740",
+   "metadata": {},
+   "source": [
+    "We can use `chain_type=\"stuff\"`, especially if using larger context window models such as:\n",
+    "\n",
+    "* 16k token OpenAI `gpt-3.5-turbo-16k` \n",
+    "* 100k token Anthropic [Claude-2](https://www.anthropic.com/index/claude-2)\n",
+    "\n",
+    "We can also supply `chain_type=\"map_reduce\"` or `chain_type=\"refine\"` (read more [here](/docs/modules/chains/document/refine))."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "fd271681",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'The article discusses the concept of building autonomous agents powered by large language models (LLMs). It explores the components of such agents, including planning, memory, and tool use. The article provides case studies and proof-of-concept examples of LLM-powered agents in various domains. It also highlights the challenges and limitations of using LLMs in agent systems.'"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.document_loaders import WebBaseLoader\n",
+    "from langchain.chains.summarize import load_summarize_chain\n",
+    "\n",
+    "loader = WebBaseLoader(\"https://lilianweng.github.io/posts/2023-06-23-agent/\")\n",
+    "docs = loader.load()\n",
+    "\n",
+    "llm = ChatOpenAI(temperature=0, model_name=\"gpt-3.5-turbo-16k\")\n",
+    "chain = load_summarize_chain(llm, chain_type=\"stuff\")\n",
+    "\n",
+    "chain.run(docs)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "615b36e1",
+   "metadata": {},
+   "source": [
+    "## Option 1. Stuff\n",
+    "\n",
+    "When we use `load_summarize_chain` with `chain_type=\"stuff\"`, we will use the [StuffDocumentsChain](/docs/modules/chains/document/stuff).\n",
+    "\n",
+    "The chain will take a list of documents, inserts them all into a prompt, and passes that prompt to an LLM:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "ef45585d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The article discusses the concept of building autonomous agents powered by large language models (LLMs). It explores the components of such agents, including planning, memory, and tool use. The article provides case studies and examples of proof-of-concept demos, highlighting the challenges and limitations of LLM-powered agents. It also includes references to related research papers and provides a citation for the article.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain.chains.llm import LLMChain\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains.combine_documents.stuff import StuffDocumentsChain\n",
+    "\n",
+    "# Define prompt\n",
+    "prompt_template = \"\"\"Write a concise summary of the following:\n",
+    "\"{text}\"\n",
+    "CONCISE SUMMARY:\"\"\"\n",
+    "prompt = PromptTemplate.from_template(prompt_template)\n",
+    "\n",
+    "# Define LLM chain\n",
+    "llm = ChatOpenAI(temperature=0, model_name=\"gpt-3.5-turbo-16k\")\n",
+    "llm_chain = LLMChain(llm=llm, prompt=prompt)\n",
+    "\n",
+    "# Define StuffDocumentsChain\n",
+    "stuff_chain = StuffDocumentsChain(\n",
+    "    llm_chain=llm_chain, document_variable_name=\"text\"\n",
+    ")\n",
+    "\n",
+    "docs = loader.load()\n",
+    "print(stuff_chain.run(docs))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4e4e4a43",
+   "metadata": {},
+   "source": [
+    "Great! We can see that we reproduce the earlier result using the `load_summarize_chain`.\n",
+    "\n",
+    "### Go deeper\n",
+    "\n",
+    "* You can easily customize the prompt. \n",
+    "* You can easily try different LLMs, (e.g., [Claude](/docs/integrations/chat/anthropic)) via the `llm` parameter."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ad6cabee",
+   "metadata": {},
+   "source": [
+    "## Option 2. Map-Reduce\n",
+    "\n",
+    "Let's unpack the map reduce approach. For this, we'll first map each document to an individual summary using an `LLMChain`. Then we'll use a `ReduceDocumentsChain` to combine those summaries into a single global summary.\n",
+    " \n",
+    "First, we specfy the LLMChain to use for mapping each document to an individual summary:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "a1e6773c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains.mapreduce import MapReduceChain\n",
+    "from langchain.text_splitter import CharacterTextSplitter\n",
+    "from langchain.chains import ReduceDocumentsChain, MapReduceDocumentsChain\n",
+    "\n",
+    "llm = ChatOpenAI(temperature=0)\n",
+    "\n",
+    "# Map\n",
+    "map_template = \"\"\"The following is a set of documents\n",
+    "{docs}\n",
+    "Based on this list of docs, please identify the main themes \n",
+    "Helpful Answer:\"\"\"\n",
+    "map_prompt = PromptTemplate.from_template(map_template)\n",
+    "map_chain = LLMChain(llm=llm, prompt=map_prompt)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bee3c331",
+   "metadata": {},
+   "source": [
+    "The `ReduceDocumentsChain` handles taking the document mapping results and reducing them into a single output. It wraps a generic `CombineDocumentsChain` (like `StuffDocumentsChain`) but adds the ability to collapse documents before passing it to the `CombineDocumentsChain` if their cumulative size exceeds `token_max`. In this example, we can actually re-use our chain for combining our docs to also collapse our docs.\n",
+    "\n",
+    "So if the cumulative number of tokens in our mapped documents exceeds 4000 tokens, then we'll recursively pass in the documents in batches of < 4000 tokens to our `StuffDocumentsChain` to create batched summaries. And once those batched summaries are cumulatively less than 4000 tokens, we'll pass them all one last time to the `StuffDocumentsChain` to create the final summary."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "1edb1b0d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Reduce\n",
+    "reduce_template = \"\"\"The following is set of summaries:\n",
+    "{doc_summaries}\n",
+    "Take these and distill it into a final, consolidated summary of the main themes. \n",
+    "Helpful Answer:\"\"\"\n",
+    "reduce_prompt = PromptTemplate.from_template(reduce_template)\n",
+    "reduce_chain = LLMChain(llm=llm, prompt=reduce_prompt)\n",
+    "\n",
+    "# Takes a list of documents, combines them into a single string, and passes this to an LLMChain\n",
+    "combine_documents_chain = StuffDocumentsChain(\n",
+    "    llm_chain=reduce_chain, document_variable_name=\"doc_summaries\"\n",
+    ")\n",
+    "\n",
+    "# Combines and iteravely reduces the mapped documents\n",
+    "reduce_documents_chain = ReduceDocumentsChain(\n",
+    "    # This is final chain that is called.\n",
+    "    combine_documents_chain=combine_documents_chain,\n",
+    "    # If documents exceed context for `StuffDocumentsChain`\n",
+    "    collapse_documents_chain=combine_documents_chain,\n",
+    "    # The maximum number of tokens to group documents into.\n",
+    "    token_max=4000,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fdb5ae1a",
+   "metadata": {},
+   "source": [
+    "Combining our map and reduce chains into one:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "22f1cdc2",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Created a chunk of size 1003, which is longer than the specified 1000\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Combining documents by mapping a chain over them, then combining results\n",
+    "map_reduce_chain = MapReduceDocumentsChain(\n",
+    "    # Map chain\n",
+    "    llm_chain=map_chain,\n",
+    "    # Reduce chain\n",
+    "    reduce_documents_chain=reduce_documents_chain,\n",
+    "    # The variable name in the llm_chain to put the documents in\n",
+    "    document_variable_name=\"docs\",\n",
+    "    # Return the results of the map steps in the output\n",
+    "    return_intermediate_steps=False,\n",
+    ")\n",
+    "\n",
+    "text_splitter = CharacterTextSplitter.from_tiktoken_encoder(\n",
+    "    chunk_size=1000, chunk_overlap=0\n",
+    ")\n",
+    "split_docs = text_splitter.split_documents(docs)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "c7afb8c3",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The main themes identified in the provided set of documents are:\n",
+      "\n",
+      "1. LLM-powered autonomous agent systems: The documents discuss the concept of building autonomous agents with large language models (LLMs) as the core controller. They explore the potential of LLMs beyond content generation and present them as powerful problem solvers.\n",
+      "\n",
+      "2. Components of the agent system: The documents outline the key components of LLM-powered agent systems, including planning, memory, and tool use. Each component is described in detail, highlighting its role in enhancing the agent's capabilities.\n",
+      "\n",
+      "3. Planning and task decomposition: The planning component focuses on task decomposition and self-reflection. The agent breaks down complex tasks into smaller subgoals and learns from past actions to improve future results.\n",
+      "\n",
+      "4. Memory and learning: The memory component includes short-term memory for in-context learning and long-term memory for retaining and recalling information over extended periods. The use of external vector stores for fast retrieval is also mentioned.\n",
+      "\n",
+      "5. Tool use and external APIs: The agent learns to utilize external APIs for accessing additional information, code execution, and proprietary sources. This enhances the agent's knowledge and problem-solving abilities.\n",
+      "\n",
+      "6. Case studies and proof-of-concept examples: The documents provide case studies and examples to demonstrate the application of LLM-powered agents in scientific discovery, generative simulations, and other domains. These examples serve as proof-of-concept for the effectiveness of the agent system.\n",
+      "\n",
+      "7. Challenges and limitations: The documents mention challenges associated with building LLM-powered autonomous agents, such as the limitations of finite context length, difficulties in long-term planning, and reliability issues with natural language interfaces.\n",
+      "\n",
+      "8. Citation and references: The documents include a citation and reference section for acknowledging the sources and inspirations for the concepts discussed.\n",
+      "\n",
+      "Overall, the main themes revolve around the development and capabilities of LLM-powered autonomous agent systems, including their components, planning and task decomposition, memory and learning mechanisms, tool use and external APIs, case studies and proof-of-concept examples, challenges and limitations, and the importance of proper citation and references.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(map_reduce_chain.run(split_docs))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e62c21cf",
+   "metadata": {},
+   "source": [
+    "### Go deeper\n",
+    " \n",
+    "**Customization** \n",
+    "\n",
+    "* As shown above, you can customize the LLMs and prompts for map and reduce stages.\n",
+    "\n",
+    "**Real-world use-case**\n",
+    "\n",
+    "* See [this blog post](https://blog.langchain.dev/llms-to-improve-documentation/) case-study on analyzing user interactions (questions about LangChain documentation)!  \n",
+    "* The blog post and associated [repo](https://github.com/mendableai/QA_clustering) also introduce clustering as a means of summarization.\n",
+    "* This opens up a third path beyond the `stuff` or `map-reduce` approaches that is worth considering.\n",
+    "\n",
+    "![Image description](/img/summarization_use_case_3.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f08ff365",
+   "metadata": {},
+   "source": [
+    "## Option 3. Refine\n",
+    " \n",
+    "[Refine](/docs/modules/chains/document/refine) is similar to map-reduce:\n",
+    "\n",
+    "> The refine documents chain constructs a response by looping over the input documents and iteratively updating its answer. For each document, it passes all non-document inputs, the current document, and the latest intermediate answer to an LLM chain to get a new answer.\n",
+    "\n",
+    "This can be easily run with the `chain_type=\"refine\"` specified."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "de1dc10e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'The GPT-Engineer project aims to create a repository of code for specific tasks specified in natural language. It involves breaking down tasks into smaller components and seeking clarification from the user when needed. The project emphasizes the importance of implementing every detail of the architecture as code and provides guidelines for file organization, code structure, and dependencies. However, there are challenges in long-term planning and task decomposition, as well as the reliability of the natural language interface. The system has limited communication bandwidth and struggles to adjust plans when faced with unexpected errors. The reliability of model outputs is questionable, as formatting errors and rebellious behavior can occur. The conversation also includes instructions for writing the code, including laying out the core classes, functions, and methods, and providing the code in a markdown code block format. The user is reminded to ensure that the code is fully functional and follows best practices for file naming, imports, and types. The project is powered by LLM (Large Language Models) and incorporates prompting techniques from various research papers.'"
+      ]
+     },
+     "execution_count": 22,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain = load_summarize_chain(llm, chain_type=\"refine\")\n",
+    "chain.run(split_docs)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5b46f44d",
+   "metadata": {},
+   "source": [
+    "It's also possible to supply a prompt and return intermediate steps."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "id": "f86c8072",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt_template = \"\"\"Write a concise summary of the following:\n",
+    "{text}\n",
+    "CONCISE SUMMARY:\"\"\"\n",
+    "prompt = PromptTemplate.from_template(prompt_template)\n",
+    "\n",
+    "refine_template = (\n",
+    "    \"Your job is to produce a final summary\\n\"\n",
+    "    \"We have provided an existing summary up to a certain point: {existing_answer}\\n\"\n",
+    "    \"We have the opportunity to refine the existing summary\"\n",
+    "    \"(only if needed) with some more context below.\\n\"\n",
+    "    \"------------\\n\"\n",
+    "    \"{text}\\n\"\n",
+    "    \"------------\\n\"\n",
+    "    \"Given the new context, refine the original summary in Italian\"\n",
+    "    \"If the context isn't useful, return the original summary.\"\n",
+    ")\n",
+    "refine_prompt = PromptTemplate.from_template(refine_template)\n",
+    "chain = load_summarize_chain(\n",
+    "    llm=llm,\n",
+    "    chain_type=\"refine\",\n",
+    "    question_prompt=prompt,\n",
+    "    refine_prompt=refine_prompt,\n",
+    "    return_intermediate_steps=True,\n",
+    "    input_key=\"input_documents\",\n",
+    "    output_key=\"output_text\",\n",
+    ")\n",
+    "result = chain({\"input_documents\": split_docs}, return_only_outputs=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "id": "d9600b67-79d4-4f85-aba2-9fe81fa29f49",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "L'articolo discute il concetto di costruire agenti autonomi utilizzando LLM (large language model) come controller principale. Esplora i diversi componenti di un sistema di agenti alimentato da LLM, inclusa la pianificazione, la memoria e l'uso di strumenti. Dimostrazioni di concetto come AutoGPT mostrano la possibilità di creare agenti autonomi con LLM come controller principale. Approcci come Chain of Thought, Tree of Thoughts, LLM+P, ReAct e Reflexion consentono agli agenti autonomi di pianificare, riflettere su se stessi e migliorare iterativamente. Tuttavia, ci sono sfide legate alla lunghezza del contesto, alla pianificazione a lungo termine e alla decomposizione delle attività. Inoltre, l'affidabilità dell'interfaccia di linguaggio naturale tra LLM e componenti esterni come la memoria e gli strumenti è incerta. Nonostante ciò, l'uso di LLM come router per indirizzare le richieste ai moduli esperti più adatti è stato proposto come architettura neuro-simbolica per agenti autonomi nel sistema MRKL. L'articolo fa riferimento a diverse pubblicazioni che approfondiscono l'argomento, tra cui Chain of Thought, Tree of Thoughts, LLM+P, ReAct, Reflexion, e MRKL Systems.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(result[\"output_text\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "5f91a8eb-daa5-4191-ace4-01765801db3e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "This article discusses the concept of building autonomous agents using LLM (large language model) as the core controller. The article explores the different components of an LLM-powered agent system, including planning, memory, and tool use. It also provides examples of proof-of-concept demos and highlights the potential of LLM as a general problem solver.\n",
+      "\n",
+      "Questo articolo discute del concetto di costruire agenti autonomi utilizzando LLM (large language model) come controller principale. L'articolo esplora i diversi componenti di un sistema di agenti alimentato da LLM, inclusa la pianificazione, la memoria e l'uso degli strumenti. Vengono anche forniti esempi di dimostrazioni di proof-of-concept e si evidenzia il potenziale di LLM come risolutore generale di problemi. Inoltre, vengono presentati approcci come Chain of Thought, Tree of Thoughts, LLM+P, ReAct e Reflexion che consentono agli agenti autonomi di pianificare, riflettere su se stessi e migliorare iterativamente.\n",
+      "\n",
+      "Questo articolo discute del concetto di costruire agenti autonomi utilizzando LLM (large language model) come controller principale. L'articolo esplora i diversi componenti di un sistema di agenti alimentato da LLM, inclusa la pianificazione, la memoria e l'uso degli strumenti. Vengono anche forniti esempi di dimostrazioni di proof-of-concept e si evidenzia il potenziale di LLM come risolutore generale di problemi. Inoltre, vengono presentati approcci come Chain of Thought, Tree of Thoughts, LLM+P, ReAct e Reflexion che consentono agli agenti autonomi di pianificare, riflettere su se stessi e migliorare iterativamente. Il nuovo contesto riguarda l'approccio Chain of Hindsight (CoH) che permette al modello di migliorare autonomamente i propri output attraverso un processo di apprendimento supervisionato. Viene anche presentato l'approccio Algorithm Distillation (AD) che applica lo stesso concetto alle traiettorie di apprendimento per compiti di reinforcement learning.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\n\\n\".join(result[\"intermediate_steps\"][:3]))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0ddd522e-30dc-4f6a-b993-c4f97e656c4f",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/extras/use_cases/summarization/index.mdx

@@ -1,22 +0,0 @@
----
-sidebar_position: 5
----
-
-# Summarization
-
-Summarization involves creating a smaller summary of multiple longer documents.
-This can be useful for distilling long documents into the core pieces of information.
-
-The recommended way to get started using a summarization chain is:
-
-```python
-from langchain.chains.summarize import load_summarize_chain
-chain = load_summarize_chain(llm, chain_type="map_reduce")
-chain.run(docs)
-```
-
-The following resources exist:
-- [Summarization notebook](/docs/use_cases/summarization/summarize.html): A notebook walking through how to accomplish this task.
-
-Additional related resources include:
-- [Modules for working with documents](/docs/modules/data_connection): Core components for working with documents.

docs/extras/use_cases/tabular/adding_memory_to_sql.ipynb

@@ -1,217 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "488d6ee8",
-   "metadata": {},
-   "source": [
-    "# Adding Memory to SQL Database Chain\n",
-    "\n",
-    "This notebook shows how to add memory to a SQLDatabaseChain."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "6ef6918e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.llms import OpenAI\n",
-    "from langchain.utilities import SQLDatabase\n",
-    "from langchain_experimental.sql import SQLDatabaseChain"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "600aedb5",
-   "metadata": {},
-   "source": [
-    "Set up the SQLDatabase and LLM"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "b54c24c2",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "db = SQLDatabase.from_uri(\"sqlite:///../../../../notebooks/Chinook.db\")\n",
-    "llm = OpenAI(temperature=0, verbose=True)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "96a1543f",
-   "metadata": {},
-   "source": [
-    "Set up the memory"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "fc103f91",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.memory import ConversationBufferMemory\n",
-    "memory = ConversationBufferMemory()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "af31b91d",
-   "metadata": {},
-   "source": [
-    "Now we need add to a place for memory in the prompt template"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "debcff82",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.prompts import PromptTemplate\n",
-    "PROMPT_SUFFIX = \"\"\"Only use the following tables:\n",
-    "{table_info}\n",
-    "\n",
-    "Previous Conversation:\n",
-    "{history}\n",
-    "\n",
-    "Question: {input}\"\"\"\n",
-    "\n",
-    "_DEFAULT_TEMPLATE = \"\"\"Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer. Unless the user specifies in his question a specific number of examples he wishes to obtain, always limit your query to at most {top_k} results. You can order the results by a relevant column to return the most interesting examples in the database.\n",
-    "\n",
-    "Never query for all the columns from a specific table, only ask for a the few relevant columns given the question.\n",
-    "\n",
-    "Pay attention to use only the column names that you can see in the schema description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.\n",
-    "\n",
-    "Use the following format:\n",
-    "\n",
-    "Question: Question here\n",
-    "SQLQuery: SQL Query to run\n",
-    "SQLResult: Result of the SQLQuery\n",
-    "Answer: Final answer here\n",
-    "\n",
-    "\"\"\"\n",
-    "\n",
-    "PROMPT = PromptTemplate.from_template(\n",
-    "    _DEFAULT_TEMPLATE + PROMPT_SUFFIX,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "7f6115f4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT, verbose=True, memory=memory)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "b4753f69",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new SQLDatabaseChain chain...\u001b[0m\n",
-      "name one employee\n",
-      "SQLQuery:\u001b[32;1m\u001b[1;3mSELECT FirstName, LastName FROM Employee LIMIT 1\u001b[0m\n",
-      "SQLResult: \u001b[33;1m\u001b[1;3m[('Andrew', 'Adams')]\u001b[0m\n",
-      "Answer:\u001b[32;1m\u001b[1;3mAndrew Adams\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'Andrew Adams'"
-      ]
-     },
-     "execution_count": 6,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "db_chain.run(\"name one employee\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "aa1100c8",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new SQLDatabaseChain chain...\u001b[0m\n",
-      "how many letters in their name?\n",
-      "SQLQuery:\u001b[32;1m\u001b[1;3mSELECT LENGTH(FirstName) + LENGTH(LastName) AS 'NameLength' FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adams'\u001b[0m\n",
-      "SQLResult: \u001b[33;1m\u001b[1;3m[(11,)]\u001b[0m\n",
-      "Answer:\u001b[32;1m\u001b[1;3mAndrew Adams has 11 letters in their name.\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'Andrew Adams has 11 letters in their name.'"
-      ]
-     },
-     "execution_count": 7,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "db_chain.run(\"how many letters in their name?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "11525db8",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.1"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/snippets/get_started/quickstart/llm_chain.mdx

@@ -17,7 +17,7 @@ class CommaSeparatedListOutputParser(BaseOutputParser):
         return text.strip().split(", ")
 
 template = """You are a helpful assistant who generates comma separated lists.
-A user will pass in a category, and you should generated 5 objects in that category in a comma separated list.
+A user will pass in a category, and you should generate 5 objects in that category in a comma separated list.
 ONLY return a comma separated list, and nothing more."""
 system_message_prompt = SystemMessagePromptTemplate.from_template(template)
 human_template = "{text}"

docs/snippets/modules/chains/popular/summarize.mdx

@@ -1,384 +0,0 @@
-## Prepare Data
-First we prepare the data. For this example we create multiple documents from one long one, but these documents could be fetched in any manner (the point of this notebook to highlight what to do AFTER you fetch the documents).
-
-```python
-from langchain import OpenAI, PromptTemplate, LLMChain
-from langchain.text_splitter import CharacterTextSplitter
-from langchain.chains.mapreduce import MapReduceChain
-from langchain.prompts import PromptTemplate
-
-llm = OpenAI(temperature=0)
-
-text_splitter = CharacterTextSplitter()
-```
-
-
-```python
-with open("../../state_of_the_union.txt") as f:
-    state_of_the_union = f.read()
-texts = text_splitter.split_text(state_of_the_union)
-```
-
-
-```python
-from langchain.docstore.document import Document
-
-docs = [Document(page_content=t) for t in texts[:3]]
-```
-
-## Quickstart
-If you just want to get started as quickly as possible, this is the recommended way to do it:
-
-
-```python
-from langchain.chains.summarize import load_summarize_chain
-```
-
-
-```python
-chain = load_summarize_chain(llm, chain_type="map_reduce")
-chain.run(docs)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    ' In response to Russian aggression in Ukraine, the United States and its allies are taking action to hold Putin accountable, including economic sanctions, asset seizures, and military assistance. The US is also providing economic and humanitarian aid to Ukraine, and has passed the American Rescue Plan and the Bipartisan Infrastructure Law to help struggling families and create jobs. The US remains unified and determined to protect Ukraine and the free world.'
-```
-
-</CodeOutputBlock>
-
-If you want more control and understanding over what is happening, please see the information below.
-
-## The `stuff` Chain
-
-This sections shows results of using the `stuff` Chain to do summarization.
-
-
-```python
-chain = load_summarize_chain(llm, chain_type="stuff")
-```
-
-
-```python
-chain.run(docs)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    ' In his speech, President Biden addressed the crisis in Ukraine, the American Rescue Plan, and the Bipartisan Infrastructure Law. He discussed the need to invest in America, educate Americans, and build the economy from the bottom up. He also announced the release of 60 million barrels of oil from reserves around the world, and the creation of a dedicated task force to go after the crimes of Russian oligarchs. He concluded by emphasizing the need to Buy American and use taxpayer dollars to rebuild America.'
-```
-
-</CodeOutputBlock>
-
-**Custom Prompts**
-
-You can also use your own prompts with this chain. In this example, we will respond in Italian.
-
-
-```python
-prompt_template = """Write a concise summary of the following:
-
-
-{text}
-
-
-CONCISE SUMMARY IN ITALIAN:"""
-PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])
-chain = load_summarize_chain(llm, chain_type="stuff", prompt=PROMPT)
-chain.run(docs)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    "\n\nIn questa serata, il Presidente degli Stati Uniti ha annunciato una serie di misure per affrontare la crisi in Ucraina, causata dall'aggressione di Putin. Ha anche annunciato l'invio di aiuti economici, militari e umanitari all'Ucraina. Ha anche annunciato che gli Stati Uniti e i loro alleati stanno imponendo sanzioni economiche a Putin e stanno rilasciando 60 milioni di barili di petrolio dalle riserve di tutto il mondo. Inoltre, ha annunciato che il Dipartimento di Giustizia degli Stati Uniti sta creando una task force dedicata ai crimini degli oligarchi russi. Il Presidente ha anche annunciato l'approvazione della legge bipartitica sull'infrastruttura, che prevede investimenti per la ricostruzione dell'America. Questo porterà a creare posti"
-```
-
-</CodeOutputBlock>
-
-## The `map_reduce` Chain
-
-This sections shows results of using the `map_reduce` Chain to do summarization.
-
-
-```python
-chain = load_summarize_chain(llm, chain_type="map_reduce")
-```
-
-
-```python
-chain.run(docs)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    " In response to Russia's aggression in Ukraine, the United States and its allies have imposed economic sanctions and are taking other measures to hold Putin accountable. The US is also providing economic and military assistance to Ukraine, protecting NATO countries, and releasing oil from its Strategic Petroleum Reserve. President Biden and Vice President Harris have passed legislation to help struggling families and rebuild America's infrastructure."
-```
-
-</CodeOutputBlock>
-
-**Intermediate Steps**
-
-We can also return the intermediate steps for `map_reduce` chains, should we want to inspect them. This is done with the `return_map_steps` variable.
-
-
-```python
-chain = load_summarize_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True)
-```
-
-
-```python
-chain({"input_documents": docs}, return_only_outputs=True)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    {'map_steps': [" In response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains.",
-      ' The United States and its European allies are taking action to punish Russia for its invasion of Ukraine, including seizing assets, closing off airspace, and providing economic and military assistance to Ukraine. The US is also mobilizing forces to protect NATO countries and has released 30 million barrels of oil from its Strategic Petroleum Reserve to help blunt gas prices. The world is uniting in support of Ukraine and democracy, and the US stands with its Ukrainian-American citizens.',
-      " President Biden and Vice President Harris ran for office with a new economic vision for America, and have since passed the American Rescue Plan and the Bipartisan Infrastructure Law to help struggling families and rebuild America's infrastructure. This includes creating jobs, modernizing roads, airports, ports, and waterways, replacing lead pipes, providing affordable high-speed internet, and investing in American products to support American jobs."],
-     'output_text': " In response to Russia's aggression in Ukraine, the United States and its allies have imposed economic sanctions and are taking other measures to hold Putin accountable. The US is also providing economic and military assistance to Ukraine, protecting NATO countries, and passing legislation to help struggling families and rebuild America's infrastructure. The world is uniting in support of Ukraine and democracy, and the US stands with its Ukrainian-American citizens."}
-```
-
-</CodeOutputBlock>
-
-**Custom Prompts**
-
-You can also use your own prompts with this chain. In this example, we will respond in Italian.
-
-
-```python
-prompt_template = """Write a concise summary of the following:
-
-
-{text}
-
-
-CONCISE SUMMARY IN ITALIAN:"""
-PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])
-chain = load_summarize_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True, map_prompt=PROMPT, combine_prompt=PROMPT)
-chain({"input_documents": docs}, return_only_outputs=True)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    {'intermediate_steps': ["\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Gli Stati Uniti e i loro alleati stanno ora imponendo sanzioni economiche a Putin e stanno tagliando l'accesso della Russia alla tecnologia. Il Dipartimento di Giustizia degli Stati Uniti sta anche creando una task force dedicata per andare dopo i crimini degli oligarchi russi.",
-      "\n\nStiamo unendo le nostre forze con quelle dei nostri alleati europei per sequestrare yacht, appartamenti di lusso e jet privati di Putin. Abbiamo chiuso lo spazio aereo americano ai voli russi e stiamo fornendo più di un miliardo di dollari in assistenza all'Ucraina. Abbiamo anche mobilitato le nostre forze terrestri, aeree e navali per proteggere i paesi della NATO. Abbiamo anche rilasciato 60 milioni di barili di petrolio dalle riserve di tutto il mondo, di cui 30 milioni dalla nostra riserva strategica di petrolio. Stiamo affrontando una prova reale e ci vorrà del tempo, ma alla fine Putin non riuscirà a spegnere l'amore dei popoli per la libertà.",
-      "\n\nIl Presidente Biden ha lottato per passare l'American Rescue Plan per aiutare le persone che soffrivano a causa della pandemia. Il piano ha fornito sollievo economico immediato a milioni di americani, ha aiutato a mettere cibo sulla loro tavola, a mantenere un tetto sopra le loro teste e a ridurre il costo dell'assicurazione sanitaria. Il piano ha anche creato più di 6,5 milioni di nuovi posti di lavoro, il più alto numero di posti di lavoro creati in un anno nella storia degli Stati Uniti. Il Presidente Biden ha anche firmato la legge bipartitica sull'infrastruttura, la più ampia iniziativa di ricostruzione della storia degli Stati Uniti. Il piano prevede di modernizzare le strade, gli aeroporti, i porti e le vie navigabili in"],
-     'output_text': "\n\nIl Presidente Biden sta lavorando per aiutare le persone che soffrono a causa della pandemia attraverso l'American Rescue Plan e la legge bipartitica sull'infrastruttura. Gli Stati Uniti e i loro alleati stanno anche imponendo sanzioni economiche a Putin e tagliando l'accesso della Russia alla tecnologia. Stanno anche sequestrando yacht, appartamenti di lusso e jet privati di Putin e fornendo più di un miliardo di dollari in assistenza all'Ucraina. Alla fine, Putin non riuscirà a spegnere l'amore dei popoli per la libertà."}
-```
-
-</CodeOutputBlock>
-
-## The custom `MapReduceChain`
-
-**Multi input prompt**
-
-You can also use prompt with multi input. In this example, we will use a MapReduce chain to answer specific question about our code.
-
-
-```python
-from langchain.chains.combine_documents.map_reduce import MapReduceDocumentsChain
-from langchain.chains.combine_documents.stuff import StuffDocumentsChain
-
-map_template_string = """Give the following python code information, generate a description that explains what the code does and also mention the time complexity.
-Code:
-{code}
-
-Return the the description in the following format:
-name of the function: description of the function
-"""
-
-
-reduce_template_string = """Given the following python function names and descriptions, answer the following question
-{code_description}
-Question: {question}
-Answer:
-"""
-
-# Prompt to use in map and reduce stages 
-MAP_PROMPT = PromptTemplate(input_variables=["code"], template=map_template_string)
-REDUCE_PROMPT = PromptTemplate(input_variables=["code_description", "question"], template=reduce_template_string)
-
-# LLM to use in map and reduce stages 
-llm = OpenAI()
-map_llm_chain = LLMChain(llm=llm, prompt=MAP_PROMPT)
-reduce_llm_chain = LLMChain(llm=llm, prompt=REDUCE_PROMPT)
-
-# Takes a list of documents and combines them into a single string
-combine_documents_chain = StuffDocumentsChain(
-    llm_chain=reduce_llm_chain,
-    document_variable_name="code_description",
-)
-
-# Combines and iteravely reduces the mapped documents 
-reduce_documents_chain = ReduceDocumentsChain(
-        # This is final chain that is called.
-        combine_documents_chain=combine_documents_chain,
-        # If documents exceed context for `combine_documents_chain`
-        collapse_documents_chain=combine_documents_chain,
-        # The maximum number of tokens to group documents into
-        token_max=3000)
-
-# Combining documents by mapping a chain over them, then combining results with reduce chain
-combine_documents = MapReduceDocumentsChain(
-    # Map chain
-    llm_chain=map_llm_chain,
-     # Reduce chain
-    reduce_documents_chain=reduce_documents_chain,
-    # The variable name in the llm_chain to put the documents in
-    document_variable_name="code",
-)
-
-map_reduce = MapReduceChain(
-    combine_documents_chain=combine_documents,
-    text_splitter=CharacterTextSplitter(separator="\n##\n", chunk_size=100, chunk_overlap=0),
-)
-```
-
-
-```python
-code = """
-def bubblesort(list):
-   for iter_num in range(len(list)-1,0,-1):
-      for idx in range(iter_num):
-         if list[idx]>list[idx+1]:
-            temp = list[idx]
-            list[idx] = list[idx+1]
-            list[idx+1] = temp
-    return list
-##
-def insertion_sort(InputList):
-   for i in range(1, len(InputList)):
-      j = i-1
-      nxt_element = InputList[i]
-   while (InputList[j] > nxt_element) and (j >= 0):
-      InputList[j+1] = InputList[j]
-      j=j-1
-   InputList[j+1] = nxt_element
-   return InputList
-##
-def shellSort(input_list):
-   gap = len(input_list) // 2
-   while gap > 0:
-      for i in range(gap, len(input_list)):
-         temp = input_list[i]
-         j = i
-   while j >= gap and input_list[j - gap] > temp:
-      input_list[j] = input_list[j - gap]
-      j = j-gap
-      input_list[j] = temp
-   gap = gap//2
-   return input_list
-
-"""
-```
-
-
-```python
-map_reduce.run(input_text=code, question="Which function has a better time complexity?")
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    Created a chunk of size 247, which is longer than the specified 100
-    Created a chunk of size 267, which is longer than the specified 100
-
-
-
-
-
-    'shellSort has a better time complexity than both bubblesort and insertion_sort, as it has a time complexity of O(n^2), while the other two have a time complexity of O(n^2).'
-```
-
-</CodeOutputBlock>
-
-## The `refine` Chain
-
-This sections shows results of using the `refine` Chain to do summarization.
-
-
-```python
-chain = load_summarize_chain(llm, chain_type="refine")
-
-chain.run(docs)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. In addition, the U.S. has passed the American Rescue Plan to provide immediate economic relief for tens of millions of Americans, and the Bipartisan Infrastructure Law to rebuild America and create jobs. This investment will"
-```
-
-</CodeOutputBlock>
-
-**Intermediate Steps**
-
-We can also return the intermediate steps for `refine` chains, should we want to inspect them. This is done with the `return_refine_steps` variable.
-
-
-```python
-chain = load_summarize_chain(OpenAI(temperature=0), chain_type="refine", return_intermediate_steps=True)
-
-chain({"input_documents": docs}, return_only_outputs=True)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    {'refine_steps': [" In response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains.",
-      "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. Putin's war on Ukraine has left Russia weaker and the rest of the world stronger, with the world uniting in support of democracy and peace.",
-      "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. In addition, the U.S. has passed the American Rescue Plan to provide immediate economic relief for tens of millions of Americans, and the Bipartisan Infrastructure Law to rebuild America and create jobs. This includes investing"],
-     'output_text': "\n\nIn response to Russia's aggression in Ukraine, the United States has united with other freedom-loving nations to impose economic sanctions and hold Putin accountable. The U.S. Department of Justice is also assembling a task force to go after the crimes of Russian oligarchs and seize their ill-gotten gains. We are joining with our European allies to find and seize the assets of Russian oligarchs, including yachts, luxury apartments, and private jets. The U.S. is also closing off American airspace to all Russian flights, further isolating Russia and adding an additional squeeze on their economy. The U.S. and its allies are providing support to the Ukrainians in their fight for freedom, including military, economic, and humanitarian assistance. The U.S. is also mobilizing ground forces, air squadrons, and ship deployments to protect NATO countries. The U.S. and its allies are also releasing 60 million barrels of oil from reserves around the world, with the U.S. contributing 30 million barrels from its own Strategic Petroleum Reserve. In addition, the U.S. has passed the American Rescue Plan to provide immediate economic relief for tens of millions of Americans, and the Bipartisan Infrastructure Law to rebuild America and create jobs. This includes investing"}
-```
-
-</CodeOutputBlock>
-
-**Custom Prompts**
-
-You can also use your own prompts with this chain. In this example, we will respond in Italian.
-
-
-```python
-prompt_template = """Write a concise summary of the following:
-
-
-{text}
-
-
-CONCISE SUMMARY IN ITALIAN:"""
-PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])
-refine_template = (
-    "Your job is to produce a final summary\n"
-    "We have provided an existing summary up to a certain point: {existing_answer}\n"
-    "We have the opportunity to refine the existing summary"
-    "(only if needed) with some more context below.\n"
-    "------------\n"
-    "{text}\n"
-    "------------\n"
-    "Given the new context, refine the original summary in Italian"
-    "If the context isn't useful, return the original summary."
-)
-refine_prompt = PromptTemplate(
-    input_variables=["existing_answer", "text"],
-    template=refine_template,
-)
-chain = load_summarize_chain(OpenAI(temperature=0), chain_type="refine", return_intermediate_steps=True, question_prompt=PROMPT, refine_prompt=refine_prompt)
-chain({"input_documents": docs}, return_only_outputs=True)
-```
-
-<CodeOutputBlock lang="python">
-
-```
-    {'intermediate_steps': ["\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia e bloccando i suoi più grandi istituti bancari dal sistema finanziario internazionale. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi.",
-      "\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia, bloccando i suoi più grandi istituti bancari dal sistema finanziario internazionale e chiudendo lo spazio aereo americano a tutti i voli russi. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi. Stiamo fornendo più di un miliardo di dollari in assistenza diretta all'Ucraina e fornendo assistenza militare,",
-      "\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia, bloccando i suoi più grandi istituti bancari dal sistema finanziario internazionale e chiudendo lo spazio aereo americano a tutti i voli russi. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi. Stiamo fornendo più di un miliardo di dollari in assistenza diretta all'Ucraina e fornendo assistenza militare."],
-     'output_text': "\n\nQuesta sera, ci incontriamo come democratici, repubblicani e indipendenti, ma soprattutto come americani. La Russia di Putin ha cercato di scuotere le fondamenta del mondo libero, ma ha sottovalutato la forza della gente ucraina. Insieme ai nostri alleati, stiamo imponendo sanzioni economiche, tagliando l'accesso della Russia alla tecnologia, bloccando i suoi più grandi istituti bancari dal sistema finanziario internazionale e chiudendo lo spazio aereo americano a tutti i voli russi. Il Dipartimento di Giustizia degli Stati Uniti sta anche assemblando una task force dedicata per andare dopo i crimini degli oligarchi russi. Stiamo fornendo più di un miliardo di dollari in assistenza diretta all'Ucraina e fornendo assistenza militare."}
-```
-
-</CodeOutputBlock>

docs/snippets/modules/chains/popular/vector_db_qa.mdx

@@ -117,3 +117,38 @@ qa.run(query)
 ```
 
 </CodeOutputBlock>
+
+## Vectorstore Retriever Options
+You can adjust how documents are retrieved from your vectorstore depending on the specific task.
+
+There are two main ways to retrieve documents relevant to a query- Similarity Search and Max Marginal Relevance Search (MMR Search). Similarity Search is the default, but you can use MMR by adding the `search_type` parameter:
+
+```python
+docsearch.as_retriever(search_type="mmr")
+```
+
+You can also modify the search by passing specific search arguments through the retriever to the search function, using the `search_kwargs` keyword argument.
+
+- `k` defines how many documents are returned; defaults to 4.
+- `score_threshold` allows you to set a minimum relevance for documents returned by the retriever, if you are using the "similarity_score_threshold" search type.
+- `fetch_k` determines the amount of documents to pass to the MMR algorithm; defaults to 20. 
+- `lambda_mult` controls the diversity of results returned by the MMR algorithm, with 1 being minimum diversity and 0 being maximum. Defaults to 0.5.
+- `filter` allows you to define a filter on what documents should be retrieved, based on the documents' metadata. This has no effect if the Vectorstore doesn't store any metadata.
+
+Some examples for how these parameters can be used:
+```python
+# Retrieve more documents with higher diversity- useful if your dataset has many similar documents
+docsearch.as_retriever(search_type="mmr", search_kwargs={'k': 6, 'lambda_mult': 0.25})
+
+# Fetch more documents for the MMR algorithm to consider, but only return the top 5
+docsearch.as_retriever(search_type="mmr", search_kwargs={'k': 5, 'fetch_k': 50})
+
+# Only retrieve documents that have a relevance score above a certain threshold
+docsearch.as_retriever(search_type="similarity_score_threshold", search_kwargs={'score_threshold': 0.8})
+
+# Only get the single most similar document from the dataset
+docsearch.as_retriever(search_kwargs={'k': 1})
+
+# Use a filter to only retrieve documents from a specific paper 
+docsearch.as_retriever(search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}})
+```

docs/snippets/modules/chains/popular/vector_db_qa_with_sources.mdx

@@ -3,7 +3,7 @@ Additionally, we can return the source documents used to answer the question by
 
 
 ```python
-qa = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff", retriever=docsearch.as_retriever(), return_source_documents=True)
+qa = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff", retriever=docsearch.as_retriever(search_type="mmr", search_kwargs={'fetch_k': 30}), return_source_documents=True)
 ```
 
 

docs/snippets/modules/data_connection/document_transformers/text_splitters/character_text_splitter.mdx

@@ -12,6 +12,7 @@ text_splitter = CharacterTextSplitter(
     chunk_size = 1000,
     chunk_overlap  = 200,
     length_function = len,
+    is_separator_regex = False,
 )
 ```
 

docs/snippets/modules/data_connection/document_transformers/text_splitters/recursive_text_splitter.mdx

@@ -16,6 +16,7 @@ text_splitter = RecursiveCharacterTextSplitter(
     chunk_size = 100,
     chunk_overlap  = 20,
     length_function = len,
+    is_separator_regex = False,
 )
 ```
 

docs/snippets/modules/data_connection/retrievers/self_query/get_started.mdx

@@ -3,7 +3,7 @@ We'll use a Pinecone vector store in this example.
 
 First we'll want to create a `Pinecone` VectorStore and seed it with some data. We've created a small demo set of documents that contain summaries of movies.
 
-To use Pinecone, you to have `pinecone` package installed and you must have an API key and an Environment. Here are the [installation instructions](https://docs.pinecone.io/docs/quickstart).
+To use Pinecone, you need to have `pinecone` package installed and you must have an API key and an Environment. Here are the [installation instructions](https://docs.pinecone.io/docs/quickstart).
 
 NOTE: The self-query retriever requires you to have `lark` package installed.
 
@@ -85,7 +85,7 @@ retriever = SelfQueryRetriever.from_llm(llm, vectorstore, document_content_descr
 ```
 
 ## Testing it out
-And now we can try actually using our retriever!
+And now we can actually try using our retriever!
 
 
 ```python
@@ -198,4 +198,4 @@ retriever = SelfQueryRetriever.from_llm(
 ```python
 # This example only specifies a relevant query
 retriever.get_relevant_documents("What are two movies about dinosaurs")
-```
+```

docs/snippets/modules/memory/types/summary.mdx

@@ -60,7 +60,7 @@ memory.predict_new_summary(messages, previous_summary)
 
 </CodeOutputBlock>
 
-## Initializing with messages
+## Initializing with messages/existing summary
 
 If you have messages outside this class, you can easily initialize the class with ChatMessageHistory. During loading, a summary will be calculated.
 
@@ -73,7 +73,11 @@ history.add_ai_message("hi there!")
 
 
 ```python
-memory = ConversationSummaryMemory.from_messages(llm=OpenAI(temperature=0), chat_memory=history, return_messages=True)
+memory = ConversationSummaryMemory.from_messages(
+    llm=OpenAI(temperature=0),
+    chat_memory=history,
+    return_messages=True
+)
 ```
 
 
@@ -89,6 +93,17 @@ memory.buffer
 
 </CodeOutputBlock>
 
+Optionally you can speed up initialization using a previously generated summary, and avoid regenerating the summary by just initializing directly.
+
+```python
+memory = ConversationSummaryMemory(
+    llm=OpenAI(temperature=0),
+    buffer="The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good because it will help humans reach their full potential.",
+    chat_memory=history,
+    return_messages=True
+)
+```
+
 ## Using in a chain
 Let's walk through an example of using this in a chain, again setting `verbose=True` so we can see the prompt.
 

docs/snippets/modules/model_io/prompts/prompt_templates/get_started.mdx

@@ -1,140 +1,115 @@
-Here's the simplest example:
+Typically, language models expect the prompt to either be a string or else a list of chat messages.
+
+## Prompt template
+
+Use `PromptTemplate` to create a template for a string prompt.
+
+By default, `PromptTemplate` uses [Python's str.format](https://docs.python.org/3/library/stdtypes.html#str.format)
+syntax for templating; however other templating syntax is available (e.g., `jinja2`).
 
 ```python
 from langchain import PromptTemplate
 
-
-template = """\
-You are a naming consultant for new companies.
-What is a good name for a company that makes {product}?
-"""
-
-prompt = PromptTemplate.from_template(template)
-prompt.format(product="colorful socks")
+prompt_template = PromptTemplate.from_template(
+    "Tell me a {adjective} joke about {content}."
+)
+prompt_template.format(adjective="funny", content="chickens")
 ```
 
 <CodeOutputBlock lang="python">
 
 ```
-You are a naming consultant for new companies.
-What is a good name for a company that makes colorful socks?
+"Tell me a funny joke about chickens."
 ```
 
 </CodeOutputBlock>
 
-
-## Create a prompt template
-
-You can create simple hardcoded prompts using the `PromptTemplate` class. Prompt templates can take any number of input variables, and can be formatted to generate a prompt.
-
+The template supports any number of variables, including no variables:
 
 ```python
 from langchain import PromptTemplate
 
-# An example prompt with no input variables
-no_input_prompt = PromptTemplate(input_variables=[], template="Tell me a joke.")
-no_input_prompt.format()
-# -> "Tell me a joke."
-
-# An example prompt with one input variable
-one_input_prompt = PromptTemplate(input_variables=["adjective"], template="Tell me a {adjective} joke.")
-one_input_prompt.format(adjective="funny")
-# -> "Tell me a funny joke."
-
-# An example prompt with multiple input variables
-multiple_input_prompt = PromptTemplate(
-    input_variables=["adjective", "content"], 
-    template="Tell me a {adjective} joke about {content}."
+prompt_template = PromptTemplate.from_template(
+"Tell me a joke"
 )
-multiple_input_prompt.format(adjective="funny", content="chickens")
-# -> "Tell me a funny joke about chickens."
+prompt_template.format()
 ```
 
-If you do not wish to specify `input_variables` manually, you can also create a `PromptTemplate` using `from_template` class method. `langchain` will automatically infer the `input_variables` based on the `template` passed.
+For additional validation, specify `input_variables` explicitly. These variables
+will be compared against the variables present in the template string during instantiation, raising an exception if
+there is a mismatch; for example,
 
 ```python
-template = "Tell me a {adjective} joke about {content}."
+from langchain import PromptTemplate
 
-prompt_template = PromptTemplate.from_template(template)
-prompt_template.input_variables
-# -> ['adjective', 'content']
-prompt_template.format(adjective="funny", content="chickens")
-# -> Tell me a funny joke about chickens.
+invalid_prompt = PromptTemplate(
+    input_variables=["adjective"],
+    template="Tell me a {adjective} joke about {content}."
+)
 ```
 
-You can create custom prompt templates that format the prompt in any way you want. For more information, see [Custom Prompt Templates](./custom_prompt_template.html).
-
+You can create custom prompt templates that format the prompt in any way you want.
+For more information, see [Custom Prompt Templates](./custom_prompt_template.html).
 
 <!-- TODO(shreya): Add link to Jinja -->
 
 ## Chat prompt template
 
-[Chat Models](../models/chat) take a list of chat messages as input - this list commonly referred to as a `prompt`.
-These chat messages differ from raw string (which you would pass into a [LLM](/docs/modules/model_io/models/llms) model) in that every message is associated with a `role`.
-
-For example, in OpenAI [Chat Completion API](https://platform.openai.com/docs/guides/chat/introduction), a chat message can be associated with the AI, human or system role. The model is supposed to follow instruction from system chat message more closely.
-
-LangChain provides several prompt templates to make constructing and working with prompts easily. You are encouraged to use these chat related prompt templates instead of `PromptTemplate` when querying chat models to fully exploit the potential of underlying chat model.
-
-
+The prompt to [Chat Models](../models/chat) is a list of chat messages.
 
+Each chat message is associated with content, and an additional parameter called `role`.
+For example, in the OpenAI [Chat Completions API](https://platform.openai.com/docs/guides/chat/introduction), a chat message can be associated with an AI assistant, a human or a system role.
 
+Create a chat prompt template like this:
 
 ```python
-from langchain.prompts import (
-    ChatPromptTemplate,
-    PromptTemplate,
-    SystemMessagePromptTemplate,
-    AIMessagePromptTemplate,
-    HumanMessagePromptTemplate,
-)
-from langchain.schema import (
-    AIMessage,
-    HumanMessage,
-    SystemMessage
+from langchain.prompts import ChatPromptTemplate
+
+template = ChatPromptTemplate.from_messages([
+    ("system", "You are a helpful AI bot. Your name is {name}."),
+    ("human", "Hello, how are you doing?"),
+    ("ai", "I'm doing well, thanks!"),
+    ("human", "{user_input}"),
+])
+
+messages = template.format_messages(
+    name="Bob",
+    user_input="What is your name?"
 )
 ```
 
-To create a message template associated with a role, you use `MessagePromptTemplate`.
-
-For convenience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:
+`ChatPromptTemplate.from_messages` accepts a variety of message representations.
 
+For example, in addition to using the 2-tuple representation of (type, content) used
+above, you could pass in an instance of `MessagePromptTemplate` or `BaseMessage`.
 
 ```python
-template="You are a helpful assistant that translates {input_language} to {output_language}."
-system_message_prompt = SystemMessagePromptTemplate.from_template(template)
-human_template="{text}"
-human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
-```
+from langchain.prompts import ChatPromptTemplate
+from langchain.prompts.chat import SystemMessage, HumanMessagePromptTemplate
 
-If you wanted to construct the `MessagePromptTemplate` more directly, you could create a PromptTemplate outside and then pass it in, eg:
-
-
-```python
-prompt=PromptTemplate(
-    template="You are a helpful assistant that translates {input_language} to {output_language}.",
-    input_variables=["input_language", "output_language"],
+template = ChatPromptTemplate.from_messages(
+    [
+        SystemMessage(
+            content=(
+                "You are a helpful assistant that re-writes the user's text to "
+                "sound more upbeat."
+            )
+        ),
+        HumanMessagePromptTemplate.from_template("{text}"),
+    ]
 )
-system_message_prompt_2 = SystemMessagePromptTemplate(prompt=prompt)
 
-assert system_message_prompt == system_message_prompt_2
-```
+from langchain.chat_models import ChatOpenAI
 
-After that, you can build a `ChatPromptTemplate` from one or more `MessagePromptTemplates`. You can use `ChatPromptTemplate`'s `format_prompt` -- this returns a `PromptValue`, which you can convert to a string or Message object, depending on whether you want to use the formatted value as input to an llm or chat model.
-
-
-```python
-chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])
-
-# get a chat completion from the formatted messages
-chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_messages()
+llm = ChatOpenAI()
+llm(template.format_messages(text='i dont like eating tasty things.'))
 ```
 
 <CodeOutputBlock lang="python">
-
 ```
-    [SystemMessage(content='You are a helpful assistant that translates English to French.', additional_kwargs={}),
-     HumanMessage(content='I love programming.', additional_kwargs={})]
+AIMessage(content='I absolutely adore indulging in delicious treats!', additional_kwargs={}, example=False)
 ```
-
 </CodeOutputBlock>
+
+This provides you with a lot of flexibility in how you construct your chat prompts.
+

libs/experimental/langchain_experimental/sql/base.py

@@ -112,9 +112,6 @@ class SQLDatabaseChain(Chain):
             "table_info": table_info,
             "stop": ["\nSQLResult:"],
         }
-        if self.memory is not None:
-            for k in self.memory.memory_variables:
-                llm_inputs[k] = inputs[k]
         intermediate_steps: List = []
         try:
             intermediate_steps.append(llm_inputs)  # input: sql generation

libs/langchain/Makefile

@@ -92,15 +92,24 @@ spell_fix:
 ######################
 
 help:
-	@echo '----'
-	@echo 'coverage                     - run unit tests and generate coverage report'
+	@echo '===================='
+	@echo '-- DOCUMENTATION --'
+	@echo 'clean                        - run docs_clean and api_docs_clean'
 	@echo 'docs_build                   - build the documentation'
 	@echo 'docs_clean                   - clean the documentation build artifacts'
 	@echo 'docs_linkcheck               - run linkchecker on the documentation'
+	@echo 'api_docs_build               - build the API Reference documentation'
+	@echo 'api_docs_clean               - clean the API Reference documentation build artifacts'
+	@echo 'api_docs_linkcheck           - run linkchecker on the API Reference documentation'
+	@echo '-- LINTING --'
 	@echo 'format                       - run code formatters'
 	@echo 'lint                         - run linters'
+	@echo 'spell_check               	- run codespell on the project'
+	@echo 'spell_fix               		- run codespell on the project and fix the errors'
+	@echo '-- TESTS --'
+	@echo 'coverage                     - run unit tests and generate coverage report'
 	@echo 'test                         - run unit tests'
-	@echo 'tests                        - run unit tests'
+	@echo 'tests                        - run unit tests (alias for "make test")'
 	@echo 'test TEST_FILE=<test_file>   - run all tests in file'
 	@echo 'extended_tests               - run only extended unit tests'
 	@echo 'test_watch                   - run unit tests in watch mode'

libs/langchain/README.md

@@ -18,8 +18,8 @@
 
 Looking for the JS/TS version? Check out [LangChain.js](https://github.com/hwchase17/langchainjs).
 
-**Production Support:** As you move your LangChains into production, we'd love to offer more comprehensive support.
-Please fill out [this form](https://6w1pwbss0py.typeform.com/to/rrbrdTH2) and we'll set up a dedicated support Slack channel.
+**Production Support:** As you move your LangChains into production, we'd love to offer more hands-on support.
+Fill out [this form](https://airtable.com/appwQzlErAS2qiP0L/shrGtGaVBVAz7NcV2) to share more about what you're building, and our team will get in touch.
 
 ## Quick Install
 

libs/langchain/langchain/agents/__init__.py

@@ -1,4 +1,33 @@
-"""Interface for agents."""
+"""
+**Agent** is a class that uses an LLM to choose a sequence of actions to take.
+
+In Chains, a sequence of actions is hardcoded. In Agents,
+a language model is used as a reasoning engine to determine which actions
+to take and in which order.
+
+Agents select and use **Tools** and **Toolkits** for actions.
+
+**Class hierarchy:**
+
+.. code-block::
+
+    BaseSingleActionAgent --> LLMSingleActionAgent
+                              OpenAIFunctionsAgent
+                              XMLAgent
+                              Agent --> <name>Agent  # Examples: ZeroShotAgent, ChatAgent
+                                        
+
+    BaseMultiActionAgent  --> OpenAIMultiFunctionsAgent
+    
+    
+**Main helpers:**
+
+.. code-block::
+
+    AgentType, AgentExecutor, AgentOutputParser, AgentExecutorIterator,
+    AgentAction, AgentFinish
+    
+"""  # noqa: E501
 from langchain.agents.agent import (
     Agent,
     AgentExecutor,

libs/langchain/langchain/agents/agent.py

@@ -897,7 +897,10 @@ s
             else:
                 tool_run_kwargs = self.agent.tool_run_logging_kwargs()
                 observation = InvalidTool().run(
-                    agent_action.tool,
+                    {
+                        "requested_tool_name": agent_action.tool,
+                        "available_tool_names": list(name_to_tool_map.keys()),
+                    },
                     verbose=self.verbose,
                     color=None,
                     callbacks=run_manager.get_child() if run_manager else None,
@@ -992,7 +995,10 @@ s
             else:
                 tool_run_kwargs = self.agent.tool_run_logging_kwargs()
                 observation = await InvalidTool().arun(
-                    agent_action.tool,
+                    {
+                        "requested_tool_name": agent_action.tool,
+                        "available_tool_names": list(name_to_tool_map.keys()),
+                    },
                     verbose=self.verbose,
                     color=None,
                     callbacks=run_manager.get_child() if run_manager else None,