cr

Added a use case with parallelise on batches. Simplified text.

.devcontainer/README.md

@@ -5,17 +5,21 @@ This project includes a [dev container](https://containers.dev/), which lets you
 You can use the dev container configuration in this folder to build and run the app without needing to install any of its tools locally! You can use it in [GitHub Codespaces](https://github.com/features/codespaces) or the [VS Code Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers).
 
 ## GitHub Codespaces
-[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hwchase17/langchain)
+[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/langchain-ai/langchain)
 
 You may use the button above, or follow these steps to open this repo in a Codespace:
-1. Click the **Code** drop-down menu at the top of https://github.com/hwchase17/langchain.
+1. Click the **Code** drop-down menu at the top of https://github.com/langchain-ai/langchain.
 1. Click on the **Codespaces** tab.
 1. Click **Create codespace on master** .
 
 For more info, check out the [GitHub documentation](https://docs.github.com/en/free-pro-team@latest/github/developing-online-with-codespaces/creating-a-codespace#creating-a-codespace).
   
 ## VS Code Dev Containers
-[![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/hwchase17/langchain)
+[![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/langchain-ai/langchain)
+
+Note: If you click this link you will open the main repo and not your local cloned repo, you can use this link and replace with your username and cloned repo name: 
+https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/<yourusername>/<yourclonedreponame>
+
 
 If you already have VS Code and Docker installed, you can use the button above to get started. This will cause VS Code to automatically install the Dev Containers extension if needed, clone the source code into a container volume, and spin up a dev container for use.
 
@@ -25,7 +29,7 @@ You can also follow these steps to open this repo in a container using the VS Co
 
 2. Open a locally cloned copy of the code:
 
-   - Clone this repository to your local filesystem.
+   - Fork and Clone this repository to your local filesystem.
    - Press <kbd>F1</kbd> and select the **Dev Containers: Open Folder in Container...** command.
    - Select the cloned copy of this folder, wait for the container to start, and try things out!
 

.github/CONTRIBUTING.md

@@ -9,19 +9,19 @@ to contributions, whether they be in the form of new features, improved infra, b
 ### 👩‍💻 Contributing Code
 
 To contribute to this project, please follow a ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow.
-Please do not try to push directly to this repo unless you are maintainer.
+Please do not try to push directly to this repo unless you are a maintainer.
 
 Please follow the checked-in pull request template when opening pull requests. Note related issues and tag relevant
 maintainers.
 
-Pull requests cannot land without passing the formatting, linting and testing checks first. See
-[Common Tasks](#-common-tasks) for how to run these checks locally.
+Pull requests cannot land without passing the formatting, linting and testing checks first. See [Testing](#testing) and
+[Formatting and Linting](#formatting-and-linting) for how to run these checks locally.
 
 It's essential that we maintain great documentation and testing. If you:
 - Fix a bug
   - Add a relevant unit or integration test when possible. These live in `tests/unit_tests` and `tests/integration_tests`.
 - Make an improvement
-  - Update any affected example notebooks and documentation. These lives in `docs`.
+  - Update any affected example notebooks and documentation. These live in `docs`.
   - Update unit and integration tests when relevant.
 - Add a feature
   - Add a demo notebook in `docs/modules`.
@@ -32,8 +32,8 @@ best way to get our attention.
 
 ### 🚩GitHub Issues
 
-Our [issues](https://github.com/hwchase17/langchain/issues) page is kept up to date
-with bugs, improvements, and feature requests. 
+Our [issues](https://github.com/langchain-ai/langchain/issues) page is kept up to date
+with bugs, improvements, and feature requests.
 
 There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help
 organize issues.
@@ -43,8 +43,8 @@ If you start working on an issue, please assign it to yourself.
 If you are adding an issue, please try to keep it focused on a single, modular bug/improvement/feature.
 If two issues are related, or blocking, please link them rather than combining them.
 
-We will try to keep these issues as up to date as possible, though
-with the rapid rate of develop in this field some may get out of date.
+We will try to keep these issues as up-to-date as possible, though
+with the rapid rate of development in this field some may get out of date.
 If you notice this happening, please let us know.
 
 ### 🙋Getting Help
@@ -59,43 +59,85 @@ we do not want these to get in the way of getting good code into the codebase.
 
 ## 🚀 Quick Start
 
-> **Note:** You can run this repository locally (which is described below) or in a [development container](https://containers.dev/) (which is described in the [.devcontainer folder](https://github.com/hwchase17/langchain/tree/master/.devcontainer)).
+This quick start describes running the repository locally.
+For a [development container](https://containers.dev/), see the [.devcontainer folder](https://github.com/langchain-ai/langchain/tree/master/.devcontainer).
 
-This project uses [Poetry](https://python-poetry.org/) as a dependency manager. Check out Poetry's [documentation on how to install it](https://python-poetry.org/docs/#installation) on your system before proceeding.
+### Dependency Management: Poetry and other env/dependency managers
 
-❗Note: If you use `Conda` or `Pyenv` as your environment / package manager, avoid dependency conflicts by doing the following first:
-1. *Before installing Poetry*, create and activate a new Conda env (e.g. `conda create -n langchain python=3.9`)
-2. Install Poetry (see above)
-3. Tell Poetry to use the virtualenv python environment (`poetry config virtualenvs.prefer-active-python true`)
-4. Continue with the following steps.
+This project uses [Poetry](https://python-poetry.org/) v1.6.1+ as a dependency manager.
+
+❗Note: *Before installing Poetry*, if you use `Conda`, create and activate a new Conda env (e.g. `conda create -n langchain python=3.9`)
+
+Install Poetry: **[documentation on how to install it](https://python-poetry.org/docs/#installation)**.
+
+❗Note: If you use `Conda` or `Pyenv` as your environment/package manager, after installing Poetry,
+tell Poetry to use the virtualenv python environment (`poetry config virtualenvs.prefer-active-python true`)
+
+### Core vs. Experimental
 
 There are two separate projects in this repository:
 - `langchain`: core langchain code, abstractions, and use cases
-- `langchain.experimental`: more experimental code
+- `langchain.experimental`: see the [Experimental README](../libs/experimental/README.md) for more information.
 
-Each of these has their OWN development environment. 
-In order to run any of the commands below, please move into their respective directories.
-For example, to contribute to `langchain` run `cd libs/langchain` before getting started with the below.
+Each of these has their own development environment. Docs are run from the top-level makefile, but development
+is split across separate test & release flows.
 
-To install requirements:
+For this quickstart, start with langchain core:
 
 ```bash
-poetry install -E all
+cd libs/langchain
 ```
 
-This will install all requirements for running the package, examples, linting, formatting, tests, and coverage. Note the `-E all` flag will install all optional dependencies necessary for integration testing.
+### Local Development Dependencies
 
-❗Note: If you're running Poetry 1.4.1 and receive a `WheelFileValidationError` for `debugpy` during installation, you can try either downgrading to Poetry 1.4.0 or disabling "modern installation" (`poetry config installer.modern-installation false`) and re-install requirements. See [this `debugpy` issue](https://github.com/microsoft/debugpy/issues/1246) for more details.
+Install langchain development requirements (for running langchain, running examples, linting, formatting, tests, and coverage):
 
-Now, you should be able to run the common tasks in the following section. To double check, run `make test`, all tests should pass. If they don't you may need to pip install additional dependencies, such as `numexpr` and `openapi_schema_pydantic`.
+```bash
+poetry install --with test
+```
 
-## ✅ Common Tasks
+Then verify dependency installation:
 
-Type `make` for a list of common tasks.
+```bash
+make test
+```
 
-### Code Formatting
+If the tests don't pass, you may need to pip install additional dependencies, such as `numexpr` and `openapi_schema_pydantic`.
 
-Formatting for this project is done via a combination of [Black](https://black.readthedocs.io/en/stable/) and [isort](https://pycqa.github.io/isort/).
+If during installation you receive a `WheelFileValidationError` for `debugpy`, please make sure you are running
+Poetry v1.6.1+. This bug was present in older versions of Poetry (e.g. 1.4.1) and has been resolved in newer releases.
+If you are still seeing this bug on v1.6.1, you may also try disabling "modern installation"
+(`poetry config installer.modern-installation false`) and re-installing requirements.
+See [this `debugpy` issue](https://github.com/microsoft/debugpy/issues/1246) for more details.
+
+### Testing
+
+_some test dependencies are optional; see section about optional dependencies_.
+
+Unit tests cover modular logic that does not require calls to outside APIs.
+If you add new logic, please add a unit test.
+
+To run unit tests:
+
+```bash
+make test
+```
+
+To run unit tests in Docker:
+
+```bash
+make docker_tests
+```
+
+There are also [integration tests and code-coverage](../libs/langchain/tests/README.md) available.
+
+### Formatting and Linting
+
+Run these locally before submitting a PR; the CI system will check also.
+
+#### Code Formatting
+
+Formatting for this project is done via a combination of [Black](https://black.readthedocs.io/en/stable/) and [ruff](https://docs.astral.sh/ruff/rules/).
 
 To run formatting for this project:
 
@@ -111,9 +153,9 @@ make format_diff
 
 This is especially useful when you have made changes to a subset of the project and want to ensure your changes are properly formatted without affecting the rest of the codebase.
 
-### Linting
+#### Linting
 
-Linting for this project is done via a combination of [Black](https://black.readthedocs.io/en/stable/), [isort](https://pycqa.github.io/isort/), [flake8](https://flake8.pycqa.org/en/latest/), and [mypy](http://mypy-lang.org/).
+Linting for this project is done via a combination of [Black](https://black.readthedocs.io/en/stable/), [ruff](https://docs.astral.sh/ruff/rules/), and [mypy](http://mypy-lang.org/).
 
 To run linting for this project:
 
@@ -131,10 +173,10 @@ This can be very helpful when you've made changes to only certain parts of the p
 
 We recognize linting can be annoying - if you do not want to do it, please contact a project maintainer, and they can help you with it. We do not want this to be a blocker for good code getting contributed.
 
-### Spellcheck
+#### Spellcheck
 
 Spellchecking for this project is done via [codespell](https://github.com/codespell-project/codespell).
-Note that `codespell` finds common typos, so could have false-positive (correctly spelled but rarely used) and false-negatives (not finding misspelled) words.
+Note that `codespell` finds common typos, so it could have false-positive (correctly spelled but rarely used) and false-negatives (not finding misspelled) words.
 
 To check spelling for this project:
 
@@ -157,27 +199,17 @@ If codespell is incorrectly flagging a word, you can skip spellcheck for that wo
 ignore-words-list = 'momento,collison,ned,foor,reworkd,parth,whats,aapply,mysogyny,unsecure'
 ```
 
-### Coverage
-
-Code coverage (i.e. the amount of code that is covered by unit tests) helps identify areas of the code that are potentially more or less brittle.
-
-To get a report of current coverage, run the following:
-
-```bash
-make coverage
-```
-
-### Working with Optional Dependencies
+## Working with Optional Dependencies
 
 Langchain relies heavily on optional dependencies to keep the Langchain package lightweight.
 
 If you're adding a new dependency to Langchain, assume that it will be an optional dependency, and
 that most users won't have it installed.
 
-Users that do not have the dependency installed should be able to **import** your code without
-any side effects (no warnings, no errors, no exceptions). 
+Users who do not have the dependency installed should be able to **import** your code without
+any side effects (no warnings, no errors, no exceptions).
 
-To introduce the dependency to the pyproject.toml file correctly, please do the following: 
+To introduce the dependency to the pyproject.toml file correctly, please do the following:
 
 1. Add the dependency to the main group as an optional dependency
   ```bash
@@ -188,57 +220,13 @@ To introduce the dependency to the pyproject.toml file correctly, please do the
   ```bash
   poetry lock --no-update
   ```
-4. Add a unit test that the very least attempts to import the new code. Ideally the unit
+4. Add a unit test that the very least attempts to import the new code. Ideally, the unit
 test makes use of lightweight fixtures to test the logic of the code.
 5. Please use the `@pytest.mark.requires(package_name)` decorator for any tests that require the dependency.
 
-### Testing
+## Adding a Jupyter Notebook
 
-See section about optional dependencies.
-
-#### Unit Tests
-
-Unit tests cover modular logic that does not require calls to outside APIs.
-
-To run unit tests:
-
-```bash
-make test
-```
-
-To run unit tests in Docker:
-
-```bash
-make docker_tests
-```
-
-If you add new logic, please add a unit test.
-
-
-
-#### Integration Tests
-
-Integration tests cover logic that requires making calls to outside APIs (often integration with other services).
-
-**warning** Almost no tests should be integration tests. 
-
-  Tests that require making network connections make it difficult for other
-  developers to test the code.
-
-  Instead favor relying on `responses` library and/or mock.patch to mock
-  requests using small fixtures.
-
-To run integration tests:
-
-```bash
-make integration_tests
-```
-
-If you add support for a new external API, please add a new integration test.
-
-### Adding a Jupyter Notebook
-
-If you are adding a Jupyter notebook example, you'll want to install the optional `dev` dependencies.
+If you are adding a Jupyter Notebook example, you'll want to install the optional `dev` dependencies.
 
 To install dev dependencies:
 
@@ -259,6 +247,12 @@ When you run `poetry install`, the `langchain` package is installed as editable
 While the code is split between `langchain` and `langchain.experimental`, the documentation is one holistic thing.
 This covers how to get started contributing to documentation.
 
+From the top-level of this repo, install documentation dependencies:
+
+```bash
+poetry install
+```
+
 ### Contribute Documentation
 
 The docs directory contains Documentation and API Reference.
@@ -295,6 +289,13 @@ make docs_linkcheck
 make api_docs_linkcheck
 ```
 
+### Verify Documentation changes
+
+After pushing documentation changes to the repository, you can preview and verify that the changes are 
+what you wanted by clicking the `View deployment` or `Visit Preview` buttons on the pull request `Conversation` page. 
+This will take you to a preview of the documentation changes.
+This preview is created by [Vercel](https://vercel.com/docs/getting-started-with-vercel).
+
 ## 🏭 Release Process
 
 As of now, LangChain has an ad hoc release process: releases are cut with high frequency by
@@ -307,4 +308,3 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
 
 If your contribution has made its way into a release, we will want to give you credit on Twitter (only if you want though)!
 If you have a Twitter account you would like us to mention, please let us know in the PR or in another manner.
-

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -1,5 +1,5 @@
 name: "\U0001F41B Bug Report"
-description: Submit a bug report to help us improve LangChain
+description: Submit a bug report to help us improve LangChain. To report a security issue, please instead use the security option below.
 labels: ["02 Bug Report"]
 body:
   - type: markdown

.github/ISSUE_TEMPLATE/feature-request.yml

@@ -27,4 +27,4 @@ body:
     attributes:
       label: Your contribution
       description: |
-        Is there any way that you could help, e.g. by submitting a PR? Make sure to read the CONTRIBUTING.MD [readme](https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md)
+        Is there any way that you could help, e.g. by submitting a PR? Make sure to read the CONTRIBUTING.MD [readme](https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md)

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,28 +1,20 @@
 <!-- Thank you for contributing to LangChain!
 
-Replace this comment with:
-  - Description: a description of the change, 
-  - Issue: the issue # it fixes (if applicable),
-  - Dependencies: any dependencies required for this change,
-  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
-  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!
+Replace this entire comment with:
+  - **Description:** a description of the change, 
+  - **Issue:** the issue # it fixes (if applicable),
+  - **Dependencies:** any dependencies required for this change,
+  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
+  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!
 
-Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.
+Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.
+
+See contribution guidelines for more information on how to write/run tests, lint, etc: 
+https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md
 
 If you're adding a new integration, please include:
   1. a test for the integration, preferably unit tests that do not rely on network access,
-  2. an example notebook showing its use.
+  2. an example notebook showing its use. It lives in `docs/extras` directory.
 
-Maintainer responsibilities:
-  - General / Misc / if you don't know who to tag: @baskaryan
-  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
-  - Models / Prompts: @hwchase17, @baskaryan
-  - Memory: @hwchase17
-  - Agents / Tools / Toolkits: @hinthornw
-  - Tracing / Callbacks: @agola11
-  - Async: @agola11
-
-If no one reviews your PR within a few days, feel free to @-mention the same people again.
-
-See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
+If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
  -->

.github/actions/poetry_setup/action.yml

@@ -15,64 +15,77 @@ inputs:
     description: Poetry version
     required: true
 
-  install-command:
-    description: Command run for installing dependencies
-    required: false
-    default: poetry install
-
   cache-key:
     description: Cache key to use for manual handling of caching
     required: true
 
   working-directory:
-    description: Directory to run install-command in
-    required: false
-    default: ""
+    description: Directory whose poetry.lock file should be cached
+    required: true
 
 runs:
   using: composite
   steps:
     - uses: actions/setup-python@v4
-      name: Setup python $${ inputs.python-version }}
+      name: Setup python ${{ inputs.python-version }}
       with:
         python-version: ${{ inputs.python-version }}
 
     - uses: actions/cache@v3
-      id: cache-pip
-      name: Cache Pip ${{ inputs.python-version }}
+      id: cache-bin-poetry
+      name: Cache Poetry binary - Python ${{ inputs.python-version }}
       env:
-        SEGMENT_DOWNLOAD_TIMEOUT_MIN: "15"
+        SEGMENT_DOWNLOAD_TIMEOUT_MIN: "1"
+      with:
+        path: |
+          /opt/pipx/venvs/poetry
+        # This step caches the poetry installation, so make sure it's keyed on the poetry version as well.
+        key: bin-poetry-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-${{ inputs.poetry-version }}
+
+    - name: Refresh shell hashtable and fixup softlinks
+      if: steps.cache-bin-poetry.outputs.cache-hit == 'true'
+      shell: bash
+      env:
+        POETRY_VERSION: ${{ inputs.poetry-version }}
+        PYTHON_VERSION: ${{ inputs.python-version }}
+      run: |
+        set -eux
+
+        # Refresh the shell hashtable, to ensure correct `which` output.
+        hash -r
+
+        # `actions/cache@v3` doesn't always seem able to correctly unpack softlinks.
+        # Delete and recreate the softlinks pipx expects to have.
+        rm /opt/pipx/venvs/poetry/bin/python
+        cd /opt/pipx/venvs/poetry/bin
+        ln -s "$(which "python$PYTHON_VERSION")" python
+        chmod +x python
+        cd /opt/pipx_bin/
+        ln -s /opt/pipx/venvs/poetry/bin/poetry poetry
+        chmod +x poetry
+
+        # Ensure everything got set up correctly.
+        /opt/pipx/venvs/poetry/bin/python --version
+        /opt/pipx_bin/poetry --version
+
+    - name: Install poetry
+      if: steps.cache-bin-poetry.outputs.cache-hit != 'true'
+      shell: bash
+      env:
+        POETRY_VERSION: ${{ inputs.poetry-version }}
+        PYTHON_VERSION: ${{ inputs.python-version }}
+      run: pipx install "poetry==$POETRY_VERSION" --python "python$PYTHON_VERSION" --verbose
+
+    - name: Restore pip and poetry cached dependencies
+      uses: actions/cache@v3
+      env:
+        SEGMENT_DOWNLOAD_TIMEOUT_MIN: "4"
+        WORKDIR: ${{ inputs.working-directory == '' && '.' || inputs.working-directory }}
       with:
         path: |
           ~/.cache/pip
-        key: pip-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}
-
-    - run: pipx install poetry==${{ inputs.poetry-version }} --python python${{ inputs.python-version }}
-      shell: bash
-
-    - name: Check Poetry File
-      shell: bash
-      working-directory: ${{ inputs.working-directory }}
-      run: |
-        poetry check
-
-    - name: Check lock file
-      shell: bash
-      working-directory: ${{ inputs.working-directory }}
-      run: |
-        poetry lock --check
-
-    - uses: actions/cache@v3
-      id: cache-poetry
-      env:
-        SEGMENT_DOWNLOAD_TIMEOUT_MIN: "15"
-      with:
-        path: |
           ~/.cache/pypoetry/virtualenvs
           ~/.cache/pypoetry/cache
           ~/.cache/pypoetry/artifacts
-        key: poetry-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-poetry-${{ inputs.poetry-version }}-${{ inputs.cache-key }}-${{ hashFiles('poetry.lock') }}
-
-    - run: ${{ inputs.install-command }}
-      working-directory: ${{ inputs.working-directory }}
-      shell: bash
+          ${{ env.WORKDIR }}/.venv
+        key: py-deps-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-poetry-${{ inputs.poetry-version }}-${{ inputs.cache-key }}-${{ hashFiles(format('{0}/**/poetry.lock', env.WORKDIR)) }}

.github/tools/git-restore-mtime

@@ -0,0 +1,606 @@
+#!/usr/bin/env python3
+#
+# git-restore-mtime - Change mtime of files based on commit date of last change
+#
+#    Copyright (C) 2012 Rodrigo Silva (MestreLion) <linux@rodrigosilva.com>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License as published by
+#    the Free Software Foundation, either version 3 of the License, or
+#    (at your option) any later version.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program. See <http://www.gnu.org/licenses/gpl.html>
+#
+# Source: https://github.com/MestreLion/git-tools
+# Version: July 13, 2023 (commit hash 5f832e72453e035fccae9d63a5056918d64476a2)
+"""
+Change the modification time (mtime) of files in work tree, based on the
+date of the most recent commit that modified the file, including renames.
+
+Ignores untracked files and uncommitted deletions, additions and renames, and
+by default modifications too.
+---
+Useful prior to generating release tarballs, so each file is archived with a
+date that is similar to the date when the file was actually last modified,
+assuming the actual modification date and its commit date are close.
+"""
+
+# TODO:
+# - Add -z on git whatchanged/ls-files, so we don't deal with filename decoding
+# - When Python is bumped to 3.7, use text instead of universal_newlines on subprocess
+# - Update "Statistics for some large projects" with modern hardware and repositories.
+# - Create a README.md for git-restore-mtime alone. It deserves extensive documentation
+#   - Move Statistics there
+# - See git-extras as a good example on project structure and documentation
+
+# FIXME:
+# - When current dir is outside the worktree, e.g. using --work-tree, `git ls-files`
+#   assume any relative pathspecs are to worktree root, not the current dir. As such,
+#   relative pathspecs may not work.
+# - Renames are tricky:
+#   - R100 should not change mtime, but original name is not on filelist. Should
+#     track renames until a valid (A, M) mtime found and then set on current name.
+#   - Should set mtime for both current and original directories.
+#   - Check mode changes with unchanged blobs?
+# - Check file (A, D) for the directory mtime is not sufficient:
+#   - Renames also change dir mtime, unless rename was on a parent dir
+#   - If most recent change of all files in a dir was a Modification (M),
+#     dir might not be touched at all.
+#   - Dirs containing only subdirectories but no direct files will also
+#     not be touched. They're files' [grand]parent dir, but never their dirname().
+#   - Some solutions:
+#     - After files done, perform some dir processing for missing dirs, finding latest
+#       file (A, D, R)
+#     - Simple approach: dir mtime is the most recent child (dir or file) mtime
+#     - Use a virtual concept of "created at most at" to fill missing info, bubble up
+#       to parents and grandparents
+#   - When handling [grand]parent dirs, stay inside <pathspec>
+# - Better handling of merge commits. `-m` is plain *wrong*. `-c/--cc` is perfect, but
+#   painfully slow. First pass without merge commits is not accurate. Maybe add a new
+#   `--accurate` mode for `--cc`?
+
+if __name__ != "__main__":
+    raise ImportError("{} should not be used as a module.".format(__name__))
+
+import argparse
+import datetime
+import logging
+import os.path
+import shlex
+import signal
+import subprocess
+import sys
+import time
+
+__version__ = "2022.12+dev"
+
+# Update symlinks only if the platform supports not following them
+UPDATE_SYMLINKS = bool(os.utime in getattr(os, 'supports_follow_symlinks', []))
+
+# Call os.path.normpath() only if not in a POSIX platform (Windows)
+NORMALIZE_PATHS = (os.path.sep != '/')
+
+# How many files to process in each batch when re-trying merge commits
+STEPMISSING = 100
+
+# (Extra) keywords for the os.utime() call performed by touch()
+UTIME_KWS = {} if not UPDATE_SYMLINKS else {'follow_symlinks': False}
+
+
+# Command-line interface ######################################################
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description=__doc__.split('\n---')[0])
+
+    group = parser.add_mutually_exclusive_group()
+    group.add_argument('--quiet', '-q', dest='loglevel',
+        action="store_const", const=logging.WARNING, default=logging.INFO,
+        help="Suppress informative messages and summary statistics.")
+    group.add_argument('--verbose', '-v', action="count", help="""
+        Print additional information for each processed file.
+        Specify twice to further increase verbosity.
+        """)
+
+    parser.add_argument('--cwd', '-C', metavar="DIRECTORY", help="""
+        Run as if %(prog)s was started in directory %(metavar)s.
+        This affects how --work-tree, --git-dir and PATHSPEC arguments are handled.
+        See 'man 1 git' or 'git --help' for more information.
+        """)
+
+    parser.add_argument('--git-dir', dest='gitdir', metavar="GITDIR", help="""
+        Path to the git repository, by default auto-discovered by searching
+        the current directory and its parents for a .git/ subdirectory.
+        """)
+
+    parser.add_argument('--work-tree', dest='workdir', metavar="WORKTREE", help="""
+        Path to the work tree root, by default the parent of GITDIR if it's
+        automatically discovered, or the current directory if GITDIR is set.
+        """)
+
+    parser.add_argument('--force', '-f', default=False, action="store_true", help="""
+        Force updating files with uncommitted modifications.
+        Untracked files and uncommitted deletions, renames and additions are
+        always ignored.
+        """)
+
+    parser.add_argument('--merge', '-m', default=False, action="store_true", help="""
+        Include merge commits.
+        Leads to more recent times and more files per commit, thus with the same
+        time, which may or may not be what you want.
+        Including merge commits may lead to fewer commits being evaluated as files
+        are found sooner, which can improve performance, sometimes substantially.
+        But as merge commits are usually huge, processing them may also take longer.
+        By default, merge commits are only used for files missing from regular commits.
+        """)
+
+    parser.add_argument('--first-parent', default=False, action="store_true", help="""
+        Consider only the first parent, the "main branch", when evaluating merge commits.
+        Only effective when merge commits are processed, either when --merge is
+        used or when finding missing files after the first regular log search.
+        See --skip-missing.
+        """)
+
+    parser.add_argument('--skip-missing', '-s', dest="missing", default=True,
+        action="store_false", help="""
+        Do not try to find missing files.
+        If merge commits were not evaluated with --merge and some files were
+        not found in regular commits, by default %(prog)s searches for these
+        files again in the merge commits.
+        This option disables this retry, so files found only in merge commits
+        will not have their timestamp updated.
+        """)
+
+    parser.add_argument('--no-directories', '-D', dest='dirs', default=True,
+        action="store_false", help="""
+        Do not update directory timestamps.
+        By default, use the time of its most recently created, renamed or deleted file.
+        Note that just modifying a file will NOT update its directory time.
+        """)
+
+    parser.add_argument('--test', '-t', default=False, action="store_true",
+        help="Test run: do not actually update any file timestamp.")
+
+    parser.add_argument('--commit-time', '-c', dest='commit_time', default=False,
+        action='store_true', help="Use commit time instead of author time.")
+
+    parser.add_argument('--oldest-time', '-o', dest='reverse_order', default=False,
+        action='store_true', help="""
+        Update times based on the oldest, instead of the most recent commit of a file.
+        This reverses the order in which the git log is processed to emulate a
+        file "creation" date. Note this will be inaccurate for files deleted and
+        re-created at later dates.
+        """)
+
+    parser.add_argument('--skip-older-than', metavar='SECONDS', type=int, help="""
+        Ignore files that are currently older than %(metavar)s.
+        Useful in workflows that assume such files already have a correct timestamp,
+        as it may improve performance by processing fewer files.
+        """)
+
+    parser.add_argument('--skip-older-than-commit', '-N', default=False,
+        action='store_true', help="""
+        Ignore files older than the timestamp it would be updated to.
+        Such files may be considered "original", likely in the author's repository.
+        """)
+
+    parser.add_argument('--unique-times', default=False, action="store_true", help="""
+        Set the microseconds to a unique value per commit.
+        Allows telling apart changes that would otherwise have identical timestamps,
+        as git's time accuracy is in seconds.
+        """)
+
+    parser.add_argument('pathspec', nargs='*', metavar='PATHSPEC', help="""
+        Only modify paths matching %(metavar)s, relative to current directory.
+        By default, update all but untracked files and submodules.
+        """)
+
+    parser.add_argument('--version', '-V', action='version',
+        version='%(prog)s version {version}'.format(version=get_version()))
+
+    args_ = parser.parse_args()
+    if args_.verbose:
+        args_.loglevel = max(logging.TRACE, logging.DEBUG // args_.verbose)
+    args_.debug = args_.loglevel <= logging.DEBUG
+    return args_
+
+
+def get_version(version=__version__):
+    if not version.endswith('+dev'):
+        return version
+    try:
+        cwd = os.path.dirname(os.path.realpath(__file__))
+        return Git(cwd=cwd, errors=False).describe().lstrip('v')
+    except Git.Error:
+        return '-'.join((version, "unknown"))
+
+
+# Helper functions ############################################################
+
+def setup_logging():
+    """Add TRACE logging level and corresponding method, return the root logger"""
+    logging.TRACE = TRACE = logging.DEBUG // 2
+    logging.Logger.trace = lambda _, m, *a, **k: _.log(TRACE, m, *a, **k)
+    return logging.getLogger()
+
+
+def normalize(path):
+    r"""Normalize paths from git, handling non-ASCII characters.
+
+    Git stores paths as UTF-8 normalization form C.
+    If path contains non-ASCII or non-printable characters, git outputs the UTF-8
+    in octal-escaped notation, escaping double-quotes and backslashes, and then
+    double-quoting the whole path.
+    https://git-scm.com/docs/git-config#Documentation/git-config.txt-corequotePath
+
+    This function reverts this encoding, so:
+    normalize(r'"Back\\slash_double\"quote_a\303\247a\303\255"') =>
+        r'Back\slash_double"quote_açaí')
+
+    Paths with invalid UTF-8 encoding, such as single 0x80-0xFF bytes (e.g, from
+    Latin1/Windows-1251 encoding) are decoded using surrogate escape, the same
+    method used by Python for filesystem paths. So 0xE6 ("æ" in Latin1, r'\\346'
+    from Git) is decoded as "\udce6". See https://peps.python.org/pep-0383/ and
+    https://vstinner.github.io/painful-history-python-filesystem-encoding.html
+
+    Also see notes on `windows/non-ascii-paths.txt` about path encodings on
+    non-UTF-8 platforms and filesystems.
+    """
+    if path and path[0] == '"':
+        # Python 2: path = path[1:-1].decode("string-escape")
+        # Python 3: https://stackoverflow.com/a/46650050/624066
+        path = (path[1:-1]                 # Remove enclosing double quotes
+                .encode('latin1')          # Convert to bytes, required by 'unicode-escape'
+                .decode('unicode-escape')  # Perform the actual octal-escaping decode
+                .encode('latin1')          # 1:1 mapping to bytes, UTF-8 encoded
+                .decode('utf8', 'surrogateescape'))  # Decode from UTF-8
+    if NORMALIZE_PATHS:
+        # Make sure the slash matches the OS; for Windows we need a backslash
+        path = os.path.normpath(path)
+    return path
+
+
+def dummy(*_args, **_kwargs):
+    """No-op function used in dry-run tests"""
+
+
+def touch(path, mtime):
+    """The actual mtime update"""
+    os.utime(path, (mtime, mtime), **UTIME_KWS)
+
+
+def touch_ns(path, mtime_ns):
+    """The actual mtime update, using nanoseconds for unique timestamps"""
+    os.utime(path, None, ns=(mtime_ns, mtime_ns), **UTIME_KWS)
+
+
+def isodate(secs: int):
+    # time.localtime() accepts floats, but discards fractional part
+    return time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(secs))
+
+
+def isodate_ns(ns: int):
+    # for integers fromtimestamp() is equivalent and ~16% slower than isodate()
+    return datetime.datetime.fromtimestamp(ns / 1000000000).isoformat(sep=' ')
+
+
+def get_mtime_ns(secs: int, idx: int):
+    # Time resolution for filesystems and functions:
+    # ext-4 and other POSIX filesystems: 1 nanosecond
+    # NTFS (Windows default): 100 nanoseconds
+    # datetime.datetime() (due to 64-bit float epoch): 1 microsecond
+    us = idx % 1000000  # 10**6
+    return 1000 * (1000000 * secs + us)
+
+
+def get_mtime_path(path):
+    return os.path.getmtime(path)
+
+
+# Git class and parse_log(), the heart of the script ##########################
+
+class Git:
+    def __init__(self, workdir=None, gitdir=None, cwd=None, errors=True):
+        self.gitcmd = ['git']
+        self.errors = errors
+        self._proc = None
+        if workdir: self.gitcmd.extend(('--work-tree', workdir))
+        if gitdir:  self.gitcmd.extend(('--git-dir',   gitdir))
+        if cwd:     self.gitcmd.extend(('-C',          cwd))
+        self.workdir, self.gitdir = self._get_repo_dirs()
+
+    def ls_files(self, paths: list = None):
+        return (normalize(_) for _ in self._run('ls-files --full-name', paths))
+
+    def ls_dirty(self, force=False):
+        return (normalize(_[3:].split(' -> ', 1)[-1])
+                for _ in self._run('status --porcelain')
+                if _[:2] != '??' and (not force or (_[0] in ('R', 'A')
+                                                    or _[1] == 'D')))
+
+    def log(self, merge=False, first_parent=False, commit_time=False,
+            reverse_order=False, paths: list = None):
+        cmd = 'whatchanged --pretty={}'.format('%ct' if commit_time else '%at')
+        if merge:         cmd += ' -m'
+        if first_parent:  cmd += ' --first-parent'
+        if reverse_order: cmd += ' --reverse'
+        return self._run(cmd, paths)
+
+    def describe(self):
+        return self._run('describe --tags', check=True)[0]
+
+    def terminate(self):
+        if self._proc is None:
+            return
+        try:
+            self._proc.terminate()
+        except OSError:
+            # Avoid errors on OpenBSD
+            pass
+
+    def _get_repo_dirs(self):
+        return (os.path.normpath(_) for _ in
+            self._run('rev-parse --show-toplevel --absolute-git-dir', check=True))
+
+    def _run(self, cmdstr: str, paths: list = None, output=True, check=False):
+        cmdlist = self.gitcmd + shlex.split(cmdstr)
+        if paths:
+            cmdlist.append('--')
+            cmdlist.extend(paths)
+        popen_args = dict(universal_newlines=True, encoding='utf8')
+        if not self.errors:
+            popen_args['stderr'] = subprocess.DEVNULL
+        log.trace("Executing: %s", ' '.join(cmdlist))
+        if not output:
+            return subprocess.call(cmdlist, **popen_args)
+        if check:
+            try:
+                stdout: str = subprocess.check_output(cmdlist, **popen_args)
+                return stdout.splitlines()
+            except subprocess.CalledProcessError as e:
+                raise self.Error(e.returncode, e.cmd, e.output, e.stderr)
+        self._proc = subprocess.Popen(cmdlist, stdout=subprocess.PIPE, **popen_args)
+        return (_.rstrip() for _ in self._proc.stdout)
+
+    def __del__(self):
+        self.terminate()
+
+    class Error(subprocess.CalledProcessError):
+        """Error from git executable"""
+
+
+def parse_log(filelist, dirlist, stats, git, merge=False, filterlist=None):
+    mtime = 0
+    datestr = isodate(0)
+    for line in git.log(
+            merge,
+            args.first_parent,
+            args.commit_time,
+            args.reverse_order,
+            filterlist
+    ):
+        stats['loglines'] += 1
+
+        # Blank line between Date and list of files
+        if not line:
+            continue
+
+        # Date line
+        if line[0] != ':':  # Faster than `not line.startswith(':')`
+            stats['commits'] += 1
+            mtime = int(line)
+            if args.unique_times:
+                mtime = get_mtime_ns(mtime, stats['commits'])
+            if args.debug:
+                datestr = isodate(mtime)
+            continue
+
+        # File line: three tokens if it describes a renaming, otherwise two
+        tokens = line.split('\t')
+
+        # Possible statuses:
+        # M: Modified (content changed)
+        # A: Added (created)
+        # D: Deleted
+        # T: Type changed: to/from regular file, symlinks, submodules
+        # R099: Renamed (moved), with % of unchanged content. 100 = pure rename
+        # Not possible in log: C=Copied, U=Unmerged, X=Unknown, B=pairing Broken
+        status = tokens[0].split(' ')[-1]
+        file = tokens[-1]
+
+        # Handles non-ASCII chars and OS path separator
+        file = normalize(file)
+
+        def do_file():
+            if args.skip_older_than_commit and get_mtime_path(file) <= mtime:
+                stats['skip'] += 1
+                return
+            if args.debug:
+                log.debug("%d\t%d\t%d\t%s\t%s",
+                          stats['loglines'], stats['commits'], stats['files'],
+                          datestr, file)
+            try:
+                touch(os.path.join(git.workdir, file), mtime)
+                stats['touches'] += 1
+            except Exception as e:
+                log.error("ERROR: %s: %s", e, file)
+                stats['errors'] += 1
+
+        def do_dir():
+            if args.debug:
+                log.debug("%d\t%d\t-\t%s\t%s",
+                          stats['loglines'], stats['commits'],
+                          datestr, "{}/".format(dirname or '.'))
+            try:
+                touch(os.path.join(git.workdir, dirname), mtime)
+                stats['dirtouches'] += 1
+            except Exception as e:
+                log.error("ERROR: %s: %s", e, dirname)
+                stats['direrrors'] += 1
+
+        if file in filelist:
+            stats['files'] -= 1
+            filelist.remove(file)
+            do_file()
+
+        if args.dirs and status in ('A', 'D'):
+            dirname = os.path.dirname(file)
+            if dirname in dirlist:
+                dirlist.remove(dirname)
+                do_dir()
+
+        # All files done?
+        if not stats['files']:
+            git.terminate()
+            return
+
+
+# Main Logic ##################################################################
+
+def main():
+    start = time.time()  # yes, Wall time. CPU time is not realistic for users.
+    stats = {_: 0 for _ in ('loglines', 'commits', 'touches', 'skip', 'errors',
+                            'dirtouches', 'direrrors')}
+
+    logging.basicConfig(level=args.loglevel, format='%(message)s')
+    log.trace("Arguments: %s", args)
+
+    # First things first: Where and Who are we?
+    if args.cwd:
+        log.debug("Changing directory: %s", args.cwd)
+        try:
+            os.chdir(args.cwd)
+        except OSError as e:
+            log.critical(e)
+            return e.errno
+    # Using both os.chdir() and `git -C` is redundant, but might prevent side effects
+    # `git -C` alone could be enough if we make sure that:
+    # - all paths, including args.pathspec, are processed by git: ls-files, rev-parse
+    # - touch() / os.utime() path argument is always prepended with git.workdir
+    try:
+        git = Git(workdir=args.workdir, gitdir=args.gitdir, cwd=args.cwd)
+    except Git.Error as e:
+        # Not in a git repository, and git already informed user on stderr. So we just...
+        return e.returncode
+
+    # Get the files managed by git and build file list to be processed
+    if UPDATE_SYMLINKS and not args.skip_older_than:
+        filelist = set(git.ls_files(args.pathspec))
+    else:
+        filelist = set()
+        for path in git.ls_files(args.pathspec):
+            fullpath = os.path.join(git.workdir, path)
+
+            # Symlink (to file, to dir or broken - git handles the same way)
+            if not UPDATE_SYMLINKS and os.path.islink(fullpath):
+                log.warning("WARNING: Skipping symlink, no OS support for updates: %s",
+                            path)
+                continue
+
+            # skip files which are older than given threshold
+            if (args.skip_older_than
+                    and start - get_mtime_path(fullpath) > args.skip_older_than):
+                continue
+
+            # Always add files relative to worktree root
+            filelist.add(path)
+
+    # If --force, silently ignore uncommitted deletions (not in the filesystem)
+    # and renames / additions (will not be found in log anyway)
+    if args.force:
+        filelist -= set(git.ls_dirty(force=True))
+    # Otherwise, ignore any dirty files
+    else:
+        dirty = set(git.ls_dirty())
+        if dirty:
+            log.warning("WARNING: Modified files in the working directory were ignored."
+                "\nTo include such files, commit your changes or use --force.")
+            filelist -= dirty
+
+    # Build dir list to be processed
+    dirlist = set(os.path.dirname(_) for _ in filelist) if args.dirs else set()
+
+    stats['totalfiles'] = stats['files'] = len(filelist)
+    log.info("{0:,} files to be processed in work dir".format(stats['totalfiles']))
+
+    if not filelist:
+        # Nothing to do. Exit silently and without errors, just like git does
+        return
+
+    # Process the log until all files are 'touched'
+    log.debug("Line #\tLog #\tF.Left\tModification Time\tFile Name")
+    parse_log(filelist, dirlist, stats, git, args.merge, args.pathspec)
+
+    # Missing files
+    if filelist:
+        # Try to find them in merge logs, if not done already
+        # (usually HUGE, thus MUCH slower!)
+        if args.missing and not args.merge:
+            filterlist = list(filelist)
+            missing = len(filterlist)
+            log.info("{0:,} files not found in log, trying merge commits".format(missing))
+            for i in range(0, missing, STEPMISSING):
+                parse_log(filelist, dirlist, stats, git,
+                          merge=True, filterlist=filterlist[i:i + STEPMISSING])
+
+        # Still missing some?
+        for file in filelist:
+            log.warning("WARNING: not found in the log: %s", file)
+
+    # Final statistics
+    # Suggestion: use git-log --before=mtime to brag about skipped log entries
+    def log_info(msg, *a, width=13):
+        ifmt = '{:%d,}'    % (width,)  # not using 'n' for consistency with ffmt
+        ffmt = '{:%d,.2f}' % (width,)
+        # %-formatting lacks a thousand separator, must pre-render with .format()
+        log.info(msg.replace('%d', ifmt).replace('%f', ffmt).format(*a))
+
+    log_info(
+        "Statistics:\n"
+        "%f seconds\n"
+        "%d log lines processed\n"
+        "%d commits evaluated",
+        time.time() - start, stats['loglines'], stats['commits'])
+
+    if args.dirs:
+        if stats['direrrors']: log_info("%d directory update errors", stats['direrrors'])
+        log_info("%d directories updated", stats['dirtouches'])
+
+    if stats['touches'] != stats['totalfiles']:
+                        log_info("%d files",              stats['totalfiles'])
+    if stats['skip']:   log_info("%d files skipped",      stats['skip'])
+    if stats['files']:  log_info("%d files missing",      stats['files'])
+    if stats['errors']: log_info("%d file update errors", stats['errors'])
+
+    log_info("%d files updated", stats['touches'])
+
+    if args.test:
+        log.info("TEST RUN - No files modified!")
+
+
+# Keep only essential, global assignments here. Any other logic must be in main()
+log = setup_logging()
+args = parse_args()
+
+# Set the actual touch() and other functions based on command-line arguments
+if args.unique_times:
+    touch = touch_ns
+    isodate = isodate_ns
+
+# Make sure this is always set last to ensure --test behaves as intended
+if args.test:
+    touch = dummy
+
+# UI done, it's showtime!
+try:
+    sys.exit(main())
+except KeyboardInterrupt:
+    log.info("\nAborting")
+    signal.signal(signal.SIGINT, signal.SIG_DFL)
+    os.kill(os.getpid(), signal.SIGINT)

.github/workflows/_lint.yml

@@ -9,38 +9,142 @@ on:
         description: "From which folder this pipeline executes"
 
 env:
-  POETRY_VERSION: "1.4.2"
+  POETRY_VERSION: "1.6.1"
+  WORKDIR: ${{ inputs.working-directory == '' && '.' || inputs.working-directory }}
 
 jobs:
   build:
-    defaults:
-      run:
-        working-directory: ${{ inputs.working-directory }}
     runs-on: ubuntu-latest
+    env:
+      # This number is set "by eye": we want it to be big enough
+      # so that it's bigger than the number of commits in any reasonable PR,
+      # and also as small as possible since increasing the number makes
+      # the initial `git fetch` slower.
+      FETCH_DEPTH: 50
     strategy:
       matrix:
+        # Only lint on the min and max supported Python versions.
+        # It's extremely unlikely that there's a lint issue on any version in between
+        # that doesn't show up on the min or max versions.
+        #
+        # GitHub rate-limits how many jobs can be running at any one time.
+        # Starting new jobs is also relatively slow,
+        # so linting on fewer versions makes CI faster.
         python-version:
           - "3.8"
-          - "3.9"
-          - "3.10"
           - "3.11"
     steps:
       - uses: actions/checkout@v3
-      - name: Install poetry
+        with:
+          # Fetch the last FETCH_DEPTH commits, so the mtime-changing script
+          # can accurately set the mtimes of files modified in the last FETCH_DEPTH commits.
+          fetch-depth: ${{ env.FETCH_DEPTH }}
+      - name: Restore workdir file mtimes to last-edited commit date
+        id: restore-mtimes
+        # This is needed to make black caching work.
+        # Black's cache uses file (mtime, size) to check whether a lookup is a cache hit.
+        # Without this command, files in the repo would have the current time as the modified time,
+        # since the previous action step just created them.
+        # This command resets the mtime to the last time the files were modified in git instead,
+        # which is a high-quality and stable representation of the last modification date.
         run: |
-          pipx install poetry==$POETRY_VERSION
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+          # Important considerations:
+          # - These commands run at base of the repo, since we never `cd` to the `WORKDIR`.
+          # - We only want to alter mtimes for Python files, since that's all black checks.
+          # - We don't need to alter mtimes for directories, since black doesn't look at those.
+          # - We also only alter mtimes inside the `WORKDIR` since that's all we'll lint.
+          # - This should run before `poetry install`, because poetry's venv also contains
+          #   Python files, and we don't want to alter their mtimes since they aren't linted.
+
+          # Ensure we fail on non-zero exits and on undefined variables.
+          # Also print executed commands, for easier debugging.
+          set -eux
+
+          # Restore the mtimes of Python files in the workdir based on git history.
+          .github/tools/git-restore-mtime --no-directories "$WORKDIR/**/*.py"
+
+          # Since CI only does a partial fetch (to `FETCH_DEPTH`) for efficiency,
+          # the local git repo doesn't have full history. There are probably files
+          # that were last modified in a commit *older than* the oldest fetched commit.
+          # After `git-restore-mtime`, such files have a mtime set to the oldest fetched commit.
+          #
+          # As new commits get added, that timestamp will keep moving forward.
+          # If left unchanged, this will make `black` think that the files were edited
+          # more recently than its cache suggests. Instead, we can set their mtime
+          # to a fixed date in the far past that won't change and won't cause cache misses in black.
+          #
+          # For all workdir Python files modified in or before the oldest few fetched commits,
+          # make their mtime be 2000-01-01 00:00:00.
+          OLDEST_COMMIT="$(git log --reverse '--pretty=format:%H' | head -1)"
+          OLDEST_COMMIT_TIME="$(git show -s '--format=%ai' "$OLDEST_COMMIT")"
+          find "$WORKDIR" -name '*.py' -type f -not -newermt "$OLDEST_COMMIT_TIME" -exec touch -c -m -t '200001010000' '{}' '+'
+
+          echo "oldest-commit=$OLDEST_COMMIT" >> "$GITHUB_OUTPUT"
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
         with:
           python-version: ${{ matrix.python-version }}
-          cache: poetry
-      - name: Install dependencies
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: ${{ inputs.working-directory }}
+          cache-key: lint-with-extras
+
+      - name: Check Poetry File
+        shell: bash
+        working-directory: ${{ inputs.working-directory }}
         run: |
-          poetry install
+          poetry check
+
+      - name: Check lock file
+        shell: bash
+        working-directory: ${{ inputs.working-directory }}
+        run: |
+          poetry lock --check
+
+      - name: Install dependencies
+        # Also installs dev/lint/test/typing dependencies, to ensure we have
+        # type hints for as many of our libraries as possible.
+        # This helps catch errors that require dependencies to be spotted, for example:
+        # https://github.com/langchain-ai/langchain/pull/10249/files#diff-935185cd488d015f026dcd9e19616ff62863e8cde8c0bee70318d3ccbca98341
+        #
+        # If you change this configuration, make sure to change the `cache-key`
+        # in the `poetry_setup` action above to stop using the old cache.
+        # It doesn't matter how you change it, any change will cause a cache-bust.
+        working-directory: ${{ inputs.working-directory }}
+        run: |
+          poetry install --with dev,lint,test,typing
+
       - name: Install langchain editable
-        if: ${{ inputs.working-directory != 'langchain' }}
+        working-directory: ${{ inputs.working-directory }}
+        if: ${{ inputs.working-directory != 'libs/langchain' }}
         run: |
           pip install -e ../langchain
+
+      - name: Restore black cache
+        uses: actions/cache@v3
+        env:
+          CACHE_BASE: black-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', env.WORKDIR)) }}
+          SEGMENT_DOWNLOAD_TIMEOUT_MIN: "1"
+        with:
+          path: |
+            ${{ env.WORKDIR }}/.black_cache
+          key: ${{ env.CACHE_BASE }}-${{ steps.restore-mtimes.outputs.oldest-commit }}
+          restore-keys:
+            # If we can't find an exact match for our cache key, accept any with this prefix.
+            ${{ env.CACHE_BASE }}-
+
+      - name: Get .mypy_cache to speed up mypy
+        uses: actions/cache@v3
+        env:
+          SEGMENT_DOWNLOAD_TIMEOUT_MIN: "2"
+        with:
+          path: |
+            ${{ env.WORKDIR }}/.mypy_cache
+          key: mypy-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', env.WORKDIR)) }}
+
       - name: Analysing the code with our lint
+        working-directory: ${{ inputs.working-directory }}
+        env:
+          BLACK_CACHE_DIR: .black_cache
         run: |
           make lint

.github/workflows/_pydantic_compatibility.yml

@@ -0,0 +1,93 @@
+name: pydantic v1/v2 compatibility
+
+on:
+  workflow_call:
+    inputs:
+      working-directory:
+        required: true
+        type: string
+        description: "From which folder this pipeline executes"
+
+env:
+  POETRY_VERSION: "1.6.1"
+
+jobs:
+  build:
+    defaults:
+      run:
+        working-directory: ${{ inputs.working-directory }}
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version:
+          - "3.8"
+          - "3.9"
+          - "3.10"
+          - "3.11"
+    name: Pydantic v1/v2 compatibility - Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: ${{ inputs.working-directory }}
+          cache-key: pydantic-cross-compat
+
+      - name: Install dependencies
+        shell: bash
+        run: poetry install
+
+      - name: Install the opposite major version of pydantic
+        # If normal tests use pydantic v1, here we'll use v2, and vice versa.
+        shell: bash
+        run: |
+          # Determine the major part of pydantic version
+          REGULAR_VERSION=$(poetry run python -c "import pydantic; print(pydantic.__version__)" | cut -d. -f1)
+
+          if [[ "$REGULAR_VERSION" == "1" ]]; then
+            PYDANTIC_DEP=">=2.1,<3"
+            TEST_WITH_VERSION="2"
+          elif [[ "$REGULAR_VERSION" == "2" ]]; then
+            PYDANTIC_DEP="<2"
+            TEST_WITH_VERSION="1"
+          else
+            echo "Unexpected pydantic major version '$REGULAR_VERSION', cannot determine which version to use for cross-compatibility test."
+            exit 1
+          fi
+
+          # Install via `pip` instead of `poetry add` to avoid changing lockfile,
+          # which would prevent caching from working: the cache would get saved
+          # to a different key than where it gets loaded from.
+          poetry run pip install "pydantic${PYDANTIC_DEP}"
+
+          # Ensure that the correct pydantic is installed now.
+          echo "Checking pydantic version... Expecting ${TEST_WITH_VERSION}"
+
+          # Determine the major part of pydantic version
+          CURRENT_VERSION=$(poetry run python -c "import pydantic; print(pydantic.__version__)" | cut -d. -f1)
+
+          # Check that the major part of pydantic version is as expected, if not
+          # raise an error
+          if [[ "$CURRENT_VERSION" != "$TEST_WITH_VERSION" ]]; then
+            echo "Error: expected pydantic version ${CURRENT_VERSION} to have been installed, but found: ${TEST_WITH_VERSION}"
+            exit 1
+          fi
+          echo "Found pydantic version ${CURRENT_VERSION}, as expected"
+      - name: Run pydantic compatibility tests
+        shell: bash
+        run: make test
+
+      - name: Ensure the tests did not create any additional files
+        shell: bash
+        run: |
+          set -eu
+
+          STATUS="$(git status)"
+          echo "$STATUS"
+
+          # grep will exit non-zero if the target message isn't found,
+          # and `set -e` above will cause the step to fail.
+          echo "$STATUS" | grep 'nothing to commit, working tree clean'

.github/workflows/_release.yml

@@ -9,26 +9,37 @@ on:
         description: "From which folder this pipeline executes"
 
 env:
-  POETRY_VERSION: "1.4.2"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   if_release:
-    if: |
-        ${{ github.event.pull_request.merged == true }}
-        && ${{ contains(github.event.pull_request.labels.*.name, 'release') }}
+    # Disallow publishing from branches that aren't `master`.
+    if: github.ref == 'refs/heads/master'
     runs-on: ubuntu-latest
+    permissions:
+      # This permission is used for trusted publishing:
+      # https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/
+      #
+      # Trusted publishing has to also be configured on PyPI for each package:
+      # https://docs.pypi.org/trusted-publishers/adding-a-publisher/
+      id-token: write
+
+      # This permission is needed by `ncipollo/release-action` to create the GitHub release.
+      contents: write
     defaults:
       run:
         working-directory: ${{ inputs.working-directory }}
     steps:
       - uses: actions/checkout@v3
-      - name: Install poetry
-        run: pipx install poetry==$POETRY_VERSION
-      - name: Set up Python 3.10
-        uses: actions/setup-python@v4
+
+      - name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
         with:
           python-version: "3.10"
-          cache: "poetry"
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: ${{ inputs.working-directory }}
+          cache-key: release
+
       - name: Build project for distribution
         run: poetry build
       - name: Check Version
@@ -37,6 +48,7 @@ jobs:
           echo version=$(poetry version --short) >> $GITHUB_OUTPUT
       - name: Create Release
         uses: ncipollo/release-action@v1
+        if: ${{ inputs.working-directory == 'libs/langchain' }}
         with:
           artifacts: "dist/*"
           token: ${{ secrets.GITHUB_TOKEN }}
@@ -44,8 +56,9 @@ jobs:
           generateReleaseNotes: true
           tag: v${{ steps.check-version.outputs.version }}
           commit: master
-      - name: Publish to PyPI
-        env:
-          POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
-        run: |
-          poetry publish
+      - name: Publish package distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          packages-dir: ${{ inputs.working-directory }}/dist/
+          verbose: true
+          print-hash: true

.github/workflows/_release_docker.yml

@@ -0,0 +1,62 @@
+name: release_docker
+
+on:
+  workflow_call:
+    inputs:
+      dockerfile:
+        required: true
+        type: string
+        description: "Path to the Dockerfile to build"
+      image:
+        required: true
+        type: string
+        description: "Name of the image to build"
+
+env:
+  TEST_TAG: ${{ inputs.image }}:test
+  LATEST_TAG: ${{ inputs.image }}:latest
+
+jobs:
+  docker:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Get git tag
+        uses: actions-ecosystem/action-get-latest-tag@v1
+        id: get-latest-tag
+      - name: Set docker tag
+        env:
+          VERSION: ${{ steps.get-latest-tag.outputs.tag }}
+        run: |
+          echo "VERSION_TAG=${{ inputs.image }}:${VERSION#v}" >> $GITHUB_ENV
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+      - name: Login to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      - name: Build for Test
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          file: ${{ inputs.dockerfile }}
+          load: true
+          tags: ${{ env.TEST_TAG }}
+      - name: Test
+        run: |
+          docker run --rm ${{ env.TEST_TAG }} python -c "import langchain"
+      - name: Build and Push to Docker Hub
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          file: ${{ inputs.dockerfile }}
+          # We can only build for the intersection of platforms supported by
+          # QEMU and base python image, for now build only for
+          # linux/amd64 and linux/arm64
+          platforms: linux/amd64,linux/arm64
+          tags: ${{ env.LATEST_TAG }},${{ env.VERSION_TAG }}
+          push: true

.github/workflows/_test.yml

@@ -7,13 +7,9 @@ on:
         required: true
         type: string
         description: "From which folder this pipeline executes"
-      test_type:
-        type: string
-        description: "Test types to run"
-        default: '["core", "extended"]'
 
 env:
-  POETRY_VERSION: "1.4.2"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   build:
@@ -28,34 +24,34 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
-        test_type: ${{ fromJSON(inputs.test_type) }}
-    name: Python ${{ matrix.python-version }} ${{ matrix.test_type }}
+    name: Python ${{ matrix.python-version }}
     steps:
       - uses: actions/checkout@v3
-      - name: Set up Python ${{ matrix.python-version }}
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"
         with:
           python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
           working-directory: ${{ inputs.working-directory }}
-          poetry-version: "1.4.2"
-          cache-key: ${{ matrix.test_type }}
-          install-command: |
-              if [ "${{ matrix.test_type }}" == "core" ]; then
-                echo "Running core tests, installing dependencies with poetry..."
-                poetry install
-              else
-                echo "Running extended tests, installing dependencies with poetry..."
-                poetry install -E extended_testing
-              fi
-      - name: Install langchain editable
-        if: ${{ inputs.working-directory != 'langchain' }}
-        run: |
-          pip install -e ../langchain
-      - name: Run ${{matrix.test_type}} tests
-        run: |
-          if [ "${{ matrix.test_type }}" == "core" ]; then
-            make test
-          else
-            make extended_tests
-          fi
+          cache-key: core
+
+      - name: Install dependencies
         shell: bash
+        run: poetry install
+
+      - name: Run core tests
+        shell: bash
+        run: make test
+
+      - name: Ensure the tests did not create any additional files
+        shell: bash
+        run: |
+          set -eu
+
+          STATUS="$(git status)"
+          echo "$STATUS"
+
+          # grep will exit non-zero if the target message isn't found,
+          # and `set -e` above will cause the step to fail.
+          echo "$STATUS" | grep 'nothing to commit, working tree clean'

.github/workflows/codespell.yml

@@ -18,7 +18,19 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v3
+
+      - name: Install Dependencies
+        run: |
+          pip install toml
+
+      - name: Extract Ignore Words List
+        run: |
+          # Use a Python script to extract the ignore words list from pyproject.toml
+          python .github/workflows/extract_ignored_words_list.py
+        id: extract_ignore_words
+
       - name: Codespell
         uses: codespell-project/actions-codespell@v2
         with:
           skip: guide_imports.json
+          ignore_words_list: ${{ steps.extract_ignore_words.outputs.ignore_words_list }}

.github/workflows/doc_lint.yml

@@ -0,0 +1,22 @@
+---
+name: Documentation Lint
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v2
+
+    - name: Run import check
+      run: |
+        # We should not encourage imports directly from main init file
+        # Expect for hub
+        git grep 'from langchain import' docs/{docs,snippets} | grep -vE 'from langchain import (hub)' && exit 1 || exit 0

.github/workflows/extract_ignored_words_list.py

@@ -0,0 +1,8 @@
+import toml
+
+pyproject_toml = toml.load("pyproject.toml")
+
+# Extract the ignore words list (adjust the key as per your TOML structure)
+ignore_words_list = pyproject_toml.get("tool", {}).get("codespell", {}).get("ignore-words-list")
+
+print(f"::set-output name=ignore_words_list::{ignore_words_list}")

.github/workflows/langchain_ci.yml

@@ -6,12 +6,29 @@ on:
     branches: [ master ]
   pull_request:
     paths:
+      - '.github/actions/poetry_setup/action.yml'
+      - '.github/tools/**'
       - '.github/workflows/_lint.yml'
       - '.github/workflows/_test.yml'
+      - '.github/workflows/_pydantic_compatibility.yml'
       - '.github/workflows/langchain_ci.yml'
       - 'libs/langchain/**'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
+# If another push to the same PR or branch happens while this workflow is still running,
+# cancel the earlier run in favor of the next run.
+#
+# There's no point in testing an outdated version of the code. GitHub only allows
+# a limited number of job runners to be active at the same time, so it's better to cancel
+# pointless jobs early so that more useful jobs can run sooner.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  POETRY_VERSION: "1.6.1"
+  WORKDIR: "libs/langchain"
+
 jobs:
   lint:
     uses:
@@ -19,9 +36,62 @@ jobs:
     with:
       working-directory: libs/langchain
     secrets: inherit
+
   test:
     uses:
       ./.github/workflows/_test.yml
     with:
       working-directory: libs/langchain
-    secrets: inherit
+    secrets: inherit
+
+  pydantic-compatibility:
+    uses:
+      ./.github/workflows/_pydantic_compatibility.yml
+    with:
+      working-directory: libs/langchain
+    secrets: inherit
+
+  extended-tests:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ${{ env.WORKDIR }}
+    strategy:
+      matrix:
+        python-version:
+          - "3.8"
+          - "3.9"
+          - "3.10"
+          - "3.11"
+    name: Python ${{ matrix.python-version }} extended tests
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: libs/langchain
+          cache-key: extended
+
+      - name: Install dependencies
+        shell: bash
+        run: |
+          echo "Running extended tests, installing dependencies with poetry..."
+          poetry install -E extended_testing
+
+      - name: Run extended tests
+        run: make extended_tests
+
+      - name: Ensure the tests did not create any additional files
+        shell: bash
+        run: |
+          set -eu
+
+          STATUS="$(git status)"
+          echo "$STATUS"
+
+          # grep will exit non-zero if the target message isn't found,
+          # and `set -e` above will cause the step to fail.
+          echo "$STATUS" | grep 'nothing to commit, working tree clean'

.github/workflows/langchain_experimental_ci.yml

@@ -1,11 +1,13 @@
 ---
-name: libs/langchain-experimental CI
+name: libs/experimental CI
 
 on:
   push:
     branches: [ master ]
   pull_request:
     paths:
+      - '.github/actions/poetry_setup/action.yml'
+      - '.github/tools/**'
       - '.github/workflows/_lint.yml'
       - '.github/workflows/_test.yml'
       - '.github/workflows/langchain_experimental_ci.yml'
@@ -13,6 +15,20 @@ on:
       - 'libs/experimental/**'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
+# If another push to the same PR or branch happens while this workflow is still running,
+# cancel the earlier run in favor of the next run.
+#
+# There's no point in testing an outdated version of the code. GitHub only allows
+# a limited number of job runners to be active at the same time, so it's better to cancel
+# pointless jobs early so that more useful jobs can run sooner.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  POETRY_VERSION: "1.6.1"
+  WORKDIR: "libs/experimental"
+
 jobs:
   lint:
     uses:
@@ -20,10 +36,94 @@ jobs:
     with:
       working-directory: libs/experimental
     secrets: inherit
+
   test:
     uses:
       ./.github/workflows/_test.yml
     with:
       working-directory: libs/experimental
-      test_type: '["core"]'
-    secrets: inherit
+    secrets: inherit
+
+  # It's possible that langchain-experimental works fine with the latest *published* langchain,
+  # but is broken with the langchain on `master`.
+  #
+  # We want to catch situations like that *before* releasing a new langchain, hence this test.
+  test-with-latest-langchain:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ${{ env.WORKDIR }}
+    strategy:
+      matrix:
+        python-version:
+          - "3.8"
+          - "3.9"
+          - "3.10"
+          - "3.11"
+    name: test with unpublished langchain - Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: ${{ env.WORKDIR }}
+          cache-key: unpublished-langchain
+
+      - name: Install dependencies
+        shell: bash
+        run: |
+          echo "Running tests with unpublished langchain, installing dependencies with poetry..."
+          poetry install
+
+          echo "Editably installing langchain outside of poetry, to avoid messing up lockfile..."
+          poetry run pip install -e ../langchain
+
+      - name: Run tests
+        run: make test
+  extended-tests:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ${{ env.WORKDIR }}
+    strategy:
+      matrix:
+        python-version:
+          - "3.8"
+          - "3.9"
+          - "3.10"
+          - "3.11"
+    name: Python ${{ matrix.python-version }} extended tests
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: libs/experimental
+          cache-key: extended
+
+      - name: Install dependencies
+        shell: bash
+        run: |
+          echo "Running extended tests, installing dependencies with poetry..."
+          poetry install -E extended_testing
+
+      - name: Run extended tests
+        run: make extended_tests
+
+      - name: Ensure the tests did not create any additional files
+        shell: bash
+        run: |
+          set -eu
+
+          STATUS="$(git status)"
+          echo "$STATUS"
+
+          # grep will exit non-zero if the target message isn't found,
+          # and `set -e` above will cause the step to fail.
+          echo "$STATUS" | grep 'nothing to commit, working tree clean'

.github/workflows/langchain_experimental_release.yml

@@ -1,14 +1,7 @@
 ---
-name: libs/langchain-experimental Release
+name: libs/experimental Release
 
 on:
-  pull_request:
-    types:
-      - closed
-    branches:
-      - master
-    paths:
-      - 'libs/experimental/pyproject.toml'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
 jobs:
@@ -17,4 +10,4 @@ jobs:
       ./.github/workflows/_release.yml
     with:
       working-directory: libs/experimental
-    secrets: inherit
+    secrets: inherit

.github/workflows/langchain_release.yml

@@ -2,13 +2,6 @@
 name: libs/langchain Release
 
 on:
-  pull_request:
-    types:
-      - closed
-    branches:
-      - master
-    paths:
-      - 'libs/langchain/pyproject.toml'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
 jobs:
@@ -17,4 +10,18 @@ jobs:
       ./.github/workflows/_release.yml
     with:
       working-directory: libs/langchain
-    secrets: inherit
+    secrets: inherit
+
+  # N.B.: It's possible that PyPI doesn't make the new release visible / available
+  #       immediately after publishing. If that happens, the docker build might not
+  #       create a new docker image for the new release, since it won't see it.
+  #
+  #       If this ends up being a problem, add a check to the end of the `_release.yml`
+  #       workflow that prevents the workflow from finishing until the new release
+  #       is visible and installable on PyPI.
+  release-docker:
+    needs:
+      - release
+    uses:
+      ./.github/workflows/langchain_release_docker.yml
+    secrets: inherit

.github/workflows/langchain_release_docker.yml

@@ -0,0 +1,14 @@
+---
+name: docker/langchain/langchain Release
+
+on:
+  workflow_dispatch: # Allows to trigger the workflow manually in GitHub UI
+  workflow_call: # Allows triggering from another workflow
+
+jobs:
+  release:
+    uses: ./.github/workflows/_release_docker.yml
+    with:
+      dockerfile: docker/Dockerfile.base
+      image: langchain/langchain
+    secrets: inherit

.github/workflows/scheduled_test.yml

@@ -0,0 +1,81 @@
+name: Scheduled tests
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+  schedule:
+    - cron:  '0 13 * * *'
+
+env:
+  POETRY_VERSION: "1.6.1"
+
+jobs:
+  build:
+    defaults:
+      run:
+        working-directory: libs/langchain
+    runs-on: ubuntu-latest
+    environment: Scheduled testing
+    strategy:
+      matrix:
+        python-version:
+          - "3.8"
+          - "3.9"
+          - "3.10"
+          - "3.11"
+    name: Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: libs/langchain
+          cache-key: scheduled
+
+      - name: 'Authenticate to Google Cloud'
+        id: 'auth'
+        uses: 'google-github-actions/auth@v1'
+        with:
+          credentials_json: '${{ secrets.GOOGLE_CREDENTIALS }}'
+
+      - name: Configure AWS Credentials
+        uses: aws-actions/configure-aws-credentials@v4
+        with:
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          aws-region: ${{ vars.AWS_REGION }}
+
+      - name: Install dependencies
+        working-directory: libs/langchain
+        shell: bash
+        run: |
+          echo "Running scheduled tests, installing dependencies with poetry..."
+          poetry install --with=test_integration
+          poetry run pip install google-cloud-aiplatform
+          poetry run pip install "boto3>=1.28.57"
+
+      - name: Run tests
+        shell: bash
+        env:
+          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          AZURE_OPENAI_API_VERSION: ${{ secrets.AZURE_OPENAI_API_VERSION }}
+          AZURE_OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_API_BASE }}
+          AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
+          AZURE_OPENAI_DEPLOYMENT_NAME:  ${{ secrets.AZURE_OPENAI_DEPLOYMENT_NAME }}
+        run: |
+          make scheduled_tests
+
+      - name: Ensure the tests did not create any additional files
+        shell: bash
+        run: |
+          set -eu
+
+          STATUS="$(git status)"
+          echo "$STATUS"
+
+          # grep will exit non-zero if the target message isn't found,
+          # and `set -e` above will cause the step to fail.
+          echo "$STATUS" | grep 'nothing to commit, working tree clean'

.gitignore

@@ -30,6 +30,12 @@ share/python-wheels/
 *.egg
 MANIFEST
 
+# Google GitHub Actions credentials files created by:
+# https://github.com/google-github-actions/auth
+#
+# That action recommends adding this gitignore to prevent accidentally committing keys.
+gha-creds-*.json
+
 # PyInstaller
 #  Usually these files are written by a python script from a template
 #  before PyInstaller builds the exe, so as to inject date/other infos into it.
@@ -162,11 +168,13 @@ docs/.docusaurus/
 docs/.cache-loader/
 docs/_dist
 docs/api_reference/api_reference.rst
+docs/api_reference/experimental_api_reference.rst
 docs/api_reference/_build
 docs/api_reference/*/
 !docs/api_reference/_static/
 !docs/api_reference/templates/
 !docs/api_reference/themes/
-docs/docs_skeleton/build
-docs/docs_skeleton/node_modules
-docs/docs_skeleton/yarn.lock
+docs/docs/build
+docs/docs/node_modules
+docs/docs/yarn.lock
+_dist

.gitmodules

@@ -1,4 +0,0 @@
-[submodule "docs/_docs_skeleton"]
-	path = docs/_docs_skeleton
-	url = https://github.com/langchain-ai/langchain-shared-docs
-	branch = main

.readthedocs.yaml

@@ -9,9 +9,14 @@ build:
   os: ubuntu-22.04
   tools:
     python: "3.11"
-  jobs:
-    pre_build:
+  commands:
+      - python -mvirtualenv $READTHEDOCS_VIRTUALENV_PATH
+      - python -m pip install --upgrade --no-cache-dir pip setuptools
+      - python -m pip install --upgrade --no-cache-dir sphinx readthedocs-sphinx-ext
+      - python -m pip install --exists-action=w --no-cache-dir -r docs/api_reference/requirements.txt
       - python docs/api_reference/create_api_rst.py
+      - cat docs/api_reference/conf.py
+      - python -m sphinx -T -E -b html -d _build/doctrees -c docs/api_reference docs/api_reference $READTHEDOCS_OUTPUT/html -j auto
 
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
@@ -25,5 +30,3 @@ sphinx:
 python:
    install:
    - requirements: docs/api_reference/requirements.txt
-   - method: pip
-     path: .

CITATION.cff

@@ -5,4 +5,4 @@ authors:
   given-names: "Harrison"
 title: "LangChain"
 date-released: 2022-10-17
-url: "https://github.com/hwchase17/langchain"
+url: "https://github.com/langchain-ai/langchain"

MIGRATE.md

@@ -43,6 +43,10 @@ Now:
 
 `from langchain_experimental.sql import SQLDatabaseChain`
 
+Alternatively, if you are just interested in using the query generation part of the SQL chain, you can check out [`create_sql_query_chain`](https://github.com/langchain-ai/langchain/blob/master/docs/extras/use_cases/tabular/sql_query.ipynb)
+
+`from langchain.chains import create_sql_query_chain`
+
 ## `load_prompt` for Python files
 
 Note: this only applies if you want to load Python files as prompts.

Makefile

@@ -15,10 +15,10 @@ docs_build:
 	docs/.local_build.sh
 
 docs_clean:
-	rm -r docs/_dist
+	rm -r _dist
 
 docs_linkcheck:
-	poetry run linkchecker docs/_dist/docs_skeleton/ --ignore-url node_modules
+	poetry run linkchecker _dist/docs/ --ignore-url node_modules
 
 api_docs_build:
 	poetry run python docs/api_reference/create_api_rst.py
@@ -42,8 +42,15 @@ spell_fix:
 ######################
 
 help:
-	@echo '----'
-	@echo 'coverage                     - run unit tests and generate coverage report'
+	@echo '===================='
+	@echo '-- DOCUMENTATION --'
+	@echo 'clean                        - run docs_clean and api_docs_clean'
 	@echo 'docs_build                   - build the documentation'
 	@echo 'docs_clean                   - clean the documentation build artifacts'
 	@echo 'docs_linkcheck               - run linkchecker on the documentation'
+	@echo 'api_docs_build               - build the API Reference documentation'
+	@echo 'api_docs_clean               - clean the API Reference documentation build artifacts'
+	@echo 'api_docs_linkcheck           - run linkchecker on the API Reference documentation'
+	@echo 'spell_check               	- run codespell on the project'
+	@echo 'spell_fix               		- run codespell on the project and fix the errors'
+	@echo '-- TEST and LINT tasks are within libs/*/ per-package --'

README.md

@@ -2,31 +2,32 @@
 
 ⚡ Building applications with LLMs through composability ⚡
 
-[![Release Notes](https://img.shields.io/github/release/hwchase17/langchain)](https://github.com/hwchase17/langchain/releases)
-[![CI](https://github.com/hwchase17/langchain/actions/workflows/langchain_ci.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/langchain_ci.yml)
-[![Experimental CI](https://github.com/hwchase17/langchain/actions/workflows/langchain_experimental_ci.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/langchain_experimental_ci.yml)
+[![Release Notes](https://img.shields.io/github/release/langchain-ai/langchain)](https://github.com/langchain-ai/langchain/releases)
+[![CI](https://github.com/langchain-ai/langchain/actions/workflows/langchain_ci.yml/badge.svg)](https://github.com/langchain-ai/langchain/actions/workflows/langchain_ci.yml)
+[![Experimental CI](https://github.com/langchain-ai/langchain/actions/workflows/langchain_experimental_ci.yml/badge.svg)](https://github.com/langchain-ai/langchain/actions/workflows/langchain_experimental_ci.yml)
 [![Downloads](https://static.pepy.tech/badge/langchain/month)](https://pepy.tech/project/langchain)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![Twitter](https://img.shields.io/twitter/url/https/twitter.com/langchainai.svg?style=social&label=Follow%20%40LangChainAI)](https://twitter.com/langchainai)
 [![](https://dcbadge.vercel.app/api/server/6adMQxSpJS?compact=true&style=flat)](https://discord.gg/6adMQxSpJS)
-[![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/hwchase17/langchain)
-[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hwchase17/langchain)
-[![GitHub star chart](https://img.shields.io/github/stars/hwchase17/langchain?style=social)](https://star-history.com/#hwchase17/langchain)
-[![Dependency Status](https://img.shields.io/librariesio/github/hwchase17/langchain)](https://libraries.io/github/hwchase17/langchain)
-[![Open Issues](https://img.shields.io/github/issues-raw/hwchase17/langchain)](https://github.com/hwchase17/langchain/issues)
+[![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/langchain-ai/langchain)
+[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/langchain-ai/langchain)
+[![GitHub star chart](https://img.shields.io/github/stars/langchain-ai/langchain?style=social)](https://star-history.com/#langchain-ai/langchain)
+[![Dependency Status](https://img.shields.io/librariesio/github/langchain-ai/langchain)](https://libraries.io/github/langchain-ai/langchain)
+[![Open Issues](https://img.shields.io/github/issues-raw/langchain-ai/langchain)](https://github.com/langchain-ai/langchain/issues)
 
 
-Looking for the JS/TS version? Check out [LangChain.js](https://github.com/hwchase17/langchainjs).
+Looking for the JS/TS version? Check out [LangChain.js](https://github.com/langchain-ai/langchainjs).
 
-**Production Support:** As you move your LangChains into production, we'd love to offer more comprehensive support.
-Please fill out [this form](https://6w1pwbss0py.typeform.com/to/rrbrdTH2) and we'll set up a dedicated support Slack channel.
+To help you ship LangChain apps to production faster, check out [LangSmith](https://smith.langchain.com). 
+[LangSmith](https://smith.langchain.com) is a unified developer platform for building, testing, and monitoring LLM applications. 
+Fill out [this form](https://airtable.com/appwQzlErAS2qiP0L/shrGtGaVBVAz7NcV2) to get off the waitlist or speak with our sales team
 
-## 🚨Breaking Changes for select chains (SQLDatabase) on 7/28
+## 🚨Breaking Changes for select chains (SQLDatabase) on 7/28/23
 
 In an effort to make `langchain` leaner and safer, we are moving select chains to `langchain_experimental`.
 This migration has already started, but we are remaining backwards compatible until 7/28.
 On that date, we will remove functionality from `langchain`.
-Read more about the motivation and the progress [here](https://github.com/hwchase17/langchain/discussions/8043).
+Read more about the motivation and the progress [here](https://github.com/langchain-ai/langchain/discussions/8043).
 Read how to migrate your code [here](MIGRATE.md).
 
 ## Quick Install
@@ -49,7 +50,7 @@ This library aims to assist in the development of those types of applications. C
 **💬 Chatbots**
 
 - [Documentation](https://python.langchain.com/docs/use_cases/chatbots/)
-- End-to-end Example: [Chat-LangChain](https://github.com/hwchase17/chat-langchain)
+- End-to-end Example: [Chat-LangChain](https://github.com/langchain-ai/chat-langchain)
 
 **🤖 Agents**
 

SECURITY.md

@@ -0,0 +1,6 @@
+# Security Policy
+
+## Reporting a Vulnerability
+
+Please report security vulnerabilities by email to `security@langchain.dev`.
+This email is an alias to a subset of our maintainers, and will ensure the issue is promptly triaged and acted upon as needed.

cookbook/LLaMA2_sql_chat.ipynb

@@ -0,0 +1,400 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "fc935871-7640-41c6-b798-58514d860fe0",
+   "metadata": {},
+   "source": [
+    "## LLaMA2 chat with SQL\n",
+    "\n",
+    "Open source, local LLMs are great to consider for any application that demands data privacy.\n",
+    "\n",
+    "SQL is one good example. \n",
+    "\n",
+    "This cookbook shows how to perform text-to-SQL using various local versions of LLaMA2 run locally.\n",
+    "\n",
+    "## Packages"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "81adcf8b-395a-4f02-8749-ac976942b446",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "! pip install langchain replicate"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8e13ed66-300b-4a23-b8ac-44df68ee4733",
+   "metadata": {},
+   "source": [
+    "## LLM\n",
+    "\n",
+    "There are a few ways to access LLaMA2.\n",
+    "\n",
+    "To run locally, we use Ollama.ai. \n",
+    "\n",
+    "See [here](https://python.langchain.com/docs/integrations/chat/ollama) for details on installation and setup.\n",
+    "\n",
+    "Also, see [here](https://python.langchain.com/docs/guides/local_llms) for our full guide on local LLMs.\n",
+    " \n",
+    "To use an external API, which is not private, we can use Replicate."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "6a75a5c6-34ee-4ab9-a664-d9b432d812ee",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Init param `input` is deprecated, please use `model_kwargs` instead.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Local \n",
+    "from langchain.chat_models import ChatOllama\n",
+    "llama2_chat = ChatOllama(model=\"llama2:13b-chat\")\n",
+    "llama2_code = ChatOllama(model=\"codellama:7b-instruct\")\n",
+    "\n",
+    "# API\n",
+    "from getpass import getpass\n",
+    "from langchain.llms import Replicate\n",
+    "# REPLICATE_API_TOKEN = getpass()\n",
+    "# os.environ[\"REPLICATE_API_TOKEN\"] = REPLICATE_API_TOKEN\n",
+    "replicate_id = \"meta/llama-2-13b-chat:f4e2de70d66816a838a89eeeb621910adffb0dd0baba3976c96980970978018d\"\n",
+    "llama2_chat_replicate = Replicate(\n",
+    "    model=replicate_id,\n",
+    "    input={\"temperature\": 0.01, \n",
+    "           \"max_length\": 500, \n",
+    "           \"top_p\": 1}\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "ce96f7ea-b3d5-44e1-9fa5-a79e04a9e1fb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Simply set the LLM we want to use\n",
+    "llm = llama2_chat"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "80222165-f353-4e35-a123-5f70fd70c6c8",
+   "metadata": {},
+   "source": [
+    "## DB\n",
+    "\n",
+    "Connect to a SQLite DB.\n",
+    "\n",
+    "To create this particular DB, you can use the code and follow the steps shown [here](https://github.com/facebookresearch/llama-recipes/blob/main/demo_apps/StructuredLlama.ipynb)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "025bdd82-3bb1-4948-bc7c-c3ccd94fd05c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.utilities import SQLDatabase\n",
+    "db = SQLDatabase.from_uri(\"sqlite:///nba_roster.db\", sample_rows_in_table_info= 0)\n",
+    "\n",
+    "def get_schema(_):\n",
+    "    return db.get_table_info()\n",
+    "\n",
+    "def run_query(query):\n",
+    "    return db.run(query)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "654b3577-baa2-4e12-a393-f40e5db49ac7",
+   "metadata": {},
+   "source": [
+    "## Query a SQL DB \n",
+    "\n",
+    "Follow the runnables workflow [here](https://python.langchain.com/docs/expression_language/cookbook/sql_db)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "5a4933ea-d9c0-4b0a-8177-ba4490c6532b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "' SELECT \"Team\" FROM nba_roster WHERE \"NAME\" = \\'Klay Thompson\\';'"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Prompt\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "template = \"\"\"Based on the table schema below, write a SQL query that would answer the user's question:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query:\"\"\"\n",
+    "prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question, convert it to a SQL query. No pre-amble.\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "# Chain to query\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
+    "from langchain.schema.runnable import RunnablePassthrough\n",
+    "\n",
+    "sql_response = (\n",
+    "        RunnablePassthrough.assign(schema=get_schema)\n",
+    "        | prompt\n",
+    "        | llm.bind(stop=[\"\\nSQLResult:\"])\n",
+    "        | StrOutputParser()\n",
+    "    )\n",
+    "\n",
+    "sql_response.invoke({\"question\": \"What team is Klay Thompson on?\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a0e9e2c8-9b88-4853-ac86-001bc6cc6695",
+   "metadata": {},
+   "source": [
+    "We can review the results:\n",
+    "\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/afa56a06-b4e2-469a-a60f-c1746e75e42b/r) LLaMA2-13 Replicate API\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/2d4ecc72-6b8f-4523-8f0b-ea95c6b54a1d/r) LLaMA2-13 local \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "2a2825e3-c1b6-4f7d-b9c9-d9835de323bb",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=' Based on the table schema and SQL query, there are 30 unique teams in the NBA.')"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Chain to answer\n",
+    "template = \"\"\"Based on the table schema below, question, sql query, and sql response, write a natural language response:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query: {query}\n",
+    "SQL Response: {response}\"\"\"\n",
+    "prompt_response = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question and SQL response, convert it to a natural langugae answer. No pre-amble.\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "full_chain = (\n",
+    "    RunnablePassthrough.assign(query=sql_response) \n",
+    "    | RunnablePassthrough.assign(\n",
+    "        schema=get_schema,\n",
+    "        response=lambda x: db.run(x[\"query\"]),\n",
+    "    )\n",
+    "    | prompt_response \n",
+    "    | llm\n",
+    ")\n",
+    "\n",
+    "full_chain.invoke({\"question\": \"How many unique teams are there?\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ec17b3ee-6618-4681-b6df-089bbb5ffcd7",
+   "metadata": {},
+   "source": [
+    "We can review the results:\n",
+    "\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/10420721-746a-4806-8ecf-d6dc6399d739/r) LLaMA2-13 Replicate API\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/5265ebab-0a22-4f37-936b-3300f2dfa1c1/r) LLaMA2-13 local "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1e85381b-1edc-4bb3-a7bd-2ab23f81e54d",
+   "metadata": {},
+   "source": [
+    "## Chat with a SQL DB \n",
+    "\n",
+    "Next, we can add memory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "1985aa1c-eb8f-4fb1-a54f-c8aa10744687",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "' SELECT \"Team\" FROM nba_roster WHERE \"NAME\" = \\'Klay Thompson\\';'"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Prompt\n",
+    "from langchain.memory import ConversationBufferMemory\n",
+    "from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
+    "template = \"\"\"Based on the table schema below, write a SQL query that would answer the user's question:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query:\"\"\"\n",
+    "prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question, convert it to a SQL query. No pre-amble.\"),\n",
+    "    MessagesPlaceholder(variable_name=\"history\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "memory = ConversationBufferMemory(return_messages=True)\n",
+    "\n",
+    "# Chain to query with memory \n",
+    "from langchain.schema.runnable import RunnableLambda\n",
+    "\n",
+    "sql_chain = (\n",
+    "    RunnablePassthrough.assign(\n",
+    "       schema=get_schema,\n",
+    "       history=RunnableLambda(lambda x: memory.load_memory_variables(x)[\"history\"])\n",
+    "    )| prompt\n",
+    "    | llm.bind(stop=[\"\\nSQLResult:\"])\n",
+    "    | StrOutputParser()\n",
+    ")\n",
+    "\n",
+    "def save(input_output):\n",
+    "    output = {\"output\": input_output.pop(\"output\")}\n",
+    "    memory.save_context(input_output, output)\n",
+    "    return output['output']\n",
+    "    \n",
+    "sql_response_memory = RunnablePassthrough.assign(output=sql_chain) | save\n",
+    "sql_response_memory.invoke({\"question\": \"What team is Klay Thompson on?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "0b45818a-1498-441d-b82d-23c29428c2bb",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "' SELECT \"SALARY\" FROM nba_roster WHERE \"NAME\" = \\'Klay Thompson\\';'"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "sql_response_memory.invoke({\"question\": \"What is his salary?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "800a7a3b-f411-478b-af51-2310cd6e0425",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=' Sure! Here\\'s the natural language response based on the given input:\\n\\n\"Klay Thompson\\'s salary is $43,219,440.\"')"
+      ]
+     },
+     "execution_count": 21,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Chain to answer\n",
+    "template = \"\"\"Based on the table schema below, question, sql query, and sql response, write a natural language response:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query: {query}\n",
+    "SQL Response: {response}\"\"\"\n",
+    "prompt_response = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question and SQL response, convert it to a natural langugae answer. No pre-amble.\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "full_chain = (\n",
+    "    RunnablePassthrough.assign(query=sql_response_memory) \n",
+    "    | RunnablePassthrough.assign(\n",
+    "        schema=get_schema,\n",
+    "        response=lambda x: db.run(x[\"query\"]),\n",
+    "    )\n",
+    "    | prompt_response \n",
+    "    | llm\n",
+    ")\n",
+    "\n",
+    "full_chain.invoke({\"question\": \"What is his salary?\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b77fee61-f4da-4bb1-8285-14101e505518",
+   "metadata": {},
+   "source": [
+    "Here is the [trace](https://smith.langchain.com/public/54794d18-2337-4ce2-8b9f-3d8a2df89e51/r)."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/README.md

@@ -0,0 +1,52 @@
+# LangChain cookbook
+
+Example code for building applications with LangChain, with an emphasis on more applied and end-to-end examples than contained in the [main documentation](https://python.langchain.com).
+
+Notebook | Description
+:- | :-
+[LLaMA2_sql_chat.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/LLaMA2_sql_chat.ipynb) | Build a chat application that interacts with a sql database using an open source llm (llama2), specifically demonstrated on a sqlite database containing nba rosters.
+[Semi_Structured_RAG.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_Structured_RAG.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data, including text and tables, using unstructured for parsing, multi-vector retriever for storing, and lcel for implementing chains.
+[Semi_structured_and_multi_moda...](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_structured_and_multi_modal_RAG.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using unstructured for parsing, multi-vector retriever for storage and retrieval, and lcel for implementing chains.
+[Semi_structured_multi_modal_RA...](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_structured_multi_modal_RAG_LLaMA2.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using various tools and methods such as unstructured for parsing, multi-vector retriever for storing, lcel for implementing chains, and open source language models like llama2, llava, and gpt4all.
+[autogpt/autogpt.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/autogpt/autogpt.ipynb) | Implement autogpt, a language model, with langchain primitives such as llms, prompttemplates, vectorstores, embeddings, and tools.
+[autogpt/marathon_times.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/autogpt/marathon_times.ipynb) | Implement autogpt for finding winning marathon times.
+[baby_agi.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/baby_agi.ipynb) | Implement babyagi, an ai agent that can generate and execute tasks based on a given objective, with the flexibility to swap out specific vectorstores/model providers.
+[baby_agi_with_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/baby_agi_with_agent.ipynb) | Swap out the execution chain in the babyagi notebook with an agent that has access to tools, aiming to obtain more reliable information.
+[camel_role_playing.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/camel_role_playing.ipynb) | Implement the camel framework for creating autonomous cooperative agents in large scale language models, using role-playing and inception prompting to guide chat agents towards task completion.
+[causal_program_aided_language_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/causal_program_aided_language_model.ipynb) | Implement the causal program-aided language (cpal) chain, which improves upon the program-aided language (pal) by incorporating causal structure to prevent hallucination in language models, particularly when dealing with complex narratives and math problems with nested dependencies.
+[code-analysis-deeplake.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/code-analysis-deeplake.ipynb) | Analyze its own code base with the help of gpt and activeloop's deep lake.
+[custom_agent_with_plugin_retri...](https://github.com/langchain-ai/langchain/tree/master/cookbook/custom_agent_with_plugin_retrieval.ipynb) | Build a custom agent that can interact with ai plugins by retrieving tools and creating natural language wrappers around openapi endpoints.
+[custom_agent_with_plugin_retri...](https://github.com/langchain-ai/langchain/tree/master/cookbook/custom_agent_with_plugin_retrieval_using_plugnplai.ipynb) | Build a custom agent with plugin retrieval functionality, utilizing ai plugins from the `plugnplai` directory.
+[databricks_sql_db.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/databricks_sql_db.ipynb) | Connect to databricks runtimes and databricks sql.
+[deeplake_semantic_search_over_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/deeplake_semantic_search_over_chat.ipynb) | Perform semantic search and question-answering over a group chat using activeloop's deep lake with gpt4.
+[elasticsearch_db_qa.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/elasticsearch_db_qa.ipynb) | Interact with elasticsearch analytics databases in natural language and build search queries via the elasticsearch dsl api.
+[forward_looking_retrieval_augm...](https://github.com/langchain-ai/langchain/tree/master/cookbook/forward_looking_retrieval_augmented_generation.ipynb) | Implement the forward-looking active retrieval augmented generation (flare) method, which generates answers to questions, identifies uncertain tokens, generates hypothetical questions based on these tokens, and retrieves relevant documents to continue generating the answer.
+[generative_agents_interactive_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/generative_agents_interactive_simulacra_of_human_behavior.ipynb) | Implement a generative agent that simulates human behavior, based on a research paper, using a time-weighted memory object backed by a langchain retriever.
+[gymnasium_agent_simulation.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/gymnasium_agent_simulation.ipynb) | Create a simple agent-environment interaction loop in simulated environments like text-based games with gymnasium.
+[hugginggpt.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/hugginggpt.ipynb) | Implement hugginggpt, a system that connects language models like chatgpt with the machine learning community via hugging face.
+[hypothetical_document_embeddin...](https://github.com/langchain-ai/langchain/tree/master/cookbook/hypothetical_document_embeddings.ipynb) | Improve document indexing with hypothetical document embeddings (hyde), an embedding technique that generates and embeds hypothetical answers to queries.
+[learned_prompt_optimization.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/learned_prompt_optimization.ipynb) | Automatically enhance language model prompts by injecting specific terms using reinforcement learning, which can be used to personalize responses based on user preferences.
+[llm_bash.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_bash.ipynb) | Perform simple filesystem commands using language learning models (llms) and a bash process.
+[llm_checker.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_checker.ipynb) | Create a self-checking chain using the llmcheckerchain function.
+[llm_math.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_math.ipynb) | Solve complex word math problems using language models and python repls.
+[llm_summarization_checker.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_summarization_checker.ipynb) | Check the accuracy of text summaries, with the option to run the checker multiple times for improved results.
+[llm_symbolic_math.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_symbolic_math.ipynb) | Solve algebraic equations with the help of llms (language learning models) and sympy, a python library for symbolic mathematics.
+[meta_prompt.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/meta_prompt.ipynb) | Implement the meta-prompt concept, which is a method for building self-improving agents that reflect on their own performance and modify their instructions accordingly.
+[multi_modal_output_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multi_modal_output_agent.ipynb) | Generate multi-modal outputs, specifically images and text.
+[multi_player_dnd.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multi_player_dnd.ipynb) | Simulate multi-player dungeons & dragons games, with a custom function determining the speaking schedule of the agents.
+[multiagent_authoritarian.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multiagent_authoritarian.ipynb) | Implement a multi-agent simulation where a privileged agent controls the conversation, including deciding who speaks and when the conversation ends, in the context of a simulated news network.
+[multiagent_bidding.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multiagent_bidding.ipynb) | Implement a multi-agent simulation where agents bid to speak, with the highest bidder speaking next, demonstrated through a fictitious presidential debate example.
+[myscale_vector_sql.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/myscale_vector_sql.ipynb) | Access and interact with the myscale integrated vector database, which can enhance the performance of language model (llm) applications.
+[openai_functions_retrieval_qa....](https://github.com/langchain-ai/langchain/tree/master/cookbook/openai_functions_retrieval_qa.ipynb) | Structure response output in a question answering system by incorporating openai functions into a retrieval pipeline.
+[petting_zoo.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/petting_zoo.ipynb) | Create multi-agent simulations with simulated environments using the petting zoo library.
+[plan_and_execute_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/plan_and_execute_agent.ipynb) | Create plan-and-execute agents that accomplish objectives by planning tasks with a language model (llm) and executing them with a separate agent.
+[press_releases.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/press_releases.ipynb) | Retrieve and query company press release data powered by [Kay.ai](https://kay.ai).
+[program_aided_language_model.i...](https://github.com/langchain-ai/langchain/tree/master/cookbook/program_aided_language_model.ipynb) | Implement program-aided language models as described in the provided research paper.
+[sales_agent_with_context.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/sales_agent_with_context.ipynb) | Implement a context-aware ai sales agent, salesgpt, that can have natural sales conversations, interact with other systems, and use a product knowledge base to discuss a company's offerings.
+[self_query_hotel_search.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/self_query_hotel_search.ipynb) | Build a hotel room search feature with self-querying retrieval, using a specific hotel recommendation dataset.
+[smart_llm.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/smart_llm.ipynb) | Implement a smartllmchain, a self-critique chain that generates multiple output proposals, critiques them to find the best one, and then improves upon it to produce a final output.
+[tree_of_thought.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/tree_of_thought.ipynb) | Query a large language model using the tree of thought technique.
+[twitter-the-algorithm-analysis...](https://github.com/langchain-ai/langchain/tree/master/cookbook/twitter-the-algorithm-analysis-deeplake.ipynb) | Analyze the source code of the twitter algorithm with the help of gpt4 and activeloop's deep lake.
+[two_agent_debate_tools.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_agent_debate_tools.ipynb) | Simulate multi-agent dialogues where the agents can utilize various tools.
+[two_player_dnd.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_player_dnd.ipynb) | Simulate a two-player dungeons & dragons game, where a dialoguesimulator class is used to coordinate the dialogue between the protagonist and the dungeon master.
+[wikibase_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/wikibase_agent.ipynb) | Create a simple wikibase agent that utilizes sparql generation, with testing done on http://wikidata.org.

cookbook/autogpt/autogpt.ipynb

@@ -100,7 +100,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.experimental import AutoGPT\n",
+    "from langchain_experimental.autonomous_agents import AutoGPT\n",
     "from langchain.chat_models import ChatOpenAI"
    ]
   },
@@ -124,43 +124,47 @@
   },
   {
    "cell_type": "markdown",
+   "id": "f0f208d9",
+   "metadata": {
+    "collapsed": false
+   },
    "source": [
     "## Run an example\n",
     "\n",
     "Here we will make it write a weather report for SF"
-   ],
-   "metadata": {
-    "collapsed": false
-   },
-   "id": "f0f208d9"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "outputs": [],
-   "source": [
-    "agent.run([\"write a weather report for SF today\"])"
-   ],
+   "id": "d119d788",
    "metadata": {
     "collapsed": false
    },
-   "id": "d119d788"
+   "outputs": [],
+   "source": [
+    "agent.run([\"write a weather report for SF today\"])"
+   ]
   },
   {
    "cell_type": "markdown",
+   "id": "f13f8322",
+   "metadata": {
+    "collapsed": false
+   },
    "source": [
     "## Chat History Memory\n",
     "\n",
     "In addition to the memory that holds the agent immediate steps, we also have a chat history memory. By default, the agent will use 'ChatMessageHistory' and it can be changed. This is useful when you want to use a different type of memory for example 'FileChatHistoryMemory'"
-   ],
-   "metadata": {
-    "collapsed": false
-   },
-   "id": "f13f8322"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "2a81f5ad",
+   "metadata": {
+    "collapsed": false
+   },
    "outputs": [],
    "source": [
     "from langchain.memory.chat_message_histories import FileChatMessageHistory\n",
@@ -173,19 +177,15 @@
     "    memory=vectorstore.as_retriever(),\n",
     "    chat_history_memory=FileChatMessageHistory(\"chat_history.txt\"),\n",
     ")"
-   ],
-   "metadata": {
-    "collapsed": false
-   },
-   "id": "2a81f5ad"
+   ]
   },
   {
    "cell_type": "markdown",
-   "source": [],
+   "id": "b1403008",
    "metadata": {
     "collapsed": false
    },
-   "id": "b1403008"
+   "source": []
   }
  ],
  "metadata": {
@@ -209,4 +209,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}

cookbook/autogpt/marathon_times.ipynb

@@ -36,7 +36,7 @@
     "# General\n",
     "import os\n",
     "import pandas as pd\n",
-    "from langchain.experimental.autonomous_agents.autogpt.agent import AutoGPT\n",
+    "from langchain_experimental.autonomous_agents import AutoGPT\n",
     "from langchain.chat_models import ChatOpenAI\n",
     "\n",
     "from langchain.agents.agent_toolkits.pandas.base import create_pandas_dataframe_agent\n",

cookbook/baby_agi.ipynb

@@ -33,13 +33,13 @@
     "from collections import deque\n",
     "from typing import Dict, List, Optional, Any\n",
     "\n",
-    "from langchain import LLMChain, OpenAI, PromptTemplate\n",
+    "from langchain.chains import LLMChain\nfrom langchain.llms import OpenAI\nfrom langchain.prompts import PromptTemplate\n",
     "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.llms import BaseLLM\n",
-    "from langchain.vectorstores.base import VectorStore\n",
+    "from langchain.schema.vectorstore import VectorStore\n",
     "from pydantic import BaseModel, Field\n",
     "from langchain.chains.base import Chain\n",
-    "from langchain.experimental import BabyAGI"
+    "from langchain_experimental.autonomous_agents import BabyAGI"
    ]
   },
   {
@@ -249,7 +249,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.9.16"
   }
  },
  "nbformat": 4,

cookbook/baby_agi_with_agent.ipynb

@@ -29,13 +29,13 @@
     "from collections import deque\n",
     "from typing import Dict, List, Optional, Any\n",
     "\n",
-    "from langchain import LLMChain, OpenAI, PromptTemplate\n",
+    "from langchain.chains import LLMChain\nfrom langchain.llms import OpenAI\nfrom langchain.prompts import PromptTemplate\n",
     "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.llms import BaseLLM\n",
-    "from langchain.vectorstores.base import VectorStore\n",
+    "from langchain.schema.vectorstore import VectorStore\n",
     "from pydantic import BaseModel, Field\n",
     "from langchain.chains.base import Chain\n",
-    "from langchain.experimental import BabyAGI"
+    "from langchain_experimental.autonomous_agents import BabyAGI"
    ]
   },
   {
@@ -111,7 +111,7 @@
    "outputs": [],
    "source": [
     "from langchain.agents import ZeroShotAgent, Tool, AgentExecutor\n",
-    "from langchain import OpenAI, SerpAPIWrapper, LLMChain\n",
+    "from langchain.llms import OpenAI\nfrom langchain.utilities import SerpAPIWrapper\nfrom langchain.chains import LLMChain\n",
     "\n",
     "todo_prompt = PromptTemplate.from_template(\n",
     "    \"You are a planner who is an expert at coming up with a todo list for a given objective. Come up with a todo list for this objective: {objective}\"\n",

cookbook/camel_role_playing.ipynb

@@ -282,7 +282,7 @@
     "user_agent.reset()\n",
     "\n",
     "# Initialize chats\n",
-    "assistant_msg = HumanMessage(\n",
+    "user_msg = HumanMessage(\n",
     "    content=(\n",
     "        f\"{user_sys_msg.content}. \"\n",
     "        \"Now start to give me introductions one by one. \"\n",
@@ -290,8 +290,8 @@
     "    )\n",
     ")\n",
     "\n",
-    "user_msg = HumanMessage(content=f\"{assistant_sys_msg.content}\")\n",
-    "user_msg = assistant_agent.step(user_msg)"
+    "assistant_msg = HumanMessage(content=f\"{assistant_sys_msg.content}\")\n",
+    "assistant_msg = assistant_agent.step(user_msg)"
    ]
   },
   {

cookbook/custom_agent_with_plugin_retrieval.ipynb

@@ -10,7 +10,7 @@
     "This notebook combines two concepts in order to build a custom agent that can interact with AI Plugins:\n",
     "\n",
     "1. [Custom Agent with Tool Retrieval](/docs/modules/agents/how_to/custom_agent_with_tool_retrieval.html): This introduces the concept of retrieving many tools, which is useful when trying to work with arbitrarily many plugins.\n",
-    "2. [Natural Language API Chains](/docs/modules/chains/additional/openapi.html): This creates Natural Language wrappers around OpenAPI endpoints. This is useful because (1) plugins use OpenAPI endpoints under the hood, (2) wrapping them in an NLAChain allows the router agent to call it more easily.\n",
+    "2. [Natural Language API Chains](/docs/use_cases/apis/openapi.html): This creates Natural Language wrappers around OpenAPI endpoints. This is useful because (1) plugins use OpenAPI endpoints under the hood, (2) wrapping them in an NLAChain allows the router agent to call it more easily.\n",
     "\n",
     "The novel idea introduced in this notebook is the idea of using retrieval to select not the tools explicitly, but the set of OpenAPI specs to use. We can then generate tools from those OpenAPI specs. The use case for this is when trying to get agents to use plugins. It may be more efficient to choose plugins first, then the endpoints, rather than the endpoints directly. This is because the plugins may contain more useful information for selection."
    ]
@@ -39,7 +39,7 @@
     "    AgentOutputParser,\n",
     ")\n",
     "from langchain.prompts import StringPromptTemplate\n",
-    "from langchain import OpenAI, SerpAPIWrapper, LLMChain\n",
+    "from langchain.llms import OpenAI\nfrom langchain.utilities import SerpAPIWrapper\nfrom langchain.chains import LLMChain\n",
     "from typing import List, Union\n",
     "from langchain.schema import AgentAction, AgentFinish\n",
     "from langchain.agents.agent_toolkits import NLAToolkit\n",

cookbook/custom_agent_with_plugin_retrieval_using_plugnplai.ipynb

@@ -63,7 +63,7 @@
     "    AgentOutputParser,\n",
     ")\n",
     "from langchain.prompts import StringPromptTemplate\n",
-    "from langchain import OpenAI, SerpAPIWrapper, LLMChain\n",
+    "from langchain.llms import OpenAI\nfrom langchain.utilities import SerpAPIWrapper\nfrom langchain.chains import LLMChain\n",
     "from typing import List, Union\n",
     "from langchain.schema import AgentAction, AgentFinish\n",
     "from langchain.agents.agent_toolkits import NLAToolkit\n",

cookbook/databricks_sql_db.ipynb

@@ -80,7 +80,7 @@
    "outputs": [],
    "source": [
     "# Connecting to Databricks with SQLDatabase wrapper\n",
-    "from langchain import SQLDatabase\n",
+    "from langchain.utilities import SQLDatabase\n",
     "\n",
     "db = SQLDatabase.from_databricks(catalog=\"samples\", schema=\"nyctaxi\")"
    ]
@@ -115,7 +115,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain import SQLDatabaseChain\n",
+    "from langchain.utilities import SQLDatabaseChain\n",
     "\n",
     "db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)"
    ]
@@ -166,7 +166,7 @@
    "source": [
     "### SQL Database Agent example\n",
     "\n",
-    "This example demonstrates the use of the [SQL Database Agent](/docs/modules/agents/toolkits/sql_database.html) for answering questions over a Databricks database."
+    "This example demonstrates the use of the [SQL Database Agent](/docs/integrations/toolkits/sql_database.html) for answering questions over a Databricks database."
    ]
   },
   {

cookbook/deeplake_semantic_search_over_chat.ipynb

@@ -5,7 +5,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Question answering over a group chat messages using Activeloop's DeepLake\n",
+    "# QA using Activeloop's DeepLake\n",
     "In this tutorial, we are going to use Langchain + Activeloop's Deep Lake with GPT4 to semantically search and ask questions over a group chat.\n",
     "\n",
     "View a working demo [here](https://twitter.com/thisissukh_/status/1647223328363679745)"
@@ -102,9 +102,52 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 4,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[Document(page_content='Participants:\\n\\nJerry: Loves movies and is a bit of a klutz.\\nSamantha: Enthusiastic about food and always trying new restaurants.\\nBarry: A nature lover, but always manages to get lost.\\nJerry: Hey, guys! You won\\'t believe what happened to me at the Times Square AMC theater. I tripped over my own feet and spilled popcorn everywhere! 🍿💥\\n\\nSamantha: LOL, that\\'s so you, Jerry! Was the floor buttery enough for you to ice skate on after that? 😂\\n\\nBarry: Sounds like a regular Tuesday for you, Jerry. Meanwhile, I tried to find that new hiking trail in Central Park. You know, the one that\\'s supposed to be impossible to get lost on? Well, guess what...\\n\\nJerry: You found a hidden treasure?\\n\\nBarry: No, I got lost. AGAIN. 🧭🙄\\n\\nSamantha: Barry, you\\'d get lost in your own backyard! But speaking of treasures, I found this new sushi place in Little Tokyo. \"Samantha\\'s Sushi Symphony\" it\\'s called. Coincidence? I think not!\\n\\nJerry: Maybe they named it after your ability to eat your body weight in sushi. 🍣', metadata={}), Document(page_content='Barry: How do you even FIND all these places, Samantha?\\n\\nSamantha: Simple, I don\\'t rely on Barry\\'s navigation skills. 😉 But seriously, the wasabi there was hotter than Jerry\\'s love for Marvel movies!\\n\\nJerry: Hey, nothing wrong with a little superhero action. By the way, did you guys see the new \"Captain Crunch: Breakfast Avenger\" trailer?\\n\\nSamantha: Captain Crunch? Are you sure you didn\\'t get that from one of your Saturday morning cereal binges?\\n\\nBarry: Yeah, and did he defeat his arch-enemy, General Mills? 😆\\n\\nJerry: Ha-ha, very funny. Anyway, that sushi place sounds awesome, Samantha. Next time, let\\'s go together, and maybe Barry can guide us... if we want a city-wide tour first.\\n\\nBarry: As long as we\\'re not hiking, I\\'ll get us there... eventually. 😅\\n\\nSamantha: It\\'s a date! But Jerry, you\\'re banned from carrying any food items.\\n\\nJerry: Deal! Just promise me no wasabi challenges. I don\\'t want to end up like the time I tried Sriracha ice cream.', metadata={}), Document(page_content=\"Barry: Wait, what happened with Sriracha ice cream?\\n\\nJerry: Let's just say it was a hot situation. Literally. 🔥\\n\\nSamantha: 🤣 I still have the video!\\n\\nJerry: Samantha, if you value our friendship, that video will never see the light of day.\\n\\nSamantha: No promises, Jerry. No promises. 🤐😈\\n\\nBarry: I foresee a fun weekend ahead! 🎉\", metadata={})]\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Your Deep Lake dataset has been successfully created!\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\\"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Dataset(path='hub://adilkhan/data', tensors=['embedding', 'id', 'metadata', 'text'])\n",
+      "\n",
+      "  tensor      htype      shape     dtype  compression\n",
+      "  -------    -------    -------   -------  ------- \n",
+      " embedding  embedding  (3, 1536)  float32   None   \n",
+      "    id        text      (3, 1)      str     None   \n",
+      " metadata     json      (3, 1)      str     None   \n",
+      "   text       text      (3, 1)      str     None   \n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      " \r"
+     ]
+    }
+   ],
    "source": [
     "with open(\"messages.txt\") as f:\n",
     "    state_of_the_union = f.read()\n",
@@ -116,7 +159,7 @@
     "\n",
     "print(texts)\n",
     "\n",
-    "dataset_path = \"hub://\" + org + \"/data\"\n",
+    "dataset_path = \"hub://\" + org_id + \"/data\"\n",
     "embeddings = OpenAIEmbeddings()\n",
     "db = DeepLake.from_documents(\n",
     "    texts, embeddings, dataset_path=dataset_path, overwrite=True\n",
@@ -133,7 +176,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -150,7 +193,7 @@
     "# dataset_path = \"hub://\" + org + \"/data\"\n",
     "# embeddings = OpenAIEmbeddings()\n",
     "# db = DeepLake.from_documents(\n",
-    "#     texts, embeddings, dataset_path=dataset_path, overwrite=True, runtime=\"tensor_db\"\n",
+    "#     texts, embeddings, dataset_path=dataset_path, overwrite=True, runtime={\"tensor_db\": True}\n",
     "# )"
    ]
   },
@@ -170,7 +213,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "db = DeepLake(dataset_path=dataset_path, read_only=True, embedding_function=embeddings)\n",
+    "db = DeepLake(dataset_path=dataset_path, read_only=True, embedding=embeddings)\n",
     "\n",
     "retriever = db.as_retriever()\n",
     "retriever.search_kwargs[\"distance_metric\"] = \"cos\"\n",
@@ -188,13 +231,6 @@
     "\n",
     "print(ans)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
@@ -213,7 +249,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

cookbook/elasticsearch_db_qa.ipynb

@@ -2,41 +2,50 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "dd7ec7af",
    "metadata": {},
    "source": [
-    "# Elasticsearch database\n",
+    "# Elasticsearch\n",
     "\n",
-    "Interact with Elasticsearch analytics database via Langchain. This chain builds search queries via the Elasticsearch DSL API (filters and aggregations).\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/qa_structured/integrations/elasticsearch.ipynb)\n",
+    "\n",
+    "We can use LLMs to interact with Elasticsearch analytics databases in natural language.\n",
+    "\n",
+    "This chain builds search queries via the Elasticsearch DSL API (filters and aggregations).\n",
     "\n",
     "The Elasticsearch client must have permissions for index listing, mapping description and search queries.\n",
     "\n",
-    "See [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html) for instructions on how to run Elasticsearch locally.\n",
-    "\n",
-    "Make sure to install the Elasticsearch Python client before:\n",
-    "\n",
-    "```sh\n",
-    "pip install elasticsearch\n",
-    "```"
+    "See [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html) for instructions on how to run Elasticsearch locally."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
-   "id": "dd8eae75",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "! pip install langchain langchain-experimental openai elasticsearch\n",
+    "\n",
+    "# Set env var OPENAI_API_KEY or load from a .env file\n",
+    "# import dotenv\n",
+    "\n",
+    "# dotenv.load_dotenv()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
    "metadata": {},
    "outputs": [],
    "source": [
     "from elasticsearch import Elasticsearch\n",
     "\n",
-    "from langchain.chains.elasticsearch_database import ElasticsearchDatabaseChain\n",
-    "from langchain.chat_models import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.chains.elasticsearch_database import ElasticsearchDatabaseChain"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
-   "id": "5cde03bc",
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -48,7 +57,6 @@
   },
   {
    "cell_type": "markdown",
-   "id": "74a41374",
    "metadata": {},
    "source": [
     "Uncomment the next cell to initially populate your db."
@@ -56,8 +64,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
-   "id": "430ada0f",
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -74,8 +81,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
-   "id": "f36ae0d8",
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -85,35 +91,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
-   "id": "b5d22d9d",
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new ElasticsearchDatabaseChain chain...\u001b[0m\n",
-      "What are the first names of all the customers?\n",
-      "ESQuery:\u001b[32;1m\u001b[1;3m{'size': 10, 'query': {'match_all': {}}, '_source': ['firstname']}\u001b[0m\n",
-      "ESResult: \u001b[33;1m\u001b[1;3m{'took': 5, 'timed_out': False, '_shards': {'total': 1, 'successful': 1, 'skipped': 0, 'failed': 0}, 'hits': {'total': {'value': 6, 'relation': 'eq'}, 'max_score': 1.0, 'hits': [{'_index': 'customers', '_id': '0', '_score': 1.0, '_source': {'firstname': 'Jennifer'}}, {'_index': 'customers', '_id': '1', '_score': 1.0, '_source': {'firstname': 'Monica'}}, {'_index': 'customers', '_id': '2', '_score': 1.0, '_source': {'firstname': 'Carol'}}, {'_index': 'customers', '_id': '3', '_score': 1.0, '_source': {'firstname': 'Wanda'}}, {'_index': 'customers', '_id': '4', '_score': 1.0, '_source': {'firstname': 'Jennifer'}}, {'_index': 'customers', '_id': 'firstname', '_score': 1.0, '_source': {'firstname': 'Jennifer'}}]}}\u001b[0m\n",
-      "Answer:\u001b[32;1m\u001b[1;3mThe first names of all the customers are Jennifer, Monica, Carol, Wanda, and Jennifer.\u001b[0m\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'The first names of all the customers are Jennifer, Monica, Carol, Wanda, and Jennifer.'"
-      ]
-     },
-     "execution_count": 13,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "question = \"What are the first names of all the customers?\"\n",
     "chain.run(question)"
@@ -121,18 +101,14 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9b4bfada",
    "metadata": {},
    "source": [
-    "## Custom prompt\n",
-    "\n",
-    "For best results you'll likely need to customize the prompt."
+    "We can customize the prompt."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
-   "id": "0a494f5b",
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -156,37 +132,13 @@
     ")\n",
     "chain = ElasticsearchDatabaseChain.from_llm(llm=llm, database=db, query_prompt=PROMPT)"
    ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "372b8f93",
-   "metadata": {},
-   "source": [
-    "## Adding example rows from each index\n",
-    "\n",
-    "Sometimes, the format of the data is not obvious and it is optimal to include a sample of rows from the indices in the prompt to allow the LLM to understand the data before providing a final query. Here we will use this feature to let the LLM know that artists are saved with their full names by providing ten rows from the index."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "eef818de",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = ElasticsearchDatabaseChain.from_llm(\n",
-    "    llm=ChatOpenAI(temperature=0),\n",
-    "    database=db,\n",
-    "    sample_documents_in_index_info=2,  # 2 rows from each index will be included in the prompt as sample data\n",
-    ")"
-   ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "venv",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "venv"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -198,9 +150,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 5
+ "nbformat_minor": 4
 }

cookbook/forward_looking_retrieval_augmented_generation.ipynb

@@ -5,7 +5,7 @@
    "id": "0f0b9afa",
    "metadata": {},
    "source": [
-    "# FLARE\n",
+    "# Retrieve as you generate with FLARE\n",
     "\n",
     "This notebook is an implementation of Forward-Looking Active REtrieval augmented generation (FLARE).\n",
     "\n",
@@ -56,8 +56,7 @@
    "source": [
     "import os\n",
     "\n",
-    "os.environ[\"SERPER_API_KEY\"] = \"\"",
-    "os.environ[\"OPENAI_API_KEY\"] = \"\""
+    "os.environ[\"SERPER_API_KEY\"] = \"\"os.environ[\"OPENAI_API_KEY\"] = \"\""
    ]
   },
   {
@@ -136,9 +135,9 @@
    "outputs": [],
    "source": [
     "# We set this so we can see what exactly is going on\n",
-    "import langchain\n",
+    "from langchain.globals import set_verbose\n",
     "\n",
-    "langchain.verbose = True"
+    "set_verbose(True)"
    ]
   },
   {
@@ -490,7 +489,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.1"
   }
  },
  "nbformat": 4,

cookbook/generative_agents_interactive_simulacra_of_human_behavior.ipynb

@@ -101,7 +101,7 @@
    },
    "outputs": [],
    "source": [
-    "from langchain.experimental.generative_agents import (\n",
+    "from langchain_experimental.generative_agents import (\n",
     "    GenerativeAgent,\n",
     "    GenerativeAgentMemory,\n",
     ")"

cookbook/hypothetical_document_embeddings.ipynb

@@ -5,7 +5,7 @@
    "id": "ccb74c9b",
    "metadata": {},
    "source": [
-    "# Hypothetical Document Embeddings\n",
+    "# Improve document indexing with HyDE\n",
     "This notebook goes over how to use Hypothetical Document Embeddings (HyDE), as described in [this paper](https://arxiv.org/abs/2212.10496). \n",
     "\n",
     "At a high level, HyDE is an embedding technique that takes queries, generates a hypothetical answer, and then embeds that generated document and uses that as the final example. \n",
@@ -255,7 +255,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.11.3"
   },
   "vscode": {
    "interpreter": {

cookbook/llm_bash.ipynb

@@ -10,7 +10,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [
     {
@@ -37,13 +37,13 @@
        "'Hello World\\n'"
       ]
      },
-     "execution_count": 9,
+     "execution_count": 1,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "from langchain.chains import LLMBashChain\n",
+    "from langchain_experimental.llm_bash.base import LLMBashChain\n",
     "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
@@ -65,7 +65,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -98,7 +98,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -125,7 +125,7 @@
        "'Hello World\\n'"
       ]
      },
-     "execution_count": 11,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -149,7 +149,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -166,28 +166,24 @@
       "cd ..\n",
       "```\u001b[0m\n",
       "Code: \u001b[33;1m\u001b[1;3m['ls', 'cd ..']\u001b[0m\n",
-      "Answer: \u001b[33;1m\u001b[1;3mapi.html\t\t\tllm_summarization_checker.html\n",
-      "constitutional_chain.html\tmoderation.html\n",
-      "llm_bash.html\t\t\topenai_openapi.yaml\n",
-      "llm_checker.html\t\topenapi.html\n",
-      "llm_math.html\t\t\tpal.html\n",
-      "llm_requests.html\t\tsqlite.html\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3mcpal.ipynb  llm_bash.ipynb  llm_symbolic_math.ipynb\n",
+      "index.mdx   llm_math.ipynb  pal.ipynb\u001b[0m\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
     },
     {
      "data": {
       "text/plain": [
-       "'api.html\\t\\t\\tllm_summarization_checker.html\\r\\nconstitutional_chain.html\\tmoderation.html\\r\\nllm_bash.html\\t\\t\\topenai_openapi.yaml\\r\\nllm_checker.html\\t\\topenapi.html\\r\\nllm_math.html\\t\\t\\tpal.html\\r\\nllm_requests.html\\t\\tsqlite.html'"
+       "'cpal.ipynb  llm_bash.ipynb  llm_symbolic_math.ipynb\\r\\nindex.mdx   llm_math.ipynb  pal.ipynb'"
       ]
      },
-     "execution_count": 12,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "from langchain.utilities.bash import BashProcess\n",
+    "from langchain_experimental.llm_bash.bash import BashProcess\n",
     "\n",
     "\n",
     "persistent_process = BashProcess(persistent=True)\n",
@@ -200,7 +196,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -217,18 +213,19 @@
       "cd ..\n",
       "```\u001b[0m\n",
       "Code: \u001b[33;1m\u001b[1;3m['ls', 'cd ..']\u001b[0m\n",
-      "Answer: \u001b[33;1m\u001b[1;3mexamples\t\tgetting_started.html\tindex_examples\n",
-      "generic\t\t\thow_to_guides.rst\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3m_category_.yml\tdata_generation.ipynb\t\t   self_check\n",
+      "agents\t\tgraph\n",
+      "code_writing\tlearned_prompt_optimization.ipynb\u001b[0m\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
     },
     {
      "data": {
       "text/plain": [
-       "'examples\\t\\tgetting_started.html\\tindex_examples\\r\\ngeneric\\t\\t\\thow_to_guides.rst'"
+       "'_category_.yml\\tdata_generation.ipynb\\t\\t   self_check\\r\\nagents\\t\\tgraph\\r\\ncode_writing\\tlearned_prompt_optimization.ipynb'"
       ]
      },
-     "execution_count": 13,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -237,13 +234,6 @@
     "# Run the same command again and see that the state is maintained between calls\n",
     "bash_chain.run(text)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
@@ -262,7 +252,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.11.4"
   }
  },
  "nbformat": 4,

cookbook/llm_math.ipynb

@@ -45,7 +45,7 @@
     }
    ],
    "source": [
-    "from langchain import OpenAI, LLMMathChain\n",
+    "from langchain.llms import OpenAI\nfrom langchain.chains import LLMMathChain\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "llm_math = LLMMathChain.from_llm(llm, verbose=True)\n",

cookbook/llm_symbolic_math.ipynb

@@ -10,12 +10,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
     "from langchain.llms import OpenAI\n",
-    "from langchain.chains.llm_symbolic_math.base import LLMSymbolicMathChain\n",
+    "from langchain_experimental.llm_symbolic_math.base import LLMSymbolicMathChain\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "llm_symbolic_math = LLMSymbolicMathChain.from_llm(llm)"
@@ -30,7 +30,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 23,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -39,7 +39,7 @@
        "'Answer: exp(x)*sin(x) + exp(x)*cos(x)'"
       ]
      },
-     "execution_count": 23,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -50,7 +50,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -59,7 +59,7 @@
        "'Answer: exp(x)*sin(x)'"
       ]
      },
-     "execution_count": 18,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -79,7 +79,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
@@ -88,7 +88,7 @@
        "'Answer: Eq(y(t), C2*exp(-t) + (C1 + t/2)*exp(t))'"
       ]
      },
-     "execution_count": 19,
+     "execution_count": 6,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -99,7 +99,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
@@ -108,7 +108,7 @@
        "'Answer: {0, -sqrt(3)*I/3, sqrt(3)*I/3}'"
       ]
      },
-     "execution_count": 21,
+     "execution_count": 7,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -119,7 +119,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [
     {
@@ -128,7 +128,7 @@
        "'Answer: (3 - sqrt(7), -sqrt(7) - 2, 1 - sqrt(7)), (sqrt(7) + 3, -2 + sqrt(7), 1 + sqrt(7))'"
       ]
      },
-     "execution_count": 22,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -140,9 +140,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "venv",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "venv"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -154,9 +154,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.11.4"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }

cookbook/meta_prompt.ipynb

@@ -56,7 +56,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain import OpenAI, LLMChain, PromptTemplate\n",
+    "from langchain.llms import OpenAI\nfrom langchain.chains import LLMChain\nfrom langchain.prompts import PromptTemplate\n",
     "from langchain.memory import ConversationBufferWindowMemory"
    ]
   },

cookbook/multi_modal_output_agent.ipynb

@@ -0,0 +1,188 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "cd835d40",
+   "metadata": {},
+   "source": [
+    "# Multi-modal outputs: Image & Text"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fa88e03a",
+   "metadata": {},
+   "source": [
+    "This notebook shows how non-text producing tools can be used to create multi-modal agents.\n",
+    "\n",
+    "This example is limited to text and image outputs and uses UUIDs to transfer content across tools and agents. \n",
+    "\n",
+    "This example uses Steamship to generate and store generated images. Generated are auth protected by default. \n",
+    "\n",
+    "You can get your Steamship api key here: https://steamship.com/account/api"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0653da01",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from steamship import Block, Steamship\n",
+    "import re\n",
+    "from IPython.display import Image"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f6933033",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import OpenAI\n",
+    "from langchain.agents import initialize_agent\n",
+    "from langchain.agents import AgentType\n",
+    "from langchain.tools import SteamshipImageGenerationTool"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "71e51e53",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(temperature=0)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a9fc769d",
+   "metadata": {},
+   "source": [
+    "## Dall-E "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cd177dfe",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tools = [SteamshipImageGenerationTool(model_name=\"dall-e\")]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c71b1e46",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "mrkl = initialize_agent(\n",
+    "    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "603aeb9a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "output = mrkl.run(\"How would you visualize a parot playing soccer?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "25eb4efe",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def show_output(output):\n",
+    "    \"\"\"Display the multi-modal output from the agent.\"\"\"\n",
+    "    UUID_PATTERN = re.compile(\n",
+    "        r\"([0-9A-Za-z]{8}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{12})\"\n",
+    "    )\n",
+    "\n",
+    "    outputs = UUID_PATTERN.split(output)\n",
+    "    outputs = [\n",
+    "        re.sub(r\"^\\W+\", \"\", el) for el in outputs\n",
+    "    ]  # Clean trailing and leading non-word characters\n",
+    "\n",
+    "    for output in outputs:\n",
+    "        maybe_block_id = UUID_PATTERN.search(output)\n",
+    "        if maybe_block_id:\n",
+    "            display(Image(Block.get(Steamship(), _id=maybe_block_id.group()).raw()))\n",
+    "        else:\n",
+    "            print(output, end=\"\\n\\n\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e247b2c4",
+   "metadata": {},
+   "source": [
+    "## StableDiffusion "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "315025e7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tools = [SteamshipImageGenerationTool(model_name=\"stable-diffusion\")]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7930064a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "mrkl = initialize_agent(\n",
+    "    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "611a833d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "output = mrkl.run(\"How would you visualize a parot playing soccer?\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/multiagent_bidding.ipynb

@@ -24,7 +24,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain import PromptTemplate\n",
+    "from langchain.prompts import PromptTemplate\n",
     "import re\n",
     "import tenacity\n",
     "from typing import List, Dict, Callable\n",
@@ -414,7 +414,7 @@
     "1. define a format they will produce their outputs in\n",
     "2. parse their outputs\n",
     "\n",
-    "We can subclass the [RegexParser](https://github.com/hwchase17/langchain/blob/master/langchain/output_parsers/regex.py) to implement our own custom output parser for bids."
+    "We can subclass the [RegexParser](https://github.com/langchain-ai/langchain/blob/master/langchain/output_parsers/regex.py) to implement our own custom output parser for bids."
    ]
   },
   {

cookbook/myscale_vector_sql.ipynb

@@ -0,0 +1,200 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "245065c6",
+   "metadata": {},
+   "source": [
+    "# Vector SQL Retriever with MyScale\n",
+    "\n",
+    ">[MyScale](https://docs.myscale.com/en/) is an integrated vector database. You can access your database in SQL and also from here, LangChain. MyScale can make a use of [various data types and functions for filters](https://blog.myscale.com/2023/06/06/why-integrated-database-solution-can-boost-your-llm-apps/#filter-on-anything-without-constraints). It will boost up your LLM app no matter if you are scaling up your data or expand your system to broader application."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0246c5bf",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip3 install clickhouse-sqlalchemy InstructorEmbedding sentence_transformers openai langchain-experimental"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7585d2c3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n",
+    "from os import environ\n",
+    "import getpass\n",
+    "from typing import Dict, Any\n",
+    "from langchain.llms import OpenAI\nfrom langchain.utilities import SQLDatabase\nfrom langchain.chains import LLMChain\n",
+    "from langchain_experimental.sql.vector_sql import VectorSQLDatabaseChain\n",
+    "from sqlalchemy import create_engine, Column, MetaData\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "\n",
+    "\n",
+    "from sqlalchemy import create_engine\n",
+    "\n",
+    "MYSCALE_HOST = \"msc-1decbcc9.us-east-1.aws.staging.myscale.cloud\"\n",
+    "MYSCALE_PORT = 443\n",
+    "MYSCALE_USER = \"chatdata\"\n",
+    "MYSCALE_PASSWORD = \"myscale_rocks\"\n",
+    "OPENAI_API_KEY = getpass.getpass(\"OpenAI API Key:\")\n",
+    "\n",
+    "engine = create_engine(\n",
+    "    f\"clickhouse://{MYSCALE_USER}:{MYSCALE_PASSWORD}@{MYSCALE_HOST}:{MYSCALE_PORT}/default?protocol=https\"\n",
+    ")\n",
+    "metadata = MetaData(bind=engine)\n",
+    "environ[\"OPENAI_API_KEY\"] = OPENAI_API_KEY"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e08d9ddc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.embeddings import HuggingFaceInstructEmbeddings\n",
+    "from langchain_experimental.sql.vector_sql import VectorSQLOutputParser\n",
+    "\n",
+    "output_parser = VectorSQLOutputParser.from_embeddings(\n",
+    "    model=HuggingFaceInstructEmbeddings(\n",
+    "        model_name=\"hkunlp/instructor-xl\", model_kwargs={\"device\": \"cpu\"}\n",
+    "    )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "84b705b2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.callbacks import StdOutCallbackHandler\n",
+    "\n",
+    "from langchain.utilities.sql_database import SQLDatabase\n",
+    "from langchain_experimental.sql.prompt import MYSCALE_PROMPT\n",
+    "from langchain_experimental.sql.vector_sql import VectorSQLDatabaseChain\n",
+    "\n",
+    "chain = VectorSQLDatabaseChain(\n",
+    "    llm_chain=LLMChain(\n",
+    "        llm=OpenAI(openai_api_key=OPENAI_API_KEY, temperature=0),\n",
+    "        prompt=MYSCALE_PROMPT,\n",
+    "    ),\n",
+    "    top_k=10,\n",
+    "    return_direct=True,\n",
+    "    sql_cmd_parser=output_parser,\n",
+    "    database=SQLDatabase(engine, None, metadata),\n",
+    ")\n",
+    "\n",
+    "import pandas as pd\n",
+    "\n",
+    "pd.DataFrame(\n",
+    "    chain.run(\n",
+    "        \"Please give me 10 papers to ask what is PageRank?\",\n",
+    "        callbacks=[StdOutCallbackHandler()],\n",
+    "    )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c09cda0",
+   "metadata": {},
+   "source": [
+    "## SQL Database as Retriever"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "734d7ff5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.chains.qa_with_sources.retrieval import RetrievalQAWithSourcesChain\n",
+    "\n",
+    "from langchain_experimental.sql.vector_sql import VectorSQLDatabaseChain\n",
+    "from langchain_experimental.retrievers.vector_sql_database \\\n",
+    "    import VectorSQLDatabaseChainRetriever\n",
+    "from langchain_experimental.sql.prompt import MYSCALE_PROMPT\n",
+    "from langchain_experimental.sql.vector_sql import VectorSQLRetrieveAllOutputParser\n",
+    "\n",
+    "output_parser_retrieve_all = VectorSQLRetrieveAllOutputParser.from_embeddings(\n",
+    "    output_parser.model\n",
+    ")\n",
+    "\n",
+    "chain = VectorSQLDatabaseChain.from_llm(\n",
+    "    llm=OpenAI(openai_api_key=OPENAI_API_KEY, temperature=0),\n",
+    "    prompt=MYSCALE_PROMPT,\n",
+    "    top_k=10,\n",
+    "    return_direct=True,\n",
+    "    db=SQLDatabase(engine, None, metadata),\n",
+    "    sql_cmd_parser=output_parser_retrieve_all,\n",
+    "    native_format=True,\n",
+    ")\n",
+    "\n",
+    "# You need all those keys to get docs\n",
+    "retriever = VectorSQLDatabaseChainRetriever(sql_db_chain=chain, page_content_key=\"abstract\")\n",
+    "\n",
+    "document_with_metadata_prompt = PromptTemplate(\n",
+    "    input_variables=[\"page_content\", \"id\", \"title\", \"authors\", \"pubdate\", \"categories\"],\n",
+    "    template=\"Content:\\n\\tTitle: {title}\\n\\tAbstract: {page_content}\\n\\tAuthors: {authors}\\n\\tDate of Publication: {pubdate}\\n\\tCategories: {categories}\\nSOURCE: {id}\",\n",
+    ")\n",
+    "\n",
+    "chain = RetrievalQAWithSourcesChain.from_chain_type(\n",
+    "    ChatOpenAI(\n",
+    "        model_name=\"gpt-3.5-turbo-16k\", openai_api_key=OPENAI_API_KEY, temperature=0.6\n",
+    "    ),\n",
+    "    retriever=retriever,\n",
+    "    chain_type=\"stuff\",\n",
+    "    chain_type_kwargs={\n",
+    "        \"document_prompt\": document_with_metadata_prompt,\n",
+    "    },\n",
+    "    return_source_documents=True,\n",
+    ")\n",
+    "ans = chain(\"Please give me 10 papers to ask what is PageRank?\",\n",
+    "            callbacks=[StdOutCallbackHandler()])\n",
+    "print(ans[\"answer\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4948ff25",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/openai_functions_retrieval_qa.ipynb

@@ -5,7 +5,7 @@
    "id": "71a43144",
    "metadata": {},
    "source": [
-    "# Retrieval QA using OpenAI functions\n",
+    "# Structure answers with OpenAI functions\n",
     "\n",
     "OpenAI functions allows for structuring of response output. This is often useful in question answering when you want to not only get the final answer but also supporting evidence, citations, etc.\n",
     "\n",
@@ -337,7 +337,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "ac9e4626",
    "metadata": {},
@@ -431,7 +430,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -445,7 +444,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.5"
+   "version": "3.11.3"
   }
  },
  "nbformat": 4,

cookbook/plan_and_execute_agent.ipynb

@@ -0,0 +1,252 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "0ddfef23-3c74-444c-81dd-6753722997fa",
+   "metadata": {},
+   "source": [
+    "# Plan-and-execute\n",
+    "\n",
+    "Plan-and-execute agents accomplish an objective by first planning what to do, then executing the sub tasks. This idea is largely inspired by [BabyAGI](https://github.com/yoheinakajima/babyagi) and then the [\"Plan-and-Solve\" paper](https://arxiv.org/abs/2305.04091).\n",
+    "\n",
+    "The planning is almost always done by an LLM.\n",
+    "\n",
+    "The execution is usually done by a separate agent (equipped with tools)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a7ecb22a-7009-48ec-b14e-f0fa5aac1cd0",
+   "metadata": {},
+   "source": [
+    "## Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "5fbbd4ee-bfe8-4a25-afe4-8d1a552a3d2e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents.tools import Tool\n",
+    "from langchain.chains import LLMMathChain\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.utilities import DuckDuckGoSearchAPIWrapper\n",
+    "from langchain_experimental.plan_and_execute import PlanAndExecute, load_agent_executor, load_chat_planner"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e0e995e5-af9d-4988-bcd0-467a2a2e18cd",
+   "metadata": {},
+   "source": [
+    "## Tools"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "1d789f4e-54e3-4602-891a-f076e0ab9594",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "search = DuckDuckGoSearchAPIWrapper()\n",
+    "llm = OpenAI(temperature=0)\n",
+    "llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)\n",
+    "tools = [\n",
+    " Tool(\n",
+    "     name=\"Search\",\n",
+    "     func=search.run,\n",
+    "     description=\"useful for when you need to answer questions about current events\"\n",
+    " ),\n",
+    " Tool(\n",
+    "     name=\"Calculator\",\n",
+    "     func=llm_math_chain.run,\n",
+    "     description=\"useful for when you need to answer questions about math\"\n",
+    " ),\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "04dc6452-a07f-49f9-be12-95be1e2afccc",
+   "metadata": {},
+   "source": [
+    "## Planner, Executor, and Agent\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "d8f49c03-c804-458b-8122-c92b26c7b7dd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model = ChatOpenAI(temperature=0)\n",
+    "planner = load_chat_planner(model)\n",
+    "executor = load_agent_executor(model, tools, verbose=True)\n",
+    "agent = PlanAndExecute(planner=planner, executor=executor)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "78ba03dd-0322-4927-b58d-a7e2027fdbb3",
+   "metadata": {},
+   "source": [
+    "## Run example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "a57f7efe-7866-47a7-bce5-9c7b1047964e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"current prime minister of the UK\"\n",
+      "}\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"current prime minister of the UK\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mBottom right: Rishi Sunak is the current prime minister and the first non-white prime minister. The prime minister of the United Kingdom is the principal minister of the crown of His Majesty's Government, and the head of the British Cabinet. 3 min. British Prime Minister Rishi Sunak asserted his stance on gender identity in a speech Wednesday, stating it was \"common sense\" that \"a man is a man and a woman is a woman\" — a ... The former chancellor Rishi Sunak is the UK's new prime minister. Here's what you need to know about him. He won after running for the second time this year He lost to Liz Truss in September,... Isaeli Prime Minister Benjamin Netanyahu spoke with US President Joe Biden on Wednesday, the prime minister's office said in a statement. Netanyahu \"thanked the President for the powerful words of ... By Yasmeen Serhan/London Updated: October 25, 2022 12:56 PM EDT | Originally published: October 24, 2022 9:17 AM EDT S top me if you've heard this one before: After a tumultuous period of political...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mThe search results indicate that Rishi Sunak is the current prime minister of the UK. However, it's important to note that this information may not be accurate or up to date.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"current age of the prime minister of the UK\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mHow old is Rishi Sunak? Mr Sunak was born on 12 May, 1980, making him 42 years old. He first became an MP in 2015, aged 34, and has served the constituency of Richmond in Yorkshire ever since. He... Prime Ministers' ages when they took office From oldest to youngest, the ages of the PMs were as follows: Winston Churchill - 65 years old James Callaghan - 64 years old Clement Attlee - 62 years... Anna Kaufman USA TODAY Just a few days after Liz Truss resigned as prime minister, the UK has a new prime minister. Truss, who lasted a mere 45 days in office, will be replaced by Rishi... Advertisement Rishi Sunak is the youngest British prime minister of modern times. Mr. Sunak is 42 and started out in Parliament in 2015. Rishi Sunak was appointed as chancellor of the Exchequer... The first prime minister of the current United Kingdom of Great Britain and Northern Ireland upon its effective creation in 1922 (when 26 Irish counties seceded and created the Irish Free State) was Bonar Law, [10] although the country was not renamed officially until 1927, when Stanley Baldwin was the serving prime minister. [11]\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mBased on the search results, it seems that Rishi Sunak is the current prime minister of the UK. However, I couldn't find any specific information about his age. Would you like me to search again for the current age of the prime minister?\n",
+      "\n",
+      "Action:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"age of Rishi Sunak\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mRishi Sunak is 42 years old, making him the youngest person to hold the office of prime minister in modern times. How tall is Rishi Sunak? How Old Is Rishi Sunak? Rishi Sunak was born on May 12, 1980, in Southampton, England. Parents and Nationality Sunak's parents were born to Indian-origin families in East Africa before... Born on May 12, 1980, Rishi is currently 42 years old. He has been a member of parliament since 2015 where he was an MP for Richmond and has served in roles including Chief Secretary to the Treasury and the Chancellor of Exchequer while Boris Johnson was PM. Family Murty, 42, is the daughter of the Indian billionaire NR Narayana Murthy, often described as the Bill Gates of India, who founded the software company Infosys. According to reports, his... Sunak became the first non-White person to lead the country and, at age 42, the youngest to take on the role in more than a century. Like most politicians, Sunak is revered by some and...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mBased on the search results, Rishi Sunak is currently 42 years old. He was born on May 12, 1980.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mThought: To calculate the age raised to the power of 0.43, I can use the calculator tool.\n",
+      "\n",
+      "Action:\n",
+      "```json\n",
+      "{\n",
+      "  \"action\": \"Calculator\",\n",
+      "  \"action_input\": \"42^0.43\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n",
+      "42^0.43\u001b[32;1m\u001b[1;3m```text\n",
+      "42**0.43\n",
+      "```\n",
+      "...numexpr.evaluate(\"42**0.43\")...\n",
+      "\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3m4.9888126515157\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "Observation: \u001b[33;1m\u001b[1;3mAnswer: 4.9888126515157\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mThe age raised to the power of 0.43 is approximately 4.9888126515157.\n",
+      "\n",
+      "Final Answer:\n",
+      "```json\n",
+      "{\n",
+      "  \"action\": \"Final Answer\",\n",
+      "  \"action_input\": \"The age raised to the power of 0.43 is approximately 4.9888126515157.\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Final Answer\",\n",
+      "  \"action_input\": \"The current prime minister of the UK is Rishi Sunak. His age raised to the power of 0.43 is approximately 4.9888126515157.\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'The current prime minister of the UK is Rishi Sunak. His age raised to the power of 0.43 is approximately 4.9888126515157.'"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.run(\"Who is the current prime minister of the UK? What is their current age raised to the 0.43 power?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0ef78a07-1a2a-46f8-9bc9-ae45f9bd706c",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "poetry-venv",
+   "language": "python",
+   "name": "poetry-venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/press_releases.ipynb

@@ -0,0 +1,152 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "62ee82e4-2ad8-498b-8438-fac388afe1a2",
+   "metadata": {},
+   "source": [
+    "Press Releases Data\n",
+    "=\n",
+    "\n",
+    "Press Releases data powered by [Kay.ai](https://kay.ai).\n",
+    "\n",
+    ">Press releases are used by companies to announce something noteworthy, including product launches, financial performance reports, partnerships, and other significant news. They are widely used by analysts to track corporate strategy, operational updates and financial performance.\n",
+    "Kay.ai obtains press releases of all US public companies from a variety of sources, which include the company's official press room and partnerships with various data API providers. \n",
+    "This data is updated till Sept 30th for free access, if you want to access the real-time feed, reach out to us at hello@kay.ai or [tweet at us](https://twitter.com/vishalrohra_)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8183d85d-365f-4672-a963-52b533547de0",
+   "metadata": {},
+   "source": [
+    "Setup\n",
+    "=\n",
+    "\n",
+    "First you will need to install the `kay` package. You will also need an API key: you can get one for free at [https://kay.ai](https://kay.ai/). Once you have an API key, you must set it as an environment variable `KAY_API_KEY`.\n",
+    "\n",
+    "In this example we're going to use the `KayAiRetriever`. Take a look at the [kay notebook](/docs/integrations/retrievers/kay) for more detailed information for the parmeters that it accepts."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02ec21c7-49fe-4844-b58a-bf064ad40b2a",
+   "metadata": {},
+   "source": [
+    "Examples\n",
+    "="
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "bf0395f7-6ebe-4136-8b0d-00b9dea3becd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      " ········\n",
+      " ········\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Setup API keys for Kay and OpenAI\n",
+    "from getpass import getpass\n",
+    "KAY_API_KEY = getpass()\n",
+    "OPENAI_API_KEY = getpass()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "f7fcaf70-29a4-444b-8f07-9784f808c300",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "os.environ[\"KAY_API_KEY\"] = KAY_API_KEY\n",
+    "os.environ[\"OPENAI_API_KEY\"] = OPENAI_API_KEY"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "ac00bf93-3635-4ffe-b9a6-a8b4f35c0c85",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains import ConversationalRetrievalChain\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.retrievers import KayAiRetriever\n",
+    "\n",
+    "model = ChatOpenAI(model_name=\"gpt-3.5-turbo\")\n",
+    "retriever = KayAiRetriever.create(dataset_id=\"company\", data_types=[\"PressRelease\"], num_contexts=6)\n",
+    "qa = ConversationalRetrievalChain.from_llm(model, retriever=retriever)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "8d9d927c-35b2-4a7b-8ea7-4d0350797941",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "-> **Question**: How is the healthcare industry adopting generative AI tools? \n",
+      "\n",
+      "**Answer**: The healthcare industry is adopting generative AI tools to improve various aspects of patient care and administrative tasks. Companies like HCA Healthcare Inc, Amazon Com Inc, and Mayo Clinic have collaborated with technology providers like Google Cloud, AWS, and Microsoft to implement generative AI solutions.\n",
+      "\n",
+      "HCA Healthcare is testing a nurse handoff tool that generates draft reports quickly and accurately, which nurses have shown interest in using. They are also exploring the use of Google's medically-tuned Med-PaLM 2 LLM to support caregivers in asking complex medical questions.\n",
+      "\n",
+      "Amazon Web Services (AWS) has introduced AWS HealthScribe, a generative AI-powered service that automatically creates clinical documentation. However, integrating multiple AI systems into a cohesive solution requires significant engineering resources, including access to AI experts, healthcare data, and compute capacity.\n",
+      "\n",
+      "Mayo Clinic is among the first healthcare organizations to deploy Microsoft 365 Copilot, a generative AI service that combines large language models with organizational data from Microsoft 365. This tool has the potential to automate tasks like form-filling, relieving administrative burdens on healthcare providers and allowing them to focus more on patient care.\n",
+      "\n",
+      "Overall, the healthcare industry is recognizing the potential benefits of generative AI tools in improving efficiency, automating tasks, and enhancing patient care. \n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "# More sample questions in the Playground on https://kay.ai\n",
+    "questions = [\n",
+    "    \"How is the healthcare industry adopting generative AI tools?\",\n",
+    "    #\"What are some recent challenges faced by the renewable energy sector?\",\n",
+    "]\n",
+    "chat_history = []\n",
+    "\n",
+    "for question in questions:\n",
+    "    result = qa({\"question\": question, \"chat_history\": chat_history})\n",
+    "    chat_history.append((question, result[\"answer\"]))\n",
+    "    print(f\"-> **Question**: {question} \\n\")\n",
+    "    print(f\"**Answer**: {result['answer']} \\n\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/program_aided_language_model.ipynb

@@ -17,8 +17,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.chains import PALChain\n",
-    "from langchain import OpenAI"
+    "from langchain_experimental.pal_chain import PALChain\n",
+    "from langchain.llms import OpenAI"
    ]
   },
   {

cookbook/sales_agent_with_context.ipynb

@@ -12,14 +12,14 @@
     "\n",
     "SalesGPT is context-aware, which means it can understand what section of a sales conversation it is in and act accordingly.\n",
     " \n",
-    "As such, this agent can have a natural sales conversation with a prospect and behaves based on the conversation stage. Hence, this notebook demonstrates how we can use AI to automate sales development representatives activites, such as outbound sales calls. \n",
+    "As such, this agent can have a natural sales conversation with a prospect and behaves based on the conversation stage. Hence, this notebook demonstrates how we can use AI to automate sales development representatives activities, such as outbound sales calls. \n",
     "\n",
     "Additionally, the AI Sales agent has access to tools, which allow it to interact with other systems.\n",
     "\n",
     "Here, we show how the AI Sales Agent can use a **Product Knowledge Base** to speak about a particular's company offerings,\n",
     "hence increasing relevance and reducing hallucinations.\n",
     "\n",
-    "We leverage the [`langchain`](https://github.com/hwchase17/langchain) library in this implementation, specifically [Custom Agent Configuration](https://langchain-langchain.vercel.app/docs/modules/agents/how_to/custom_agent_with_tool_retrieval) and are inspired by [BabyAGI](https://github.com/yoheinakajima/babyagi) architecture ."
+    "We leverage the [`langchain`](https://github.com/langchain-ai/langchain) library in this implementation, specifically [Custom Agent Configuration](https://langchain-langchain.vercel.app/docs/modules/agents/how_to/custom_agent_with_tool_retrieval) and are inspired by [BabyAGI](https://github.com/yoheinakajima/babyagi) architecture ."
    ]
   },
   {
@@ -44,7 +44,7 @@
     "\n",
     "from typing import Dict, List, Any, Union, Callable\n",
     "from pydantic import BaseModel, Field\n",
-    "from langchain import LLMChain, PromptTemplate\n",
+    "from langchain.chains import LLMChain\nfrom langchain.prompts import PromptTemplate\n",
     "from langchain.llms import BaseLLM\n",
     "from langchain.chains.base import Chain\n",
     "from langchain.chat_models import ChatOpenAI\n",
@@ -66,7 +66,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# install aditional dependencies\n",
+    "# install additional dependencies\n",
     "# ! pip install chromadb openai tiktoken"
    ]
   },
@@ -150,7 +150,7 @@
     "            {conversation_history}\n",
     "            ===\n",
     "\n",
-    "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting ony from the following options:\n",
+    "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting only from the following options:\n",
     "            1. Introduction: Start the conversation by introducing yourself and your company. Be polite and respectful while keeping the tone of the conversation professional.\n",
     "            2. Qualification: Qualify the prospect by confirming if they are the right person to talk to regarding your product/service. Ensure that they have the authority to make purchasing decisions.\n",
     "            3. Value proposition: Briefly explain how your product/service can benefit the prospect. Focus on the unique selling points and value proposition of your product/service that sets it apart from competitors.\n",
@@ -277,7 +277,7 @@
       "            \n",
       "            ===\n",
       "\n",
-      "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting ony from the following options:\n",
+      "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting only from the following options:\n",
       "            1. Introduction: Start the conversation by introducing yourself and your company. Be polite and respectful while keeping the tone of the conversation professional.\n",
       "            2. Qualification: Qualify the prospect by confirming if they are the right person to talk to regarding your product/service. Ensure that they have the authority to make purchasing decisions.\n",
       "            3. Value proposition: Briefly explain how your product/service can benefit the prospect. Focus on the unique selling points and value proposition of your product/service that sets it apart from competitors.\n",

cookbook/smart_llm.ipynb

@@ -0,0 +1,281 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "9e9b7651",
+   "metadata": {},
+   "source": [
+    "# How to use a SmartLLMChain\n",
+    "\n",
+    "A SmartLLMChain is a form of self-critique chain that can help you if have particularly complex questions to answer. Instead of doing a single LLM pass, it instead performs these 3 steps:\n",
+    "1. Ideation: Pass the user prompt n times through the LLM to get n output proposals (called \"ideas\"), where n is a parameter you can set \n",
+    "2. Critique: The LLM critiques all ideas to find possible flaws and picks the best one \n",
+    "3. Resolve: The LLM tries to improve upon the best idea (as chosen in the critique step) and outputs it. This is then the final output.\n",
+    "\n",
+    "SmartLLMChains are based on the SmartGPT workflow proposed in https://youtu.be/wVzuvf9D9BU.\n",
+    "\n",
+    "Note that SmartLLMChains\n",
+    "- use more LLM passes (ie n+2 instead of just 1)\n",
+    "- only work then the underlying LLM has the capability for reflection, which smaller models often don't\n",
+    "- only work with underlying models that return exactly 1 output, not multiple\n",
+    "\n",
+    "This notebook demonstrates how to use a SmartLLMChain."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "714dede0",
+   "metadata": {},
+   "source": [
+    "##### Same LLM for all steps"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "d3f7fb22",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"OPENAI_API_KEY\"] = \"...\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "10e5ece6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain_experimental.smart_llm import SmartLLMChain"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "1780da51",
+   "metadata": {},
+   "source": [
+    "As example question, we will use \"I have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?\". This is an example from the original SmartGPT video (https://youtu.be/wVzuvf9D9BU?t=384). While this seems like a very easy question, LLMs struggle do these kinds of questions that involve numbers and physical reasoning.\n",
+    "\n",
+    "As we will see, all 3 initial ideas are completely wrong - even though we're using GPT4! Only when using self-reflection do we get a correct answer. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "054af6b1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "hard_question = \"I have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?\""
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "8049cecd",
+   "metadata": {},
+   "source": [
+    "So, we first create an LLM and prompt template"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "811ea8e1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt = PromptTemplate.from_template(hard_question)\n",
+    "llm = ChatOpenAI(temperature=0, model_name=\"gpt-4\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "50b602e4",
+   "metadata": {},
+   "source": [
+    "Now we can create a SmartLLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "8cd49199",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = SmartLLMChain(llm=llm, prompt=prompt, n_ideas=3, verbose=True)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "6a72f276",
+   "metadata": {},
+   "source": [
+    "Now we can use the SmartLLM as a drop-in replacement for our LLM. E.g.:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "074e5e75",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new SmartLLMChain chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mI have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?\u001b[0m\n",
+      "Idea 1:\n",
+      "\u001b[36;1m\u001b[1;3m1. Fill the 6-liter jug completely.\n",
+      "2. Pour the water from the 6-liter jug into the 12-liter jug.\n",
+      "3. Fill the 6-liter jug again.\n",
+      "4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full.\n",
+      "5. The amount of water left in the 6-liter jug will be exactly 6 liters.\u001b[0m\n",
+      "Idea 2:\n",
+      "\u001b[36;1m\u001b[1;3m1. Fill the 6-liter jug completely.\n",
+      "2. Pour the water from the 6-liter jug into the 12-liter jug.\n",
+      "3. Fill the 6-liter jug again.\n",
+      "4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full.\n",
+      "5. Since the 12-liter jug is now full, there will be 2 liters of water left in the 6-liter jug.\n",
+      "6. Empty the 12-liter jug.\n",
+      "7. Pour the 2 liters of water from the 6-liter jug into the 12-liter jug.\n",
+      "8. Fill the 6-liter jug completely again.\n",
+      "9. Pour the water from the 6-liter jug into the 12-liter jug, which already has 2 liters in it.\n",
+      "10. Now, the 12-liter jug will have exactly 6 liters of water (2 liters from before + 4 liters from the 6-liter jug).\u001b[0m\n",
+      "Idea 3:\n",
+      "\u001b[36;1m\u001b[1;3m1. Fill the 6-liter jug completely.\n",
+      "2. Pour the water from the 6-liter jug into the 12-liter jug.\n",
+      "3. Fill the 6-liter jug again.\n",
+      "4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full.\n",
+      "5. The amount of water left in the 6-liter jug will be exactly 6 liters.\u001b[0m\n",
+      "Critique:\n",
+      "\u001b[33;1m\u001b[1;3mIdea 1:\n",
+      "1. Fill the 6-liter jug completely. (No flaw)\n",
+      "2. Pour the water from the 6-liter jug into the 12-liter jug. (No flaw)\n",
+      "3. Fill the 6-liter jug again. (No flaw)\n",
+      "4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. (Flaw: The 12-liter jug will never be full in this step, as it can hold 12 liters and we are only pouring 6 liters into it.)\n",
+      "5. The amount of water left in the 6-liter jug will be exactly 6 liters. (Flaw: This statement is incorrect, as there will be no water left in the 6-liter jug after pouring it into the 12-liter jug.)\n",
+      "\n",
+      "Idea 2:\n",
+      "1. Fill the 6-liter jug completely. (No flaw)\n",
+      "2. Pour the water from the 6-liter jug into the 12-liter jug. (No flaw)\n",
+      "3. Fill the 6-liter jug again. (No flaw)\n",
+      "4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. (Flaw: The 12-liter jug will never be full in this step, as it can hold 12 liters and we are only pouring 6 liters into it.)\n",
+      "5. Since the 12-liter jug is now full, there will be 2 liters of water left in the 6-liter jug. (Flaw: This statement is incorrect, as the 12-liter jug will not be full and there will be no water left in the 6-liter jug.)\n",
+      "6. Empty the 12-liter jug. (No flaw)\n",
+      "7. Pour the 2 liters of water from the 6-liter jug into the 12-liter jug. (Flaw: This step is based on the incorrect assumption that there are 2 liters of water left in the 6-liter jug.)\n",
+      "8. Fill the 6-liter jug completely again. (No flaw)\n",
+      "9. Pour the water from the 6-liter jug into the 12-liter jug, which already has 2 liters in it. (Flaw: This step is based on the incorrect assumption that there are 2 liters of water in the 12-liter jug.)\n",
+      "10. Now, the 12-liter jug will have exactly 6 liters of water (2 liters from before + 4 liters from the 6-liter jug). (Flaw: This conclusion is based on the incorrect assumptions made in the previous steps.)\n",
+      "\n",
+      "Idea 3:\n",
+      "1. Fill the 6-liter jug completely. (No flaw)\n",
+      "2. Pour the water from the 6-liter jug into the 12-liter jug. (No flaw)\n",
+      "3. Fill the 6-liter jug again. (No flaw)\n",
+      "4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. (Flaw: The 12-liter jug will never be full in this step, as it can hold 12 liters and we are only pouring 6 liters into it.)\n",
+      "5. The amount of water left in the 6-liter jug will be exactly 6 liters. (Flaw: This statement is incorrect, as there will be no water left in the 6-liter jug after pouring it into the 12-liter jug.)\u001b[0m\n",
+      "Resolution:\n",
+      "\u001b[32;1m\u001b[1;3m1. Fill the 12-liter jug completely.\n",
+      "2. Pour the water from the 12-liter jug into the 6-liter jug until the 6-liter jug is full.\n",
+      "3. The amount of water left in the 12-liter jug will be exactly 6 liters.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'1. Fill the 12-liter jug completely.\\n2. Pour the water from the 12-liter jug into the 6-liter jug until the 6-liter jug is full.\\n3. The amount of water left in the 12-liter jug will be exactly 6 liters.'"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.run({})"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "bbfebea1",
+   "metadata": {},
+   "source": [
+    "##### Different LLM for different steps"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "5be6ec08",
+   "metadata": {},
+   "source": [
+    "You can also use different LLMs for the different steps by passing `ideation_llm`, `critique_llm` and `resolve_llm`. You might want to do this to use a more creative (i.e., high-temperature) model for ideation and a more strict (i.e., low-temperature) model for critique and resolution."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "9c33fa19",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = SmartLLMChain(\n",
+    "    ideation_llm=ChatOpenAI(temperature=0.9, model_name=\"gpt-4\"),\n",
+    "    llm=ChatOpenAI(\n",
+    "        temperature=0, model_name=\"gpt-4\"\n",
+    "    ),  # will be used for critique and resolution as no specific llms are given\n",
+    "    prompt=prompt,\n",
+    "    n_ideas=3,\n",
+    "    verbose=True,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "886c1cc1",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/sql_db_qa.mdx

@@ -1,3 +1,7 @@
+# SQL Database Chain
+
+This example demonstrates the use of the `SQLDatabaseChain` for answering questions over a SQL database.
+
 Under the hood, LangChain uses SQLAlchemy to connect to SQL databases. The `SQLDatabaseChain` can therefore be used with any SQL dialect supported by SQLAlchemy, such as MS SQL, MySQL, MariaDB, PostgreSQL, Oracle SQL, [Databricks](/docs/ecosystem/integrations/databricks.html) and SQLite. Please refer to the SQLAlchemy documentation for more information about requirements for connecting to your database. For example, a connection to MySQL requires an appropriate connector such as PyMySQL. A URI for a MySQL connection might look like: `mysql+pymysql://user:pass@some_mysql_db_address/db_name`.
 
 This demonstration uses SQLite and the example Chinook database.
@@ -5,7 +9,9 @@ To set it up, follow the instructions on https://database.guide/2-sample-databas
 
 
 ```python
-from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
+from langchain.llms import OpenAI
+from langchain.utilities import SQLDatabase
+from langchain_experimental.sql import SQLDatabaseChain
 ```
 
 
@@ -29,8 +35,8 @@ db_chain.run("How many employees are there?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many employees are there?
     SQLQuery:
@@ -69,8 +75,8 @@ db_chain.run("How many albums by Aerosmith?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many albums by Aerosmith?
     SQLQuery:SELECT COUNT(*) FROM Album WHERE ArtistId = 3;
@@ -127,8 +133,8 @@ db_chain.run("How many employees are there in the foobar table?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many employees are there in the foobar table?
     SQLQuery:SELECT COUNT(*) FROM Employee;
@@ -163,8 +169,8 @@ result["intermediate_steps"]
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many employees are there in the foobar table?
     SQLQuery:SELECT COUNT(*) FROM Employee;
@@ -189,6 +195,112 @@ result["intermediate_steps"]
 
 </CodeOutputBlock>
 
+## Adding Memory
+
+How to add memory to a SQLDatabaseChain:
+
+```python
+from langchain.llms import OpenAI
+from langchain.utilities import SQLDatabase
+from langchain_experimental.sql import SQLDatabaseChain
+```
+
+Set up the SQLDatabase and LLM
+
+```python
+db = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db")
+llm = OpenAI(temperature=0, verbose=True)
+```
+
+Set up the memory
+
+```python
+from langchain.memory import ConversationBufferMemory
+memory = ConversationBufferMemory()
+```
+
+Now we need to add a place for memory in the prompt template
+
+```python
+from langchain.prompts import PromptTemplate
+PROMPT_SUFFIX = """Only use the following tables:
+{table_info}
+
+Previous Conversation:
+{history}
+
+Question: {input}"""
+
+_DEFAULT_TEMPLATE = """Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer. Unless the user specifies in his question a specific number of examples he wishes to obtain, always limit your query to at most {top_k} results. You can order the results by a relevant column to return the most interesting examples in the database.
+
+Never query for all the columns from a specific table, only ask for a the few relevant columns given the question.
+
+Pay attention to use only the column names that you can see in the schema description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
+
+Use the following format:
+
+Question: Question here
+SQLQuery: SQL Query to run
+SQLResult: Result of the SQLQuery
+Answer: Final answer here
+
+"""
+
+PROMPT = PromptTemplate.from_template(
+    _DEFAULT_TEMPLATE + PROMPT_SUFFIX,
+)
+```
+
+Now let's create and run out chain
+
+```python
+db_chain = SQLDatabaseChain.from_llm(llm, db, prompt=PROMPT, verbose=True, memory=memory)
+db_chain.run("name one employee")
+```
+
+<CodeOutputBlock lang="python">
+
+```
+    > Entering new SQLDatabaseChain chain...
+    name one employee
+    SQLQuery:SELECT FirstName, LastName FROM Employee LIMIT 1
+    SQLResult: [('Andrew', 'Adams')]
+    Answer:Andrew Adams
+    > Finished chain.
+
+
+
+
+
+    'Andrew Adams'
+```
+
+</CodeOutputBlock>
+
+```python
+db_chain.run("how many letters in their name?")
+```
+
+<CodeOutputBlock lang="python">
+
+```
+    > Entering new SQLDatabaseChain chain...
+    how many letters in their name?
+    SQLQuery:SELECT LENGTH(FirstName) + LENGTH(LastName) AS 'NameLength' FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adams'
+    SQLResult: [(11,)]
+    Answer:Andrew Adams has 11 letters in their name.
+    > Finished chain.
+
+
+
+
+
+    'Andrew Adams has 11 letters in their name.'
+```
+
+</CodeOutputBlock>
+
+
 ## Choosing how to limit the number of rows returned
 If you are querying for several rows of a table you can select the maximum number of results you want to get by using the 'top_k' parameter (default is 10). This is useful for avoiding query results that exceed the prompt max length or consume tokens unnecessarily.
 
@@ -205,8 +317,8 @@ db_chain.run("What are some example tracks by composer Johann Sebastian Bach?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     What are some example tracks by composer Johann Sebastian Bach?
     SQLQuery:SELECT Name FROM Track WHERE Composer = 'Johann Sebastian Bach' LIMIT 3
@@ -244,23 +356,23 @@ print(db.table_info)
 <CodeOutputBlock lang="python">
 
 ```
-    
+
     CREATE TABLE "Track" (
-    	"TrackId" INTEGER NOT NULL, 
-    	"Name" NVARCHAR(200) NOT NULL, 
-    	"AlbumId" INTEGER, 
-    	"MediaTypeId" INTEGER NOT NULL, 
-    	"GenreId" INTEGER, 
-    	"Composer" NVARCHAR(220), 
-    	"Milliseconds" INTEGER NOT NULL, 
-    	"Bytes" INTEGER, 
-    	"UnitPrice" NUMERIC(10, 2) NOT NULL, 
-    	PRIMARY KEY ("TrackId"), 
-    	FOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"), 
-    	FOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"), 
+    	"TrackId" INTEGER NOT NULL,
+    	"Name" NVARCHAR(200) NOT NULL,
+    	"AlbumId" INTEGER,
+    	"MediaTypeId" INTEGER NOT NULL,
+    	"GenreId" INTEGER,
+    	"Composer" NVARCHAR(220),
+    	"Milliseconds" INTEGER NOT NULL,
+    	"Bytes" INTEGER,
+    	"UnitPrice" NUMERIC(10, 2) NOT NULL,
+    	PRIMARY KEY ("TrackId"),
+    	FOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"),
+    	FOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"),
     	FOREIGN KEY("AlbumId") REFERENCES "Album" ("AlbumId")
     )
-    
+
     /*
     2 rows from Track table:
     TrackId	Name	AlbumId	MediaTypeId	GenreId	Composer	Milliseconds	Bytes	UnitPrice
@@ -284,8 +396,8 @@ db_chain.run("What are some example tracks by Bach?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     What are some example tracks by Bach?
     SQLQuery:SELECT "Name", "Composer" FROM "Track" WHERE "Composer" LIKE '%Bach%' LIMIT 5
@@ -303,7 +415,7 @@ db_chain.run("What are some example tracks by Bach?")
 </CodeOutputBlock>
 
 ### Custom Table Info
-In some cases, it can be useful to provide custom table information instead of using the automatically generated table definitions and the first `sample_rows_in_table_info` sample rows. For example, if you know that the first few rows of a table are uninformative, it could help to manually provide example rows that are more diverse or provide more information to the model. It is also possible to limit the columns that will be visible to the model if there are unnecessary columns. 
+In some cases, it can be useful to provide custom table information instead of using the automatically generated table definitions and the first `sample_rows_in_table_info` sample rows. For example, if you know that the first few rows of a table are uninformative, it could help to manually provide example rows that are more diverse or provide more information to the model. It is also possible to limit the columns that will be visible to the model if there are unnecessary columns.
 
 This information can be provided as a dictionary with table names as the keys and table information as the values. For example, let's provide a custom definition and sample rows for the Track table with only a few columns:
 
@@ -311,7 +423,7 @@ This information can be provided as a dictionary with table names as the keys an
 ```python
 custom_table_info = {
     "Track": """CREATE TABLE Track (
-	"TrackId" INTEGER NOT NULL, 
+	"TrackId" INTEGER NOT NULL,
 	"Name" NVARCHAR(200) NOT NULL,
 	"Composer" NVARCHAR(220),
 	PRIMARY KEY ("TrackId")
@@ -340,22 +452,22 @@ print(db.table_info)
 <CodeOutputBlock lang="python">
 
 ```
-    
+
     CREATE TABLE "Playlist" (
-    	"PlaylistId" INTEGER NOT NULL, 
-    	"Name" NVARCHAR(120), 
+    	"PlaylistId" INTEGER NOT NULL,
+    	"Name" NVARCHAR(120),
     	PRIMARY KEY ("PlaylistId")
     )
-    
+
     /*
     2 rows from Playlist table:
     PlaylistId	Name
     1	Music
     2	Movies
     */
-    
+
     CREATE TABLE Track (
-    	"TrackId" INTEGER NOT NULL, 
+    	"TrackId" INTEGER NOT NULL,
     	"Name" NVARCHAR(200) NOT NULL,
     	"Composer" NVARCHAR(220),
     	PRIMARY KEY ("TrackId")
@@ -382,8 +494,8 @@ db_chain.run("What are some example tracks by Bach?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     What are some example tracks by Bach?
     SQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE '%Bach%' LIMIT 5;
@@ -393,31 +505,31 @@ db_chain.run("What are some example tracks by Bach?")
     Unless the user specifies in the question a specific number of examples to obtain, query for at most 5 results using the LIMIT clause as per SQLite. You can order the results to return the most informative data in the database.
     Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.
     Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
-    
+
     Use the following format:
-    
+
     Question: "Question here"
     SQLQuery: "SQL Query to run"
     SQLResult: "Result of the SQLQuery"
     Answer: "Final answer here"
-    
+
     Only use the following tables:
-    
+
     CREATE TABLE "Playlist" (
-    	"PlaylistId" INTEGER NOT NULL, 
-    	"Name" NVARCHAR(120), 
+    	"PlaylistId" INTEGER NOT NULL,
+    	"Name" NVARCHAR(120),
     	PRIMARY KEY ("PlaylistId")
     )
-    
+
     /*
     2 rows from Playlist table:
     PlaylistId	Name
     1	Music
     2	Movies
     */
-    
+
     CREATE TABLE Track (
-    	"TrackId" INTEGER NOT NULL, 
+    	"TrackId" INTEGER NOT NULL,
     	"Name" NVARCHAR(200) NOT NULL,
     	"Composer" NVARCHAR(220),
     	PRIMARY KEY ("TrackId")
@@ -429,7 +541,7 @@ db_chain.run("What are some example tracks by Bach?")
     2	Balls to the Wall	None
     3	My favorite song ever	The coolest composer of all time
     */
-    
+
     Question: What are some example tracks by Bach?
     SQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE '%Bach%' LIMIT 5;
     SQLResult: [('American Woman',), ('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace',), ('Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria',), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude',), ('Toccata and Fugue in D Minor, BWV 565: I. Toccata',)]
@@ -449,7 +561,7 @@ db_chain.run("What are some example tracks by Bach?")
 
 ### SQL Views
 
-In some case, the table schema can be hidden behind a JSON or JSONB column. Adding row samples into the prompt might help won't always describe the data perfectly. 
+In some case, the table schema can be hidden behind a JSON or JSONB column. Adding row samples into the prompt might help won't always describe the data perfectly.
 
 For this reason, a custom SQL views can help.
 
@@ -484,7 +596,7 @@ This is useful in cases where the number of tables in the database is large.
 
 
 ```python
-from langchain.chains import SQLDatabaseSequentialChain
+from langchain_experimental.sql import SQLDatabaseSequentialChain
 db = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db")
 ```
 
@@ -501,19 +613,19 @@ chain.run("How many employees are also customers?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseSequentialChain chain...
     Table names to use:
     ['Employee', 'Customer']
-    
+
     > Entering new SQLDatabaseChain chain...
     How many employees are also customers?
     SQLQuery:SELECT COUNT(*) FROM Employee e INNER JOIN Customer c ON e.EmployeeId = c.SupportRepId;
     SQLResult: [(59,)]
     Answer:59 employees are also customers.
     > Finished chain.
-    
+
     > Finished chain.
 
 
@@ -535,7 +647,7 @@ Sometimes you may not have the luxury of using OpenAI or other service-hosted la
 import logging
 import torch
 from transformers import AutoTokenizer, GPT2TokenizerFast, pipeline, AutoModelForSeq2SeqLM, AutoModelForCausalLM
-from langchain import HuggingFacePipeline
+from langchain.llms import HuggingFacePipeline
 
 # Note: This model requires a large GPU, e.g. an 80GB A100. See documentation for other ways to run private non-OpenAI models.
 model_id = "google/flan-ul2"
@@ -567,7 +679,8 @@ local_llm = HuggingFacePipeline(pipeline=pipe)
 
 
 ```python
-from langchain import SQLDatabase, SQLDatabaseChain
+from langchain.utilities import SQLDatabase
+from langchain_experimental.sql import SQLDatabaseChain
 
 db = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db", include_tables=['Customer'])
 local_chain = SQLDatabaseChain.from_llm(local_llm, db, verbose=True, return_intermediate_steps=True, use_query_checker=True)
@@ -583,8 +696,8 @@ local_chain("How many customers are there?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are there?
     SQLQuery:
@@ -625,7 +738,7 @@ local_chain("How many customers are there?")
 
 </CodeOutputBlock>
 
-Even this relatively large model will most likely fail to generate more complicated SQL by itself. However, you can log its inputs and outputs so that you can hand-correct them and use the corrected examples for few shot prompt examples later. In practice, you could log any executions of your chain that raise exceptions (as shown in the example below) or get direct user feedback in cases where the results are incorrect (but did not raise an exception).
+Even this relatively large model will most likely fail to generate more complicated SQL by itself. However, you can log its inputs and outputs so that you can hand-correct them and use the corrected examples for few-shot prompt examples later. In practice, you could log any executions of your chain that raise exceptions (as shown in the example below) or get direct user feedback in cases where the results are incorrect (but did not raise an exception).
 
 
 ```bash
@@ -770,8 +883,8 @@ print("\n" + yaml_example)
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     List all the customer first names that start with 'a'
     SQLQuery:
@@ -791,7 +904,7 @@ print("\n" + yaml_example)
     [('François', 'Frantiek', 'Helena', 'Astrid', 'Daan', 'Kara', 'Eduardo', 'Alexandre', 'Fernanda', 'Mark', 'Frank', 'Jack', 'Dan', 'Kathy', 'Heather', 'Frank', 'Richard', 'Patrick', 'Julia', 'Edward', 'Martha', 'Aaron', 'Madalena', 'Hannah', 'Niklas', 'Camille', 'Marc', 'Wyatt', 'Isabelle', 'Ladislav', 'Lucas', 'Johannes', 'Stanisaw', 'Joakim', 'Emma', 'Mark', 'Manoj', 'Puja']
     > Finished chain.
     *** Query succeeded
-    
+
     answer: '[(''François'', ''Frantiek'', ''Helena'', ''Astrid'', ''Daan'', ''Kara'',
       ''Eduardo'', ''Alexandre'', ''Fernanda'', ''Mark'', ''Frank'', ''Jack'', ''Dan'',
       ''Kathy'', ''Heather'', ''Frank'', ''Richard'', ''Patrick'', ''Julia'', ''Edward'',
@@ -822,7 +935,7 @@ print("\n" + yaml_example)
       None\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n3\tFrançois\t\
       Tremblay\tNone\t1498 rue Bélanger\tMontréal\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\t\
       None\tftremblay@gmail.com\t3\n*/"
-    
+
 ```
 
 </CodeOutputBlock>
@@ -835,20 +948,20 @@ YAML_EXAMPLES = """
 - input: How many customers are not from Brazil?
   table_info: |
     CREATE TABLE "Customer" (
-      "CustomerId" INTEGER NOT NULL, 
-      "FirstName" NVARCHAR(40) NOT NULL, 
-      "LastName" NVARCHAR(20) NOT NULL, 
-      "Company" NVARCHAR(80), 
-      "Address" NVARCHAR(70), 
-      "City" NVARCHAR(40), 
-      "State" NVARCHAR(40), 
-      "Country" NVARCHAR(40), 
-      "PostalCode" NVARCHAR(10), 
-      "Phone" NVARCHAR(24), 
-      "Fax" NVARCHAR(24), 
-      "Email" NVARCHAR(60) NOT NULL, 
-      "SupportRepId" INTEGER, 
-      PRIMARY KEY ("CustomerId"), 
+      "CustomerId" INTEGER NOT NULL,
+      "FirstName" NVARCHAR(40) NOT NULL,
+      "LastName" NVARCHAR(20) NOT NULL,
+      "Company" NVARCHAR(80),
+      "Address" NVARCHAR(70),
+      "City" NVARCHAR(40),
+      "State" NVARCHAR(40),
+      "Country" NVARCHAR(40),
+      "PostalCode" NVARCHAR(10),
+      "Phone" NVARCHAR(24),
+      "Fax" NVARCHAR(24),
+      "Email" NVARCHAR(60) NOT NULL,
+      "SupportRepId" INTEGER,
+      PRIMARY KEY ("CustomerId"),
       FOREIGN KEY("SupportRepId") REFERENCES "Employee" ("EmployeeId")
     )
   sql_cmd: SELECT COUNT(*) FROM "Customer" WHERE NOT "Country" = "Brazil";
@@ -857,8 +970,8 @@ YAML_EXAMPLES = """
 - input: list all the genres that start with 'r'
   table_info: |
     CREATE TABLE "Genre" (
-      "GenreId" INTEGER NOT NULL, 
-      "Name" NVARCHAR(120), 
+      "GenreId" INTEGER NOT NULL,
+      "Name" NVARCHAR(120),
       PRIMARY KEY ("GenreId")
     )
 
@@ -871,15 +984,15 @@ YAML_EXAMPLES = """
     */
   sql_cmd: SELECT "Name" FROM "Genre" WHERE "Name" LIKE 'r%';
   sql_result: "[('Rock',), ('Rock and Roll',), ('Reggae',), ('R&B/Soul',)]"
-  answer: The genres that start with 'r' are Rock, Rock and Roll, Reggae and R&B/Soul. 
+  answer: The genres that start with 'r' are Rock, Rock and Roll, Reggae and R&B/Soul.
 """
 ```
 
-Now that you have some examples (with manually corrected output SQL), you can do few shot prompt seeding the usual way:
+Now that you have some examples (with manually corrected output SQL), you can do few-shot prompt seeding the usual way:
 
 
 ```python
-from langchain import FewShotPromptTemplate, PromptTemplate
+from langchain.prompts import FewShotPromptTemplate, PromptTemplate
 from langchain.chains.sql_database.prompt import _sqlite_prompt, PROMPT_SUFFIX
 from langchain.embeddings.huggingface import HuggingFaceEmbeddings
 from langchain.prompts.example_selector.semantic_similarity import SemanticSimilarityExampleSelector
@@ -922,7 +1035,7 @@ few_shot_prompt = FewShotPromptTemplate(
 
 </CodeOutputBlock>
 
-The model should do better now with this few shot prompt, especially for inputs similar to the examples you have seeded it with.
+The model should do better now with this few-shot prompt, especially for inputs similar to the examples you have seeded it with.
 
 
 ```python
@@ -937,8 +1050,8 @@ result = local_chain("How many customers are from Brazil?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are from Brazil?
     SQLQuery:SELECT count(*) FROM Customer WHERE Country = "Brazil";
@@ -957,8 +1070,8 @@ result = local_chain("How many customers are not from Brazil?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are not from Brazil?
     SQLQuery:SELECT count(*) FROM customer WHERE country NOT IN (SELECT country FROM customer WHERE country = 'Brazil')
@@ -977,8 +1090,8 @@ result = local_chain("How many customers are there in total?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are there in total?
     SQLQuery:SELECT count(*) FROM Customer;

cookbook/tree_of_thought.ipynb

@@ -4,9 +4,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Tree of Thouht (ToT) example\n",
+    "# Tree of Thought (ToT) example\n",
     "\n",
-    "The Tree of Thought (ToT) is a chain that allows you to query a Large Language Model (LLM) using the Tree of Thought technique. This is based on the papaer [\"Large Language Model Guided Tree-of-Thought\"](https://arxiv.org/pdf/2305.08291.pdf)"
+    "The Tree of Thought (ToT) is a chain that allows you to query a Large Language Model (LLM) using the Tree of Thought technique. This is based on the paper [\"Large Language Model Guided Tree-of-Thought\"](https://arxiv.org/pdf/2305.08291.pdf)"
    ]
   },
   {

cookbook/wikibase_agent.ipynb

@@ -123,7 +123,7 @@
     "wikidata_user_agent_header = (\n",
     "    None\n",
     "    if not config.has_section(\"WIKIDATA\")\n",
-    "    else config[\"WIKIDATA\"][\"WIKIDAtA_USER_AGENT_HEADER\"]\n",
+    "    else config[\"WIKIDATA\"][\"WIKIDATA_USER_AGENT_HEADER\"]\n",
     ")"
    ]
   },
@@ -396,7 +396,7 @@
     "    AgentOutputParser,\n",
     ")\n",
     "from langchain.prompts import StringPromptTemplate\n",
-    "from langchain import OpenAI, LLMChain\n",
+    "from langchain.llms import OpenAI\nfrom langchain.chains import LLMChain\n",
     "from typing import List, Union\n",
     "from langchain.schema import AgentAction, AgentFinish\n",
     "import re"

docker/Dockerfile.base

@@ -0,0 +1,3 @@
+FROM python:3.11
+
+RUN pip install langchain

docs/.local_build.sh

@@ -8,10 +8,12 @@ set -o xtrace
 SCRIPT_DIR="$(cd "$(dirname "$0")"; pwd)"
 cd "${SCRIPT_DIR}"
 
-mkdir -p _dist/docs_skeleton
-cp -r {docs_skeleton,snippets} _dist
-cp -r extras/* _dist/docs_skeleton/docs
-cd _dist/docs_skeleton
-poetry run nbdoc_build
+mkdir -p ../_dist
+cp -r . ../_dist
+cd ../_dist
+poetry run python scripts/model_feat_table.py
+poetry run nbdoc_build --srcdir docs
+cp ../cookbook/README.md src/pages/cookbook.mdx
+poetry run python scripts/generate_api_reference_links.py
 yarn install
 yarn start

docs/README.md

@@ -42,7 +42,7 @@ If you are using GitHub pages for hosting, this command is a convenient way to b
 
 ### Continuous Integration
 
-Some common defaults for linting/formatting have been set for you. If you integrate your project with an open source Continuous Integration system (e.g. Travis CI, CircleCI), you may check for issues using the following command.
+Some common defaults for linting/formatting have been set for you. If you integrate your project with an open-source Continuous Integration system (e.g. Travis CI, CircleCI), you may check for issues using the following command.
 
 ```
 $ yarn ci

docs/api_reference/Makefile

@@ -3,7 +3,7 @@
 
 # You can set these variables from the command line, and also
 # from the environment for the first two.
-SPHINXOPTS    ?= 
+SPHINXOPTS    ?= -j auto 
 SPHINXBUILD   ?= sphinx-build
 SPHINXAUTOBUILD   ?= sphinx-autobuild
 SOURCEDIR     = .

docs/api_reference/conf.py

@@ -23,6 +23,7 @@ from sphinx.util.docutils import SphinxDirective
 _DIR = Path(__file__).parent.absolute()
 sys.path.insert(0, os.path.abspath("."))
 sys.path.insert(0, os.path.abspath("../../libs/langchain"))
+sys.path.insert(0, os.path.abspath("../../libs/experimental"))
 
 with (_DIR.parents[1] / "libs" / "langchain" / "pyproject.toml").open("r") as f:
     data = toml.load(f)
@@ -99,6 +100,9 @@ extensions = [
 ]
 source_suffix = [".rst"]
 
+# some autodoc pydantic options are repeated in the actual template.
+# potentially user error, but there may be bugs in the sphinx extension
+# with options not being passed through correctly (from either the location in the code)
 autodoc_pydantic_model_show_json = False
 autodoc_pydantic_field_list_validators = False
 autodoc_pydantic_config_members = False
@@ -111,13 +115,6 @@ autodoc_member_order = "groupwise"
 autoclass_content = "both"
 autodoc_typehints_format = "short"
 
-autodoc_default_options = {
-    "members": True,
-    "show-inheritance": True,
-    "inherited-members": "BaseModel",
-    "undoc-members": True,
-    "special-members": "__call__",
-}
 # autodoc_typehints = "description"
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["templates"]
@@ -159,7 +156,7 @@ html_context = {
 html_static_path = ["_static"]
 
 # These paths are either relative to html_static_path
-# or fully qualified paths (eg. https://...)
+# or fully qualified paths (e.g. https://...)
 html_css_files = [
     "css/custom.css",
 ]

docs/api_reference/create_api_rst.py

@@ -1,76 +1,274 @@
-"""Script for auto-generating api_reference.rst"""
-import glob
-import re
+"""Script for auto-generating api_reference.rst."""
+import importlib
+import inspect
+import typing
 from pathlib import Path
+from typing import TypedDict, Sequence, List, Dict, Literal, Union, Optional
+from enum import Enum
+
+from pydantic import BaseModel
 
 ROOT_DIR = Path(__file__).parents[2].absolute()
+HERE = Path(__file__).parent
+
 PKG_DIR = ROOT_DIR / "libs" / "langchain" / "langchain"
-WRITE_FILE = Path(__file__).parent / "api_reference.rst"
+EXP_DIR = ROOT_DIR / "libs" / "experimental" / "langchain_experimental"
+WRITE_FILE = HERE / "api_reference.rst"
+EXP_WRITE_FILE = HERE / "experimental_api_reference.rst"
 
 
-def load_members() -> dict:
-    members: dict = {}
-    for py in glob.glob(str(PKG_DIR) + "/**/*.py", recursive=True):
-        module = py[len(str(PKG_DIR)) + 1 :].replace(".py", "").replace("/", ".")
-        top_level = module.split(".")[0]
-        if top_level not in members:
-            members[top_level] = {"classes": [], "functions": []}
-        with open(py, "r") as f:
-            for line in f.readlines():
-                cls = re.findall(r"^class ([^_].*)\(", line)
-                members[top_level]["classes"].extend([module + "." + c for c in cls])
-                func = re.findall(r"^def ([^_].*)\(", line)
-                afunc = re.findall(r"^async def ([^_].*)\(", line)
-                func_strings = [module + "." + f for f in func + afunc]
-                members[top_level]["functions"].extend(func_strings)
-    return members
+ClassKind = Literal["TypedDict", "Regular", "Pydantic", "enum"]
 
 
-def construct_doc(members: dict) -> str:
-    full_doc = """\
-.. _api_reference:
+class ClassInfo(TypedDict):
+    """Information about a class."""
 
-=============
-API Reference
-=============
+    name: str
+    """The name of the class."""
+    qualified_name: str
+    """The fully qualified name of the class."""
+    kind: ClassKind
+    """The kind of the class."""
+    is_public: bool
+    """Whether the class is public or not."""
+
+
+class FunctionInfo(TypedDict):
+    """Information about a function."""
+
+    name: str
+    """The name of the function."""
+    qualified_name: str
+    """The fully qualified name of the function."""
+    is_public: bool
+    """Whether the function is public or not."""
+
+
+class ModuleMembers(TypedDict):
+    """A dictionary of module members."""
+
+    classes_: Sequence[ClassInfo]
+    functions: Sequence[FunctionInfo]
+
+
+def _load_module_members(module_path: str, namespace: str) -> ModuleMembers:
+    """Load all members of a module.
+
+    Args:
+        module_path: Path to the module.
+        namespace: the namespace of the module.
+
+    Returns:
+        list: A list of loaded module objects.
+    """
+    classes_: List[ClassInfo] = []
+    functions: List[FunctionInfo] = []
+    module = importlib.import_module(module_path)
+    for name, type_ in inspect.getmembers(module):
+        if not hasattr(type_, "__module__"):
+            continue
+        if type_.__module__ != module_path:
+            continue
+
+        if inspect.isclass(type_):
+            if type(type_) == typing._TypedDictMeta:  # type: ignore
+                kind: ClassKind = "TypedDict"
+            elif issubclass(type_, Enum):
+                kind = "enum"
+            elif issubclass(type_, BaseModel):
+                kind = "Pydantic"
+            else:
+                kind = "Regular"
+
+            classes_.append(
+                ClassInfo(
+                    name=name,
+                    qualified_name=f"{namespace}.{name}",
+                    kind=kind,
+                    is_public=not name.startswith("_"),
+                )
+            )
+        elif inspect.isfunction(type_):
+            functions.append(
+                FunctionInfo(
+                    name=name,
+                    qualified_name=f"{namespace}.{name}",
+                    is_public=not name.startswith("_"),
+                )
+            )
+        else:
+            continue
+
+    return ModuleMembers(
+        classes_=classes_,
+        functions=functions,
+    )
+
+
+def _merge_module_members(
+    module_members: Sequence[ModuleMembers],
+) -> ModuleMembers:
+    """Merge module members."""
+    classes_: List[ClassInfo] = []
+    functions: List[FunctionInfo] = []
+    for module in module_members:
+        classes_.extend(module["classes_"])
+        functions.extend(module["functions"])
+
+    return ModuleMembers(
+        classes_=classes_,
+        functions=functions,
+    )
+
+
+def _load_package_modules(
+    package_directory: Union[str, Path], submodule: Optional[str] = None
+) -> Dict[str, ModuleMembers]:
+    """Recursively load modules of a package based on the file system.
+
+    Traversal based on the file system makes it easy to determine which
+    of the modules/packages are part of the package vs. 3rd party or built-in.
+
+    Parameters:
+        package_directory: Path to the package directory.
+        submodule: Optional name of submodule to load.
+
+    Returns:
+        list: A list of loaded module objects.
+    """
+    package_path = (
+        Path(package_directory)
+        if isinstance(package_directory, str)
+        else package_directory
+    )
+    modules_by_namespace = {}
+
+    # Get the high level package name
+    package_name = package_path.name
+
+    # If we are loading a submodule, add it in
+    if submodule is not None:
+        package_path = package_path / submodule
+
+    for file_path in package_path.rglob("*.py"):
+        if file_path.name.startswith("_"):
+            continue
+
+        relative_module_name = file_path.relative_to(package_path)
+
+        # Skip if any module part starts with an underscore
+        if any(part.startswith("_") for part in relative_module_name.parts):
+            continue
+
+        # Get the full namespace of the module
+        namespace = str(relative_module_name).replace(".py", "").replace("/", ".")
+        # Keep only the top level namespace
+        top_namespace = namespace.split(".")[0]
+
+        try:
+            # If submodule is present, we need to construct the paths in a slightly
+            # different way
+            if submodule is not None:
+                module_members = _load_module_members(
+                    f"{package_name}.{submodule}.{namespace}",
+                    f"{submodule}.{namespace}",
+                )
+            else:
+                module_members = _load_module_members(
+                    f"{package_name}.{namespace}", namespace
+                )
+            # Merge module members if the namespace already exists
+            if top_namespace in modules_by_namespace:
+                existing_module_members = modules_by_namespace[top_namespace]
+                _module_members = _merge_module_members(
+                    [existing_module_members, module_members]
+                )
+            else:
+                _module_members = module_members
+
+            modules_by_namespace[top_namespace] = _module_members
+
+        except ImportError as e:
+            print(f"Error: Unable to import module '{namespace}' with error: {e}")
+
+    return modules_by_namespace
+
+
+def _construct_doc(pkg: str, members_by_namespace: Dict[str, ModuleMembers]) -> str:
+    """Construct the contents of the reference.rst file for the given package.
+
+    Args:
+        pkg: The package name
+        members_by_namespace: The members of the package, dict organized by top level
+                              module contains a list of classes and functions
+                              inside of the top level namespace.
+
+    Returns:
+        The contents of the reference.rst file.
+    """
+    full_doc = f"""\
+=======================
+``{pkg}`` API Reference
+=======================
 
 """
-    for module, _members in sorted(members.items(), key=lambda kv: kv[0]):
-        classes = _members["classes"]
+    namespaces = sorted(members_by_namespace)
+
+    for module in namespaces:
+        _members = members_by_namespace[module]
+        classes = _members["classes_"]
         functions = _members["functions"]
         if not (classes or functions):
             continue
-        section = f":mod:`langchain.{module}`"
+        section = f":mod:`{pkg}.{module}`"
+        underline = "=" * (len(section) + 1)
         full_doc += f"""\
 {section}
-{'=' * (len(section) + 1)}
+{underline}
 
-.. automodule:: langchain.{module}
+.. automodule:: {pkg}.{module}
     :no-members:
     :no-inherited-members:
 
 """
 
         if classes:
-            cstring = "\n    ".join(sorted(classes))
             full_doc += f"""\
 Classes
 --------------
-.. currentmodule:: langchain
+.. currentmodule:: {pkg}
 
 .. autosummary::
     :toctree: {module}
-    :template: class.rst
-
-    {cstring}
-
 """
+
+            for class_ in sorted(classes, key=lambda c: c["qualified_name"]):
+                if not class_["is_public"]:
+                    continue
+
+                if class_["kind"] == "TypedDict":
+                    template = "typeddict.rst"
+                elif class_["kind"] == "enum":
+                    template = "enum.rst"
+                elif class_["kind"] == "Pydantic":
+                    template = "pydantic.rst"
+                else:
+                    template = "class.rst"
+
+                full_doc += f"""\
+    :template: {template}
+    
+    {class_["qualified_name"]}
+    
+"""
+
         if functions:
-            fstring = "\n    ".join(sorted(functions))
+            _functions = [f["qualified_name"] for f in functions if f["is_public"]]
+            fstring = "\n    ".join(sorted(_functions))
             full_doc += f"""\
 Functions
 --------------
-.. currentmodule:: langchain
+.. currentmodule:: {pkg}
 
 .. autosummary::
     :toctree: {module}
@@ -82,11 +280,46 @@ Functions
     return full_doc
 
 
-def main() -> None:
-    members = load_members()
-    full_doc = construct_doc(members)
+def _document_langchain_experimental() -> None:
+    """Document the langchain_experimental package."""
+    # Generate experimental_api_reference.rst
+    exp_members = _load_package_modules(EXP_DIR)
+    exp_doc = ".. _experimental_api_reference:\n\n" + _construct_doc(
+        "langchain_experimental", exp_members
+    )
+    with open(EXP_WRITE_FILE, "w") as f:
+        f.write(exp_doc)
+
+
+def _document_langchain_core() -> None:
+    """Document the main langchain package."""
+    # load top level module members
+    lc_members = _load_package_modules(PKG_DIR)
+
+    # Add additional packages
+    tools = _load_package_modules(PKG_DIR, "tools")
+    agents = _load_package_modules(PKG_DIR, "agents")
+    schema = _load_package_modules(PKG_DIR, "schema")
+
+    lc_members.update(
+        {
+            "agents.output_parsers": agents["output_parsers"],
+            "agents.format_scratchpad": agents["format_scratchpad"],
+            "tools.render": tools["render"],
+            "schema.runnable": schema["runnable"],
+        }
+    )
+
+    lc_doc = ".. _api_reference:\n\n" + _construct_doc("langchain", lc_members)
+
     with open(WRITE_FILE, "w") as f:
-        f.write(full_doc)
+        f.write(lc_doc)
+
+
+def main() -> None:
+    """Generate the reference.rst file for each package."""
+    _document_langchain_core()
+    _document_langchain_experimental()
 
 
 if __name__ == "__main__":

docs/api_reference/requirements.txt

@@ -1,4 +1,6 @@
 -e libs/langchain
+-e libs/experimental
+pydantic<2
 autodoc_pydantic==1.8.0
 myst_parser
 nbsphinx==0.8.9
@@ -10,4 +12,4 @@ sphinx-panels
 toml
 myst_nb
 sphinx_copybutton
-pydata-sphinx-theme==0.13.1
+pydata-sphinx-theme==0.13.1

docs/api_reference/templates/class.rst

@@ -5,17 +5,6 @@
 
 .. autoclass:: {{ objname }}
 
-   {% block methods %}
-   {% if methods %}
-   .. rubric:: {{ _('Methods') }}
-
-   .. autosummary::
-   {% for item in methods %}
-      ~{{ name }}.{{ item }}
-   {%- endfor %}
-   {% endif %}
-   {% endblock %}
-
    {% block attributes %}
    {% if attributes %}
    .. rubric:: {{ _('Attributes') }}
@@ -27,4 +16,21 @@
    {% endif %}
    {% endblock %}
 
+   {% block methods %}
+   {% if methods %}
+   .. rubric:: {{ _('Methods') }}
+
+   .. autosummary::
+   {% for item in methods %}
+      ~{{ name }}.{{ item }}
+   {%- endfor %}
+
+   {% for item in methods %}
+   .. automethod:: {{ name }}.{{ item }}
+   {%- endfor %}
+
+   {% endif %}
+   {% endblock %}
+
+
 .. example_links:: {{ objname }}

docs/api_reference/templates/enum.rst

@@ -0,0 +1,14 @@
+:mod:`{{module}}`.{{objname}}
+{{ underline }}==============
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+
+    {% block attributes %}
+    {% for item in attributes %}
+    .. autoattribute:: {{ item }}
+    {% endfor %}
+    {% endblock %}
+
+.. example_links:: {{ objname }}

docs/api_reference/templates/pydantic.rst

@@ -0,0 +1,22 @@
+:mod:`{{module}}`.{{objname}}
+{{ underline }}==============
+
+.. currentmodule:: {{ module }}
+
+.. autopydantic_model:: {{ objname }}
+    :model-show-json: False
+    :model-show-config-summary: False
+    :model-show-validator-members: False
+    :model-show-field-summary: False
+    :field-signature-prefix: param
+    :members:
+    :undoc-members:
+    :inherited-members:
+    :member-order: groupwise
+    :show-inheritance: True
+    :special-members: __call__
+
+    {% block attributes %}
+    {% endblock %}
+
+.. example_links:: {{ objname }}

docs/api_reference/templates/redirects.html

@@ -5,9 +5,10 @@
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <meta http-equiv="Refresh" content="0; url={{ redirect }}" />
-    <meta name="Description" content="scikit-learn: machine learning in Python">
+    <meta name="robots" content="follow, index">
+    <meta name="Description" content="Python API reference for LangChain.">
     <link rel="canonical" href="{{ redirect }}" />
-    <title>scikit-learn: machine learning in Python</title>
+    <title>LangChain Python API Reference Documentation.</title>
   </head>
   <body>
     <p>You will be automatically redirected to the <a href="{{ redirect }}">new location of this page</a>.</p>

docs/api_reference/templates/typeddict.rst

@@ -0,0 +1,14 @@
+:mod:`{{module}}`.{{objname}}
+{{ underline }}==============
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+
+    {% block attributes %}
+   {% for item in attributes %}
+  .. autoattribute:: {{ item }}
+   {% endfor %}
+   {% endblock %}
+
+.. example_links:: {{ objname }}

docs/api_reference/themes/scikit-learn-modern/layout.html

@@ -19,7 +19,7 @@
   {% block htmltitle %}
   <title>{{ title|striptags|e }}{{ titlesuffix }}</title>
   {% endblock %}
-  <link rel="canonical" href="http://scikit-learn.org/stable/{{pagename}}.html" />
+  <link rel="canonical" href="https://api.python.langchain.com/en/latest/{{pagename}}.html" />
 
   {% if favicon_url %}
   <link rel="shortcut icon" href="{{ favicon_url|e }}"/>

docs/api_reference/themes/scikit-learn-modern/nav.html

@@ -6,17 +6,6 @@
   {%- set top_container_cls = "sk-landing-container" %}
 {%- endif %}
 
-{% if theme_link_to_live_contributing_page|tobool %}
-{# Link to development page for live builds #}
-  {%- set development_link = "https://scikit-learn.org/dev/developers/index.html" %}
-{# Open on a new development page in new window/tab for live builds #}
-  {%- set development_attrs = 'target="_blank" rel="noopener noreferrer"' %}
-{%- else %}
-  {%- set development_link = pathto('developers/index') %}
-  {%- set development_attrs = '' %}
-{%- endif %}
-
-
 <nav id="navbar" class="{{ nav_bar_class }} navbar navbar-expand-md navbar-light bg-light py-0">
   <div class="container-fluid {{ top_container_cls }} px-0">
     {%- if logo_url %}
@@ -45,6 +34,9 @@
         <li class="nav-item">
           <a class="sk-nav-link nav-link" href="{{ pathto('api_reference') }}">API</a>
         </li>
+        <li class="nav-item">
+          <a class="sk-nav-link nav-link" href="{{ pathto('experimental_api_reference') }}">Experimental</a>
+        </li>
         <li class="nav-item">
           <a class="sk-nav-link nav-link" target="_blank" rel="noopener noreferrer" href="https://python.langchain.com/">Python Docs</a>
         </li>