Bagatur/bump 325 (#12440)

See discussion here:
https://github.com/langchain-ai/langchain/discussions/11680

The code is available for usage from langchain_experimental. The reason
for the deprecation is that the agents are relying on a Python REPL. The
code can only be run safely with appropriate sandboxing.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,132 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+conduct@langchain.dev.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

.github/CONTRIBUTING.md

@@ -1,20 +1,19 @@
 # Contributing to LangChain
 
 Hi there! Thank you for even being interested in contributing to LangChain.
-As an open source project in a rapidly developing field, we are extremely open
-to contributions, whether they be in the form of new features, improved infra, better documentation, or bug fixes.
+As an open-source project in a rapidly developing field, we are extremely open to contributions, whether they involve new features, improved infrastructure, better documentation, or bug fixes.
 
 ## 🗺️ Guidelines
 
 ### 👩‍💻 Contributing Code
 
-To contribute to this project, please follow a ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow.
+To contribute to this project, please follow the ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow.
 Please do not try to push directly to this repo unless you are a maintainer.
 
 Please follow the checked-in pull request template when opening pull requests. Note related issues and tag relevant
 maintainers.
 
-Pull requests cannot land without passing the formatting, linting and testing checks first. See [Testing](#testing) and
+Pull requests cannot land without passing the formatting, linting, and testing checks first. See [Testing](#testing) and
 [Formatting and Linting](#formatting-and-linting) for how to run these checks locally.
 
 It's essential that we maintain great documentation and testing. If you:
@@ -27,16 +26,14 @@ It's essential that we maintain great documentation and testing. If you:
   - Add a demo notebook in `docs/modules`.
   - Add unit and integration tests.
 
-We're a small, building-oriented team. If there's something you'd like to add or change, opening a pull request is the
+We are a small, progress-oriented team. If there's something you'd like to add or change, opening a pull request is the
 best way to get our attention.
 
 ### 🚩GitHub Issues
 
-Our [issues](https://github.com/langchain-ai/langchain/issues) page is kept up to date
-with bugs, improvements, and feature requests.
+Our [issues](https://github.com/langchain-ai/langchain/issues) page is kept up to date with bugs, improvements, and feature requests.
 
-There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help
-organize issues.
+There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help organize issues.
 
 If you start working on an issue, please assign it to yourself.
 
@@ -59,12 +56,12 @@ we do not want these to get in the way of getting good code into the codebase.
 
 ## 🚀 Quick Start
 
-This quick start describes running the repository locally.
+This quick start guide explains how to run the repository locally.
 For a [development container](https://containers.dev/), see the [.devcontainer folder](https://github.com/langchain-ai/langchain/tree/master/.devcontainer).
 
 ### Dependency Management: Poetry and other env/dependency managers
 
-This project uses [Poetry](https://python-poetry.org/) v1.6.1+ as a dependency manager.
+This project utilizes [Poetry](https://python-poetry.org/) v1.6.1+ as a dependency manager.
 
 ❗Note: *Before installing Poetry*, if you use `Conda`, create and activate a new Conda env (e.g. `conda create -n langchain python=3.9`)
 
@@ -75,11 +72,11 @@ tell Poetry to use the virtualenv python environment (`poetry config virtualenvs
 
 ### Core vs. Experimental
 
-There are two separate projects in this repository:
-- `langchain`: core langchain code, abstractions, and use cases
-- `langchain.experimental`: see the [Experimental README](../libs/experimental/README.md) for more information.
+This repository contains two separate projects:
+- `langchain`: core langchain code, abstractions, and use cases.
+- `langchain.experimental`: see the [Experimental README](https://github.com/langchain-ai/langchain/tree/master/libs/experimental/README.md) for more information.
 
-Each of these has their own development environment. Docs are run from the top-level makefile, but development
+Each of these has its own development environment. Docs are run from the top-level makefile, but development
 is split across separate test & release flows.
 
 For this quickstart, start with langchain core:
@@ -129,7 +126,7 @@ To run unit tests in Docker:
 make docker_tests
 ```
 
-There are also [integration tests and code-coverage](../libs/langchain/tests/README.md) available.
+There are also [integration tests and code-coverage](https://github.com/langchain-ai/langchain/tree/master/libs/langchain/tests/README.md) available.
 
 ### Formatting and Linting
 
@@ -282,13 +279,20 @@ make docs_build
 make api_docs_build
 ```
 
-Finally, you can run the linkchecker to make sure all links are valid:
+Finally, run the link checker to ensure all links are valid:
 
 ```bash
 make docs_linkcheck
 make api_docs_linkcheck
 ```
 
+### Verify Documentation changes
+
+After pushing documentation changes to the repository, you can preview and verify that the changes are 
+what you wanted by clicking the `View deployment` or `Visit Preview` buttons on the pull request `Conversation` page. 
+This will take you to a preview of the documentation changes.
+This preview is created by [Vercel](https://vercel.com/docs/getting-started-with-vercel).
+
 ## 🏭 Release Process
 
 As of now, LangChain has an ad hoc release process: releases are cut with high frequency by
@@ -300,4 +304,4 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
 ### 🌟 Recognition
 
 If your contribution has made its way into a release, we will want to give you credit on Twitter (only if you want though)!
-If you have a Twitter account you would like us to mention, please let us know in the PR or in another manner.
+If you have a Twitter account you would like us to mention, please let us know in the PR or through another means.

.github/workflows/_compile_integration_test.yml

@@ -0,0 +1,57 @@
+name: compile-integration-test
+
+on:
+  workflow_call:
+    inputs:
+      working-directory:
+        required: true
+        type: string
+        description: "From which folder this pipeline executes"
+
+env:
+  POETRY_VERSION: "1.6.1"
+
+jobs:
+  build:
+    defaults:
+      run:
+        working-directory: ${{ inputs.working-directory }}
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version:
+          - "3.8"
+          - "3.9"
+          - "3.10"
+          - "3.11"
+    name: Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: ${{ matrix.python-version }}
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: ${{ inputs.working-directory }}
+          cache-key: compile-integration
+
+      - name: Install integration dependencies
+        shell: bash
+        run: poetry install --with=test_integration
+
+      - name: Check integration tests compile
+        shell: bash
+        run: poetry run pytest -m compile tests/integration_tests
+
+      - name: Ensure the tests did not create any additional files
+        shell: bash
+        run: |
+          set -eu
+
+          STATUS="$(git status)"
+          echo "$STATUS"
+
+          # grep will exit non-zero if the target message isn't found,
+          # and `set -e` above will cause the step to fail.
+          echo "$STATUS" | grep 'nothing to commit, working tree clean'

.github/workflows/_lint.yml

@@ -7,6 +7,10 @@ on:
         required: true
         type: string
         description: "From which folder this pipeline executes"
+      langchain-location:
+        required: false
+        type: string
+        description: "Relative path to the langchain library folder"
 
 env:
   POETRY_VERSION: "1.6.1"
@@ -34,7 +38,7 @@ jobs:
           - "3.8"
           - "3.11"
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
         with:
           # Fetch the last FETCH_DEPTH commits, so the mtime-changing script
           # can accurately set the mtimes of files modified in the last FETCH_DEPTH commits.
@@ -117,8 +121,10 @@ jobs:
       - name: Install langchain editable
         working-directory: ${{ inputs.working-directory }}
         if: ${{ inputs.working-directory != 'libs/langchain' }}
+        env:
+          LANGCHAIN_LOCATION: ${{ inputs.langchain-location || '../langchain'}}
         run: |
-          pip install -e ../langchain
+          pip install -e "$LANGCHAIN_LOCATION"
 
       - name: Restore black cache
         uses: actions/cache@v3

.github/workflows/_pydantic_compatibility.yml

@@ -26,7 +26,7 @@ jobs:
           - "3.11"
     name: Pydantic v1/v2 compatibility - Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/_release.yml

@@ -30,7 +30,7 @@ jobs:
       run:
         working-directory: ${{ inputs.working-directory }}
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/_test.yml

@@ -26,7 +26,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/_test_release.yml

@@ -0,0 +1,50 @@
+name: test-release
+
+on:
+  workflow_call:
+    inputs:
+      working-directory:
+        required: true
+        type: string
+        description: "From which folder this pipeline executes"
+
+env:
+  POETRY_VERSION: "1.6.1"
+
+jobs:
+  publish_to_test_pypi:
+    runs-on: ubuntu-latest
+    permissions:
+      # This permission is used for trusted publishing:
+      # https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/
+      #
+      # Trusted publishing has to also be configured on PyPI for each package:
+      # https://docs.pypi.org/trusted-publishers/adding-a-publisher/
+      id-token: write
+    defaults:
+      run:
+        working-directory: ${{ inputs.working-directory }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
+        uses: "./.github/actions/poetry_setup"
+        with:
+          python-version: "3.10"
+          poetry-version: ${{ env.POETRY_VERSION }}
+          working-directory: ${{ inputs.working-directory }}
+          cache-key: release
+
+      - name: Build project for distribution
+        run: poetry build
+      - name: Check Version
+        id: check-version
+        run: |
+          echo version=$(poetry version --short) >> $GITHUB_OUTPUT
+      - name: Publish package to TestPyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          repository-url: https://test.pypi.org/legacy/
+          packages-dir: ${{ inputs.working-directory }}/dist/
+          verbose: true
+          print-hash: true

.github/workflows/codespell.yml

@@ -17,7 +17,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Install Dependencies
         run: |

.github/workflows/doc_lint.yml

@@ -19,4 +19,4 @@ jobs:
       run: |
         # We should not encourage imports directly from main init file
         # Expect for hub
-        git grep 'from langchain import' docs/{extras,docs_skeleton,snippets} | grep -vE 'from langchain import (hub)' && exit 1 || exit 0
+        git grep 'from langchain import' docs/{docs,snippets} | grep -vE 'from langchain import (hub)' && exit 1 || exit 0

.github/workflows/langchain_ci.yml

@@ -12,6 +12,7 @@ on:
       - '.github/workflows/_test.yml'
       - '.github/workflows/_pydantic_compatibility.yml'
       - '.github/workflows/langchain_ci.yml'
+      - 'libs/*'
       - 'libs/langchain/**'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
@@ -44,6 +45,13 @@ jobs:
       working-directory: libs/langchain
     secrets: inherit
 
+  compile-integration-tests:
+    uses:
+      ./.github/workflows/_compile_integration_test.yml
+    with:
+      working-directory: libs/langchain
+    secrets: inherit
+
   pydantic-compatibility:
     uses:
       ./.github/workflows/_pydantic_compatibility.yml
@@ -65,7 +73,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }} extended tests
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/langchain_cli_ci.yml

@@ -0,0 +1,53 @@
+---
+name: libs/cli CI
+
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+    paths:
+      - '.github/actions/poetry_setup/action.yml'
+      - '.github/tools/**'
+      - '.github/workflows/_lint.yml'
+      - '.github/workflows/_test.yml'
+      - '.github/workflows/_pydantic_compatibility.yml'
+      - '.github/workflows/langchain_cli_ci.yml'
+      - 'libs/cli/**'
+      - 'libs/*'
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+# If another push to the same PR or branch happens while this workflow is still running,
+# cancel the earlier run in favor of the next run.
+#
+# There's no point in testing an outdated version of the code. GitHub only allows
+# a limited number of job runners to be active at the same time, so it's better to cancel
+# pointless jobs early so that more useful jobs can run sooner.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  POETRY_VERSION: "1.6.1"
+  WORKDIR: "libs/cli"
+
+jobs:
+  lint:
+    uses:
+      ./.github/workflows/_lint.yml
+    with:
+      working-directory: libs/cli
+    secrets: inherit
+
+  test:
+    uses:
+      ./.github/workflows/_test.yml
+    with:
+      working-directory: libs/cli
+    secrets: inherit
+
+  pydantic-compatibility:
+    uses:
+      ./.github/workflows/_pydantic_compatibility.yml
+    with:
+      working-directory: libs/cli
+    secrets: inherit

.github/workflows/langchain_experimental_ci.yml

@@ -11,7 +11,7 @@ on:
       - '.github/workflows/_lint.yml'
       - '.github/workflows/_test.yml'
       - '.github/workflows/langchain_experimental_ci.yml'
-      - 'libs/langchain/**'
+      - 'libs/*'
       - 'libs/experimental/**'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
@@ -44,6 +44,13 @@ jobs:
       working-directory: libs/experimental
     secrets: inherit
 
+  compile-integration-tests:
+    uses:
+      ./.github/workflows/_compile_integration_test.yml
+    with:
+      working-directory: libs/experimental
+    secrets: inherit
+
   # It's possible that langchain-experimental works fine with the latest *published* langchain,
   # but is broken with the langchain on `master`.
   #
@@ -62,7 +69,7 @@ jobs:
           - "3.11"
     name: test with unpublished langchain - Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"
@@ -97,7 +104,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }} extended tests
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/langchain_experimental_test_release.yml

@@ -0,0 +1,13 @@
+---
+name: Experimental Test Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_test_release.yml
+    with:
+      working-directory: libs/experimental
+    secrets: inherit

.github/workflows/langchain_release.yml

@@ -24,3 +24,4 @@ jobs:
       - release
     uses:
       ./.github/workflows/langchain_release_docker.yml
+    secrets: inherit

.github/workflows/langchain_test_release.yml

@@ -0,0 +1,13 @@
+---
+name: Test Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_test_release.yml
+    with:
+      working-directory: libs/langchain
+    secrets: inherit

.github/workflows/scheduled_test.yml

@@ -24,7 +24,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }}
         uses: "./.github/actions/poetry_setup"
@@ -55,12 +55,21 @@ jobs:
           poetry install --with=test_integration
           poetry run pip install google-cloud-aiplatform
           poetry run pip install "boto3>=1.28.57"
+          if [[ ${{ matrix.python-version }} != "3.8" ]]
+          then
+            poetry run pip install fireworks-ai
+          fi
 
       - name: Run tests
         shell: bash
         env:
           OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          AZURE_OPENAI_API_VERSION: ${{ secrets.AZURE_OPENAI_API_VERSION }}
+          AZURE_OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_API_BASE }}
+          AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
+          AZURE_OPENAI_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_NAME }}
+          FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
         run: |
           make scheduled_tests
 

.github/workflows/templates_ci.yml

@@ -0,0 +1,37 @@
+---
+name: templates CI
+
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+    paths:
+      - '.github/actions/poetry_setup/action.yml'
+      - '.github/tools/**'
+      - '.github/workflows/_lint.yml'
+      - '.github/workflows/templates_ci.yml'
+      - 'templates/**'
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+# If another push to the same PR or branch happens while this workflow is still running,
+# cancel the earlier run in favor of the next run.
+#
+# There's no point in testing an outdated version of the code. GitHub only allows
+# a limited number of job runners to be active at the same time, so it's better to cancel
+# pointless jobs early so that more useful jobs can run sooner.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  POETRY_VERSION: "1.6.1"
+  WORKDIR: "templates"
+
+jobs:
+  lint:
+    uses:
+      ./.github/workflows/_lint.yml
+    with:
+      working-directory: templates
+      langchain-location: ../libs/langchain
+    secrets: inherit

.gitignore

@@ -174,6 +174,7 @@ docs/api_reference/*/
 !docs/api_reference/_static/
 !docs/api_reference/templates/
 !docs/api_reference/themes/
-docs/docs_skeleton/build
-docs/docs_skeleton/node_modules
-docs/docs_skeleton/yarn.lock
+docs/docs/build
+docs/docs/node_modules
+docs/docs/yarn.lock
+_dist

.gitmodules

@@ -1,4 +0,0 @@
-[submodule "docs/_docs_skeleton"]
-	path = docs/_docs_skeleton
-	url = https://github.com/langchain-ai/langchain-shared-docs
-	branch = main

.readthedocs.yaml

@@ -9,9 +9,14 @@ build:
   os: ubuntu-22.04
   tools:
     python: "3.11"
-  jobs:
-    pre_build:
+  commands:
+      - python -mvirtualenv $READTHEDOCS_VIRTUALENV_PATH
+      - python -m pip install --upgrade --no-cache-dir pip setuptools
+      - python -m pip install --upgrade --no-cache-dir sphinx readthedocs-sphinx-ext
+      - python -m pip install --exists-action=w --no-cache-dir -r docs/api_reference/requirements.txt
       - python docs/api_reference/create_api_rst.py
+      - cat docs/api_reference/conf.py
+      - python -m sphinx -T -E -b html -d _build/doctrees -c docs/api_reference docs/api_reference $READTHEDOCS_OUTPUT/html -j auto
 
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
@@ -25,5 +30,3 @@ sphinx:
 python:
    install:
    - requirements: docs/api_reference/requirements.txt
-   - method: pip
-     path: .

Makefile

@@ -15,10 +15,10 @@ docs_build:
 	docs/.local_build.sh
 
 docs_clean:
-	rm -r docs/_dist
+	rm -r _dist
 
 docs_linkcheck:
-	poetry run linkchecker docs/_dist/docs_skeleton/ --ignore-url node_modules
+	poetry run linkchecker _dist/docs/ --ignore-url node_modules
 
 api_docs_build:
 	poetry run python docs/api_reference/create_api_rst.py
@@ -53,4 +53,4 @@ help:
 	@echo 'api_docs_linkcheck           - run linkchecker on the API Reference documentation'
 	@echo 'spell_check               	- run codespell on the project'
 	@echo 'spell_fix               		- run codespell on the project and fix the errors'
-	@echo '-- TEST and LINT tasks are within libs/*/ per-package --'
+	@echo '-- TEST and LINT tasks are within libs/*/ per-package --'

README.md

@@ -18,8 +18,9 @@
 
 Looking for the JS/TS version? Check out [LangChain.js](https://github.com/langchain-ai/langchainjs).
 
-**Production Support:** As you move your LangChains into production, we'd love to offer more hands-on support.
-Fill out [this form](https://airtable.com/appwQzlErAS2qiP0L/shrGtGaVBVAz7NcV2) to share more about what you're building, and our team will get in touch.
+To help you ship LangChain apps to production faster, check out [LangSmith](https://smith.langchain.com). 
+[LangSmith](https://smith.langchain.com) is a unified developer platform for building, testing, and monitoring LLM applications. 
+Fill out [this form](https://airtable.com/appwQzlErAS2qiP0L/shrGtGaVBVAz7NcV2) to get off the waitlist or speak with our sales team
 
 ## 🚨Breaking Changes for select chains (SQLDatabase) on 7/28/23
 
@@ -92,7 +93,7 @@ Memory refers to persisting state between calls of a chain/agent. LangChain prov
 
 **🧐 Evaluation:**
 
-[BETA] Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.
+[BETA] Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is by using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.
 
 For more information on these concepts, please see our [full documentation](https://python.langchain.com).
 

cookbook/LLaMA2_sql_chat.ipynb

@@ -0,0 +1,400 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "fc935871-7640-41c6-b798-58514d860fe0",
+   "metadata": {},
+   "source": [
+    "## LLaMA2 chat with SQL\n",
+    "\n",
+    "Open source, local LLMs are great to consider for any application that demands data privacy.\n",
+    "\n",
+    "SQL is one good example. \n",
+    "\n",
+    "This cookbook shows how to perform text-to-SQL using various local versions of LLaMA2 run locally.\n",
+    "\n",
+    "## Packages"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "81adcf8b-395a-4f02-8749-ac976942b446",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "! pip install langchain replicate"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8e13ed66-300b-4a23-b8ac-44df68ee4733",
+   "metadata": {},
+   "source": [
+    "## LLM\n",
+    "\n",
+    "There are a few ways to access LLaMA2.\n",
+    "\n",
+    "To run locally, we use Ollama.ai. \n",
+    "\n",
+    "See [here](https://python.langchain.com/docs/integrations/chat/ollama) for details on installation and setup.\n",
+    "\n",
+    "Also, see [here](https://python.langchain.com/docs/guides/local_llms) for our full guide on local LLMs.\n",
+    " \n",
+    "To use an external API, which is not private, we can use Replicate."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "6a75a5c6-34ee-4ab9-a664-d9b432d812ee",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Init param `input` is deprecated, please use `model_kwargs` instead.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Local \n",
+    "from langchain.chat_models import ChatOllama\n",
+    "llama2_chat = ChatOllama(model=\"llama2:13b-chat\")\n",
+    "llama2_code = ChatOllama(model=\"codellama:7b-instruct\")\n",
+    "\n",
+    "# API\n",
+    "from getpass import getpass\n",
+    "from langchain.llms import Replicate\n",
+    "# REPLICATE_API_TOKEN = getpass()\n",
+    "# os.environ[\"REPLICATE_API_TOKEN\"] = REPLICATE_API_TOKEN\n",
+    "replicate_id = \"meta/llama-2-13b-chat:f4e2de70d66816a838a89eeeb621910adffb0dd0baba3976c96980970978018d\"\n",
+    "llama2_chat_replicate = Replicate(\n",
+    "    model=replicate_id,\n",
+    "    input={\"temperature\": 0.01, \n",
+    "           \"max_length\": 500, \n",
+    "           \"top_p\": 1}\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "ce96f7ea-b3d5-44e1-9fa5-a79e04a9e1fb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Simply set the LLM we want to use\n",
+    "llm = llama2_chat"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "80222165-f353-4e35-a123-5f70fd70c6c8",
+   "metadata": {},
+   "source": [
+    "## DB\n",
+    "\n",
+    "Connect to a SQLite DB.\n",
+    "\n",
+    "To create this particular DB, you can use the code and follow the steps shown [here](https://github.com/facebookresearch/llama-recipes/blob/main/demo_apps/StructuredLlama.ipynb)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "025bdd82-3bb1-4948-bc7c-c3ccd94fd05c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.utilities import SQLDatabase\n",
+    "db = SQLDatabase.from_uri(\"sqlite:///nba_roster.db\", sample_rows_in_table_info= 0)\n",
+    "\n",
+    "def get_schema(_):\n",
+    "    return db.get_table_info()\n",
+    "\n",
+    "def run_query(query):\n",
+    "    return db.run(query)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "654b3577-baa2-4e12-a393-f40e5db49ac7",
+   "metadata": {},
+   "source": [
+    "## Query a SQL DB \n",
+    "\n",
+    "Follow the runnables workflow [here](https://python.langchain.com/docs/expression_language/cookbook/sql_db)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "5a4933ea-d9c0-4b0a-8177-ba4490c6532b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "' SELECT \"Team\" FROM nba_roster WHERE \"NAME\" = \\'Klay Thompson\\';'"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Prompt\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "template = \"\"\"Based on the table schema below, write a SQL query that would answer the user's question:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query:\"\"\"\n",
+    "prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question, convert it to a SQL query. No pre-amble.\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "# Chain to query\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
+    "from langchain.schema.runnable import RunnablePassthrough\n",
+    "\n",
+    "sql_response = (\n",
+    "        RunnablePassthrough.assign(schema=get_schema)\n",
+    "        | prompt\n",
+    "        | llm.bind(stop=[\"\\nSQLResult:\"])\n",
+    "        | StrOutputParser()\n",
+    "    )\n",
+    "\n",
+    "sql_response.invoke({\"question\": \"What team is Klay Thompson on?\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a0e9e2c8-9b88-4853-ac86-001bc6cc6695",
+   "metadata": {},
+   "source": [
+    "We can review the results:\n",
+    "\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/afa56a06-b4e2-469a-a60f-c1746e75e42b/r) LLaMA2-13 Replicate API\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/2d4ecc72-6b8f-4523-8f0b-ea95c6b54a1d/r) LLaMA2-13 local \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "2a2825e3-c1b6-4f7d-b9c9-d9835de323bb",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=' Based on the table schema and SQL query, there are 30 unique teams in the NBA.')"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Chain to answer\n",
+    "template = \"\"\"Based on the table schema below, question, sql query, and sql response, write a natural language response:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query: {query}\n",
+    "SQL Response: {response}\"\"\"\n",
+    "prompt_response = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question and SQL response, convert it to a natural langugae answer. No pre-amble.\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "full_chain = (\n",
+    "    RunnablePassthrough.assign(query=sql_response) \n",
+    "    | RunnablePassthrough.assign(\n",
+    "        schema=get_schema,\n",
+    "        response=lambda x: db.run(x[\"query\"]),\n",
+    "    )\n",
+    "    | prompt_response \n",
+    "    | llm\n",
+    ")\n",
+    "\n",
+    "full_chain.invoke({\"question\": \"How many unique teams are there?\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ec17b3ee-6618-4681-b6df-089bbb5ffcd7",
+   "metadata": {},
+   "source": [
+    "We can review the results:\n",
+    "\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/10420721-746a-4806-8ecf-d6dc6399d739/r) LLaMA2-13 Replicate API\n",
+    "* [LangSmith trace](https://smith.langchain.com/public/5265ebab-0a22-4f37-936b-3300f2dfa1c1/r) LLaMA2-13 local "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1e85381b-1edc-4bb3-a7bd-2ab23f81e54d",
+   "metadata": {},
+   "source": [
+    "## Chat with a SQL DB \n",
+    "\n",
+    "Next, we can add memory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "1985aa1c-eb8f-4fb1-a54f-c8aa10744687",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "' SELECT \"Team\" FROM nba_roster WHERE \"NAME\" = \\'Klay Thompson\\';'"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Prompt\n",
+    "from langchain.memory import ConversationBufferMemory\n",
+    "from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
+    "template = \"\"\"Based on the table schema below, write a SQL query that would answer the user's question:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query:\"\"\"\n",
+    "prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question, convert it to a SQL query. No pre-amble.\"),\n",
+    "    MessagesPlaceholder(variable_name=\"history\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "memory = ConversationBufferMemory(return_messages=True)\n",
+    "\n",
+    "# Chain to query with memory \n",
+    "from langchain.schema.runnable import RunnableLambda\n",
+    "\n",
+    "sql_chain = (\n",
+    "    RunnablePassthrough.assign(\n",
+    "       schema=get_schema,\n",
+    "       history=RunnableLambda(lambda x: memory.load_memory_variables(x)[\"history\"])\n",
+    "    )| prompt\n",
+    "    | llm.bind(stop=[\"\\nSQLResult:\"])\n",
+    "    | StrOutputParser()\n",
+    ")\n",
+    "\n",
+    "def save(input_output):\n",
+    "    output = {\"output\": input_output.pop(\"output\")}\n",
+    "    memory.save_context(input_output, output)\n",
+    "    return output['output']\n",
+    "    \n",
+    "sql_response_memory = RunnablePassthrough.assign(output=sql_chain) | save\n",
+    "sql_response_memory.invoke({\"question\": \"What team is Klay Thompson on?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "0b45818a-1498-441d-b82d-23c29428c2bb",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "' SELECT \"SALARY\" FROM nba_roster WHERE \"NAME\" = \\'Klay Thompson\\';'"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "sql_response_memory.invoke({\"question\": \"What is his salary?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "800a7a3b-f411-478b-af51-2310cd6e0425",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=' Sure! Here\\'s the natural language response based on the given input:\\n\\n\"Klay Thompson\\'s salary is $43,219,440.\"')"
+      ]
+     },
+     "execution_count": 21,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Chain to answer\n",
+    "template = \"\"\"Based on the table schema below, question, sql query, and sql response, write a natural language response:\n",
+    "{schema}\n",
+    "\n",
+    "Question: {question}\n",
+    "SQL Query: {query}\n",
+    "SQL Response: {response}\"\"\"\n",
+    "prompt_response = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"Given an input question and SQL response, convert it to a natural langugae answer. No pre-amble.\"),\n",
+    "    (\"human\", template)\n",
+    "])\n",
+    "\n",
+    "full_chain = (\n",
+    "    RunnablePassthrough.assign(query=sql_response_memory) \n",
+    "    | RunnablePassthrough.assign(\n",
+    "        schema=get_schema,\n",
+    "        response=lambda x: db.run(x[\"query\"]),\n",
+    "    )\n",
+    "    | prompt_response \n",
+    "    | llm\n",
+    ")\n",
+    "\n",
+    "full_chain.invoke({\"question\": \"What is his salary?\"})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b77fee61-f4da-4bb1-8285-14101e505518",
+   "metadata": {},
+   "source": [
+    "Here is the [trace](https://smith.langchain.com/public/54794d18-2337-4ce2-8b9f-3d8a2df89e51/r)."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/README.md

@@ -0,0 +1,52 @@
+# LangChain cookbook
+
+Example code for building applications with LangChain, with an emphasis on more applied and end-to-end examples than contained in the [main documentation](https://python.langchain.com).
+
+Notebook | Description
+:- | :-
+[LLaMA2_sql_chat.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/LLaMA2_sql_chat.ipynb) | Build a chat application that interacts with a SQL database using an open source llm (llama2), specifically demonstrated on an SQLite database containing rosters.
+[Semi_Structured_RAG.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_Structured_RAG.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data, including text and tables, using unstructured for parsing, multi-vector retriever for storing, and lcel for implementing chains.
+[Semi_structured_and_multi_moda...](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_structured_and_multi_modal_RAG.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using unstructured for parsing, multi-vector retriever for storage and retrieval, and lcel for implementing chains.
+[Semi_structured_multi_modal_RA...](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_structured_multi_modal_RAG_LLaMA2.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using various tools and methods such as unstructured for parsing, multi-vector retriever for storing, lcel for implementing chains, and open source language models like llama2, llava, and gpt4all.
+[autogpt/autogpt.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/autogpt/autogpt.ipynb) | Implement autogpt, a language model, with langchain primitives such as llms, prompttemplates, vectorstores, embeddings, and tools.
+[autogpt/marathon_times.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/autogpt/marathon_times.ipynb) | Implement autogpt for finding winning marathon times.
+[baby_agi.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/baby_agi.ipynb) | Implement babyagi, an ai agent that can generate and execute tasks based on a given objective, with the flexibility to swap out specific vectorstores/model providers.
+[baby_agi_with_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/baby_agi_with_agent.ipynb) | Swap out the execution chain in the babyagi notebook with an agent that has access to tools, aiming to obtain more reliable information.
+[camel_role_playing.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/camel_role_playing.ipynb) | Implement the camel framework for creating autonomous cooperative agents in large-scale language models, using role-playing and inception prompting to guide chat agents towards task completion.
+[causal_program_aided_language_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/causal_program_aided_language_model.ipynb) | Implement the causal program-aided language (cpal) chain, which improves upon the program-aided language (pal) by incorporating causal structure to prevent hallucination in language models, particularly when dealing with complex narratives and math problems with nested dependencies.
+[code-analysis-deeplake.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/code-analysis-deeplake.ipynb) | Analyze its own code base with the help of gpt and activeloop's deep lake.
+[custom_agent_with_plugin_retri...](https://github.com/langchain-ai/langchain/tree/master/cookbook/custom_agent_with_plugin_retrieval.ipynb) | Build a custom agent that can interact with ai plugins by retrieving tools and creating natural language wrappers around openapi endpoints.
+[custom_agent_with_plugin_retri...](https://github.com/langchain-ai/langchain/tree/master/cookbook/custom_agent_with_plugin_retrieval_using_plugnplai.ipynb) | Build a custom agent with plugin retrieval functionality, utilizing ai plugins from the `plugnplai` directory.
+[databricks_sql_db.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/databricks_sql_db.ipynb) | Connect to databricks runtimes and databricks sql.
+[deeplake_semantic_search_over_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/deeplake_semantic_search_over_chat.ipynb) | Perform semantic search and question-answering over a group chat using activeloop's deep lake with gpt4.
+[elasticsearch_db_qa.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/elasticsearch_db_qa.ipynb) | Interact with elasticsearch analytics databases in natural language and build search queries via the elasticsearch dsl API.
+[forward_looking_retrieval_augm...](https://github.com/langchain-ai/langchain/tree/master/cookbook/forward_looking_retrieval_augmented_generation.ipynb) | Implement the forward-looking active retrieval augmented generation (flare) method, which generates answers to questions, identifies uncertain tokens, generates hypothetical questions based on these tokens, and retrieves relevant documents to continue generating the answer.
+[generative_agents_interactive_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/generative_agents_interactive_simulacra_of_human_behavior.ipynb) | Implement a generative agent that simulates human behavior, based on a research paper, using a time-weighted memory object backed by a langchain retriever.
+[gymnasium_agent_simulation.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/gymnasium_agent_simulation.ipynb) | Create a simple agent-environment interaction loop in simulated environments like text-based games with gymnasium.
+[hugginggpt.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/hugginggpt.ipynb) | Implement hugginggpt, a system that connects language models like chatgpt with the machine learning community via hugging face.
+[hypothetical_document_embeddin...](https://github.com/langchain-ai/langchain/tree/master/cookbook/hypothetical_document_embeddings.ipynb) | Improve document indexing with hypothetical document embeddings (hyde), an embedding technique that generates and embeds hypothetical answers to queries.
+[learned_prompt_optimization.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/learned_prompt_optimization.ipynb) | Automatically enhance language model prompts by injecting specific terms using reinforcement learning, which can be used to personalize responses based on user preferences.
+[llm_bash.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_bash.ipynb) | Perform simple filesystem commands using language learning models (llms) and a bash process.
+[llm_checker.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_checker.ipynb) | Create a self-checking chain using the llmcheckerchain function.
+[llm_math.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_math.ipynb) | Solve complex word math problems using language models and python repls.
+[llm_summarization_checker.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_summarization_checker.ipynb) | Check the accuracy of text summaries, with the option to run the checker multiple times for improved results.
+[llm_symbolic_math.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/llm_symbolic_math.ipynb) | Solve algebraic equations with the help of llms (language learning models) and sympy, a python library for symbolic mathematics.
+[meta_prompt.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/meta_prompt.ipynb) | Implement the meta-prompt concept, which is a method for building self-improving agents that reflect on their own performance and modify their instructions accordingly.
+[multi_modal_output_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multi_modal_output_agent.ipynb) | Generate multi-modal outputs, specifically images and text.
+[multi_player_dnd.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multi_player_dnd.ipynb) | Simulate multi-player dungeons & dragons games, with a custom function determining the speaking schedule of the agents.
+[multiagent_authoritarian.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multiagent_authoritarian.ipynb) | Implement a multi-agent simulation where a privileged agent controls the conversation, including deciding who speaks and when the conversation ends, in the context of a simulated news network.
+[multiagent_bidding.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multiagent_bidding.ipynb) | Implement a multi-agent simulation where agents bid to speak, with the highest bidder speaking next, demonstrated through a fictitious presidential debate example.
+[myscale_vector_sql.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/myscale_vector_sql.ipynb) | Access and interact with the myscale integrated vector database, which can enhance the performance of language model (llm) applications.
+[openai_functions_retrieval_qa....](https://github.com/langchain-ai/langchain/tree/master/cookbook/openai_functions_retrieval_qa.ipynb) | Structure response output in a question-answering system by incorporating openai functions into a retrieval pipeline.
+[petting_zoo.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/petting_zoo.ipynb) | Create multi-agent simulations with simulated environments using the petting zoo library.
+[plan_and_execute_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/plan_and_execute_agent.ipynb) | Create plan-and-execute agents that accomplish objectives by planning tasks with a language model (llm) and executing them with a separate agent.
+[press_releases.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/press_releases.ipynb) | Retrieve and query company press release data powered by [Kay.ai](https://kay.ai).
+[program_aided_language_model.i...](https://github.com/langchain-ai/langchain/tree/master/cookbook/program_aided_language_model.ipynb) | Implement program-aided language models as described in the provided research paper.
+[sales_agent_with_context.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/sales_agent_with_context.ipynb) | Implement a context-aware ai sales agent, salesgpt, that can have natural sales conversations, interact with other systems, and use a product knowledge base to discuss a company's offerings.
+[self_query_hotel_search.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/self_query_hotel_search.ipynb) | Build a hotel room search feature with self-querying retrieval, using a specific hotel recommendation dataset.
+[smart_llm.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/smart_llm.ipynb) | Implement a smartllmchain, a self-critique chain that generates multiple output proposals, critiques them to find the best one, and then improves upon it to produce a final output.
+[tree_of_thought.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/tree_of_thought.ipynb) | Query a large language model using the tree of thought technique.
+[twitter-the-algorithm-analysis...](https://github.com/langchain-ai/langchain/tree/master/cookbook/twitter-the-algorithm-analysis-deeplake.ipynb) | Analyze the source code of the Twitter algorithm with the help of gpt4 and activeloop's deep lake.
+[two_agent_debate_tools.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_agent_debate_tools.ipynb) | Simulate multi-agent dialogues where the agents can utilize various tools.
+[two_player_dnd.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_player_dnd.ipynb) | Simulate a two-player dungeons & dragons game, where a dialogue simulator class is used to coordinate the dialogue between the protagonist and the dungeon master.
+[wikibase_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/wikibase_agent.ipynb) | Create a simple wikibase agent that utilizes sparql generation, with testing done on http://wikidata.org.

cookbook/code-analysis-deeplake.ipynb

@@ -940,7 +940,7 @@
       "- DocArrayRetriever\n",
       "- ElasticSearchBM25Retriever\n",
       "- EnsembleRetriever\n",
-      "- GoogleCloudEnterpriseSearchRetriever\n",
+      "- GoogleVertexAISearchRetriever\n",
       "- AmazonKendraRetriever\n",
       "- KNNRetriever\n",
       "- LlamaIndexGraphRetriever and LlamaIndexRetriever\n",
@@ -992,7 +992,7 @@
     {
      "data": {
       "text/plain": [
-       "{'question': 'LangChain possesses a variety of retrievers including:\\n\\n1. ArxivRetriever\\n2. AzureCognitiveSearchRetriever\\n3. BM25Retriever\\n4. ChaindeskRetriever\\n5. ChatGPTPluginRetriever\\n6. ContextualCompressionRetriever\\n7. DocArrayRetriever\\n8. ElasticSearchBM25Retriever\\n9. EnsembleRetriever\\n10. GoogleCloudEnterpriseSearchRetriever\\n11. AmazonKendraRetriever\\n12. KNNRetriever\\n13. LlamaIndexGraphRetriever\\n14. LlamaIndexRetriever\\n15. MergerRetriever\\n16. MetalRetriever\\n17. MilvusRetriever\\n18. MultiQueryRetriever\\n19. ParentDocumentRetriever\\n20. PineconeHybridSearchRetriever\\n21. PubMedRetriever\\n22. RePhraseQueryRetriever\\n23. RemoteLangChainRetriever\\n24. SelfQueryRetriever\\n25. SVMRetriever\\n26. TFIDFRetriever\\n27. TimeWeightedVectorStoreRetriever\\n28. VespaRetriever\\n29. WeaviateHybridSearchRetriever\\n30. WebResearchRetriever\\n31. WikipediaRetriever\\n32. ZepRetriever\\n33. ZillizRetriever\\n\\nIt also includes self query translators like:\\n\\n1. ChromaTranslator\\n2. DeepLakeTranslator\\n3. MyScaleTranslator\\n4. PineconeTranslator\\n5. QdrantTranslator\\n6. WeaviateTranslator\\n\\nAnd remote retrievers like:\\n\\n1. RemoteLangChainRetriever'}"
+       "{'question': 'LangChain possesses a variety of retrievers including:\\n\\n1. ArxivRetriever\\n2. AzureCognitiveSearchRetriever\\n3. BM25Retriever\\n4. ChaindeskRetriever\\n5. ChatGPTPluginRetriever\\n6. ContextualCompressionRetriever\\n7. DocArrayRetriever\\n8. ElasticSearchBM25Retriever\\n9. EnsembleRetriever\\n10. GoogleVertexAISearchRetriever\\n11. AmazonKendraRetriever\\n12. KNNRetriever\\n13. LlamaIndexGraphRetriever\\n14. LlamaIndexRetriever\\n15. MergerRetriever\\n16. MetalRetriever\\n17. MilvusRetriever\\n18. MultiQueryRetriever\\n19. ParentDocumentRetriever\\n20. PineconeHybridSearchRetriever\\n21. PubMedRetriever\\n22. RePhraseQueryRetriever\\n23. RemoteLangChainRetriever\\n24. SelfQueryRetriever\\n25. SVMRetriever\\n26. TFIDFRetriever\\n27. TimeWeightedVectorStoreRetriever\\n28. VespaRetriever\\n29. WeaviateHybridSearchRetriever\\n30. WebResearchRetriever\\n31. WikipediaRetriever\\n32. ZepRetriever\\n33. ZillizRetriever\\n\\nIt also includes self query translators like:\\n\\n1. ChromaTranslator\\n2. DeepLakeTranslator\\n3. MyScaleTranslator\\n4. PineconeTranslator\\n5. QdrantTranslator\\n6. WeaviateTranslator\\n\\nAnd remote retrievers like:\\n\\n1. RemoteLangChainRetriever'}"
       ]
      },
      "execution_count": 31,
@@ -1124,7 +1124,7 @@
       "- DocArrayRetriever\n",
       "- ElasticSearchBM25Retriever\n",
       "- EnsembleRetriever\n",
-      "- GoogleCloudEnterpriseSearchRetriever\n",
+      "- GoogleVertexAISearchRetriever\n",
       "- AmazonKendraRetriever\n",
       "- KNNRetriever\n",
       "- LlamaIndexGraphRetriever and LlamaIndexRetriever\n",

cookbook/elasticsearch_db_qa.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Elasticsearch\n",
     "\n",
-    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/extras/use_cases/qa_structured/integrations/elasticsearch.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/qa_structured/integrations/elasticsearch.ipynb)\n",
     "\n",
     "We can use LLMs to interact with Elasticsearch analytics databases in natural language.\n",
     "\n",

cookbook/forward_looking_retrieval_augmented_generation.ipynb

@@ -135,9 +135,9 @@
    "outputs": [],
    "source": [
     "# We set this so we can see what exactly is going on\n",
-    "import langchain\n",
+    "from langchain.globals import set_verbose\n",
     "\n",
-    "langchain.verbose = True"
+    "set_verbose(True)"
    ]
   },
   {
@@ -489,7 +489,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.10.1"
   }
  },
  "nbformat": 4,

cookbook/llm_bash.ipynb

@@ -10,7 +10,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [
     {
@@ -37,13 +37,13 @@
        "'Hello World\\n'"
       ]
      },
-     "execution_count": 9,
+     "execution_count": 1,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "from langchain.chains import LLMBashChain\n",
+    "from langchain_experimental.llm_bash.base import LLMBashChain\n",
     "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
@@ -65,7 +65,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -98,7 +98,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -125,7 +125,7 @@
        "'Hello World\\n'"
       ]
      },
-     "execution_count": 11,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -149,7 +149,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -166,28 +166,24 @@
       "cd ..\n",
       "```\u001b[0m\n",
       "Code: \u001b[33;1m\u001b[1;3m['ls', 'cd ..']\u001b[0m\n",
-      "Answer: \u001b[33;1m\u001b[1;3mapi.html\t\t\tllm_summarization_checker.html\n",
-      "constitutional_chain.html\tmoderation.html\n",
-      "llm_bash.html\t\t\topenai_openapi.yaml\n",
-      "llm_checker.html\t\topenapi.html\n",
-      "llm_math.html\t\t\tpal.html\n",
-      "llm_requests.html\t\tsqlite.html\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3mcpal.ipynb  llm_bash.ipynb  llm_symbolic_math.ipynb\n",
+      "index.mdx   llm_math.ipynb  pal.ipynb\u001b[0m\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
     },
     {
      "data": {
       "text/plain": [
-       "'api.html\\t\\t\\tllm_summarization_checker.html\\r\\nconstitutional_chain.html\\tmoderation.html\\r\\nllm_bash.html\\t\\t\\topenai_openapi.yaml\\r\\nllm_checker.html\\t\\topenapi.html\\r\\nllm_math.html\\t\\t\\tpal.html\\r\\nllm_requests.html\\t\\tsqlite.html'"
+       "'cpal.ipynb  llm_bash.ipynb  llm_symbolic_math.ipynb\\r\\nindex.mdx   llm_math.ipynb  pal.ipynb'"
       ]
      },
-     "execution_count": 12,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "from langchain.utilities.bash import BashProcess\n",
+    "from langchain_experimental.llm_bash.bash import BashProcess\n",
     "\n",
     "\n",
     "persistent_process = BashProcess(persistent=True)\n",
@@ -200,7 +196,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -217,18 +213,19 @@
       "cd ..\n",
       "```\u001b[0m\n",
       "Code: \u001b[33;1m\u001b[1;3m['ls', 'cd ..']\u001b[0m\n",
-      "Answer: \u001b[33;1m\u001b[1;3mexamples\t\tgetting_started.html\tindex_examples\n",
-      "generic\t\t\thow_to_guides.rst\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3m_category_.yml\tdata_generation.ipynb\t\t   self_check\n",
+      "agents\t\tgraph\n",
+      "code_writing\tlearned_prompt_optimization.ipynb\u001b[0m\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
     },
     {
      "data": {
       "text/plain": [
-       "'examples\\t\\tgetting_started.html\\tindex_examples\\r\\ngeneric\\t\\t\\thow_to_guides.rst'"
+       "'_category_.yml\\tdata_generation.ipynb\\t\\t   self_check\\r\\nagents\\t\\tgraph\\r\\ncode_writing\\tlearned_prompt_optimization.ipynb'"
       ]
      },
-     "execution_count": 13,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -237,13 +234,6 @@
     "# Run the same command again and see that the state is maintained between calls\n",
     "bash_chain.run(text)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
@@ -262,7 +252,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.11.4"
   }
  },
  "nbformat": 4,

cookbook/llm_symbolic_math.ipynb

@@ -10,12 +10,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
     "from langchain.llms import OpenAI\n",
-    "from langchain.chains.llm_symbolic_math.base import LLMSymbolicMathChain\n",
+    "from langchain_experimental.llm_symbolic_math.base import LLMSymbolicMathChain\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "llm_symbolic_math = LLMSymbolicMathChain.from_llm(llm)"
@@ -30,7 +30,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 23,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -39,7 +39,7 @@
        "'Answer: exp(x)*sin(x) + exp(x)*cos(x)'"
       ]
      },
-     "execution_count": 23,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -50,7 +50,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 5,
    "metadata": {},
    "outputs": [
     {
@@ -59,7 +59,7 @@
        "'Answer: exp(x)*sin(x)'"
       ]
      },
-     "execution_count": 18,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -79,7 +79,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
@@ -88,7 +88,7 @@
        "'Answer: Eq(y(t), C2*exp(-t) + (C1 + t/2)*exp(t))'"
       ]
      },
-     "execution_count": 19,
+     "execution_count": 6,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -99,7 +99,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
@@ -108,7 +108,7 @@
        "'Answer: {0, -sqrt(3)*I/3, sqrt(3)*I/3}'"
       ]
      },
-     "execution_count": 21,
+     "execution_count": 7,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -119,7 +119,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [
     {
@@ -128,7 +128,7 @@
        "'Answer: (3 - sqrt(7), -sqrt(7) - 2, 1 - sqrt(7)), (sqrt(7) + 3, -2 + sqrt(7), 1 + sqrt(7))'"
       ]
      },
-     "execution_count": 22,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -140,9 +140,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "venv",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "venv"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -154,9 +154,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.11.4"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }

cookbook/plan_and_execute_agent.ipynb

@@ -0,0 +1,252 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "0ddfef23-3c74-444c-81dd-6753722997fa",
+   "metadata": {},
+   "source": [
+    "# Plan-and-execute\n",
+    "\n",
+    "Plan-and-execute agents accomplish an objective by first planning what to do, then executing the sub tasks. This idea is largely inspired by [BabyAGI](https://github.com/yoheinakajima/babyagi) and then the [\"Plan-and-Solve\" paper](https://arxiv.org/abs/2305.04091).\n",
+    "\n",
+    "The planning is almost always done by an LLM.\n",
+    "\n",
+    "The execution is usually done by a separate agent (equipped with tools)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a7ecb22a-7009-48ec-b14e-f0fa5aac1cd0",
+   "metadata": {},
+   "source": [
+    "## Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "5fbbd4ee-bfe8-4a25-afe4-8d1a552a3d2e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents.tools import Tool\n",
+    "from langchain.chains import LLMMathChain\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.utilities import DuckDuckGoSearchAPIWrapper\n",
+    "from langchain_experimental.plan_and_execute import PlanAndExecute, load_agent_executor, load_chat_planner"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e0e995e5-af9d-4988-bcd0-467a2a2e18cd",
+   "metadata": {},
+   "source": [
+    "## Tools"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "1d789f4e-54e3-4602-891a-f076e0ab9594",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "search = DuckDuckGoSearchAPIWrapper()\n",
+    "llm = OpenAI(temperature=0)\n",
+    "llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)\n",
+    "tools = [\n",
+    " Tool(\n",
+    "     name=\"Search\",\n",
+    "     func=search.run,\n",
+    "     description=\"useful for when you need to answer questions about current events\"\n",
+    " ),\n",
+    " Tool(\n",
+    "     name=\"Calculator\",\n",
+    "     func=llm_math_chain.run,\n",
+    "     description=\"useful for when you need to answer questions about math\"\n",
+    " ),\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "04dc6452-a07f-49f9-be12-95be1e2afccc",
+   "metadata": {},
+   "source": [
+    "## Planner, Executor, and Agent\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "d8f49c03-c804-458b-8122-c92b26c7b7dd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model = ChatOpenAI(temperature=0)\n",
+    "planner = load_chat_planner(model)\n",
+    "executor = load_agent_executor(model, tools, verbose=True)\n",
+    "agent = PlanAndExecute(planner=planner, executor=executor)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "78ba03dd-0322-4927-b58d-a7e2027fdbb3",
+   "metadata": {},
+   "source": [
+    "## Run example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "a57f7efe-7866-47a7-bce5-9c7b1047964e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"current prime minister of the UK\"\n",
+      "}\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"current prime minister of the UK\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mBottom right: Rishi Sunak is the current prime minister and the first non-white prime minister. The prime minister of the United Kingdom is the principal minister of the crown of His Majesty's Government, and the head of the British Cabinet. 3 min. British Prime Minister Rishi Sunak asserted his stance on gender identity in a speech Wednesday, stating it was \"common sense\" that \"a man is a man and a woman is a woman\" — a ... The former chancellor Rishi Sunak is the UK's new prime minister. Here's what you need to know about him. He won after running for the second time this year He lost to Liz Truss in September,... Isaeli Prime Minister Benjamin Netanyahu spoke with US President Joe Biden on Wednesday, the prime minister's office said in a statement. Netanyahu \"thanked the President for the powerful words of ... By Yasmeen Serhan/London Updated: October 25, 2022 12:56 PM EDT | Originally published: October 24, 2022 9:17 AM EDT S top me if you've heard this one before: After a tumultuous period of political...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mThe search results indicate that Rishi Sunak is the current prime minister of the UK. However, it's important to note that this information may not be accurate or up to date.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"current age of the prime minister of the UK\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mHow old is Rishi Sunak? Mr Sunak was born on 12 May, 1980, making him 42 years old. He first became an MP in 2015, aged 34, and has served the constituency of Richmond in Yorkshire ever since. He... Prime Ministers' ages when they took office From oldest to youngest, the ages of the PMs were as follows: Winston Churchill - 65 years old James Callaghan - 64 years old Clement Attlee - 62 years... Anna Kaufman USA TODAY Just a few days after Liz Truss resigned as prime minister, the UK has a new prime minister. Truss, who lasted a mere 45 days in office, will be replaced by Rishi... Advertisement Rishi Sunak is the youngest British prime minister of modern times. Mr. Sunak is 42 and started out in Parliament in 2015. Rishi Sunak was appointed as chancellor of the Exchequer... The first prime minister of the current United Kingdom of Great Britain and Northern Ireland upon its effective creation in 1922 (when 26 Irish counties seceded and created the Irish Free State) was Bonar Law, [10] although the country was not renamed officially until 1927, when Stanley Baldwin was the serving prime minister. [11]\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mBased on the search results, it seems that Rishi Sunak is the current prime minister of the UK. However, I couldn't find any specific information about his age. Would you like me to search again for the current age of the prime minister?\n",
+      "\n",
+      "Action:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Search\",\n",
+      "  \"action_input\": \"age of Rishi Sunak\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mRishi Sunak is 42 years old, making him the youngest person to hold the office of prime minister in modern times. How tall is Rishi Sunak? How Old Is Rishi Sunak? Rishi Sunak was born on May 12, 1980, in Southampton, England. Parents and Nationality Sunak's parents were born to Indian-origin families in East Africa before... Born on May 12, 1980, Rishi is currently 42 years old. He has been a member of parliament since 2015 where he was an MP for Richmond and has served in roles including Chief Secretary to the Treasury and the Chancellor of Exchequer while Boris Johnson was PM. Family Murty, 42, is the daughter of the Indian billionaire NR Narayana Murthy, often described as the Bill Gates of India, who founded the software company Infosys. According to reports, his... Sunak became the first non-White person to lead the country and, at age 42, the youngest to take on the role in more than a century. Like most politicians, Sunak is revered by some and...\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mBased on the search results, Rishi Sunak is currently 42 years old. He was born on May 12, 1980.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mThought: To calculate the age raised to the power of 0.43, I can use the calculator tool.\n",
+      "\n",
+      "Action:\n",
+      "```json\n",
+      "{\n",
+      "  \"action\": \"Calculator\",\n",
+      "  \"action_input\": \"42^0.43\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Entering new LLMMathChain chain...\u001b[0m\n",
+      "42^0.43\u001b[32;1m\u001b[1;3m```text\n",
+      "42**0.43\n",
+      "```\n",
+      "...numexpr.evaluate(\"42**0.43\")...\n",
+      "\u001b[0m\n",
+      "Answer: \u001b[33;1m\u001b[1;3m4.9888126515157\u001b[0m\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "Observation: \u001b[33;1m\u001b[1;3mAnswer: 4.9888126515157\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3mThe age raised to the power of 0.43 is approximately 4.9888126515157.\n",
+      "\n",
+      "Final Answer:\n",
+      "```json\n",
+      "{\n",
+      "  \"action\": \"Final Answer\",\n",
+      "  \"action_input\": \"The age raised to the power of 0.43 is approximately 4.9888126515157.\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3mAction:\n",
+      "```\n",
+      "{\n",
+      "  \"action\": \"Final Answer\",\n",
+      "  \"action_input\": \"The current prime minister of the UK is Rishi Sunak. His age raised to the power of 0.43 is approximately 4.9888126515157.\"\n",
+      "}\n",
+      "```\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'The current prime minister of the UK is Rishi Sunak. His age raised to the power of 0.43 is approximately 4.9888126515157.'"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.run(\"Who is the current prime minister of the UK? What is their current age raised to the 0.43 power?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0ef78a07-1a2a-46f8-9bc9-ae45f9bd706c",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "poetry-venv",
+   "language": "python",
+   "name": "poetry-venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/press_releases.ipynb

@@ -0,0 +1,152 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "62ee82e4-2ad8-498b-8438-fac388afe1a2",
+   "metadata": {},
+   "source": [
+    "Press Releases Data\n",
+    "=\n",
+    "\n",
+    "Press Releases data powered by [Kay.ai](https://kay.ai).\n",
+    "\n",
+    ">Press releases are used by companies to announce something noteworthy, including product launches, financial performance reports, partnerships, and other significant news. They are widely used by analysts to track corporate strategy, operational updates and financial performance.\n",
+    "Kay.ai obtains press releases of all US public companies from a variety of sources, which include the company's official press room and partnerships with various data API providers. \n",
+    "This data is updated till Sept 30th for free access, if you want to access the real-time feed, reach out to us at hello@kay.ai or [tweet at us](https://twitter.com/vishalrohra_)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8183d85d-365f-4672-a963-52b533547de0",
+   "metadata": {},
+   "source": [
+    "Setup\n",
+    "=\n",
+    "\n",
+    "First you will need to install the `kay` package. You will also need an API key: you can get one for free at [https://kay.ai](https://kay.ai/). Once you have an API key, you must set it as an environment variable `KAY_API_KEY`.\n",
+    "\n",
+    "In this example we're going to use the `KayAiRetriever`. Take a look at the [kay notebook](/docs/integrations/retrievers/kay) for more detailed information for the parmeters that it accepts."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02ec21c7-49fe-4844-b58a-bf064ad40b2a",
+   "metadata": {},
+   "source": [
+    "Examples\n",
+    "="
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "bf0395f7-6ebe-4136-8b0d-00b9dea3becd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      " ········\n",
+      " ········\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Setup API keys for Kay and OpenAI\n",
+    "from getpass import getpass\n",
+    "KAY_API_KEY = getpass()\n",
+    "OPENAI_API_KEY = getpass()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "f7fcaf70-29a4-444b-8f07-9784f808c300",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "os.environ[\"KAY_API_KEY\"] = KAY_API_KEY\n",
+    "os.environ[\"OPENAI_API_KEY\"] = OPENAI_API_KEY"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "ac00bf93-3635-4ffe-b9a6-a8b4f35c0c85",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains import ConversationalRetrievalChain\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.retrievers import KayAiRetriever\n",
+    "\n",
+    "model = ChatOpenAI(model_name=\"gpt-3.5-turbo\")\n",
+    "retriever = KayAiRetriever.create(dataset_id=\"company\", data_types=[\"PressRelease\"], num_contexts=6)\n",
+    "qa = ConversationalRetrievalChain.from_llm(model, retriever=retriever)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "8d9d927c-35b2-4a7b-8ea7-4d0350797941",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "-> **Question**: How is the healthcare industry adopting generative AI tools? \n",
+      "\n",
+      "**Answer**: The healthcare industry is adopting generative AI tools to improve various aspects of patient care and administrative tasks. Companies like HCA Healthcare Inc, Amazon Com Inc, and Mayo Clinic have collaborated with technology providers like Google Cloud, AWS, and Microsoft to implement generative AI solutions.\n",
+      "\n",
+      "HCA Healthcare is testing a nurse handoff tool that generates draft reports quickly and accurately, which nurses have shown interest in using. They are also exploring the use of Google's medically-tuned Med-PaLM 2 LLM to support caregivers in asking complex medical questions.\n",
+      "\n",
+      "Amazon Web Services (AWS) has introduced AWS HealthScribe, a generative AI-powered service that automatically creates clinical documentation. However, integrating multiple AI systems into a cohesive solution requires significant engineering resources, including access to AI experts, healthcare data, and compute capacity.\n",
+      "\n",
+      "Mayo Clinic is among the first healthcare organizations to deploy Microsoft 365 Copilot, a generative AI service that combines large language models with organizational data from Microsoft 365. This tool has the potential to automate tasks like form-filling, relieving administrative burdens on healthcare providers and allowing them to focus more on patient care.\n",
+      "\n",
+      "Overall, the healthcare industry is recognizing the potential benefits of generative AI tools in improving efficiency, automating tasks, and enhancing patient care. \n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "# More sample questions in the Playground on https://kay.ai\n",
+    "questions = [\n",
+    "    \"How is the healthcare industry adopting generative AI tools?\",\n",
+    "    #\"What are some recent challenges faced by the renewable energy sector?\",\n",
+    "]\n",
+    "chat_history = []\n",
+    "\n",
+    "for question in questions:\n",
+    "    result = qa({\"question\": question, \"chat_history\": chat_history})\n",
+    "    chat_history.append((question, result[\"answer\"]))\n",
+    "    print(f\"-> **Question**: {question} \\n\")\n",
+    "    print(f\"**Answer**: {result['answer']} \\n\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/rag_fusion.ipynb

@@ -0,0 +1,263 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "993c2768",
+   "metadata": {},
+   "source": [
+    "# RAG Fusion\n",
+    "\n",
+    "Re-implemented from [this GitHub repo](https://github.com/Raudaschl/rag-fusion), all credit to original author\n",
+    "\n",
+    "> RAG-Fusion, a search methodology that aims to bridge the gap between traditional search paradigms and the multifaceted dimensions of human queries. Inspired by the capabilities of Retrieval Augmented Generation (RAG), this project goes a step further by employing multiple query generation and Reciprocal Rank Fusion to re-rank search results."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ebcc6791",
+   "metadata": {},
+   "source": [
+    "## Setup\n",
+    "\n",
+    "For this example, we will use Pinecone and some fake data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "661a1c36",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import pinecone\n",
+    "from langchain.vectorstores import Pinecone\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "\n",
+    "pinecone.init(api_key=\"...\",environment=\"...\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "48ef7e93",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "all_documents = {\n",
+    "    \"doc1\": \"Climate change and economic impact.\",\n",
+    "    \"doc2\": \"Public health concerns due to climate change.\",\n",
+    "    \"doc3\": \"Climate change: A social perspective.\",\n",
+    "    \"doc4\": \"Technological solutions to climate change.\",\n",
+    "    \"doc5\": \"Policy changes needed to combat climate change.\",\n",
+    "    \"doc6\": \"Climate change and its impact on biodiversity.\",\n",
+    "    \"doc7\": \"Climate change: The science and models.\",\n",
+    "    \"doc8\": \"Global warming: A subset of climate change.\",\n",
+    "    \"doc9\": \"How climate change affects daily weather.\",\n",
+    "    \"doc10\": \"The history of climate change activism.\"\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fde89f0b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "vectorstore = Pinecone.from_texts(list(all_documents.values()), OpenAIEmbeddings(), index_name='rag-fusion')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "22ddd041",
+   "metadata": {},
+   "source": [
+    "## Define the Query Generator\n",
+    "\n",
+    "We will now define a chain to do the query generation"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "1d547524",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain.schema.output_parser import StrOutputParser"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 68,
+   "id": "af9ab4db",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import hub\n",
+    "\n",
+    "prompt = hub.pull('langchain-ai/rag-fusion-query-generation')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "3628b552",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# prompt = ChatPromptTemplate.from_messages([\n",
+    "#     (\"system\", \"You are a helpful assistant that generates multiple search queries based on a single input query.\"),\n",
+    "#     (\"user\", \"Generate multiple search queries related to: {original_query}\"),\n",
+    "#     (\"user\", \"OUTPUT (4 queries):\")\n",
+    "# ])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "8d6cbb73",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "generate_queries = prompt | ChatOpenAI(temperature=0) | StrOutputParser() | (lambda x: x.split(\"\\n\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ee2824cd",
+   "metadata": {},
+   "source": [
+    "## Define the full chain\n",
+    "\n",
+    "We can now put it all together and define the full chain. This chain:\n",
+    "    \n",
+    "    1. Generates a bunch of queries\n",
+    "    2. Looks up each query in the retriever\n",
+    "    3. Joins all the results together using reciprocal rank fusion\n",
+    "    \n",
+    "    \n",
+    "Note that it does NOT do a final generation step"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 50,
+   "id": "ca0bfec4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "original_query = \"impact of climate change\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 75,
+   "id": "02437d65",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "vectorstore = Pinecone.from_existing_index(\"rag-fusion\", OpenAIEmbeddings())\n",
+    "retriever = vectorstore.as_retriever()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 76,
+   "id": "46a9a0e6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.load import dumps, loads\n",
+    "def reciprocal_rank_fusion(results: list[list], k=60):\n",
+    "    fused_scores = {}\n",
+    "    for docs in results:\n",
+    "        # Assumes the docs are returned in sorted order of relevance\n",
+    "        for rank, doc in enumerate(docs):\n",
+    "            doc_str = dumps(doc)\n",
+    "            if doc_str not in fused_scores:\n",
+    "                fused_scores[doc_str] = 0\n",
+    "            previous_score = fused_scores[doc_str]\n",
+    "            fused_scores[doc_str] += 1 / (rank + k)\n",
+    "            \n",
+    "    reranked_results = [(loads(doc), score) for doc, score in sorted(fused_scores.items(), key=lambda x: x[1], reverse=True)]\n",
+    "    return reranked_results "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 77,
+   "id": "3f9d4502",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = generate_queries | retriever.map() | reciprocal_rank_fusion"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 78,
+   "id": "d70c4fcd",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[(Document(page_content='Climate change and economic impact.'),\n",
+       "  0.06558258417063283),\n",
+       " (Document(page_content='Climate change: A social perspective.'),\n",
+       "  0.06400409626216078),\n",
+       " (Document(page_content='How climate change affects daily weather.'),\n",
+       "  0.04787506400409626),\n",
+       " (Document(page_content='Climate change and its impact on biodiversity.'),\n",
+       "  0.03306010928961749),\n",
+       " (Document(page_content='Public health concerns due to climate change.'),\n",
+       "  0.016666666666666666),\n",
+       " (Document(page_content='Technological solutions to climate change.'),\n",
+       "  0.016666666666666666),\n",
+       " (Document(page_content='Policy changes needed to combat climate change.'),\n",
+       "  0.01639344262295082)]"
+      ]
+     },
+     "execution_count": 78,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke({\"original_query\": original_query})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7866e551",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/rewrite.ipynb

@@ -0,0 +1,351 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "260629f9",
+   "metadata": {},
+   "source": [
+    "# Rewrite-Retrieve-Read\n",
+    "\n",
+    "**Rewrite-Retrieve-Read** is a method proposed in the paper [Query Rewriting for Retrieval-Augmented Large Language Models](https://arxiv.org/pdf/2305.14283.pdf)\n",
+    "\n",
+    "> Because the original query can not be always optimal to retrieve for the LLM, especially in the real world... we first prompt an LLM to rewrite the queries, then conduct retrieval-augmented reading\n",
+    "\n",
+    "We show how you can easily do that with LangChain Expression Language"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eda93712",
+   "metadata": {},
+   "source": [
+    "## Baseline\n",
+    "\n",
+    "Baseline RAG (**Retrieve-and-read**) can be done like the following:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "1d2edbd2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from operator import itemgetter\n",
+    "\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
+    "from langchain.schema.runnable import RunnablePassthrough, RunnableLambda\n",
+    "from langchain.utilities import DuckDuckGoSearchAPIWrapper"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "86a46aa9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "template = \"\"\"Answer the users question based only on the following context:\n",
+    "\n",
+    "<context>\n",
+    "{context}\n",
+    "</context>\n",
+    "\n",
+    "Question: {question}\n",
+    "\"\"\"\n",
+    "prompt = ChatPromptTemplate.from_template(template)\n",
+    "\n",
+    "model = ChatOpenAI(temperature=0)\n",
+    "\n",
+    "search = DuckDuckGoSearchAPIWrapper()\n",
+    "\n",
+    "\n",
+    "def retriever(query):\n",
+    "    return search.run(query)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "8566d48e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = (\n",
+    "    {\"context\": retriever, \"question\": RunnablePassthrough()} \n",
+    "    | prompt \n",
+    "    | model \n",
+    "    | StrOutputParser()\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "5c57f9ee",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "simple_query = \"what is langchain?\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "37c5f962",
+   "metadata": {
+    "scrolled": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"LangChain is a powerful and versatile Python library that enables developers and researchers to create, experiment with, and analyze language models and agents. It simplifies the development of language-based applications by providing a suite of features for artificial general intelligence. It can be used to build chatbots, perform document analysis and summarization, and streamline interaction with various large language model providers. LangChain's unique proposition is its ability to create logical links between one or more language models, known as Chains. It is an open-source library that offers a generic interface to foundation models and allows prompt management and integration with other components and tools.\""
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke(simple_query)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "23bdb9bd",
+   "metadata": {},
+   "source": [
+    "While this is fine for well formatted queries, it can break down for more complicated queries"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "8df6a814",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "distracted_query = \"man that sam bankman fried trial was crazy! what is langchain?\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "16d7db64",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Based on the given context, there is no information provided about \"langchain.\"'"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke(distracted_query)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0b4f8b93",
+   "metadata": {},
+   "source": [
+    "This is because the retriever does a bad job with these \"distracted\" queries"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "3439d8dc",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Business She\\'s the star witness against Sam Bankman-Fried. Her testimony was explosive Gary Wang, who co-founded both FTX and Alameda Research, said Bankman-Fried directed him to change a... The Verge, following the trial\\'s Oct. 4 kickoff: \"Is Sam Bankman-Fried\\'s Defense Even Trying to Win?\". CBS Moneywatch, from Thursday: \"Sam Bankman-Fried\\'s Lawyer Struggles to Poke ... Sam Bankman-Fried, FTX\\'s founder, responded with a single word: \"Oof.\". Less than a year later, Mr. Bankman-Fried, 31, is on trial in federal court in Manhattan, fighting criminal charges ... July 19, 2023. A U.S. judge on Wednesday overruled objections by Sam Bankman-Fried\\'s lawyers and allowed jurors in the FTX founder\\'s fraud trial to see a profane message he sent to a reporter days ... Sam Bankman-Fried, who was once hailed as a virtuoso in cryptocurrency trading, is on trial over the collapse of FTX, the financial exchange he founded. Bankman-Fried is accused of...'"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "retriever(distracted_query)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7eb748ac",
+   "metadata": {},
+   "source": [
+    "## Rewrite-Retrieve-Read Implementation\n",
+    "\n",
+    "The main part is a rewriter to rewrite the search query"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "88ae702e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "template = \"\"\"Provide a better search query for \\\n",
+    "web search engine to answer the given question, end \\\n",
+    "the queries with ’**’. Question: \\\n",
+    "{x} Answer:\"\"\"\n",
+    "rewrite_prompt = ChatPromptTemplate.from_template(template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "184e1bcb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import hub\n",
+    "\n",
+    "rewrite_prompt = hub.pull(\"langchain-ai/rewrite\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "a4c23d40",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Provide a better search query for web search engine to answer the given question, end the queries with ’**’.  Question {x} Answer:\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(rewrite_prompt.template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "f55cd010",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Parser to remove the `**`\n",
+    "\n",
+    "def _parse(text):\n",
+    "    return text.strip(\"**\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "c9c34bef",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "rewriter = rewrite_prompt | ChatOpenAI(temperature=0) | StrOutputParser() | _parse"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "fb17fb3d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'What is the definition and purpose of Langchain?'"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "rewriter.invoke({\"x\": distracted_query})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "f83edb09",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "rewrite_retrieve_read_chain = (\n",
+    "    {\n",
+    "        \"context\": {\"x\": RunnablePassthrough()} | rewriter | retriever,\n",
+    "        \"question\": RunnablePassthrough()} \n",
+    "    | prompt \n",
+    "    | model \n",
+    "    | StrOutputParser()\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "43096322",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Based on the given context, LangChain is an open-source framework designed to simplify the creation of applications using large language models (LLMs). It enables LLM models to generate responses based on up-to-date online information and simplifies the organization of large volumes of data for easy access by LLMs. LangChain offers a standard interface for chains, integrations with other tools, and end-to-end chains for common applications. It is a robust library that streamlines interaction with various LLM providers. LangChain\\'s unique proposition is its ability to create logical links between one or more LLMs, known as Chains. It is an AI framework with features that simplify the development of language-based applications and offers a suite of features for artificial general intelligence. However, the context does not provide any information about the \"sam bankman fried trial\" mentioned in the question.'"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "rewrite_retrieve_read_chain.invoke(distracted_query)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "59874b4f",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

cookbook/sales_agent_with_context.ipynb

@@ -12,7 +12,7 @@
     "\n",
     "SalesGPT is context-aware, which means it can understand what section of a sales conversation it is in and act accordingly.\n",
     " \n",
-    "As such, this agent can have a natural sales conversation with a prospect and behaves based on the conversation stage. Hence, this notebook demonstrates how we can use AI to automate sales development representatives activites, such as outbound sales calls. \n",
+    "As such, this agent can have a natural sales conversation with a prospect and behaves based on the conversation stage. Hence, this notebook demonstrates how we can use AI to automate sales development representatives activities, such as outbound sales calls. \n",
     "\n",
     "Additionally, the AI Sales agent has access to tools, which allow it to interact with other systems.\n",
     "\n",
@@ -66,7 +66,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# install aditional dependencies\n",
+    "# install additional dependencies\n",
     "# ! pip install chromadb openai tiktoken"
    ]
   },
@@ -150,7 +150,7 @@
     "            {conversation_history}\n",
     "            ===\n",
     "\n",
-    "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting ony from the following options:\n",
+    "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting only from the following options:\n",
     "            1. Introduction: Start the conversation by introducing yourself and your company. Be polite and respectful while keeping the tone of the conversation professional.\n",
     "            2. Qualification: Qualify the prospect by confirming if they are the right person to talk to regarding your product/service. Ensure that they have the authority to make purchasing decisions.\n",
     "            3. Value proposition: Briefly explain how your product/service can benefit the prospect. Focus on the unique selling points and value proposition of your product/service that sets it apart from competitors.\n",
@@ -277,7 +277,7 @@
       "            \n",
       "            ===\n",
       "\n",
-      "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting ony from the following options:\n",
+      "            Now determine what should be the next immediate conversation stage for the agent in the sales conversation by selecting only from the following options:\n",
       "            1. Introduction: Start the conversation by introducing yourself and your company. Be polite and respectful while keeping the tone of the conversation professional.\n",
       "            2. Qualification: Qualify the prospect by confirming if they are the right person to talk to regarding your product/service. Ensure that they have the authority to make purchasing decisions.\n",
       "            3. Value proposition: Briefly explain how your product/service can benefit the prospect. Focus on the unique selling points and value proposition of your product/service that sets it apart from competitors.\n",

cookbook/smart_llm.ipynb

@@ -17,7 +17,7 @@
     "\n",
     "Note that SmartLLMChains\n",
     "- use more LLM passes (ie n+2 instead of just 1)\n",
-    "- only work then the underlying LLM has the capability for reflection, whicher smaller models often don't\n",
+    "- only work then the underlying LLM has the capability for reflection, which smaller models often don't\n",
     "- only work with underlying models that return exactly 1 output, not multiple\n",
     "\n",
     "This notebook demonstrates how to use a SmartLLMChain."
@@ -241,7 +241,7 @@
     "    ideation_llm=ChatOpenAI(temperature=0.9, model_name=\"gpt-4\"),\n",
     "    llm=ChatOpenAI(\n",
     "        temperature=0, model_name=\"gpt-4\"\n",
-    "    ),  # will be used for critqiue and resolution as no specific llms are given\n",
+    "    ),  # will be used for critique and resolution as no specific llms are given\n",
     "    prompt=prompt,\n",
     "    n_ideas=3,\n",
     "    verbose=True,\n",

cookbook/sql_db_qa.mdx

@@ -1,3 +1,7 @@
+# SQL Database Chain
+
+This example demonstrates the use of the `SQLDatabaseChain` for answering questions over a SQL database.
+
 Under the hood, LangChain uses SQLAlchemy to connect to SQL databases. The `SQLDatabaseChain` can therefore be used with any SQL dialect supported by SQLAlchemy, such as MS SQL, MySQL, MariaDB, PostgreSQL, Oracle SQL, [Databricks](/docs/ecosystem/integrations/databricks.html) and SQLite. Please refer to the SQLAlchemy documentation for more information about requirements for connecting to your database. For example, a connection to MySQL requires an appropriate connector such as PyMySQL. A URI for a MySQL connection might look like: `mysql+pymysql://user:pass@some_mysql_db_address/db_name`.
 
 This demonstration uses SQLite and the example Chinook database.
@@ -31,8 +35,8 @@ db_chain.run("How many employees are there?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many employees are there?
     SQLQuery:
@@ -71,8 +75,8 @@ db_chain.run("How many albums by Aerosmith?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many albums by Aerosmith?
     SQLQuery:SELECT COUNT(*) FROM Album WHERE ArtistId = 3;
@@ -129,8 +133,8 @@ db_chain.run("How many employees are there in the foobar table?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many employees are there in the foobar table?
     SQLQuery:SELECT COUNT(*) FROM Employee;
@@ -165,8 +169,8 @@ result["intermediate_steps"]
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many employees are there in the foobar table?
     SQLQuery:SELECT COUNT(*) FROM Employee;
@@ -313,8 +317,8 @@ db_chain.run("What are some example tracks by composer Johann Sebastian Bach?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     What are some example tracks by composer Johann Sebastian Bach?
     SQLQuery:SELECT Name FROM Track WHERE Composer = 'Johann Sebastian Bach' LIMIT 3
@@ -352,23 +356,23 @@ print(db.table_info)
 <CodeOutputBlock lang="python">
 
 ```
-    
+
     CREATE TABLE "Track" (
-    	"TrackId" INTEGER NOT NULL, 
-    	"Name" NVARCHAR(200) NOT NULL, 
-    	"AlbumId" INTEGER, 
-    	"MediaTypeId" INTEGER NOT NULL, 
-    	"GenreId" INTEGER, 
-    	"Composer" NVARCHAR(220), 
-    	"Milliseconds" INTEGER NOT NULL, 
-    	"Bytes" INTEGER, 
-    	"UnitPrice" NUMERIC(10, 2) NOT NULL, 
-    	PRIMARY KEY ("TrackId"), 
-    	FOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"), 
-    	FOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"), 
+    	"TrackId" INTEGER NOT NULL,
+    	"Name" NVARCHAR(200) NOT NULL,
+    	"AlbumId" INTEGER,
+    	"MediaTypeId" INTEGER NOT NULL,
+    	"GenreId" INTEGER,
+    	"Composer" NVARCHAR(220),
+    	"Milliseconds" INTEGER NOT NULL,
+    	"Bytes" INTEGER,
+    	"UnitPrice" NUMERIC(10, 2) NOT NULL,
+    	PRIMARY KEY ("TrackId"),
+    	FOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"),
+    	FOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"),
     	FOREIGN KEY("AlbumId") REFERENCES "Album" ("AlbumId")
     )
-    
+
     /*
     2 rows from Track table:
     TrackId	Name	AlbumId	MediaTypeId	GenreId	Composer	Milliseconds	Bytes	UnitPrice
@@ -392,8 +396,8 @@ db_chain.run("What are some example tracks by Bach?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     What are some example tracks by Bach?
     SQLQuery:SELECT "Name", "Composer" FROM "Track" WHERE "Composer" LIKE '%Bach%' LIMIT 5
@@ -411,7 +415,7 @@ db_chain.run("What are some example tracks by Bach?")
 </CodeOutputBlock>
 
 ### Custom Table Info
-In some cases, it can be useful to provide custom table information instead of using the automatically generated table definitions and the first `sample_rows_in_table_info` sample rows. For example, if you know that the first few rows of a table are uninformative, it could help to manually provide example rows that are more diverse or provide more information to the model. It is also possible to limit the columns that will be visible to the model if there are unnecessary columns. 
+In some cases, it can be useful to provide custom table information instead of using the automatically generated table definitions and the first `sample_rows_in_table_info` sample rows. For example, if you know that the first few rows of a table are uninformative, it could help to manually provide example rows that are more diverse or provide more information to the model. It is also possible to limit the columns that will be visible to the model if there are unnecessary columns.
 
 This information can be provided as a dictionary with table names as the keys and table information as the values. For example, let's provide a custom definition and sample rows for the Track table with only a few columns:
 
@@ -419,7 +423,7 @@ This information can be provided as a dictionary with table names as the keys an
 ```python
 custom_table_info = {
     "Track": """CREATE TABLE Track (
-	"TrackId" INTEGER NOT NULL, 
+	"TrackId" INTEGER NOT NULL,
 	"Name" NVARCHAR(200) NOT NULL,
 	"Composer" NVARCHAR(220),
 	PRIMARY KEY ("TrackId")
@@ -448,22 +452,22 @@ print(db.table_info)
 <CodeOutputBlock lang="python">
 
 ```
-    
+
     CREATE TABLE "Playlist" (
-    	"PlaylistId" INTEGER NOT NULL, 
-    	"Name" NVARCHAR(120), 
+    	"PlaylistId" INTEGER NOT NULL,
+    	"Name" NVARCHAR(120),
     	PRIMARY KEY ("PlaylistId")
     )
-    
+
     /*
     2 rows from Playlist table:
     PlaylistId	Name
     1	Music
     2	Movies
     */
-    
+
     CREATE TABLE Track (
-    	"TrackId" INTEGER NOT NULL, 
+    	"TrackId" INTEGER NOT NULL,
     	"Name" NVARCHAR(200) NOT NULL,
     	"Composer" NVARCHAR(220),
     	PRIMARY KEY ("TrackId")
@@ -490,8 +494,8 @@ db_chain.run("What are some example tracks by Bach?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     What are some example tracks by Bach?
     SQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE '%Bach%' LIMIT 5;
@@ -501,31 +505,31 @@ db_chain.run("What are some example tracks by Bach?")
     Unless the user specifies in the question a specific number of examples to obtain, query for at most 5 results using the LIMIT clause as per SQLite. You can order the results to return the most informative data in the database.
     Never query for all columns from a table. You must query only the columns that are needed to answer the question. Wrap each column name in double quotes (") to denote them as delimited identifiers.
     Pay attention to use only the column names you can see in the tables below. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.
-    
+
     Use the following format:
-    
+
     Question: "Question here"
     SQLQuery: "SQL Query to run"
     SQLResult: "Result of the SQLQuery"
     Answer: "Final answer here"
-    
+
     Only use the following tables:
-    
+
     CREATE TABLE "Playlist" (
-    	"PlaylistId" INTEGER NOT NULL, 
-    	"Name" NVARCHAR(120), 
+    	"PlaylistId" INTEGER NOT NULL,
+    	"Name" NVARCHAR(120),
     	PRIMARY KEY ("PlaylistId")
     )
-    
+
     /*
     2 rows from Playlist table:
     PlaylistId	Name
     1	Music
     2	Movies
     */
-    
+
     CREATE TABLE Track (
-    	"TrackId" INTEGER NOT NULL, 
+    	"TrackId" INTEGER NOT NULL,
     	"Name" NVARCHAR(200) NOT NULL,
     	"Composer" NVARCHAR(220),
     	PRIMARY KEY ("TrackId")
@@ -537,7 +541,7 @@ db_chain.run("What are some example tracks by Bach?")
     2	Balls to the Wall	None
     3	My favorite song ever	The coolest composer of all time
     */
-    
+
     Question: What are some example tracks by Bach?
     SQLQuery:SELECT "Name" FROM Track WHERE "Composer" LIKE '%Bach%' LIMIT 5;
     SQLResult: [('American Woman',), ('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace',), ('Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria',), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude',), ('Toccata and Fugue in D Minor, BWV 565: I. Toccata',)]
@@ -557,7 +561,7 @@ db_chain.run("What are some example tracks by Bach?")
 
 ### SQL Views
 
-In some case, the table schema can be hidden behind a JSON or JSONB column. Adding row samples into the prompt might help won't always describe the data perfectly. 
+In some case, the table schema can be hidden behind a JSON or JSONB column. Adding row samples into the prompt might help won't always describe the data perfectly.
 
 For this reason, a custom SQL views can help.
 
@@ -609,19 +613,19 @@ chain.run("How many employees are also customers?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseSequentialChain chain...
     Table names to use:
     ['Employee', 'Customer']
-    
+
     > Entering new SQLDatabaseChain chain...
     How many employees are also customers?
     SQLQuery:SELECT COUNT(*) FROM Employee e INNER JOIN Customer c ON e.EmployeeId = c.SupportRepId;
     SQLResult: [(59,)]
     Answer:59 employees are also customers.
     > Finished chain.
-    
+
     > Finished chain.
 
 
@@ -692,8 +696,8 @@ local_chain("How many customers are there?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are there?
     SQLQuery:
@@ -879,8 +883,8 @@ print("\n" + yaml_example)
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     List all the customer first names that start with 'a'
     SQLQuery:
@@ -900,7 +904,7 @@ print("\n" + yaml_example)
     [('François', 'Frantiek', 'Helena', 'Astrid', 'Daan', 'Kara', 'Eduardo', 'Alexandre', 'Fernanda', 'Mark', 'Frank', 'Jack', 'Dan', 'Kathy', 'Heather', 'Frank', 'Richard', 'Patrick', 'Julia', 'Edward', 'Martha', 'Aaron', 'Madalena', 'Hannah', 'Niklas', 'Camille', 'Marc', 'Wyatt', 'Isabelle', 'Ladislav', 'Lucas', 'Johannes', 'Stanisaw', 'Joakim', 'Emma', 'Mark', 'Manoj', 'Puja']
     > Finished chain.
     *** Query succeeded
-    
+
     answer: '[(''François'', ''Frantiek'', ''Helena'', ''Astrid'', ''Daan'', ''Kara'',
       ''Eduardo'', ''Alexandre'', ''Fernanda'', ''Mark'', ''Frank'', ''Jack'', ''Dan'',
       ''Kathy'', ''Heather'', ''Frank'', ''Richard'', ''Patrick'', ''Julia'', ''Edward'',
@@ -931,7 +935,7 @@ print("\n" + yaml_example)
       None\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n3\tFrançois\t\
       Tremblay\tNone\t1498 rue Bélanger\tMontréal\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\t\
       None\tftremblay@gmail.com\t3\n*/"
-    
+
 ```
 
 </CodeOutputBlock>
@@ -944,20 +948,20 @@ YAML_EXAMPLES = """
 - input: How many customers are not from Brazil?
   table_info: |
     CREATE TABLE "Customer" (
-      "CustomerId" INTEGER NOT NULL, 
-      "FirstName" NVARCHAR(40) NOT NULL, 
-      "LastName" NVARCHAR(20) NOT NULL, 
-      "Company" NVARCHAR(80), 
-      "Address" NVARCHAR(70), 
-      "City" NVARCHAR(40), 
-      "State" NVARCHAR(40), 
-      "Country" NVARCHAR(40), 
-      "PostalCode" NVARCHAR(10), 
-      "Phone" NVARCHAR(24), 
-      "Fax" NVARCHAR(24), 
-      "Email" NVARCHAR(60) NOT NULL, 
-      "SupportRepId" INTEGER, 
-      PRIMARY KEY ("CustomerId"), 
+      "CustomerId" INTEGER NOT NULL,
+      "FirstName" NVARCHAR(40) NOT NULL,
+      "LastName" NVARCHAR(20) NOT NULL,
+      "Company" NVARCHAR(80),
+      "Address" NVARCHAR(70),
+      "City" NVARCHAR(40),
+      "State" NVARCHAR(40),
+      "Country" NVARCHAR(40),
+      "PostalCode" NVARCHAR(10),
+      "Phone" NVARCHAR(24),
+      "Fax" NVARCHAR(24),
+      "Email" NVARCHAR(60) NOT NULL,
+      "SupportRepId" INTEGER,
+      PRIMARY KEY ("CustomerId"),
       FOREIGN KEY("SupportRepId") REFERENCES "Employee" ("EmployeeId")
     )
   sql_cmd: SELECT COUNT(*) FROM "Customer" WHERE NOT "Country" = "Brazil";
@@ -966,8 +970,8 @@ YAML_EXAMPLES = """
 - input: list all the genres that start with 'r'
   table_info: |
     CREATE TABLE "Genre" (
-      "GenreId" INTEGER NOT NULL, 
-      "Name" NVARCHAR(120), 
+      "GenreId" INTEGER NOT NULL,
+      "Name" NVARCHAR(120),
       PRIMARY KEY ("GenreId")
     )
 
@@ -980,7 +984,7 @@ YAML_EXAMPLES = """
     */
   sql_cmd: SELECT "Name" FROM "Genre" WHERE "Name" LIKE 'r%';
   sql_result: "[('Rock',), ('Rock and Roll',), ('Reggae',), ('R&B/Soul',)]"
-  answer: The genres that start with 'r' are Rock, Rock and Roll, Reggae and R&B/Soul. 
+  answer: The genres that start with 'r' are Rock, Rock and Roll, Reggae and R&B/Soul.
 """
 ```
 
@@ -1046,8 +1050,8 @@ result = local_chain("How many customers are from Brazil?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are from Brazil?
     SQLQuery:SELECT count(*) FROM Customer WHERE Country = "Brazil";
@@ -1066,8 +1070,8 @@ result = local_chain("How many customers are not from Brazil?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are not from Brazil?
     SQLQuery:SELECT count(*) FROM customer WHERE country NOT IN (SELECT country FROM customer WHERE country = 'Brazil')
@@ -1086,8 +1090,8 @@ result = local_chain("How many customers are there in total?")
 <CodeOutputBlock lang="python">
 
 ```
-    
-    
+
+
     > Entering new SQLDatabaseChain chain...
     How many customers are there in total?
     SQLQuery:SELECT count(*) FROM Customer;

cookbook/stepback-qa.ipynb

@@ -0,0 +1,335 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "83ef724e",
+   "metadata": {},
+   "source": [
+    "# Step-Back Prompting (Question-Answering)\n",
+    "\n",
+    "One prompting technique called \"Step-Back\" prompting can improve performance on complex questions by first asking a \"step back\" question. This can be combined with regular question-answering applications by then doing retrieval on both the original and step-back question.\n",
+    "\n",
+    "Read the paper [here](https://arxiv.org/abs/2310.06117)\n",
+    "\n",
+    "See an excellent blog post on this by Cobus Greyling [here](https://cobusgreyling.medium.com/a-new-prompt-engineering-technique-has-been-introduced-called-step-back-prompting-b00e8954cacb)\n",
+    "\n",
+    "In this cookbook we will replicate this technique. We modify the prompts used slightly to work better with chat models."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 85,
+   "id": "67b5cdac",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import ChatPromptTemplate, FewShotChatMessagePromptTemplate\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
+    "from langchain.schema.runnable import RunnableLambda"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 86,
+   "id": "7e017c44",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Few Shot Examples\n",
+    "examples = [\n",
+    "    {\n",
+    "        \"input\": \"Could the members of The Police perform lawful arrests?\",\n",
+    "        \"output\": \"what can the members of The Police do?\"\n",
+    "    },\n",
+    "    {\n",
+    "        \"input\": \"Jan Sindel’s was born in what country?\", \n",
+    "        \"output\": \"what is Jan Sindel’s personal history?\"\n",
+    "    },\n",
+    "]\n",
+    "# We now transform these to example messages\n",
+    "example_prompt = ChatPromptTemplate.from_messages(\n",
+    "    [\n",
+    "        (\"human\", \"{input}\"),\n",
+    "        (\"ai\", \"{output}\"),\n",
+    "    ]\n",
+    ")\n",
+    "few_shot_prompt = FewShotChatMessagePromptTemplate(\n",
+    "    example_prompt=example_prompt,\n",
+    "    examples=examples,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 87,
+   "id": "206415ee",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt = ChatPromptTemplate.from_messages([\n",
+    "    (\"system\", \"\"\"You are an expert at world knowledge. Your task is to step back and paraphrase a question to a more generic step-back question, which is easier to answer. Here are a few examples:\"\"\"),\n",
+    "    # Few shot examples\n",
+    "    few_shot_prompt,\n",
+    "    # New question\n",
+    "    (\"user\", \"{question}\"),\n",
+    "])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 88,
+   "id": "d643a85c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "question_gen = prompt | ChatOpenAI(temperature=0) | StrOutputParser()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 182,
+   "id": "5ba21b2a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "question = \"was chatgpt around while trump was president?\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 183,
+   "id": "5992c8ca",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'when was ChatGPT developed?'"
+      ]
+     },
+     "execution_count": 183,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "question_gen.invoke({\"question\": question})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 190,
+   "id": "32667424",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.utilities import DuckDuckGoSearchAPIWrapper\n",
+    "\n",
+    "\n",
+    "search = DuckDuckGoSearchAPIWrapper(max_results=4)\n",
+    "\n",
+    "def retriever(query):\n",
+    "    return search.run(query)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 191,
+   "id": "ffc28c91",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'This includes content about former President Donald Trump. According to further tests, ChatGPT successfully wrote poems admiring all recent U.S. presidents, but failed when we entered a query for ... On Wednesday, a Twitter user posted screenshots of him asking OpenAI\\'s chatbot, ChatGPT, to write a positive poem about former President Donald Trump, to which the chatbot declined, citing it ... While impressive in many respects, ChatGPT also has some major flaws. ... [President\\'s Name],\" refused to write a poem about ex-President Trump, but wrote one about President Biden ... During the Trump administration, Altman gained new attention as a vocal critic of the president. It was against that backdrop that he was rumored to be considering a run for California governor.'"
+      ]
+     },
+     "execution_count": 191,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "retriever(question)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 192,
+   "id": "00c77443",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"Will Douglas Heaven March 3, 2023 Stephanie Arnett/MITTR | Envato When OpenAI launched ChatGPT, with zero fanfare, in late November 2022, the San Francisco-based artificial-intelligence company... ChatGPT, which stands for Chat Generative Pre-trained Transformer, is a large language model -based chatbot developed by OpenAI and launched on November 30, 2022, which enables users to refine and steer a conversation towards a desired length, format, style, level of detail, and language. ChatGPT is an artificial intelligence (AI) chatbot built on top of OpenAI's foundational large language models (LLMs) like GPT-4 and its predecessors. This chatbot has redefined the standards of... June 4, 2023 ⋅ 4 min read 124 SHARES 13K At the end of 2022, OpenAI introduced the world to ChatGPT. Since its launch, ChatGPT hasn't shown significant signs of slowing down in developing new...\""
+      ]
+     },
+     "execution_count": 192,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "retriever(question_gen.invoke({\"question\": question}))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 193,
+   "id": "b257bc06",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# response_prompt_template = \"\"\"You are an expert of world knowledge. I am going to ask you a question. Your response should be comprehensive and not contradicted with the following context if they are relevant. Otherwise, ignore them if they are not relevant.\n",
+    "\n",
+    "# {normal_context}\n",
+    "# {step_back_context}\n",
+    "\n",
+    "# Original Question: {question}\n",
+    "# Answer:\"\"\"\n",
+    "# response_prompt = ChatPromptTemplate.from_template(response_prompt_template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 203,
+   "id": "f48c65b2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain import hub\n",
+    "\n",
+    "response_prompt = hub.pull(\"langchain-ai/stepback-answer\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 204,
+   "id": "97a6d5ab",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = {\n",
+    "    # Retrieve context using the normal question\n",
+    "    \"normal_context\": RunnableLambda(lambda x: x['question']) | retriever,\n",
+    "    # Retrieve context using the step-back question\n",
+    "    \"step_back_context\": question_gen | retriever,\n",
+    "    # Pass on the question\n",
+    "    \"question\": lambda x: x[\"question\"]\n",
+    "} | response_prompt | ChatOpenAI(temperature=0) | StrOutputParser()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 205,
+   "id": "ce554cb0",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"No, ChatGPT was not around while Donald Trump was president. ChatGPT was launched on November 30, 2022, which is after Donald Trump's presidency. The context provided mentions that during the Trump administration, Altman, the CEO of OpenAI, gained attention as a vocal critic of the president. This suggests that ChatGPT was not developed or available during that time.\""
+      ]
+     },
+     "execution_count": 205,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke({\"question\": question})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a9fb8dd2",
+   "metadata": {},
+   "source": [
+    "## Baseline"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 206,
+   "id": "00db8a15",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "response_prompt_template = \"\"\"You are an expert of world knowledge. I am going to ask you a question. Your response should be comprehensive and not contradicted with the following context if they are relevant. Otherwise, ignore them if they are not relevant.\n",
+    "\n",
+    "{normal_context}\n",
+    "\n",
+    "Original Question: {question}\n",
+    "Answer:\"\"\"\n",
+    "response_prompt = ChatPromptTemplate.from_template(response_prompt_template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 207,
+   "id": "06335ebb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = {\n",
+    "    # Retrieve context using the normal question (only the first 3 results)\n",
+    "    \"normal_context\": RunnableLambda(lambda x: x['question']) | retriever,\n",
+    "    # Pass on the question\n",
+    "    \"question\": lambda x: x[\"question\"]\n",
+    "} | response_prompt | ChatOpenAI(temperature=0) | StrOutputParser()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 208,
+   "id": "15e0e741",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"Yes, ChatGPT was around while Donald Trump was president. However, it is important to note that the specific context you provided mentions that ChatGPT refused to write a positive poem about former President Donald Trump. This suggests that while ChatGPT was available during Trump's presidency, it may have had limitations or biases in its responses regarding him.\""
+      ]
+     },
+     "execution_count": 208,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke({\"question\": question})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e7b9e5d6",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docker/Dockerfile.base

@@ -1,3 +1,3 @@
-FROM python:latest
+FROM python:3.11
 
 RUN pip install langchain

docs/.local_build.sh

@@ -8,11 +8,14 @@ set -o xtrace
 SCRIPT_DIR="$(cd "$(dirname "$0")"; pwd)"
 cd "${SCRIPT_DIR}"
 
-mkdir -p _dist/docs_skeleton
-cp -r {docs_skeleton,snippets} _dist
-cp -r extras/* _dist/docs_skeleton/docs
-cd _dist/docs_skeleton
-poetry run nbdoc_build
-poetry run python generate_api_reference_links.py
+mkdir -p ../_dist
+cp -r . ../_dist
+cd ../_dist
+poetry run python scripts/model_feat_table.py
+poetry run nbdoc_build --srcdir docs
+cp ../cookbook/README.md src/pages/cookbook.mdx
+cp ../.github/CONTRIBUTING.md docs/contributing.md
+wget https://raw.githubusercontent.com/langchain-ai/langserve/main/README.md -O docs/guides/deployments/langserve.md
+poetry run python scripts/generate_api_reference_links.py
 yarn install
 yarn start

docs/README.md

@@ -42,7 +42,7 @@ If you are using GitHub pages for hosting, this command is a convenient way to b
 
 ### Continuous Integration
 
-Some common defaults for linting/formatting have been set for you. If you integrate your project with an open source Continuous Integration system (e.g. Travis CI, CircleCI), you may check for issues using the following command.
+Some common defaults for linting/formatting have been set for you. If you integrate your project with an open-source Continuous Integration system (e.g. Travis CI, CircleCI), you may check for issues using the following command.
 
 ```
 $ yarn ci

docs/api_reference/Makefile

@@ -3,7 +3,7 @@
 
 # You can set these variables from the command line, and also
 # from the environment for the first two.
-SPHINXOPTS    ?= 
+SPHINXOPTS    ?= -j auto 
 SPHINXBUILD   ?= sphinx-build
 SPHINXAUTOBUILD   ?= sphinx-autobuild
 SOURCEDIR     = .

docs/api_reference/create_api_rst.py

@@ -122,8 +122,7 @@ def _merge_module_members(
 
 
 def _load_package_modules(
-    package_directory: Union[str, Path],
-    submodule: Optional[str] = None
+    package_directory: Union[str, Path], submodule: Optional[str] = None
 ) -> Dict[str, ModuleMembers]:
     """Recursively load modules of a package based on the file system.
 
@@ -171,7 +170,8 @@ def _load_package_modules(
             # different way
             if submodule is not None:
                 module_members = _load_module_members(
-                    f"{package_name}.{submodule}.{namespace}", f"{submodule}.{namespace}"
+                    f"{package_name}.{submodule}.{namespace}",
+                    f"{submodule}.{namespace}",
                 )
             else:
                 module_members = _load_module_members(
@@ -280,18 +280,9 @@ Functions
     return full_doc
 
 
-def main() -> None:
-    """Generate the reference.rst file for each package."""
-    lc_members = _load_package_modules(PKG_DIR)
-    # Put some packages at top level
-    tools = _load_package_modules(PKG_DIR, "tools")
-    lc_members['tools.render'] = tools['render']
-    agents = _load_package_modules(PKG_DIR, "agents")
-    lc_members['agents.output_parsers'] = agents['output_parsers']
-    lc_members['agents.format_scratchpad'] = agents['format_scratchpad']
-    lc_doc = ".. _api_reference:\n\n" + _construct_doc("langchain", lc_members)
-    with open(WRITE_FILE, "w") as f:
-        f.write(lc_doc)
+def _document_langchain_experimental() -> None:
+    """Document the langchain_experimental package."""
+    # Generate experimental_api_reference.rst
     exp_members = _load_package_modules(EXP_DIR)
     exp_doc = ".. _experimental_api_reference:\n\n" + _construct_doc(
         "langchain_experimental", exp_members
@@ -300,5 +291,36 @@ def main() -> None:
         f.write(exp_doc)
 
 
+def _document_langchain_core() -> None:
+    """Document the main langchain package."""
+    # load top level module members
+    lc_members = _load_package_modules(PKG_DIR)
+
+    # Add additional packages
+    tools = _load_package_modules(PKG_DIR, "tools")
+    agents = _load_package_modules(PKG_DIR, "agents")
+    schema = _load_package_modules(PKG_DIR, "schema")
+
+    lc_members.update(
+        {
+            "agents.output_parsers": agents["output_parsers"],
+            "agents.format_scratchpad": agents["format_scratchpad"],
+            "tools.render": tools["render"],
+            "schema.runnable": schema["runnable"],
+        }
+    )
+
+    lc_doc = ".. _api_reference:\n\n" + _construct_doc("langchain", lc_members)
+
+    with open(WRITE_FILE, "w") as f:
+        f.write(lc_doc)
+
+
+def main() -> None:
+    """Generate the reference.rst file for each package."""
+    _document_langchain_core()
+    _document_langchain_experimental()
+
+
 if __name__ == "__main__":
     main()