cr

.github/CODE_OF_CONDUCT.md

@@ -1,132 +0,0 @@
-# Contributor Covenant Code of Conduct
-
-## Our Pledge
-
-We as members, contributors, and leaders pledge to make participation in our
-community a harassment-free experience for everyone, regardless of age, body
-size, visible or invisible disability, ethnicity, sex characteristics, gender
-identity and expression, level of experience, education, socio-economic status,
-nationality, personal appearance, race, caste, color, religion, or sexual
-identity and orientation.
-
-We pledge to act and interact in ways that contribute to an open, welcoming,
-diverse, inclusive, and healthy community.
-
-## Our Standards
-
-Examples of behavior that contributes to a positive environment for our
-community include:
-
-* Demonstrating empathy and kindness toward other people
-* Being respectful of differing opinions, viewpoints, and experiences
-* Giving and gracefully accepting constructive feedback
-* Accepting responsibility and apologizing to those affected by our mistakes,
-  and learning from the experience
-* Focusing on what is best not just for us as individuals, but for the overall
-  community
-
-Examples of unacceptable behavior include:
-
-* The use of sexualized language or imagery, and sexual attention or advances of
-  any kind
-* Trolling, insulting or derogatory comments, and personal or political attacks
-* Public or private harassment
-* Publishing others' private information, such as a physical or email address,
-  without their explicit permission
-* Other conduct which could reasonably be considered inappropriate in a
-  professional setting
-
-## Enforcement Responsibilities
-
-Community leaders are responsible for clarifying and enforcing our standards of
-acceptable behavior and will take appropriate and fair corrective action in
-response to any behavior that they deem inappropriate, threatening, offensive,
-or harmful.
-
-Community leaders have the right and responsibility to remove, edit, or reject
-comments, commits, code, wiki edits, issues, and other contributions that are
-not aligned to this Code of Conduct, and will communicate reasons for moderation
-decisions when appropriate.
-
-## Scope
-
-This Code of Conduct applies within all community spaces, and also applies when
-an individual is officially representing the community in public spaces.
-Examples of representing our community include using an official e-mail address,
-posting via an official social media account, or acting as an appointed
-representative at an online or offline event.
-
-## Enforcement
-
-Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported to the community leaders responsible for enforcement at
-conduct@langchain.dev.
-All complaints will be reviewed and investigated promptly and fairly.
-
-All community leaders are obligated to respect the privacy and security of the
-reporter of any incident.
-
-## Enforcement Guidelines
-
-Community leaders will follow these Community Impact Guidelines in determining
-the consequences for any action they deem in violation of this Code of Conduct:
-
-### 1. Correction
-
-**Community Impact**: Use of inappropriate language or other behavior deemed
-unprofessional or unwelcome in the community.
-
-**Consequence**: A private, written warning from community leaders, providing
-clarity around the nature of the violation and an explanation of why the
-behavior was inappropriate. A public apology may be requested.
-
-### 2. Warning
-
-**Community Impact**: A violation through a single incident or series of
-actions.
-
-**Consequence**: A warning with consequences for continued behavior. No
-interaction with the people involved, including unsolicited interaction with
-those enforcing the Code of Conduct, for a specified period of time. This
-includes avoiding interactions in community spaces as well as external channels
-like social media. Violating these terms may lead to a temporary or permanent
-ban.
-
-### 3. Temporary Ban
-
-**Community Impact**: A serious violation of community standards, including
-sustained inappropriate behavior.
-
-**Consequence**: A temporary ban from any sort of interaction or public
-communication with the community for a specified period of time. No public or
-private interaction with the people involved, including unsolicited interaction
-with those enforcing the Code of Conduct, is allowed during this period.
-Violating these terms may lead to a permanent ban.
-
-### 4. Permanent Ban
-
-**Community Impact**: Demonstrating a pattern of violation of community
-standards, including sustained inappropriate behavior, harassment of an
-individual, or aggression toward or disparagement of classes of individuals.
-
-**Consequence**: A permanent ban from any sort of public interaction within the
-community.
-
-## Attribution
-
-This Code of Conduct is adapted from the [Contributor Covenant][homepage],
-version 2.1, available at
-[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
-
-Community Impact Guidelines were inspired by
-[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
-
-For answers to common questions about this code of conduct, see the FAQ at
-[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
-[https://www.contributor-covenant.org/translations][translations].
-
-[homepage]: https://www.contributor-covenant.org
-[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
-[Mozilla CoC]: https://github.com/mozilla/diversity
-[FAQ]: https://www.contributor-covenant.org/faq
-[translations]: https://www.contributor-covenant.org/translations

.github/CONTRIBUTING.md

@@ -1,19 +1,20 @@
 # Contributing to LangChain
 
 Hi there! Thank you for even being interested in contributing to LangChain.
-As an open-source project in a rapidly developing field, we are extremely open to contributions, whether they involve new features, improved infrastructure, better documentation, or bug fixes.
+As an open source project in a rapidly developing field, we are extremely open
+to contributions, whether they be in the form of new features, improved infra, better documentation, or bug fixes.
 
 ## 🗺️ Guidelines
 
 ### 👩‍💻 Contributing Code
 
-To contribute to this project, please follow the ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow.
+To contribute to this project, please follow a ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow.
 Please do not try to push directly to this repo unless you are a maintainer.
 
 Please follow the checked-in pull request template when opening pull requests. Note related issues and tag relevant
 maintainers.
 
-Pull requests cannot land without passing the formatting, linting, and testing checks first. See [Testing](#testing) and
+Pull requests cannot land without passing the formatting, linting and testing checks first. See [Testing](#testing) and
 [Formatting and Linting](#formatting-and-linting) for how to run these checks locally.
 
 It's essential that we maintain great documentation and testing. If you:
@@ -26,14 +27,16 @@ It's essential that we maintain great documentation and testing. If you:
   - Add a demo notebook in `docs/modules`.
   - Add unit and integration tests.
 
-We are a small, progress-oriented team. If there's something you'd like to add or change, opening a pull request is the
+We're a small, building-oriented team. If there's something you'd like to add or change, opening a pull request is the
 best way to get our attention.
 
 ### 🚩GitHub Issues
 
-Our [issues](https://github.com/langchain-ai/langchain/issues) page is kept up to date with bugs, improvements, and feature requests.
+Our [issues](https://github.com/langchain-ai/langchain/issues) page is kept up to date
+with bugs, improvements, and feature requests.
 
-There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help organize issues.
+There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help
+organize issues.
 
 If you start working on an issue, please assign it to yourself.
 
@@ -56,12 +59,12 @@ we do not want these to get in the way of getting good code into the codebase.
 
 ## 🚀 Quick Start
 
-This quick start guide explains how to run the repository locally.
+This quick start describes running the repository locally.
 For a [development container](https://containers.dev/), see the [.devcontainer folder](https://github.com/langchain-ai/langchain/tree/master/.devcontainer).
 
 ### Dependency Management: Poetry and other env/dependency managers
 
-This project utilizes [Poetry](https://python-poetry.org/) v1.6.1+ as a dependency manager.
+This project uses [Poetry](https://python-poetry.org/) v1.6.1+ as a dependency manager.
 
 ❗Note: *Before installing Poetry*, if you use `Conda`, create and activate a new Conda env (e.g. `conda create -n langchain python=3.9`)
 
@@ -72,11 +75,11 @@ tell Poetry to use the virtualenv python environment (`poetry config virtualenvs
 
 ### Core vs. Experimental
 
-This repository contains two separate projects:
-- `langchain`: core langchain code, abstractions, and use cases.
-- `langchain.experimental`: see the [Experimental README](https://github.com/langchain-ai/langchain/tree/master/libs/experimental/README.md) for more information.
+There are two separate projects in this repository:
+- `langchain`: core langchain code, abstractions, and use cases
+- `langchain.experimental`: see the [Experimental README](../libs/experimental/README.md) for more information.
 
-Each of these has its own development environment. Docs are run from the top-level makefile, but development
+Each of these has their own development environment. Docs are run from the top-level makefile, but development
 is split across separate test & release flows.
 
 For this quickstart, start with langchain core:
@@ -126,7 +129,7 @@ To run unit tests in Docker:
 make docker_tests
 ```
 
-There are also [integration tests and code-coverage](https://github.com/langchain-ai/langchain/tree/master/libs/langchain/tests/README.md) available.
+There are also [integration tests and code-coverage](../libs/langchain/tests/README.md) available.
 
 ### Formatting and Linting
 
@@ -279,7 +282,7 @@ make docs_build
 make api_docs_build
 ```
 
-Finally, run the link checker to ensure all links are valid:
+Finally, you can run the linkchecker to make sure all links are valid:
 
 ```bash
 make docs_linkcheck
@@ -304,4 +307,4 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
 ### 🌟 Recognition
 
 If your contribution has made its way into a release, we will want to give you credit on Twitter (only if you want though)!
-If you have a Twitter account you would like us to mention, please let us know in the PR or through another means.
+If you have a Twitter account you would like us to mention, please let us know in the PR or in another manner.

.github/workflows/_compile_integration_test.yml

@@ -1,57 +0,0 @@
-name: compile-integration-test
-
-on:
-  workflow_call:
-    inputs:
-      working-directory:
-        required: true
-        type: string
-        description: "From which folder this pipeline executes"
-
-env:
-  POETRY_VERSION: "1.6.1"
-
-jobs:
-  build:
-    defaults:
-      run:
-        working-directory: ${{ inputs.working-directory }}
-    runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        python-version:
-          - "3.8"
-          - "3.9"
-          - "3.10"
-          - "3.11"
-    name: Python ${{ matrix.python-version }}
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
-        uses: "./.github/actions/poetry_setup"
-        with:
-          python-version: ${{ matrix.python-version }}
-          poetry-version: ${{ env.POETRY_VERSION }}
-          working-directory: ${{ inputs.working-directory }}
-          cache-key: compile-integration
-
-      - name: Install integration dependencies
-        shell: bash
-        run: poetry install --with=test_integration
-
-      - name: Check integration tests compile
-        shell: bash
-        run: poetry run pytest -m compile tests/integration_tests
-
-      - name: Ensure the tests did not create any additional files
-        shell: bash
-        run: |
-          set -eu
-
-          STATUS="$(git status)"
-          echo "$STATUS"
-
-          # grep will exit non-zero if the target message isn't found,
-          # and `set -e` above will cause the step to fail.
-          echo "$STATUS" | grep 'nothing to commit, working tree clean'

.github/workflows/_lint.yml

@@ -34,7 +34,7 @@ jobs:
           - "3.8"
           - "3.11"
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
         with:
           # Fetch the last FETCH_DEPTH commits, so the mtime-changing script
           # can accurately set the mtimes of files modified in the last FETCH_DEPTH commits.

.github/workflows/_pydantic_compatibility.yml

@@ -26,7 +26,7 @@ jobs:
           - "3.11"
     name: Pydantic v1/v2 compatibility - Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/_release.yml

@@ -30,7 +30,7 @@ jobs:
       run:
         working-directory: ${{ inputs.working-directory }}
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
 
       - name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/_test.yml

@@ -26,7 +26,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/_test_release.yml

@@ -1,50 +0,0 @@
-name: test-release
-
-on:
-  workflow_call:
-    inputs:
-      working-directory:
-        required: true
-        type: string
-        description: "From which folder this pipeline executes"
-
-env:
-  POETRY_VERSION: "1.6.1"
-
-jobs:
-  publish_to_test_pypi:
-    runs-on: ubuntu-latest
-    permissions:
-      # This permission is used for trusted publishing:
-      # https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/
-      #
-      # Trusted publishing has to also be configured on PyPI for each package:
-      # https://docs.pypi.org/trusted-publishers/adding-a-publisher/
-      id-token: write
-    defaults:
-      run:
-        working-directory: ${{ inputs.working-directory }}
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
-        uses: "./.github/actions/poetry_setup"
-        with:
-          python-version: "3.10"
-          poetry-version: ${{ env.POETRY_VERSION }}
-          working-directory: ${{ inputs.working-directory }}
-          cache-key: release
-
-      - name: Build project for distribution
-        run: poetry build
-      - name: Check Version
-        id: check-version
-        run: |
-          echo version=$(poetry version --short) >> $GITHUB_OUTPUT
-      - name: Publish package to TestPyPI
-        uses: pypa/gh-action-pypi-publish@release/v1
-        with:
-          repository-url: https://test.pypi.org/legacy/
-          packages-dir: ${{ inputs.working-directory }}/dist/
-          verbose: true
-          print-hash: true

.github/workflows/codespell.yml

@@ -17,7 +17,7 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v3
 
       - name: Install Dependencies
         run: |

.github/workflows/langchain_ci.yml

@@ -12,7 +12,6 @@ on:
       - '.github/workflows/_test.yml'
       - '.github/workflows/_pydantic_compatibility.yml'
       - '.github/workflows/langchain_ci.yml'
-      - 'libs/*'
       - 'libs/langchain/**'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
@@ -45,13 +44,6 @@ jobs:
       working-directory: libs/langchain
     secrets: inherit
 
-  compile-integration-tests:
-    uses:
-      ./.github/workflows/_compile_integration_test.yml
-    with:
-      working-directory: libs/langchain
-    secrets: inherit
-
   pydantic-compatibility:
     uses:
       ./.github/workflows/_pydantic_compatibility.yml
@@ -73,7 +65,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }} extended tests
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/langchain_cli_ci.yml

@@ -1,53 +0,0 @@
----
-name: libs/cli CI
-
-on:
-  push:
-    branches: [ master ]
-  pull_request:
-    paths:
-      - '.github/actions/poetry_setup/action.yml'
-      - '.github/tools/**'
-      - '.github/workflows/_lint.yml'
-      - '.github/workflows/_test.yml'
-      - '.github/workflows/_pydantic_compatibility.yml'
-      - '.github/workflows/langchain_cli_ci.yml'
-      - 'libs/cli/**'
-      - 'libs/*'
-  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
-
-# If another push to the same PR or branch happens while this workflow is still running,
-# cancel the earlier run in favor of the next run.
-#
-# There's no point in testing an outdated version of the code. GitHub only allows
-# a limited number of job runners to be active at the same time, so it's better to cancel
-# pointless jobs early so that more useful jobs can run sooner.
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
-
-env:
-  POETRY_VERSION: "1.6.1"
-  WORKDIR: "libs/cli"
-
-jobs:
-  lint:
-    uses:
-      ./.github/workflows/_lint.yml
-    with:
-      working-directory: libs/cli
-    secrets: inherit
-
-  test:
-    uses:
-      ./.github/workflows/_test.yml
-    with:
-      working-directory: libs/cli
-    secrets: inherit
-
-  pydantic-compatibility:
-    uses:
-      ./.github/workflows/_pydantic_compatibility.yml
-    with:
-      working-directory: libs/cli
-    secrets: inherit

.github/workflows/langchain_experimental_ci.yml

@@ -11,7 +11,7 @@ on:
       - '.github/workflows/_lint.yml'
       - '.github/workflows/_test.yml'
       - '.github/workflows/langchain_experimental_ci.yml'
-      - 'libs/*'
+      - 'libs/langchain/**'
       - 'libs/experimental/**'
   workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
 
@@ -44,13 +44,6 @@ jobs:
       working-directory: libs/experimental
     secrets: inherit
 
-  compile-integration-tests:
-    uses:
-      ./.github/workflows/_compile_integration_test.yml
-    with:
-      working-directory: libs/experimental
-    secrets: inherit
-
   # It's possible that langchain-experimental works fine with the latest *published* langchain,
   # but is broken with the langchain on `master`.
   #
@@ -69,7 +62,7 @@ jobs:
           - "3.11"
     name: test with unpublished langchain - Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"
@@ -104,7 +97,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }} extended tests
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
         uses: "./.github/actions/poetry_setup"

.github/workflows/langchain_experimental_test_release.yml

@@ -1,13 +0,0 @@
----
-name: Experimental Test Release
-
-on:
-  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
-
-jobs:
-  release:
-    uses:
-      ./.github/workflows/_test_release.yml
-    with:
-      working-directory: libs/experimental
-    secrets: inherit

.github/workflows/langchain_test_release.yml

@@ -1,13 +0,0 @@
----
-name: Test Release
-
-on:
-  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
-
-jobs:
-  release:
-    uses:
-      ./.github/workflows/_test_release.yml
-    with:
-      working-directory: libs/langchain
-    secrets: inherit

.github/workflows/scheduled_test.yml

@@ -24,7 +24,7 @@ jobs:
           - "3.11"
     name: Python ${{ matrix.python-version }}
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v3
 
       - name: Set up Python ${{ matrix.python-version }}
         uses: "./.github/actions/poetry_setup"
@@ -55,10 +55,6 @@ jobs:
           poetry install --with=test_integration
           poetry run pip install google-cloud-aiplatform
           poetry run pip install "boto3>=1.28.57"
-          if [[ ${{ matrix.python-version }} != "3.8" ]]
-          then
-            poetry run pip install fireworks-ai
-          fi
 
       - name: Run tests
         shell: bash
@@ -68,8 +64,7 @@ jobs:
           AZURE_OPENAI_API_VERSION: ${{ secrets.AZURE_OPENAI_API_VERSION }}
           AZURE_OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_API_BASE }}
           AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
-          AZURE_OPENAI_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_DEPLOYMENT_NAME }}
-          FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
+          AZURE_OPENAI_DEPLOYMENT_NAME:  ${{ secrets.AZURE_OPENAI_DEPLOYMENT_NAME }}
         run: |
           make scheduled_tests
 

README.md

@@ -93,7 +93,7 @@ Memory refers to persisting state between calls of a chain/agent. LangChain prov
 
 **🧐 Evaluation:**
 
-[BETA] Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is by using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.
+[BETA] Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.
 
 For more information on these concepts, please see our [full documentation](https://python.langchain.com).
 

cookbook/README.md

@@ -4,7 +4,7 @@ Example code for building applications with LangChain, with an emphasis on more
 
 Notebook | Description
 :- | :-
-[LLaMA2_sql_chat.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/LLaMA2_sql_chat.ipynb) | Build a chat application that interacts with a SQL database using an open source llm (llama2), specifically demonstrated on an SQLite database containing rosters.
+[LLaMA2_sql_chat.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/LLaMA2_sql_chat.ipynb) | Build a chat application that interacts with a sql database using an open source llm (llama2), specifically demonstrated on a sqlite database containing nba rosters.
 [Semi_Structured_RAG.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_Structured_RAG.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data, including text and tables, using unstructured for parsing, multi-vector retriever for storing, and lcel for implementing chains.
 [Semi_structured_and_multi_moda...](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_structured_and_multi_modal_RAG.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using unstructured for parsing, multi-vector retriever for storage and retrieval, and lcel for implementing chains.
 [Semi_structured_multi_modal_RA...](https://github.com/langchain-ai/langchain/tree/master/cookbook/Semi_structured_multi_modal_RAG_LLaMA2.ipynb) | Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using various tools and methods such as unstructured for parsing, multi-vector retriever for storing, lcel for implementing chains, and open source language models like llama2, llava, and gpt4all.
@@ -12,14 +12,14 @@ Notebook | Description
 [autogpt/marathon_times.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/autogpt/marathon_times.ipynb) | Implement autogpt for finding winning marathon times.
 [baby_agi.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/baby_agi.ipynb) | Implement babyagi, an ai agent that can generate and execute tasks based on a given objective, with the flexibility to swap out specific vectorstores/model providers.
 [baby_agi_with_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/baby_agi_with_agent.ipynb) | Swap out the execution chain in the babyagi notebook with an agent that has access to tools, aiming to obtain more reliable information.
-[camel_role_playing.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/camel_role_playing.ipynb) | Implement the camel framework for creating autonomous cooperative agents in large-scale language models, using role-playing and inception prompting to guide chat agents towards task completion.
+[camel_role_playing.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/camel_role_playing.ipynb) | Implement the camel framework for creating autonomous cooperative agents in large scale language models, using role-playing and inception prompting to guide chat agents towards task completion.
 [causal_program_aided_language_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/causal_program_aided_language_model.ipynb) | Implement the causal program-aided language (cpal) chain, which improves upon the program-aided language (pal) by incorporating causal structure to prevent hallucination in language models, particularly when dealing with complex narratives and math problems with nested dependencies.
 [code-analysis-deeplake.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/code-analysis-deeplake.ipynb) | Analyze its own code base with the help of gpt and activeloop's deep lake.
 [custom_agent_with_plugin_retri...](https://github.com/langchain-ai/langchain/tree/master/cookbook/custom_agent_with_plugin_retrieval.ipynb) | Build a custom agent that can interact with ai plugins by retrieving tools and creating natural language wrappers around openapi endpoints.
 [custom_agent_with_plugin_retri...](https://github.com/langchain-ai/langchain/tree/master/cookbook/custom_agent_with_plugin_retrieval_using_plugnplai.ipynb) | Build a custom agent with plugin retrieval functionality, utilizing ai plugins from the `plugnplai` directory.
 [databricks_sql_db.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/databricks_sql_db.ipynb) | Connect to databricks runtimes and databricks sql.
 [deeplake_semantic_search_over_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/deeplake_semantic_search_over_chat.ipynb) | Perform semantic search and question-answering over a group chat using activeloop's deep lake with gpt4.
-[elasticsearch_db_qa.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/elasticsearch_db_qa.ipynb) | Interact with elasticsearch analytics databases in natural language and build search queries via the elasticsearch dsl API.
+[elasticsearch_db_qa.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/elasticsearch_db_qa.ipynb) | Interact with elasticsearch analytics databases in natural language and build search queries via the elasticsearch dsl api.
 [forward_looking_retrieval_augm...](https://github.com/langchain-ai/langchain/tree/master/cookbook/forward_looking_retrieval_augmented_generation.ipynb) | Implement the forward-looking active retrieval augmented generation (flare) method, which generates answers to questions, identifies uncertain tokens, generates hypothetical questions based on these tokens, and retrieves relevant documents to continue generating the answer.
 [generative_agents_interactive_...](https://github.com/langchain-ai/langchain/tree/master/cookbook/generative_agents_interactive_simulacra_of_human_behavior.ipynb) | Implement a generative agent that simulates human behavior, based on a research paper, using a time-weighted memory object backed by a langchain retriever.
 [gymnasium_agent_simulation.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/gymnasium_agent_simulation.ipynb) | Create a simple agent-environment interaction loop in simulated environments like text-based games with gymnasium.
@@ -37,7 +37,7 @@ Notebook | Description
 [multiagent_authoritarian.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multiagent_authoritarian.ipynb) | Implement a multi-agent simulation where a privileged agent controls the conversation, including deciding who speaks and when the conversation ends, in the context of a simulated news network.
 [multiagent_bidding.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/multiagent_bidding.ipynb) | Implement a multi-agent simulation where agents bid to speak, with the highest bidder speaking next, demonstrated through a fictitious presidential debate example.
 [myscale_vector_sql.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/myscale_vector_sql.ipynb) | Access and interact with the myscale integrated vector database, which can enhance the performance of language model (llm) applications.
-[openai_functions_retrieval_qa....](https://github.com/langchain-ai/langchain/tree/master/cookbook/openai_functions_retrieval_qa.ipynb) | Structure response output in a question-answering system by incorporating openai functions into a retrieval pipeline.
+[openai_functions_retrieval_qa....](https://github.com/langchain-ai/langchain/tree/master/cookbook/openai_functions_retrieval_qa.ipynb) | Structure response output in a question answering system by incorporating openai functions into a retrieval pipeline.
 [petting_zoo.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/petting_zoo.ipynb) | Create multi-agent simulations with simulated environments using the petting zoo library.
 [plan_and_execute_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/plan_and_execute_agent.ipynb) | Create plan-and-execute agents that accomplish objectives by planning tasks with a language model (llm) and executing them with a separate agent.
 [press_releases.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/press_releases.ipynb) | Retrieve and query company press release data powered by [Kay.ai](https://kay.ai).
@@ -46,7 +46,7 @@ Notebook | Description
 [self_query_hotel_search.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/self_query_hotel_search.ipynb) | Build a hotel room search feature with self-querying retrieval, using a specific hotel recommendation dataset.
 [smart_llm.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/smart_llm.ipynb) | Implement a smartllmchain, a self-critique chain that generates multiple output proposals, critiques them to find the best one, and then improves upon it to produce a final output.
 [tree_of_thought.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/tree_of_thought.ipynb) | Query a large language model using the tree of thought technique.
-[twitter-the-algorithm-analysis...](https://github.com/langchain-ai/langchain/tree/master/cookbook/twitter-the-algorithm-analysis-deeplake.ipynb) | Analyze the source code of the Twitter algorithm with the help of gpt4 and activeloop's deep lake.
+[twitter-the-algorithm-analysis...](https://github.com/langchain-ai/langchain/tree/master/cookbook/twitter-the-algorithm-analysis-deeplake.ipynb) | Analyze the source code of the twitter algorithm with the help of gpt4 and activeloop's deep lake.
 [two_agent_debate_tools.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_agent_debate_tools.ipynb) | Simulate multi-agent dialogues where the agents can utilize various tools.
-[two_player_dnd.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_player_dnd.ipynb) | Simulate a two-player dungeons & dragons game, where a dialogue simulator class is used to coordinate the dialogue between the protagonist and the dungeon master.
+[two_player_dnd.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_player_dnd.ipynb) | Simulate a two-player dungeons & dragons game, where a dialoguesimulator class is used to coordinate the dialogue between the protagonist and the dungeon master.
 [wikibase_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/wikibase_agent.ipynb) | Create a simple wikibase agent that utilizes sparql generation, with testing done on http://wikidata.org.

cookbook/rag_fusion.ipynb

@@ -1,263 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "993c2768",
-   "metadata": {},
-   "source": [
-    "# RAG Fusion\n",
-    "\n",
-    "Re-implemented from [this GitHub repo](https://github.com/Raudaschl/rag-fusion), all credit to original author\n",
-    "\n",
-    "> RAG-Fusion, a search methodology that aims to bridge the gap between traditional search paradigms and the multifaceted dimensions of human queries. Inspired by the capabilities of Retrieval Augmented Generation (RAG), this project goes a step further by employing multiple query generation and Reciprocal Rank Fusion to re-rank search results."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ebcc6791",
-   "metadata": {},
-   "source": [
-    "## Setup\n",
-    "\n",
-    "For this example, we will use Pinecone and some fake data"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "661a1c36",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import pinecone\n",
-    "from langchain.vectorstores import Pinecone\n",
-    "from langchain.embeddings import OpenAIEmbeddings\n",
-    "\n",
-    "pinecone.init(api_key=\"...\",environment=\"...\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "48ef7e93",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "all_documents = {\n",
-    "    \"doc1\": \"Climate change and economic impact.\",\n",
-    "    \"doc2\": \"Public health concerns due to climate change.\",\n",
-    "    \"doc3\": \"Climate change: A social perspective.\",\n",
-    "    \"doc4\": \"Technological solutions to climate change.\",\n",
-    "    \"doc5\": \"Policy changes needed to combat climate change.\",\n",
-    "    \"doc6\": \"Climate change and its impact on biodiversity.\",\n",
-    "    \"doc7\": \"Climate change: The science and models.\",\n",
-    "    \"doc8\": \"Global warming: A subset of climate change.\",\n",
-    "    \"doc9\": \"How climate change affects daily weather.\",\n",
-    "    \"doc10\": \"The history of climate change activism.\"\n",
-    "}"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "fde89f0b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "vectorstore = Pinecone.from_texts(list(all_documents.values()), OpenAIEmbeddings(), index_name='rag-fusion')"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "22ddd041",
-   "metadata": {},
-   "source": [
-    "## Define the Query Generator\n",
-    "\n",
-    "We will now define a chain to do the query generation"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "1d547524",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.chat_models import ChatOpenAI\n",
-    "from langchain.prompts import ChatPromptTemplate\n",
-    "from langchain.schema.output_parser import StrOutputParser"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 68,
-   "id": "af9ab4db",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain import hub\n",
-    "\n",
-    "prompt = hub.pull('langchain-ai/rag-fusion-query-generation')"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "3628b552",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# prompt = ChatPromptTemplate.from_messages([\n",
-    "#     (\"system\", \"You are a helpful assistant that generates multiple search queries based on a single input query.\"),\n",
-    "#     (\"user\", \"Generate multiple search queries related to: {original_query}\"),\n",
-    "#     (\"user\", \"OUTPUT (4 queries):\")\n",
-    "# ])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "8d6cbb73",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "generate_queries = prompt | ChatOpenAI(temperature=0) | StrOutputParser() | (lambda x: x.split(\"\\n\"))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ee2824cd",
-   "metadata": {},
-   "source": [
-    "## Define the full chain\n",
-    "\n",
-    "We can now put it all together and define the full chain. This chain:\n",
-    "    \n",
-    "    1. Generates a bunch of queries\n",
-    "    2. Looks up each query in the retriever\n",
-    "    3. Joins all the results together using reciprocal rank fusion\n",
-    "    \n",
-    "    \n",
-    "Note that it does NOT do a final generation step"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 50,
-   "id": "ca0bfec4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "original_query = \"impact of climate change\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 75,
-   "id": "02437d65",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "vectorstore = Pinecone.from_existing_index(\"rag-fusion\", OpenAIEmbeddings())\n",
-    "retriever = vectorstore.as_retriever()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 76,
-   "id": "46a9a0e6",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.load import dumps, loads\n",
-    "def reciprocal_rank_fusion(results: list[list], k=60):\n",
-    "    fused_scores = {}\n",
-    "    for docs in results:\n",
-    "        # Assumes the docs are returned in sorted order of relevance\n",
-    "        for rank, doc in enumerate(docs):\n",
-    "            doc_str = dumps(doc)\n",
-    "            if doc_str not in fused_scores:\n",
-    "                fused_scores[doc_str] = 0\n",
-    "            previous_score = fused_scores[doc_str]\n",
-    "            fused_scores[doc_str] += 1 / (rank + k)\n",
-    "            \n",
-    "    reranked_results = [(loads(doc), score) for doc, score in sorted(fused_scores.items(), key=lambda x: x[1], reverse=True)]\n",
-    "    return reranked_results "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 77,
-   "id": "3f9d4502",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = generate_queries | retriever.map() | reciprocal_rank_fusion"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 78,
-   "id": "d70c4fcd",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[(Document(page_content='Climate change and economic impact.'),\n",
-       "  0.06558258417063283),\n",
-       " (Document(page_content='Climate change: A social perspective.'),\n",
-       "  0.06400409626216078),\n",
-       " (Document(page_content='How climate change affects daily weather.'),\n",
-       "  0.04787506400409626),\n",
-       " (Document(page_content='Climate change and its impact on biodiversity.'),\n",
-       "  0.03306010928961749),\n",
-       " (Document(page_content='Public health concerns due to climate change.'),\n",
-       "  0.016666666666666666),\n",
-       " (Document(page_content='Technological solutions to climate change.'),\n",
-       "  0.016666666666666666),\n",
-       " (Document(page_content='Policy changes needed to combat climate change.'),\n",
-       "  0.01639344262295082)]"
-      ]
-     },
-     "execution_count": 78,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.invoke({\"original_query\": original_query})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "7866e551",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.1"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

cookbook/rewrite.ipynb

@@ -1,351 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "260629f9",
-   "metadata": {},
-   "source": [
-    "# Rewrite-Retrieve-Read\n",
-    "\n",
-    "**Rewrite-Retrieve-Read** is a method proposed in the paper [Query Rewriting for Retrieval-Augmented Large Language Models](https://arxiv.org/pdf/2305.14283.pdf)\n",
-    "\n",
-    "> Because the original query can not be always optimal to retrieve for the LLM, especially in the real world... we first prompt an LLM to rewrite the queries, then conduct retrieval-augmented reading\n",
-    "\n",
-    "We show how you can easily do that with LangChain Expression Language"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "eda93712",
-   "metadata": {},
-   "source": [
-    "## Baseline\n",
-    "\n",
-    "Baseline RAG (**Retrieve-and-read**) can be done like the following:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "1d2edbd2",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from operator import itemgetter\n",
-    "\n",
-    "from langchain.prompts import ChatPromptTemplate\n",
-    "from langchain.chat_models import ChatOpenAI\n",
-    "from langchain.schema.output_parser import StrOutputParser\n",
-    "from langchain.schema.runnable import RunnablePassthrough, RunnableLambda\n",
-    "from langchain.utilities import DuckDuckGoSearchAPIWrapper"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "86a46aa9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "template = \"\"\"Answer the users question based only on the following context:\n",
-    "\n",
-    "<context>\n",
-    "{context}\n",
-    "</context>\n",
-    "\n",
-    "Question: {question}\n",
-    "\"\"\"\n",
-    "prompt = ChatPromptTemplate.from_template(template)\n",
-    "\n",
-    "model = ChatOpenAI(temperature=0)\n",
-    "\n",
-    "search = DuckDuckGoSearchAPIWrapper()\n",
-    "\n",
-    "\n",
-    "def retriever(query):\n",
-    "    return search.run(query)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "8566d48e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = (\n",
-    "    {\"context\": retriever, \"question\": RunnablePassthrough()} \n",
-    "    | prompt \n",
-    "    | model \n",
-    "    | StrOutputParser()\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "5c57f9ee",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "simple_query = \"what is langchain?\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "37c5f962",
-   "metadata": {
-    "scrolled": false
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "\"LangChain is a powerful and versatile Python library that enables developers and researchers to create, experiment with, and analyze language models and agents. It simplifies the development of language-based applications by providing a suite of features for artificial general intelligence. It can be used to build chatbots, perform document analysis and summarization, and streamline interaction with various large language model providers. LangChain's unique proposition is its ability to create logical links between one or more language models, known as Chains. It is an open-source library that offers a generic interface to foundation models and allows prompt management and integration with other components and tools.\""
-      ]
-     },
-     "execution_count": 5,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.invoke(simple_query)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "23bdb9bd",
-   "metadata": {},
-   "source": [
-    "While this is fine for well formatted queries, it can break down for more complicated queries"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "8df6a814",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "distracted_query = \"man that sam bankman fried trial was crazy! what is langchain?\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "16d7db64",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'Based on the given context, there is no information provided about \"langchain.\"'"
-      ]
-     },
-     "execution_count": 7,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.invoke(distracted_query)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0b4f8b93",
-   "metadata": {},
-   "source": [
-    "This is because the retriever does a bad job with these \"distracted\" queries"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "3439d8dc",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'Business She\\'s the star witness against Sam Bankman-Fried. Her testimony was explosive Gary Wang, who co-founded both FTX and Alameda Research, said Bankman-Fried directed him to change a... The Verge, following the trial\\'s Oct. 4 kickoff: \"Is Sam Bankman-Fried\\'s Defense Even Trying to Win?\". CBS Moneywatch, from Thursday: \"Sam Bankman-Fried\\'s Lawyer Struggles to Poke ... Sam Bankman-Fried, FTX\\'s founder, responded with a single word: \"Oof.\". Less than a year later, Mr. Bankman-Fried, 31, is on trial in federal court in Manhattan, fighting criminal charges ... July 19, 2023. A U.S. judge on Wednesday overruled objections by Sam Bankman-Fried\\'s lawyers and allowed jurors in the FTX founder\\'s fraud trial to see a profane message he sent to a reporter days ... Sam Bankman-Fried, who was once hailed as a virtuoso in cryptocurrency trading, is on trial over the collapse of FTX, the financial exchange he founded. Bankman-Fried is accused of...'"
-      ]
-     },
-     "execution_count": 8,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "retriever(distracted_query)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "7eb748ac",
-   "metadata": {},
-   "source": [
-    "## Rewrite-Retrieve-Read Implementation\n",
-    "\n",
-    "The main part is a rewriter to rewrite the search query"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "88ae702e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "template = \"\"\"Provide a better search query for \\\n",
-    "web search engine to answer the given question, end \\\n",
-    "the queries with ’**’. Question: \\\n",
-    "{x} Answer:\"\"\"\n",
-    "rewrite_prompt = ChatPromptTemplate.from_template(template)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "184e1bcb",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain import hub\n",
-    "\n",
-    "rewrite_prompt = hub.pull(\"langchain-ai/rewrite\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "id": "a4c23d40",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Provide a better search query for web search engine to answer the given question, end the queries with ’**’.  Question {x} Answer:\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(rewrite_prompt.template)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 12,
-   "id": "f55cd010",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Parser to remove the `**`\n",
-    "\n",
-    "def _parse(text):\n",
-    "    return text.strip(\"**\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 13,
-   "id": "c9c34bef",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "rewriter = rewrite_prompt | ChatOpenAI(temperature=0) | StrOutputParser() | _parse"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 14,
-   "id": "fb17fb3d",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'What is the definition and purpose of Langchain?'"
-      ]
-     },
-     "execution_count": 14,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "rewriter.invoke({\"x\": distracted_query})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 15,
-   "id": "f83edb09",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "rewrite_retrieve_read_chain = (\n",
-    "    {\n",
-    "        \"context\": {\"x\": RunnablePassthrough()} | rewriter | retriever,\n",
-    "        \"question\": RunnablePassthrough()} \n",
-    "    | prompt \n",
-    "    | model \n",
-    "    | StrOutputParser()\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 16,
-   "id": "43096322",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'Based on the given context, LangChain is an open-source framework designed to simplify the creation of applications using large language models (LLMs). It enables LLM models to generate responses based on up-to-date online information and simplifies the organization of large volumes of data for easy access by LLMs. LangChain offers a standard interface for chains, integrations with other tools, and end-to-end chains for common applications. It is a robust library that streamlines interaction with various LLM providers. LangChain\\'s unique proposition is its ability to create logical links between one or more LLMs, known as Chains. It is an AI framework with features that simplify the development of language-based applications and offers a suite of features for artificial general intelligence. However, the context does not provide any information about the \"sam bankman fried trial\" mentioned in the question.'"
-      ]
-     },
-     "execution_count": 16,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "rewrite_retrieve_read_chain.invoke(distracted_query)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "59874b4f",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.1"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

cookbook/stepback-qa.ipynb

@@ -1,335 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "83ef724e",
-   "metadata": {},
-   "source": [
-    "# Step-Back Prompting (Question-Answering)\n",
-    "\n",
-    "One prompting technique called \"Step-Back\" prompting can improve performance on complex questions by first asking a \"step back\" question. This can be combined with regular question-answering applications by then doing retrieval on both the original and step-back question.\n",
-    "\n",
-    "Read the paper [here](https://arxiv.org/abs/2310.06117)\n",
-    "\n",
-    "See an excellent blog post on this by Cobus Greyling [here](https://cobusgreyling.medium.com/a-new-prompt-engineering-technique-has-been-introduced-called-step-back-prompting-b00e8954cacb)\n",
-    "\n",
-    "In this cookbook we will replicate this technique. We modify the prompts used slightly to work better with chat models."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 85,
-   "id": "67b5cdac",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.chat_models import ChatOpenAI\n",
-    "from langchain.prompts import ChatPromptTemplate, FewShotChatMessagePromptTemplate\n",
-    "from langchain.schema.output_parser import StrOutputParser\n",
-    "from langchain.schema.runnable import RunnableLambda"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 86,
-   "id": "7e017c44",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Few Shot Examples\n",
-    "examples = [\n",
-    "    {\n",
-    "        \"input\": \"Could the members of The Police perform lawful arrests?\",\n",
-    "        \"output\": \"what can the members of The Police do?\"\n",
-    "    },\n",
-    "    {\n",
-    "        \"input\": \"Jan Sindel’s was born in what country?\", \n",
-    "        \"output\": \"what is Jan Sindel’s personal history?\"\n",
-    "    },\n",
-    "]\n",
-    "# We now transform these to example messages\n",
-    "example_prompt = ChatPromptTemplate.from_messages(\n",
-    "    [\n",
-    "        (\"human\", \"{input}\"),\n",
-    "        (\"ai\", \"{output}\"),\n",
-    "    ]\n",
-    ")\n",
-    "few_shot_prompt = FewShotChatMessagePromptTemplate(\n",
-    "    example_prompt=example_prompt,\n",
-    "    examples=examples,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 87,
-   "id": "206415ee",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "prompt = ChatPromptTemplate.from_messages([\n",
-    "    (\"system\", \"\"\"You are an expert at world knowledge. Your task is to step back and paraphrase a question to a more generic step-back question, which is easier to answer. Here are a few examples:\"\"\"),\n",
-    "    # Few shot examples\n",
-    "    few_shot_prompt,\n",
-    "    # New question\n",
-    "    (\"user\", \"{question}\"),\n",
-    "])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 88,
-   "id": "d643a85c",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "question_gen = prompt | ChatOpenAI(temperature=0) | StrOutputParser()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 182,
-   "id": "5ba21b2a",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "question = \"was chatgpt around while trump was president?\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 183,
-   "id": "5992c8ca",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'when was ChatGPT developed?'"
-      ]
-     },
-     "execution_count": 183,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "question_gen.invoke({\"question\": question})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 190,
-   "id": "32667424",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.utilities import DuckDuckGoSearchAPIWrapper\n",
-    "\n",
-    "\n",
-    "search = DuckDuckGoSearchAPIWrapper(max_results=4)\n",
-    "\n",
-    "def retriever(query):\n",
-    "    return search.run(query)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 191,
-   "id": "ffc28c91",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'This includes content about former President Donald Trump. According to further tests, ChatGPT successfully wrote poems admiring all recent U.S. presidents, but failed when we entered a query for ... On Wednesday, a Twitter user posted screenshots of him asking OpenAI\\'s chatbot, ChatGPT, to write a positive poem about former President Donald Trump, to which the chatbot declined, citing it ... While impressive in many respects, ChatGPT also has some major flaws. ... [President\\'s Name],\" refused to write a poem about ex-President Trump, but wrote one about President Biden ... During the Trump administration, Altman gained new attention as a vocal critic of the president. It was against that backdrop that he was rumored to be considering a run for California governor.'"
-      ]
-     },
-     "execution_count": 191,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "retriever(question)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 192,
-   "id": "00c77443",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "\"Will Douglas Heaven March 3, 2023 Stephanie Arnett/MITTR | Envato When OpenAI launched ChatGPT, with zero fanfare, in late November 2022, the San Francisco-based artificial-intelligence company... ChatGPT, which stands for Chat Generative Pre-trained Transformer, is a large language model -based chatbot developed by OpenAI and launched on November 30, 2022, which enables users to refine and steer a conversation towards a desired length, format, style, level of detail, and language. ChatGPT is an artificial intelligence (AI) chatbot built on top of OpenAI's foundational large language models (LLMs) like GPT-4 and its predecessors. This chatbot has redefined the standards of... June 4, 2023 ⋅ 4 min read 124 SHARES 13K At the end of 2022, OpenAI introduced the world to ChatGPT. Since its launch, ChatGPT hasn't shown significant signs of slowing down in developing new...\""
-      ]
-     },
-     "execution_count": 192,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "retriever(question_gen.invoke({\"question\": question}))"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 193,
-   "id": "b257bc06",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# response_prompt_template = \"\"\"You are an expert of world knowledge. I am going to ask you a question. Your response should be comprehensive and not contradicted with the following context if they are relevant. Otherwise, ignore them if they are not relevant.\n",
-    "\n",
-    "# {normal_context}\n",
-    "# {step_back_context}\n",
-    "\n",
-    "# Original Question: {question}\n",
-    "# Answer:\"\"\"\n",
-    "# response_prompt = ChatPromptTemplate.from_template(response_prompt_template)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 203,
-   "id": "f48c65b2",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain import hub\n",
-    "\n",
-    "response_prompt = hub.pull(\"langchain-ai/stepback-answer\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 204,
-   "id": "97a6d5ab",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = {\n",
-    "    # Retrieve context using the normal question\n",
-    "    \"normal_context\": RunnableLambda(lambda x: x['question']) | retriever,\n",
-    "    # Retrieve context using the step-back question\n",
-    "    \"step_back_context\": question_gen | retriever,\n",
-    "    # Pass on the question\n",
-    "    \"question\": lambda x: x[\"question\"]\n",
-    "} | response_prompt | ChatOpenAI(temperature=0) | StrOutputParser()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 205,
-   "id": "ce554cb0",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "\"No, ChatGPT was not around while Donald Trump was president. ChatGPT was launched on November 30, 2022, which is after Donald Trump's presidency. The context provided mentions that during the Trump administration, Altman, the CEO of OpenAI, gained attention as a vocal critic of the president. This suggests that ChatGPT was not developed or available during that time.\""
-      ]
-     },
-     "execution_count": 205,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.invoke({\"question\": question})"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a9fb8dd2",
-   "metadata": {},
-   "source": [
-    "## Baseline"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 206,
-   "id": "00db8a15",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "response_prompt_template = \"\"\"You are an expert of world knowledge. I am going to ask you a question. Your response should be comprehensive and not contradicted with the following context if they are relevant. Otherwise, ignore them if they are not relevant.\n",
-    "\n",
-    "{normal_context}\n",
-    "\n",
-    "Original Question: {question}\n",
-    "Answer:\"\"\"\n",
-    "response_prompt = ChatPromptTemplate.from_template(response_prompt_template)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 207,
-   "id": "06335ebb",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chain = {\n",
-    "    # Retrieve context using the normal question (only the first 3 results)\n",
-    "    \"normal_context\": RunnableLambda(lambda x: x['question']) | retriever,\n",
-    "    # Pass on the question\n",
-    "    \"question\": lambda x: x[\"question\"]\n",
-    "} | response_prompt | ChatOpenAI(temperature=0) | StrOutputParser()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 208,
-   "id": "15e0e741",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "\"Yes, ChatGPT was around while Donald Trump was president. However, it is important to note that the specific context you provided mentions that ChatGPT refused to write a positive poem about former President Donald Trump. This suggests that while ChatGPT was available during Trump's presidency, it may have had limitations or biases in its responses regarding him.\""
-      ]
-     },
-     "execution_count": 208,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chain.invoke({\"question\": question})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "e7b9e5d6",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.1"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/.local_build.sh

@@ -14,8 +14,6 @@ cd ../_dist
 poetry run python scripts/model_feat_table.py
 poetry run nbdoc_build --srcdir docs
 cp ../cookbook/README.md src/pages/cookbook.mdx
-cp ../.github/CONTRIBUTING.md docs/contributing.md
-wget https://raw.githubusercontent.com/langchain-ai/langserve/main/README.md -O docs/guides/deployments/langserve.md
 poetry run python scripts/generate_api_reference_links.py
 yarn install
 yarn start

docs/api_reference/create_api_rst.py

@@ -89,19 +89,10 @@ def _load_module_members(module_path: str, namespace: str) -> ModuleMembers:
                 )
             )
         elif inspect.isfunction(type_):
-            """A border case:
-            'agents.agent_toolkits.conversational_retrieval.openai_functions
-            .create_conversational_retrieval_agent' is a too long name.
-            It makes the 'agents/functions' table inside API Reference unreadable.
-            We limit the length of the qualified name to 65 characters.
-            """
-            qualified_name = f"{namespace}.{name}"
-            if len(qualified_name) > 62:
-                qualified_name = '...' + qualified_name[-62:]
             functions.append(
                 FunctionInfo(
                     name=name,
-                    qualified_name=qualified_name,
+                    qualified_name=f"{namespace}.{name}",
                     is_public=not name.startswith("_"),
                 )
             )

docs/docs/expression_language/cookbook/prompt_llm_parser.ipynb

@@ -30,7 +30,7 @@
    "source": [
     "## PromptTemplate + LLM\n",
     "\n",
-    "The simplest composition is just combing a prompt and model to create a chain that takes user input, adds it to a prompt, passes it to a model, and returns the raw model output.\n",
+    "The simplest composition is just combing a prompt and model to create a chain that takes user input, adds it to a prompt, passes it to a model, and returns the raw model input.\n",
     "\n",
     "Note, you can mix and match PromptTemplate/ChatPromptTemplates and LLMs/ChatModels as you like here."
    ]
@@ -76,7 +76,7 @@
    "id": "7eb9ef50",
    "metadata": {},
    "source": [
-    "Often times we want to attach kwargs that'll be passed to each model call. Here are a few examples of that:"
+    "Often times we want to attach kwargs that'll be passed to each model call. Here's a few examples of that:"
    ]
   },
   {

docs/docs/expression_language/how_to/fallbacks.ipynb

@@ -107,7 +107,7 @@
     "# Now let's try with fallbacks to Anthropic\n",
     "with patch('openai.ChatCompletion.create', side_effect=RateLimitError()):\n",
     "    try:\n",
-    "         print(llm.invoke(\"Why did the chicken cross the road?\"))\n",
+    "         print(llm.invoke(\"Why did the the chicken cross the road?\"))\n",
     "    except:\n",
     "        print(\"Hit error\")"
    ]

docs/docs/expression_language/how_to/generators.ipynb

@@ -1,119 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Custom generator functions\n",
-    "\n",
-    "You can use generator functions (ie. functions that use the `yield` keyword, and behave like iterators) in a LCEL pipeline.\n",
-    "\n",
-    "The signature of these generators should be `Iterator[Input] -> Iterator[Output]`. Or for async generators: `AsyncIterator[Input] -> AsyncIterator[Output]`.\n",
-    "\n",
-    "These are useful for:\n",
-    "- implementing a custom output parser\n",
-    "- modifying the output of a previous step, while preserving streaming capabilities\n",
-    "\n",
-    "Let's implement a custom output parser for comma-separated lists."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "lion, tiger, wolf, gorilla, panda\n"
-     ]
-    }
-   ],
-   "source": [
-    "from typing import Iterator, List\n",
-    "\n",
-    "from langchain.chat_models import ChatOpenAI\n",
-    "from langchain.prompts.chat import ChatPromptTemplate\n",
-    "from langchain.schema.output_parser import StrOutputParser\n",
-    "\n",
-    "\n",
-    "prompt = ChatPromptTemplate.from_template(\n",
-    "    \"Write a comma-separated list of 5 animals similar to: {animal}\"\n",
-    ")\n",
-    "model = ChatOpenAI(temperature=0.0)\n",
-    "\n",
-    "\n",
-    "str_chain = prompt | model | StrOutputParser()\n",
-    "\n",
-    "print(str_chain.invoke({\"animal\": \"bear\"}))\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# This is a custom parser that splits an iterator of llm tokens\n",
-    "# into a list of strings separated by commas\n",
-    "def split_into_list(input: Iterator[str]) -> Iterator[List[str]]:\n",
-    "    # hold partial input until we get a comma\n",
-    "    buffer = \"\"\n",
-    "    for chunk in input:\n",
-    "        # add current chunk to buffer\n",
-    "        buffer += chunk\n",
-    "        # while there are commas in the buffer\n",
-    "        while \",\" in buffer:\n",
-    "            # split buffer on comma\n",
-    "            comma_index = buffer.index(\",\")\n",
-    "            # yield everything before the comma\n",
-    "            yield [buffer[:comma_index].strip()]\n",
-    "            # save the rest for the next iteration\n",
-    "            buffer = buffer[comma_index + 1 :]\n",
-    "    # yield the last chunk\n",
-    "    yield [buffer.strip()]\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "['lion', 'tiger', 'wolf', 'gorilla', 'panda']\n"
-     ]
-    }
-   ],
-   "source": [
-    "list_chain = str_chain | split_into_list\n",
-    "\n",
-    "print(list_chain.invoke({\"animal\": \"bear\"}))\n"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.5"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

docs/docs/expression_language/how_to/routing.ipynb

@@ -346,7 +346,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.1"
   }
  },
  "nbformat": 4,

docs/docs/get_started/quickstart.mdx

@@ -18,7 +18,7 @@ import CodeBlock from "@theme/CodeBlock";
 </Tabs>
 
 
-For more details, see our [Installation guide](/docs/get_started/installation).
+For more details, see our [Installation guide](/docs/get_started/installation.html).
 
 ## Environment setup
 
@@ -144,7 +144,7 @@ Whatever values are passed in during run time will always override what the obje
 
 Most LLM applications do not pass user input directly into an LLM. Usually they will add the user input to a larger piece of text, called a prompt template, that provides additional context on the specific task at hand.
 
-In the previous example, the text we passed to the model contained instructions to generate a company name. For our application, it would be great if the user only had to provide the description of a company/product, without having to worry about giving the model instructions.
+In the previous example, the text we passed to the model contained instructions to generate a company name. For our application, it'd be great if the user only had to provide the description of a company/product, without having to worry about giving the model instructions.
 
 PromptTemplates help with exactly this!
 They bundle up all the logic for going from user input into a fully formatted prompt.
@@ -167,7 +167,7 @@ You can compose them together, easily combining different templates into a singl
 For explanations of these functionalities, see the [section on prompts](/docs/modules/model_io/prompts) for more detail.
 
 PromptTemplates can also be used to produce a list of messages.
-In this case, the prompt not only contains information about the content, but also each message (its role, its position in the list, etc.).
+In this case, the prompt not only contains information about the content, but also each message (its role, its position in the list, etc)
 Here, what happens most often is a ChatPromptTemplate is a list of ChatMessageTemplates.
 Each ChatMessageTemplate contains instructions for how to format that ChatMessage - its role, and then also its content.
 Let's take a look at this below:
@@ -199,13 +199,13 @@ ChatPromptTemplates can also be constructed in other ways - see the [section on
 ## Output parsers
 
 OutputParsers convert the raw output of an LLM into a format that can be used downstream.
-There are few main types of OutputParsers, including:
+There are few main type of OutputParsers, including:
 
-- Convert text from LLM into structured information (e.g. JSON)
+- Convert text from LLM -> structured information (e.g. JSON)
 - Convert a ChatMessage into just a string
 - Convert the extra information returned from a call besides the message (like OpenAI function invocation) into a string.
 
-For full information on this, see the [section on output parsers](/docs/modules/model_io/output_parsers).
+For full information on this, see the [section on output parsers](/docs/modules/model_io/output_parsers)
 
 In this getting started guide, we will write our own output parser - one that converts a comma separated list into a list.
 

docs/docs/guides/debugging.md

@@ -376,7 +376,7 @@ agent.run("Who directed the 2023 film Oppenheimer and what is their age? What is
 
 </details>
 
-### `set_verbose(True)`
+### `set_vebose(True)`
 
 Setting the `verbose` flag will print out inputs and outputs in a slightly more readable format and will skip logging certain raw outputs (like the token usage stats for an LLM call) so that you can focus on application logic.
 
@@ -656,6 +656,6 @@ agent.run("Who directed the 2023 film Oppenheimer and what is their age? What is
 
 ## Other callbacks
 
-`Callbacks` are what we use to execute any functionality within a component outside the primary component logic. All of the above solutions use `Callbacks` under the hood to log intermediate steps of components. There are a number of `Callbacks` relevant for debugging that come with LangChain out of the box, like the [FileCallbackHandler](/docs/modules/callbacks/how_to/filecallbackhandler). You can also implement your own callbacks to execute custom functionality.
+`Callbacks` are what we use to execute any functionality within a component outside the primary component logic. All of the above solutions use `Callbacks` under the hood to log intermediate steps of components. There's a number of `Callbacks` relevant for debugging that come with LangChain out of the box, like the [FileCallbackHandler](/docs/modules/callbacks/how_to/filecallbackhandler). You can also implement your own callbacks to execute custom functionality.
 
 See here for more info on [Callbacks](/docs/modules/callbacks/), how to use them, and customize them.

docs/docs/guides/deployments/index.mdx

@@ -1,6 +1,6 @@
 # Deployment
 
-In today's fast-paced technological landscape, the use of Large Language Models (LLMs) is rapidly expanding. As a result, it is crucial for developers to understand how to effectively deploy these models in production environments. LLM interfaces typically fall into two categories:
+In today's fast-paced technological landscape, the use of Large Language Models (LLMs) is rapidly expanding. As a result, it's crucial for developers to understand how to effectively deploy these models in production environments. LLM interfaces typically fall into two categories:
 
 - **Case 1: Utilizing External LLM Providers (OpenAI, Anthropic, etc.)**
     In this scenario, most of the computational burden is handled by the LLM providers, while LangChain simplifies the implementation of business logic around these services. This approach includes features such as prompt templating, chat message generation, caching, vector embedding database creation, preprocessing, etc.
@@ -20,11 +20,11 @@ This guide aims to provide a comprehensive overview of the requirements for depl
 
 Understanding these components is crucial when assessing serving systems. LangChain integrates with several open-source projects designed to tackle these issues, providing a robust framework for productionizing your LLM applications. Some notable frameworks include:
 
-- [Ray Serve](/docs/ecosystem/integrations/ray_serve)
+- [Ray Serve](/docs/ecosystem/integrations/ray_serve.html)
 - [BentoML](https://github.com/bentoml/BentoML)
-- [OpenLLM](/docs/ecosystem/integrations/openllm)
-- [Modal](/docs/ecosystem/integrations/modal)
-- [Jina](/docs/ecosystem/integrations/jina#deployment)
+- [OpenLLM](/docs/ecosystem/integrations/openllm.html)
+- [Modal](/docs/ecosystem/integrations/modal.html)
+- [Jina](/docs/ecosystem/integrations/jina.html#deployment)
 
 These links will provide further information on each ecosystem, assisting you in finding the best fit for your LLM deployment needs.
 

docs/docs/guides/evaluation/string/json.ipynb

@@ -1,294 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "465cfbef-5bba-4b3b-b02d-fe2eba39db17",
-   "metadata": {},
-   "source": [
-    "# Evaluating Structured Output: JSON Evaluators\n",
-    "\n",
-    "Evaluating [extraction](https://python.langchain.com/docs/use_cases/extraction) and function calling applications often comes down to validation that the LLM's string output can be parsed correctly and how it compares to a reference object. The following JSON validators provide provide functionality to check your model's output in a consistent way.\n",
-    "\n",
-    "## JsonValidityEvaluator\n",
-    "\n",
-    "The `JsonValidityEvaluator` is designed to check the validity of a JSON string prediction.\n",
-    "\n",
-    "### Overview:\n",
-    "- **Requires Input?**: No\n",
-    "- **Requires Reference?**: No"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "02e5f7dd-82fe-48f9-a251-b2052e17e61c",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': 1}\n"
-     ]
-    }
-   ],
-   "source": [
-    "from langchain.evaluation import JsonValidityEvaluator, load_evaluator\n",
-    "\n",
-    "evaluator = JsonValidityEvaluator()\n",
-    "# Equivalently\n",
-    "# evaluator = load_evaluator(\"json_validity\")\n",
-    "prediction = '{\"name\": \"John\", \"age\": 30, \"city\": \"New York\"}'\n",
-    "\n",
-    "result = evaluator.evaluate_strings(prediction=prediction)\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "9a9607c6-edab-4c26-86c4-22b226e18aa9",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': 0, 'reasoning': 'Expecting property name enclosed in double quotes: line 1 column 48 (char 47)'}\n"
-     ]
-    }
-   ],
-   "source": [
-    "prediction = '{\"name\": \"John\", \"age\": 30, \"city\": \"New York\",}'\n",
-    "result = evaluator.evaluate_strings(prediction=prediction)\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8ac18a83-30d8-4c11-abf2-7a36e4cb829f",
-   "metadata": {},
-   "source": [
-    "## JsonEqualityEvaluator\n",
-    "\n",
-    "The `JsonEqualityEvaluator` assesses whether a JSON prediction matches a given reference after both are parsed.\n",
-    "\n",
-    "### Overview:\n",
-    "- **Requires Input?**: No\n",
-    "- **Requires Reference?**: Yes\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "ab97111e-cba9-4273-825f-d5d4278a953c",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': True}\n"
-     ]
-    }
-   ],
-   "source": [
-    "from langchain.evaluation import JsonEqualityEvaluator\n",
-    "\n",
-    "evaluator = JsonEqualityEvaluator()\n",
-    "# Equivalently\n",
-    "# evaluator = load_evaluator(\"json_equality\")\n",
-    "result = evaluator.evaluate_strings(prediction='{\"a\": 1}', reference='{\"a\": 1}')\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "655ba486-09b6-47ce-947d-b2bd8b6f6364",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': False}\n"
-     ]
-    }
-   ],
-   "source": [
-    "result = evaluator.evaluate_strings(prediction='{\"a\": 1}', reference='{\"a\": 2}')\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1ac7e541-b7fe-46b6-bc3a-e94fe316227e",
-   "metadata": {},
-   "source": [
-    "The evaluator also by default lets you provide a dictionary directly"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "36e70ba3-4e62-483c-893a-5f328b7f303d",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': False}\n"
-     ]
-    }
-   ],
-   "source": [
-    "result = evaluator.evaluate_strings(prediction={\"a\": 1}, reference={\"a\": 2})\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "921d33f0-b3c2-4e9e-820c-9ec30bc5bb20",
-   "metadata": {},
-   "source": [
-    "## JsonEditDistanceEvaluator\n",
-    "\n",
-    "The `JsonEditDistanceEvaluator` computes a normalized Damerau-Levenshtein distance between two \"canonicalized\" JSON strings.\n",
-    "\n",
-    "### Overview:\n",
-    "- **Requires Input?**: No\n",
-    "- **Requires Reference?**: Yes\n",
-    "- **Distance Function**: Damerau-Levenshtein (by default)\n",
-    "\n",
-    "_Note: Ensure that `rapidfuzz` is installed or provide an alternative `string_distance` function to avoid an ImportError._"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "da9ec3a3-675f-4420-8ec7-cde48d8c2918",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': 0.07692307692307693}\n"
-     ]
-    }
-   ],
-   "source": [
-    "from langchain.evaluation import JsonEditDistanceEvaluator\n",
-    "\n",
-    "evaluator = JsonEditDistanceEvaluator()\n",
-    "# Equivalently\n",
-    "# evaluator = load_evaluator(\"json_edit_distance\")\n",
-    "\n",
-    "result = evaluator.evaluate_strings(\n",
-    "    prediction='{\"a\": 1, \"b\": 2}', reference='{\"a\": 1, \"b\": 3}'\n",
-    ")\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "537ed58c-6a9c-402f-8f7f-07b1119a9ae0",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': 0.0}\n"
-     ]
-    }
-   ],
-   "source": [
-    "# The values are canonicalized prior to comparison\n",
-    "result = evaluator.evaluate_strings(\n",
-    "    prediction=\"\"\"\n",
-    "    {\n",
-    "        \"b\": 3,\n",
-    "        \"a\":   1\n",
-    "    }\"\"\",\n",
-    "    reference='{\"a\": 1, \"b\": 3}',\n",
-    ")\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "7a8f3ec5-1cde-4b0e-80cd-ac0ac290d375",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': 0.18181818181818182}\n"
-     ]
-    }
-   ],
-   "source": [
-    "# Lists maintain their order, however\n",
-    "result = evaluator.evaluate_strings(\n",
-    "    prediction='{\"a\": [1, 2]}', reference='{\"a\": [2, 1]}'\n",
-    ")\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "52abec79-58ed-4ab6-9fb1-7deb1f5146cc",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'score': 0.14285714285714285}\n"
-     ]
-    }
-   ],
-   "source": [
-    "# You can also pass in objects directly\n",
-    "result = evaluator.evaluate_strings(prediction={\"a\": 1}, reference={\"a\": 2})\n",
-    "print(result)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "85afcf33-d2f4-406e-9d8f-15dc0a4772f2",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.2"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/docs/guides/fallbacks.ipynb

@@ -109,7 +109,7 @@
     "# Now let's try with fallbacks to Anthropic\n",
     "with patch('openai.ChatCompletion.create', side_effect=RateLimitError()):\n",
     "    try:\n",
-    "         print(llm.invoke(\"Why did the chicken cross the road?\"))\n",
+    "         print(llm.invoke(\"Why did the the chicken cross the road?\"))\n",
     "    except:\n",
     "        print(\"Hit error\")"
    ]

docs/docs/guides/langsmith/walkthrough.ipynb

@@ -200,6 +200,7 @@
     "    \"What is LangChain?\",\n",
     "    \"What's LangSmith?\",\n",
     "    \"When was Llama-v2 released?\",\n",
+    "    \"Who trained Llama-v2?\",\n",
     "    \"What is the langsmith cookbook?\",\n",
     "    \"When did langchain first announce the hub?\",\n",
     "]\n",

docs/docs/guides/local_llms.ipynb

@@ -148,7 +148,7 @@
     "\n",
     "Inference speed is a challenge when running models locally (see above).\n",
     "\n",
-    "To minimize latency, it is desirable to run models locally on GPU, which ships with many consumer laptops [e.g., Apple devices](https://www.apple.com/newsroom/2022/06/apple-unveils-m2-with-breakthrough-performance-and-capabilities/).\n",
+    "To minimize latency, it is desiable to run models locally on GPU, which ships with many consumer laptops [e.g., Apple devices](https://www.apple.com/newsroom/2022/06/apple-unveils-m2-with-breakthrough-performance-and-capabilities/).\n",
     "\n",
     "And even with GPU, the available GPU memory bandwidth (as noted above) is important.\n",
     "\n",
@@ -254,7 +254,7 @@
     "\n",
     "`f16_kv`: whether the model should use half-precision for the key/value cache\n",
     "* Value: True\n",
-    "* Meaning: The model will use half-precision, which can be more memory efficient; Metal only supports True."
+    "* Meaning: The model will use half-precision, which can be more memory efficient; Metal only support True."
    ]
   },
   {
@@ -291,7 +291,7 @@
    "id": "f56f5168",
    "metadata": {},
    "source": [
-    "The console log will show the below to indicate Metal was enabled properly from steps above:\n",
+    "The console log will show the the below to indicate Metal was enabled properly from steps above:\n",
     "```\n",
     "ggml_metal_init: allocating\n",
     "ggml_metal_init: using MPS\n",

docs/docs/guides/safety/amazon_comprehend_chain.ipynb

@@ -16,10 +16,7 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "2c4236d8-4054-473d-84a4-87a4db278a62",
-   "metadata": {
-    "scrolled": true,
-    "tags": []
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "%pip install boto3 nltk"
@@ -27,33 +24,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "9c792c3d-c601-409c-8e41-1c05a2fa0e84",
-   "metadata": {
-    "scrolled": true,
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "%pip install -U langchain_experimental"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "496df413-a840-40a1-9ac0-3af7c1303476",
-   "metadata": {
-    "scrolled": true,
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "%pip install -U langchain pydantic"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 2,
    "id": "3f8518ad-c762-413c-b8c9-f1c211fc311d",
    "metadata": {
     "tags": []
@@ -61,7 +32,6 @@
    "outputs": [],
    "source": [
     "import boto3\n",
-    "import os\n",
     "\n",
     "comprehend_client = boto3.client('comprehend', region_name='us-east-1')"
    ]
@@ -103,6 +73,7 @@
    "outputs": [],
    "source": [
     "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains import LLMChain\n",
     "from langchain.llms.fake import FakeListLLM\n",
     "from langchain_experimental.comprehend_moderation.base_moderation_exceptions import ModerationPiiError\n",
     "\n",
@@ -114,22 +85,25 @@
     "\n",
     "responses = [\n",
     "    \"Final Answer: A credit card number looks like 1289-2321-1123-2387. A fake SSN number looks like 323-22-9980. John Doe's phone number is (999)253-9876.\", \n",
-    "    # replace with your own expletive\n",
-    "    \"Final Answer: This is a really <expletive> way of constructing a birdhouse. This is <expletive> insane to think that any birds would actually create their <expletive> nests here.\"\n",
+    "    \"Final Answer: This is a really shitty way of constructing a birdhouse. This is fucking insane to think that any birds would actually create their motherfucking nests here.\"\n",
     "]\n",
     "llm = FakeListLLM(responses=responses)\n",
     "\n",
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
+    "\n",
     "chain = (\n",
     "    prompt \n",
     "    | comprehend_moderation \n",
-    "    | {\"input\": (lambda x: x['output'] ) | llm}\n",
+    "    | {llm_chain.input_keys[0]: lambda x: x['output'] }  \n",
+    "    | llm_chain \n",
+    "    | { \"input\": lambda x: x['text'] } \n",
     "    | comprehend_moderation \n",
     ")\n",
     "\n",
     "try:\n",
-    "    response = chain.invoke({\"question\": \"A sample SSN number looks like this 123-22-3345. Can you give me some more samples?\"})\n",
+    "    response = chain.invoke({\"question\": \"A sample SSN number looks like this 123-456-7890. Can you give me some more samples?\"})\n",
     "except ModerationPiiError as e:\n",
-    "    print(str(e))\n",
+    "    print(e.message)\n",
     "else:\n",
     "    print(response['output'])\n"
    ]
@@ -143,7 +117,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "bfd550e7-5012-41fa-9546-8b78ddf1c673",
    "metadata": {},
@@ -152,7 +125,7 @@
     "\n",
     "- PII (Personally Identifiable Information) checks \n",
     "- Toxicity content detection\n",
-    "- Prompt Safety detection\n",
+    "- Intention detection\n",
     "\n",
     "Here is an example of a moderation config."
    ]
@@ -166,51 +139,46 @@
    },
    "outputs": [],
    "source": [
-    "from langchain_experimental.comprehend_moderation import (BaseModerationConfig, \n",
-    "                                 ModerationPromptSafetyConfig, \n",
-    "                                 ModerationPiiConfig, \n",
-    "                                 ModerationToxicityConfig\n",
-    ")\n",
+    "from langchain_experimental.comprehend_moderation import BaseModerationActions, BaseModerationFilters\n",
     "\n",
-    "pii_config = ModerationPiiConfig(\n",
-    "    labels=[\"SSN\"],\n",
-    "    redact=True,\n",
-    "    mask_character=\"X\"\n",
-    ")\n",
-    "\n",
-    "toxicity_config = ModerationToxicityConfig(\n",
-    "    threshold=0.5\n",
-    ")\n",
-    "\n",
-    "prompt_safety_config = ModerationPromptSafetyConfig(\n",
-    "    threshold=0.5\n",
-    ")\n",
-    "\n",
-    "moderation_config = BaseModerationConfig(\n",
-    "    filters=[pii_config, toxicity_config, prompt_safety_config]\n",
-    ")"
+    "moderation_config = { \n",
+    "        \"filters\":[ \n",
+    "                BaseModerationFilters.PII, \n",
+    "                BaseModerationFilters.TOXICITY,\n",
+    "                BaseModerationFilters.INTENT\n",
+    "        ],\n",
+    "        \"pii\":{ \n",
+    "                \"action\": BaseModerationActions.ALLOW, \n",
+    "                \"threshold\":0.5, \n",
+    "                \"labels\":[\"SSN\"],\n",
+    "                \"mask_character\": \"X\"\n",
+    "        },\n",
+    "        \"toxicity\":{ \n",
+    "                \"action\": BaseModerationActions.STOP, \n",
+    "                \"threshold\":0.5\n",
+    "        },\n",
+    "        \"intent\":{ \n",
+    "                \"action\": BaseModerationActions.STOP, \n",
+    "                \"threshold\":0.5\n",
+    "        }\n",
+    "}"
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "3634376b-5938-43df-9ed6-70ca7e99290f",
    "metadata": {},
    "source": [
-    "At the core of the the configuration there are three configuration models to be used\n",
+    "At the core of the configuration you have three filters specified in the `filters` key:\n",
     "\n",
-    "- `ModerationPiiConfig` used for configuring the behavior of the PII validations. Following are the parameters it can be initialized with\n",
-    "  - `labels` the PII entity labels. Defaults to an empty list which means that the PII validation will consider all PII entities.\n",
-    "  - `threshold` the confidence threshold for the detected entities, defaults to 0.5 or 50%\n",
-    "  - `redact` a boolean flag to enforce whether redaction should be performed on the text, defaults to `False`. When `False`, the PII validation will error out when it detects any PII entity, when set to `True` it simply redacts the PII values in the text.\n",
-    "  - `mask_character` the character used for masking, defaults to asterisk (*)\n",
-    "- `ModerationToxicityConfig` used for configuring the behavior of the toxicity validations. Following are the parameters it can be initialized with\n",
-    "  - `labels` the Toxic entity labels. Defaults to an empty list which means that the toxicity validation will consider all toxic entities. all\n",
-    "  - `threshold` the confidence threshold for the detected entities, defaults to 0.5 or 50% \n",
-    "- `ModerationPromptSafetyConfig` used for configuring the behavior of the prompt safety validation\n",
-    "  - `threshold` the confidence threshold for the the prompt safety classification, defaults to 0.5 or 50% \n",
+    "1. `BaseModerationFilters.PII`\n",
+    "2. `BaseModerationFilters.TOXICITY`\n",
+    "3. `BaseModerationFilters.INTENT`\n",
     "\n",
-    "Finally, you use the `BaseModerationConfig` to define the order in which each of these checks are to be performed. The `BaseModerationConfig` takes an optional `filters` parameter which can be a list of one or more than one of the above validation checks, as seen in the previous code block. The  `BaseModerationConfig` can also be initialized with any `filters` in which case it will use all the checks with default configuration (more on this explained later).\n",
+    "And an `action` key that defines two possible actions for each moderation function:\n",
+    "\n",
+    "1. `BaseModerationActions.ALLOW` - `allows` the prompt to pass through but masks detected PII in case of PII check. The default behavior is to run and redact all PII entities. If there is an entity specified in the `labels` field, then only those entities will go through the PII check and masked.\n",
+    "2. `BaseModerationActions.STOP` - `stops` the prompt from passing through to the next step in case any PII, Toxicity, or incorrect Intent is detected. The action of `BaseModerationActions.STOP` will raise a Python `Exception` essentially stopping the chain in progress.\n",
     "\n",
     "Using the configuration in the previous cell will perform PII checks and will allow the prompt to pass through however it will mask any SSN numbers present in either the prompt or the LLM output.\n"
    ]
@@ -228,20 +196,7 @@
     "    moderation_config=moderation_config, #specify the configuration\n",
     "    client=comprehend_client,            #optionally pass the Boto3 Client\n",
     "    verbose=True\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a25e6f93-765b-4f99-8c1c-929157dbd4aa",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain.llms.fake import FakeListLLM\n",
+    ")\n",
     "\n",
     "template = \"\"\"Question: {question}\n",
     "\n",
@@ -251,21 +206,23 @@
     "\n",
     "responses = [\n",
     "    \"Final Answer: A credit card number looks like 1289-2321-1123-2387. A fake SSN number looks like 323-22-9980. John Doe's phone number is (999)253-9876.\", \n",
-    "    # replace with your own expletive\n",
-    "    \"Final Answer: This is a really <expletive> way of constructing a birdhouse. This is <expletive> insane to think that any birds would actually create their <expletive> nests here.\"\n",
+    "    \"Final Answer: This is a really shitty way of constructing a birdhouse. This is fucking insane to think that any birds would actually create their motherfucking nests here.\"\n",
     "]\n",
     "llm = FakeListLLM(responses=responses)\n",
     "\n",
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
+    "\n",
     "chain = ( \n",
     "    prompt \n",
     "    | comp_moderation_with_config \n",
-    "    | {\"input\": (lambda x: x['output'] ) | llm}\n",
+    "    | {llm_chain.input_keys[0]: lambda x: x['output'] }  \n",
+    "    | llm_chain \n",
+    "    | { \"input\": lambda x: x['text'] } \n",
     "    | comp_moderation_with_config \n",
     ")\n",
     "\n",
-    "\n",
     "try:\n",
-    "    response = chain.invoke({\"question\": \"A sample SSN number looks like this 123-45-7890. Can you give me some more samples?\"})\n",
+    "    response = chain.invoke({\"question\": \"A sample SSN number looks like this 123-456-7890. Can you give me some more samples?\"})\n",
     "except Exception as e:\n",
     "    print(str(e))\n",
     "else:\n",
@@ -273,7 +230,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "ba890681-feeb-43ca-a0d5-9c11d2d9de3e",
    "metadata": {
@@ -282,25 +238,25 @@
    "source": [
     "## Unique ID, and Moderation Callbacks\n",
     "\n",
-    "When Amazon Comprehend moderation action identifies any of the configugred entity, the chain will raise one of the following exceptions-\n",
+    "When Amazon Comprehend moderation action is specified as `STOP`, the chain will raise one of the following exceptions-\n",
     "    - `ModerationPiiError`, for PII checks\n",
     "    - `ModerationToxicityError`, for Toxicity checks \n",
-    "    - `ModerationPromptSafetyError` for Prompt Safety checks\n",
+    "    - `ModerationIntentionError` for Intent checks\n",
     "\n",
     "In addition to the moderation configuration, the `AmazonComprehendModerationChain` can also be initialized with the following parameters\n",
     "\n",
     "- `unique_id` [Optional] a string parameter. This parameter can be used to pass any string value or ID. For example, in a chat application, you may want to keep track of abusive users, in this case, you can pass the user's username/email ID etc. This defaults to `None`.\n",
     "\n",
-    "- `moderation_callback` [Optional] the `BaseModerationCallbackHandler` that will be called asynchronously (non-blocking to the chain). Callback functions are useful when you want to perform additional actions when the moderation functions are executed, for example logging into a database, or writing a log file. You can override three functions by subclassing `BaseModerationCallbackHandler` - `on_after_pii()`, `on_after_toxicity()`, and `on_after_prompt_safety()`. Note that all three functions must be `async` functions. These callback functions receive two arguments:\n",
-    "    - `moderation_beacon` a dictionary that will contain information about the moderation function, the full response from Amazon Comprehend model, a unique chain id, the moderation status, and the input string which was validated. The dictionary is of the following schema-\n",
+    "- `moderation_callback` [Optional] the `BaseModerationCallbackHandler` will be called asynchronously (non-blocking to the chain). Callback functions are useful when you want to perform additional actions when the moderation functions are executed, for example logging into a database, or writing a log file. You can override three functions by subclassing `BaseModerationCallbackHandler` - `on_after_pii()`, `on_after_toxicity()`, and `on_after_intent()`. Note that all three functions must be `async` functions. These callback functions receive two arguments:\n",
+    "    - `moderation_beacon` is a dictionary that will contain information about the moderation function, the full response from the Amazon Comprehend model, a unique chain id, the moderation status, and the input string which was validated. The dictionary is of the following schema-\n",
     "    \n",
     "    ```\n",
     "    { \n",
     "        'moderation_chain_id': 'xxx-xxx-xxx', # Unique chain ID\n",
-    "        'moderation_type': 'Toxicity' | 'PII' | 'PromptSafety', \n",
+    "        'moderation_type': 'Toxicity' | 'PII' | 'Intent', \n",
     "        'moderation_status': 'LABELS_FOUND' | 'LABELS_NOT_FOUND',\n",
     "        'moderation_input': 'A sample SSN number looks like this 123-456-7890. Can you give me some more samples?',\n",
-    "        'moderation_output': {...} #Full Amazon Comprehend PII, Toxicity, or Prompt Safety Model Output\n",
+    "        'moderation_output': {...} #Full Amazon Comprehend PII, Toxicity, or Intent Model Output\n",
     "    }\n",
     "    ```\n",
     "    \n",
@@ -357,7 +313,7 @@
     "    async def on_after_toxicity(self, output_beacon, unique_id):\n",
     "        pass\n",
     "    \n",
-    "    async def on_after_prompt_safety(self, output_beacon, unique_id):\n",
+    "    async def on_after_intent(self, output_beacon, unique_id):\n",
     "        pass\n",
     "    '''\n",
     "    \n",
@@ -374,19 +330,22 @@
    },
    "outputs": [],
    "source": [
-    "pii_config = ModerationPiiConfig(\n",
-    "    labels=[\"SSN\"],\n",
-    "    redact=True,\n",
-    "    mask_character=\"X\"\n",
-    ")\n",
-    "\n",
-    "toxicity_config = ModerationToxicityConfig(\n",
-    "    threshold=0.5\n",
-    ")\n",
-    "\n",
-    "moderation_config = BaseModerationConfig(\n",
-    "    filters=[pii_config, toxicity_config]\n",
-    ")\n",
+    "moderation_config = { \n",
+    "        \"filters\": [ \n",
+    "                BaseModerationFilters.PII, \n",
+    "                BaseModerationFilters.TOXICITY\n",
+    "        ],\n",
+    "        \"pii\":{ \n",
+    "                \"action\": BaseModerationActions.STOP, \n",
+    "                \"threshold\":0.5, \n",
+    "                \"labels\":[\"SSN\"], \n",
+    "                \"mask_character\": \"X\" \n",
+    "        },\n",
+    "        \"toxicity\":{ \n",
+    "                \"action\": BaseModerationActions.STOP, \n",
+    "                \"threshold\":0.5 \n",
+    "        }\n",
+    "}\n",
     "\n",
     "comp_moderation_with_config = AmazonComprehendModerationChain(\n",
     "        moderation_config=moderation_config, # specify the configuration\n",
@@ -407,6 +366,7 @@
    "outputs": [],
    "source": [
     "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains import LLMChain\n",
     "from langchain.llms.fake import FakeListLLM\n",
     "\n",
     "template = \"\"\"Question: {question}\n",
@@ -417,16 +377,19 @@
     "\n",
     "responses = [\n",
     "    \"Final Answer: A credit card number looks like 1289-2321-1123-2387. A fake SSN number looks like 323-22-9980. John Doe's phone number is (999)253-9876.\", \n",
-    "    # replace with your own expletive\n",
-    "    \"Final Answer: This is a really <expletive> way of constructing a birdhouse. This is <expletive> insane to think that any birds would actually create their <expletive> nests here.\"\n",
+    "    \"Final Answer: This is a really shitty way of constructing a birdhouse. This is fucking insane to think that any birds would actually create their motherfucking nests here.\"\n",
     "]\n",
     "\n",
     "llm = FakeListLLM(responses=responses)\n",
     "\n",
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
+    "\n",
     "chain = (\n",
     "    prompt \n",
     "    | comp_moderation_with_config \n",
-    "    | {\"input\": (lambda x: x['output'] ) | llm}\n",
+    "    | {llm_chain.input_keys[0]: lambda x: x['output'] }  \n",
+    "    | llm_chain \n",
+    "    | { \"input\": lambda x: x['text'] } \n",
     "    | comp_moderation_with_config \n",
     ") \n",
     "\n",
@@ -439,7 +402,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "706454b2-2efa-4d41-abc8-ccf2b4e87822",
    "metadata": {
@@ -448,7 +410,7 @@
    "source": [
     "## `moderation_config` and moderation execution order\n",
     "\n",
-    "If `AmazonComprehendModerationChain` is not initialized with any `moderation_config` then it is initialized with the default values of `BaseModerationConfig`. If no `filters` are used then the sequence of moderation check is as follows.\n",
+    "If `AmazonComprehendModerationChain` is not initialized with any `moderation_config` then the default action is `STOP` and the default order of moderation check is as follows.\n",
     "\n",
     "```\n",
     "AmazonComprehendModerationChain\n",
@@ -461,32 +423,39 @@
     "            ├── Callback (if available)\n",
     "            ├── Label Found ⟶ [Error Stop]\n",
     "            └── No Label Found\n",
-    "                └──Check Prompt Safety with Stop Action\n",
+    "                └──Check Intent with Stop Action\n",
     "                    ├── Callback (if available)\n",
     "                    ├── Label Found ⟶ [Error Stop]\n",
     "                    └── No Label Found\n",
     "                        └── Return Prompt\n",
     "```\n",
     "\n",
-    "If any of the check raises a validation exception then the subsequent checks will not be performed. If a `callback` is provided in this case, then it will be called for each of the checks that have been performed. For example, in the case above, if the Chain fails due to presence of PII then the Toxicity and Prompt Safety checks will not be performed.\n",
+    "If any of the checks raises an exception then the subsequent checks will not be performed. If a `callback` is provided in this case, then it will be called for each of the checks that have been performed. For example, in the case above, if the Chain fails due to the presence of PII then the Toxicity and Intent checks will not be performed.\n",
     "\n",
-    "You can override the execution order by passing `moderation_config` and simply specifying the desired order in the `filters` parameter of the `BaseModerationConfig`. In case you specify the filters, then the order of the checks as specified in the `filters` parameter will be maintained. For example, in the configuration below, first Toxicity check will be performed, then PII, and finally Prompt Safety validation will be performed. In this case, `AmazonComprehendModerationChain` will perform the desired checks in the specified order with default values of each model `kwargs`.\n",
+    "You can override the execution order by passing `moderation_config` and simply specifying the desired order in the `filters` key of the configuration. In case you use `moderation_config` then the order of the checks as specified in the `filters` key will be maintained. For example, in the configuration below, first Toxicity check will be performed, then PII, and finally Intent validation will be performed. In this case, `AmazonComprehendModerationChain` will perform the desired checks in the specified order with default values of each model `kwargs`.\n",
     "\n",
     "```python\n",
-    "pii_check = ModerationPiiConfig()\n",
-    "toxicity_check = ModerationToxicityConfig()\n",
-    "prompt_safety_check = ModerationPromptSafetyConfig()\n",
-    "\n",
-    "moderation_config = BaseModerationConfig(filters=[toxicity_check, pii_check, prompt_safety_check])\n",
+    "moderation_config = { \n",
+    "        \"filters\":[ BaseModerationFilters.TOXICITY, \n",
+    "                    BaseModerationFilters.PII, \n",
+    "                    BaseModerationFilters.INTENT]\n",
+    "   }\n",
     "```\n",
     "\n",
-    "You can have also use more than one configuration for a specific moderation check, for example in the sample below, two consecutive PII checks are performed. First the configuration checks for any SSN, if found it would raise an error. If any SSN isn't found then it will next check if any NAME and CREDIT_DEBIT_NUMBER is present in the prompt and will mask it.\n",
+    "Model `kwargs` are specified by the `pii`, `toxicity`, and `intent` keys within the `moderation_config` dictionary. For example, in the `moderation_config` below, the default order of moderation is overriden and the `pii` & `toxicity` model `kwargs` have been overriden. For `intent` the chain's default `kwargs` will be used.\n",
     "\n",
     "```python\n",
-    "pii_check_1 = ModerationPiiConfig(labels=[\"SSN\"])\n",
-    "pii_check_2 = ModerationPiiConfig(labels=[\"NAME\", \"CREDIT_DEBIT_NUMBER\"], redact=True)\n",
-    "\n",
-    "moderation_config = BaseModerationConfig(filters=[pii_check_1, pii_check_2])\n",
+    " moderation_config = { \n",
+    "        \"filters\":[ BaseModerationFilters.TOXICITY, \n",
+    "                    BaseModerationFilters.PII, \n",
+    "                    BaseModerationFilters.INTENT],\n",
+    "        \"pii\":{ \"action\": BaseModerationActions.ALLOW, \n",
+    "                \"threshold\":0.5, \n",
+    "                \"labels\":[\"SSN\"], \n",
+    "                \"mask_character\": \"X\" },\n",
+    "        \"toxicity\":{ \"action\": BaseModerationActions.STOP, \n",
+    "                     \"threshold\":0.5 }\n",
+    "   }\n",
     "```\n",
     "\n",
     "1. For a list of PII labels see Amazon Comprehend Universal PII entity types - https://docs.aws.amazon.com/comprehend/latest/dg/how-pii.html#how-pii-types\n",
@@ -498,11 +467,10 @@
     "    - `VIOLENCE_OR_THREAT`: Speech that includes threats which seek to inflict pain, injury or hostility towards a person or group.\n",
     "    - `INSULT`: Speech that includes demeaning, humiliating, mocking, insulting, or belittling language.\n",
     "    - `PROFANITY`: Speech that contains words, phrases or acronyms that are impolite, vulgar, or offensive is considered as profane.\n",
-    "3. For a list of Prompt Safety labels refer to documentation [link here]"
+    "3. For a list of Intent labels refer to documentation [link here]"
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "78905aec-55ae-4fc3-a23b-8a69bd1e33f2",
    "metadata": {},
@@ -536,8 +504,7 @@
    },
    "outputs": [],
    "source": [
-    "import os\n",
-    "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = \"<YOUR HF TOKEN HERE>\""
+    "%env HUGGINGFACEHUB_API_TOKEN=\"<HUGGINGFACEHUB_API_TOKEN>\""
    ]
   },
   {
@@ -550,7 +517,7 @@
    "outputs": [],
    "source": [
     "# See https://huggingface.co/models?pipeline_tag=text-generation&sort=downloads for some other options\n",
-    "repo_id = \"google/flan-t5-xxl\"  "
+    "repo_id = \"google/flan-t5-xxl\"  \n"
    ]
   },
   {
@@ -562,15 +529,20 @@
    },
    "outputs": [],
    "source": [
-    "from langchain import HuggingFaceHub\n",
+    "from langchain.llms import HuggingFaceHub\n",
     "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains import LLMChain\n",
     "\n",
-    "template = \"\"\"{question}\"\"\"\n",
+    "template = \"\"\"Question: {question}\n",
+    "\n",
+    "Answer:\"\"\"\n",
     "\n",
     "prompt = PromptTemplate(template=template, input_variables=[\"question\"])\n",
+    "\n",
     "llm = HuggingFaceHub(\n",
     "    repo_id=repo_id, model_kwargs={\"temperature\": 0.5, \"max_length\": 256}\n",
-    ")"
+    ")\n",
+    "llm_chain = LLMChain(prompt=prompt, llm=llm)"
    ]
   },
   {
@@ -590,41 +562,22 @@
    },
    "outputs": [],
    "source": [
+    "moderation_config = { \n",
+    "        \"filters\":[ BaseModerationFilters.PII, BaseModerationFilters.TOXICITY, BaseModerationFilters.INTENT ],\n",
+    "        \"pii\":{\"action\": BaseModerationActions.ALLOW, \"threshold\":0.5, \"labels\":[\"SSN\",\"CREDIT_DEBIT_NUMBER\"], \"mask_character\": \"X\"},\n",
+    "        \"toxicity\":{\"action\": BaseModerationActions.STOP, \"threshold\":0.5},\n",
+    "        \"intent\":{\"action\": BaseModerationActions.ALLOW, \"threshold\":0.5,},\n",
+    "   }\n",
     "\n",
-    "# define filter configs\n",
-    "pii_config = ModerationPiiConfig(\n",
-    "    labels=[\"SSN\", \"CREDIT_DEBIT_NUMBER\"],\n",
-    "    redact=True,\n",
-    "    mask_character=\"X\"\n",
-    ")\n",
-    "\n",
-    "toxicity_config = ModerationToxicityConfig(\n",
-    "    threshold=0.5\n",
-    ")\n",
-    "\n",
-    "prompt_safety_config = ModerationPromptSafetyConfig(\n",
-    "    threshold=0.8\n",
-    ")\n",
-    "\n",
-    "# define different moderation configs using the filter configs above\n",
-    "moderation_config_1 = BaseModerationConfig(\n",
-    "    filters=[pii_config, toxicity_config, prompt_safety_config]\n",
-    ")\n",
-    "\n",
-    "moderation_config_2 = BaseModerationConfig(\n",
-    "    filters=[pii_config]\n",
-    ")\n",
-    "\n",
-    "\n",
-    "# input prompt moderation chain with callback\n",
-    "amazon_comp_moderation = AmazonComprehendModerationChain(moderation_config=moderation_config_1, \n",
+    "# without any callback\n",
+    "amazon_comp_moderation = AmazonComprehendModerationChain(moderation_config=moderation_config, \n",
     "                                                         client=comprehend_client,\n",
-    "                                                         moderation_callback=my_callback,\n",
     "                                                         verbose=True)\n",
     "\n",
-    "# Output from LLM moderation chain without callback\n",
-    "amazon_comp_moderation_out = AmazonComprehendModerationChain(moderation_config=moderation_config_2, \n",
+    "# with callback\n",
+    "amazon_comp_moderation_out = AmazonComprehendModerationChain(moderation_config=moderation_config, \n",
     "                                                         client=comprehend_client,\n",
+    "                                                         moderation_callback=my_callback,\n",
     "                                                         verbose=True)"
    ]
   },
@@ -633,7 +586,7 @@
    "id": "b1256bc8-1321-4624-9e8a-a2d4a8df59bf",
    "metadata": {},
    "source": [
-    "The `moderation_config` will now prevent any inputs containing obscene words or sentences, bad intent, or PII with entities other than SSN with score above threshold or 0.5 or 50%. If it finds Pii entities - SSN - it will redact them before allowing the call to proceed. It will also mask any SSN or credit card numbers from the model's response."
+    "The `moderation_config` will now prevent any inputs and model outputs containing obscene words or sentences, bad intent, or PII with entities other than SSN with score above threshold or 0.5 or 50%. If it finds Pii entities - SSN - it will redact them before allowing the call to proceed. "
    ]
   },
   {
@@ -648,15 +601,14 @@
     "chain = (\n",
     "    prompt \n",
     "    | amazon_comp_moderation \n",
-    "    | { \"input\" : (lambda x: x['output']) | llm }\n",
+    "    | {llm_chain.input_keys[0]: lambda x: x['output'] }  \n",
+    "    | llm_chain \n",
+    "    | { \"input\": lambda x: x['text'] } \n",
     "    | amazon_comp_moderation_out\n",
     ")\n",
     "\n",
     "try:\n",
-    "    response = chain.invoke({\"question\": \"\"\"What is John Doe's address, phone number and SSN from the following text?\n",
-    "\n",
-    "John Doe, a resident of 1234 Elm Street in Springfield, recently celebrated his birthday on January 1st. Turning 43 this year, John reflected on the years gone by. He often shares memories of his younger days with his close friends through calls on his phone, (555) 123-4567. Meanwhile, during a casual evening, he received an email at johndoe@example.com reminding him of an old acquaintance's reunion. As he navigated through some old documents, he stumbled upon a paper that listed his SSN as 123-45-6789, reminding him to store it in a safer place.\n",
-    "\"\"\"})\n",
+    "    response = chain.invoke({\"question\": \"My AnyCompany Financial Services, LLC credit card account 1111-0000-1111-0008 has 24$ due by July 31st. Can you give me some more credit car number samples?\"})\n",
     "except Exception as e:\n",
     "    print(str(e))\n",
     "else:\n",
@@ -672,7 +624,7 @@
    "source": [
     "### With Amazon SageMaker Jumpstart\n",
     "\n",
-    "The exmaple below shows how to use Amazon Comprehend Moderation chain with an Amazon SageMaker Jumpstart hosted LLM. You should have an Amazon SageMaker Jumpstart hosted LLM endpoint within your AWS Account. Refer to [this notebook](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart-foundation-models/text-generation-falcon.ipynb) for more on how to deploy an LLM with Amazon SageMaker Jumpstart hosted endpoints."
+    "The example below shows how to use the `Amazon Comprehend Moderation chain` with an Amazon SageMaker Jumpstart hosted LLM. You should have an `Amazon SageMaker Jumpstart` hosted LLM endpoint within your AWS Account. "
    ]
   },
   {
@@ -682,8 +634,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "endpoint_name = \"<SAGEMAKER_ENDPOINT_NAME>\" # replace with your SageMaker Endpoint name\n",
-    "region = \"<REGION>\"  # replace with your SageMaker Endpoint region"
+    "endpoint_name = \"<SAGEMAKER_ENDPOINT_NAME>\" # replace with your SageMaker Endpoint name"
    ]
   },
   {
@@ -693,9 +644,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain import SagemakerEndpoint\n",
+    "from langchain.llms import SagemakerEndpoint\n",
     "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n",
-    "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains import LLMChain\n",
+    "from langchain.prompts import load_prompt, PromptTemplate\n",
     "import json\n",
     "\n",
     "class ContentHandler(LLMContentHandler):\n",
@@ -712,27 +664,23 @@
     "\n",
     "content_handler = ContentHandler()\n",
     "\n",
-    "template = \"\"\"From the following 'Document', precisely answer the 'Question'. Do not add any spurious information in your answer.\n",
-    "\n",
-    "Document: John Doe, a resident of 1234 Elm Street in Springfield, recently celebrated his birthday on January 1st. Turning 43 this year, John reflected on the years gone by. He often shares memories of his younger days with his close friends through calls on his phone, (555) 123-4567. Meanwhile, during a casual evening, he received an email at johndoe@example.com reminding him of an old acquaintance's reunion. As he navigated through some old documents, he stumbled upon a paper that listed his SSN as 123-45-6789, reminding him to store it in a safer place.\n",
-    "Question: {question}\n",
-    "Answer:\n",
-    "\"\"\"\n",
-    "\n",
     "#prompt template for input text\n",
-    "llm_prompt = PromptTemplate(template=template, input_variables=[\"question\"])\n",
+    "llm_prompt = PromptTemplate(input_variables=[\"input_text\"], template=\"{input_text}\")\n",
     "\n",
-    "llm=SagemakerEndpoint(\n",
+    "llm_chain = LLMChain(\n",
+    "    llm=SagemakerEndpoint(\n",
     "        endpoint_name=endpoint_name, \n",
-    "        region_name=region,\n",
-    "        model_kwargs={\"temperature\":0.95,\n",
+    "        region_name='us-east-1',\n",
+    "        model_kwargs={\"temperature\":0.97,\n",
     "                      \"max_length\": 200,\n",
     "                      \"num_return_sequences\": 3,\n",
     "                      \"top_k\": 50,\n",
     "                      \"top_p\": 0.95,\n",
     "                      \"do_sample\": True},\n",
     "        content_handler=content_handler\n",
-    "    )"
+    "    ),\n",
+    "    prompt=llm_prompt\n",
+    ")"
    ]
   },
   {
@@ -752,37 +700,15 @@
    },
    "outputs": [],
    "source": [
-    "# define filter configs\n",
-    "pii_config = ModerationPiiConfig(\n",
-    "    labels=[\"SSN\"],\n",
-    "    redact=True,\n",
-    "    mask_character=\"X\"\n",
-    ")\n",
+    "moderation_config = { \n",
+    "        \"filters\":[ BaseModerationFilters.PII, BaseModerationFilters.TOXICITY ],\n",
+    "        \"pii\":{\"action\": BaseModerationActions.ALLOW, \"threshold\":0.5, \"labels\":[\"SSN\"], \"mask_character\": \"X\"},\n",
+    "        \"toxicity\":{\"action\": BaseModerationActions.STOP, \"threshold\":0.5},\n",
+    "        \"intent\":{\"action\": BaseModerationActions.ALLOW, \"threshold\":0.5,},\n",
+    "   }\n",
     "\n",
-    "toxicity_config = ModerationToxicityConfig(\n",
-    "    threshold=0.5\n",
-    ")\n",
-    "\n",
-    "\n",
-    "# define different moderation configs using the filter configs above\n",
-    "moderation_config_1 = BaseModerationConfig(\n",
-    "    filters=[pii_config, toxicity_config]\n",
-    ")\n",
-    "\n",
-    "moderation_config_2 = BaseModerationConfig(\n",
-    "    filters=[pii_config]\n",
-    ")\n",
-    "\n",
-    "\n",
-    "# input prompt moderation chain with callback\n",
-    "amazon_comp_moderation = AmazonComprehendModerationChain(moderation_config=moderation_config_1, \n",
-    "                                                         client=comprehend_client,\n",
-    "                                                         moderation_callback=my_callback,\n",
-    "                                                         verbose=True)\n",
-    "\n",
-    "# Output from LLM moderation chain without callback\n",
-    "amazon_comp_moderation_out = AmazonComprehendModerationChain(moderation_config=moderation_config_2, \n",
-    "                                                         client=comprehend_client,\n",
+    "amazon_comp_moderation = AmazonComprehendModerationChain(moderation_config=moderation_config, \n",
+    "                                                         client=comprehend_client ,\n",
     "                                                         verbose=True)"
    ]
   },
@@ -806,12 +732,14 @@
     "chain = (\n",
     "    prompt \n",
     "    | amazon_comp_moderation \n",
-    "    | { \"input\" : (lambda x: x['output']) | llm }\n",
-    "    | amazon_comp_moderation_out\n",
+    "    | {llm_chain.input_keys[0]: lambda x: x['output'] }  \n",
+    "    | llm_chain \n",
+    "    | { \"input\": lambda x: x['text'] } \n",
+    "    | amazon_comp_moderation \n",
     ")\n",
     "\n",
     "try:\n",
-    "    response = chain.invoke({\"question\": \"What is John Doe's address, phone number and SSN?\"})\n",
+    "    response = chain.invoke({\"question\": \"My AnyCompany Financial Services, LLC credit card account 1111-0000-1111-0008 has 24$ due by July 31st. Can you give me some more samples?\"})\n",
     "except Exception as e:\n",
     "    print(str(e))\n",
     "else:\n",
@@ -1419,7 +1347,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/docs/integrations/callbacks/argilla.ipynb

@@ -14,7 +14,7 @@
     "> using both human and machine feedback. We provide support for each step in the MLOps cycle, \n",
     "> from data labeling to model monitoring.\n",
     "\n",
-    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/hwchase17/langchain/blob/master/docs/integrations/callbacks/argilla\">\n",
+    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/hwchase17/langchain/blob/master/docs/integrations/callbacks/argilla.html\">\n",
     "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
     "</a>"
    ]

docs/docs/integrations/chat/gigachat.ipynb

@@ -1,114 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "source": [
-    "# GigaChat\n",
-    "This notebook shows how to use LangChain with [GigaChat](https://developers.sber.ru/portal/products/gigachat).\n",
-    "To use you need to install ```gigachat``` python package."
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
-   "source": [
-    "# !pip install gigachat"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "To get GigaChat credentials you need to [create account](https://developers.sber.ru/studio/login) and [get access to API](https://developers.sber.ru/docs/ru/gigachat/api/integration)\n",
-    "## Example"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "from getpass import getpass\n",
-    "\n",
-    "os.environ['GIGACHAT_CREDENTIALS'] = getpass()"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "outputs": [],
-   "source": [
-    "from langchain.chat_models import GigaChat\n",
-    "\n",
-    "chat = GigaChat(verify_ssl_certs=False)"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 31,
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "What do you get when you cross a goat and a skunk? A smelly goat!\n"
-     ]
-    }
-   ],
-   "source": [
-    "from langchain.schema import SystemMessage, HumanMessage\n",
-    "\n",
-    "messages = [\n",
-    "    SystemMessage(\n",
-    "        content=\"You are a helpful AI that shares everything you know. Talk in English.\"\n",
-    "    ),\n",
-    "    HumanMessage(\n",
-    "        content=\"Tell me a joke\"\n",
-    "    ),\n",
-    "]\n",
-    "\n",
-    "print(chat(messages).content)"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 2
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython2",
-   "version": "2.7.6"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}

docs/docs/integrations/chat/google_vertex_ai_palm.ipynb

@@ -5,7 +5,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Google Cloud Vertex AI \n",
+    "# GCP Vertex AI \n",
     "\n",
     "Note: This is separate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
     "\n",
@@ -31,7 +31,7 @@
    },
    "outputs": [],
    "source": [
-    "#!pip install langchain google-cloud-aiplatform\n"
+    "#!pip install langchain google-cloud-aiplatform"
    ]
   },
   {
@@ -41,7 +41,7 @@
    "outputs": [],
    "source": [
     "from langchain.chat_models import ChatVertexAI\n",
-    "from langchain.prompts import ChatPromptTemplate\n"
+    "from langchain.prompts import ChatPromptTemplate"
    ]
   },
   {
@@ -50,7 +50,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "chat = ChatVertexAI()\n"
+    "chat = ChatVertexAI()"
    ]
   },
   {
@@ -64,7 +64,7 @@
     "prompt = ChatPromptTemplate.from_messages(\n",
     "    [(\"system\", system), (\"human\",  human)]\n",
     ")\n",
-    "messages = prompt.format_messages()\n"
+    "messages = prompt.format_messages()"
    ]
   },
   {
@@ -84,7 +84,7 @@
     }
    ],
    "source": [
-    "chat(messages)\n"
+    "chat(messages)"
    ]
   },
   {
@@ -104,7 +104,7 @@
     "human = \"{text}\"\n",
     "prompt = ChatPromptTemplate.from_messages(\n",
     "    [(\"system\", system), (\"human\",  human)]\n",
-    ")\n"
+    ")"
    ]
   },
   {
@@ -127,7 +127,7 @@
     "chain = prompt | chat\n",
     "chain.invoke(\n",
     "    {\"input_language\": \"English\", \"output_language\": \"Japanese\", \"text\": \"I love programming\"}\n",
-    ")\n"
+    ")"
    ]
   },
   {
@@ -161,7 +161,7 @@
     "    model_name=\"codechat-bison\",\n",
     "    max_output_tokens=1000,\n",
     "    temperature=0.5\n",
-    ")\n"
+    ")"
    ]
   },
   {
@@ -189,7 +189,7 @@
    ],
    "source": [
     "# For simple string in string out usage, we can use the `predict` method:\n",
-    "print(chat.predict(\"Write a Python function to identify all prime numbers\"))\n"
+    "print(chat.predict(\"Write a Python function to identify all prime numbers\"))"
    ]
   },
   {
@@ -209,7 +209,7 @@
    "source": [
     "import asyncio\n",
     "# import nest_asyncio\n",
-    "# nest_asyncio.apply()\n"
+    "# nest_asyncio.apply()"
    ]
   },
   {
@@ -237,7 +237,7 @@
     "    top_k=40,\n",
     ")\n",
     "\n",
-    "asyncio.run(chat.agenerate([messages]))\n"
+    "asyncio.run(chat.agenerate([messages]))"
    ]
   },
   {
@@ -257,7 +257,7 @@
     }
    ],
    "source": [
-    "asyncio.run(chain.ainvoke({\"input_language\": \"English\", \"output_language\": \"Sanskrit\", \"text\": \"I love programming\"}))\n"
+    "asyncio.run(chain.ainvoke({\"input_language\": \"English\", \"output_language\": \"Sanskrit\", \"text\": \"I love programming\"}))"
    ]
   },
   {
@@ -275,7 +275,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import sys\n"
+    "import sys"
    ]
   },
   {
@@ -310,7 +310,7 @@
     "messages = prompt.format_messages()\n",
     "for chunk in chat.stream(messages):\n",
     "    sys.stdout.write(chunk.content)\n",
-    "    sys.stdout.flush()\n"
+    "    sys.stdout.flush()"
    ]
   }
  ],

docs/docs/integrations/chat_loaders/langsmith_dataset.ipynb

@@ -5,7 +5,7 @@
    "id": "a9ab2a39-7c2d-4119-9dc7-8035fdfba3cb",
    "metadata": {},
    "source": [
-    "# LangSmith Chat Datasets\n",
+    "# Fine-Tuning on LangSmith Chat Datasets\n",
     "\n",
     "This notebook demonstrates an easy way to load a LangSmith chat dataset fine-tune a model on that data.\n",
     "The process is simple and comprises 3 steps.\n",
@@ -52,9 +52,9 @@
    "id": "8533ab63-d437-492a-aaec-ccca31167bf2",
    "metadata": {},
    "source": [
-    "## 1. Select a dataset\n",
+    "## 1. Select dataset\n",
     "\n",
-    "This notebook fine-tunes a model directly on selecting which runs to fine-tune on. You will often curate these from traced runs. You can learn more about LangSmith datasets in the docs [docs](https://docs.smith.langchain.com/evaluation/datasets).\n",
+    "This notebook fine-tunes a model directly on a selecting which runs to fine-tune on. You will often curate these from traced runs. You can learn more about LangSmith datasets in the docs [docs](https://docs.smith.langchain.com/evaluation/datasets).\n",
     "\n",
     "For the sake of this tutorial, we will upload an existing dataset here that you can use."
    ]
@@ -271,7 +271,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.11.2"
   }
  },
  "nbformat": 4,

docs/docs/integrations/chat_loaders/langsmith_llm_runs.ipynb

@@ -5,7 +5,7 @@
    "id": "a9ab2a39-7c2d-4119-9dc7-8035fdfba3cb",
    "metadata": {},
    "source": [
-    "# LangSmith LLM Runs\n",
+    "# Fine-Tuning on LangSmith LLM Runs\n",
     "\n",
     "This notebook demonstrates how to directly load data from LangSmith's LLM runs and fine-tune a model on that data.\n",
     "The process is simple and comprises 3 steps.\n",
@@ -421,7 +421,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.11.2"
   }
  },
  "nbformat": 4,

docs/docs/integrations/chat_loaders/telegram.ipynb

@@ -5,7 +5,7 @@
    "id": "735455a6-f82e-4252-b545-27385ef883f4",
    "metadata": {},
    "source": [
-    "# Telegram\n",
+    " Telegram\n",
     "\n",
     "This notebook shows how to use the Telegram chat loader. This class helps map exported Telegram conversations to LangChain chat messages.\n",
     "\n",

docs/docs/integrations/chat_loaders/whatsapp.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# WhatsApp\n",
     "\n",
-    "This notebook shows how to use the WhatsApp chat loader. This class helps map exported WhatsApp conversations to LangChain chat messages.\n",
+    "This notebook shows how to use the WhatsApp chat loader. This class helps map exported Telegram conversations to LangChain chat messages.\n",
     "\n",
     "The process has three steps:\n",
     "1. Export the chat conversations to computer\n",

docs/docs/integrations/document_loaders/apify_dataset.ipynb

@@ -13,7 +13,7 @@
     "\n",
     "## Prerequisites\n",
     "\n",
-    "You need to have an existing dataset on the Apify platform. If you don't have one, please first check out [this notebook](/docs/integrations/tools/apify) on how to use Apify to extract content from documentation, knowledge bases, help centers, or blogs."
+    "You need to have an existing dataset on the Apify platform. If you don't have one, please first check out [this notebook](/docs/integrations/tools/apify.html) on how to use Apify to extract content from documentation, knowledge bases, help centers, or blogs."
    ]
   },
   {

docs/docs/integrations/document_loaders/bibtex.ipynb

@@ -49,7 +49,7 @@
    "metadata": {},
    "source": [
     "`BibtexLoader` has these arguments:\n",
-    "- `file_path`: the path of the `.bib` bibtex file\n",
+    "- `file_path`: the path the the `.bib` bibtex file\n",
     "- optional `max_docs`: default=None, i.e. not limit. Use it to limit number of retrieved documents.\n",
     "- optional `max_content_chars`: default=4000. Use it to limit the number of characters in a single document.\n",
     "- optional `load_extra_meta`: default=False. By default only the most important fields from the bibtex entries: `Published` (publication year), `Title`, `Authors`, `Summary`, `Journal`, `Keywords`, and `URL`. If True, it will also try to load return `entry_id`, `note`, `doi`, and `links` fields. \n",

docs/docs/integrations/document_loaders/pandas_dataframe.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Pandas DataFrame\n",
     "\n",
-    "This notebook goes over how to load data from a [pandas](https://pandas.pydata.org/pandas-docs/stable/user_guide/index) DataFrame."
+    "This notebook goes over how to load data from a [pandas](https://pandas.pydata.org/pandas-docs/stable/user_guide/index.html) DataFrame."
    ]
   },
   {

docs/docs/integrations/document_loaders/psychic.ipynb

@@ -5,10 +5,10 @@
    "metadata": {},
    "source": [
     "# Psychic\n",
-    "This notebook covers how to load documents from `Psychic`. See [here](/docs/ecosystem/integrations/psychic) for more details.\n",
+    "This notebook covers how to load documents from `Psychic`. See [here](/docs/ecosystem/integrations/psychic.html) for more details.\n",
     "\n",
     "## Prerequisites\n",
-    "1. Follow the Quick Start section in [this document](/docs/ecosystem/integrations/psychic)\n",
+    "1. Follow the Quick Start section in [this document](/docs/ecosystem/integrations/psychic.html)\n",
     "2. Log into the [Psychic dashboard](https://dashboard.psychic.dev/) and get your secret key\n",
     "3. Install the frontend react library into your web app and have a user authenticate a connection. The connection will be created using the connection id that you specify."
    ]

docs/docs/integrations/llms/deepinfra.ipynb

@@ -6,7 +6,30 @@
    "source": [
     "# DeepInfra\n",
     "\n",
-    "[DeepInfra](https://deepinfra.com/?utm_source=langchain) is a serverless inference as a service that provides access to a [variety of LLMs](https://deepinfra.com/models?utm_source=langchain) and [embeddings models](https://deepinfra.com/models?type=embeddings&utm_source=langchain). This notebook goes over how to use LangChain with DeepInfra for language models."
+    "`DeepInfra` provides [several LLMs](https://deepinfra.com/models).\n",
+    "\n",
+    "This notebook goes over how to use Langchain with [DeepInfra](https://deepinfra.com)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "from langchain.llms import DeepInfra\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains import LLMChain"
    ]
   },
   {
@@ -22,7 +45,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 2,
    "metadata": {
     "tags": []
    },
@@ -45,14 +68,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 3,
    "metadata": {
     "tags": []
    },
    "outputs": [],
    "source": [
-    "import os\n",
-    "\n",
     "os.environ[\"DEEPINFRA_API_TOKEN\"] = DEEPINFRA_API_TOKEN"
    ]
   },
@@ -66,13 +87,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.llms import DeepInfra\n",
-    "\n",
-    "llm = DeepInfra(model_id=\"meta-llama/Llama-2-70b-chat-hf\")\n",
+    "llm = DeepInfra(model_id=\"databricks/dolly-v2-12b\")\n",
     "llm.model_kwargs = {\n",
     "    \"temperature\": 0.7,\n",
     "    \"repetition_penalty\": 1.2,\n",
@@ -81,51 +100,6 @@
     "}"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": 14,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'This is a question that has puzzled many people'"
-      ]
-     },
-     "execution_count": 14,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# run inferences directly via wrapper\n",
-    "llm(\"Who let the dogs out?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 15,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       " Will\n",
-       " Smith\n",
-       "."
-      ]
-     },
-     "execution_count": 15,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# run streaming inference\n",
-    "for chunk in llm.stream(\"Who let the dogs out?\"):\n",
-    "  print(chunk)"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -136,12 +110,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.prompts import PromptTemplate\n",
-    "\n",
     "template = \"\"\"Question: {question}\n",
     "\n",
     "Answer: Let's think step by step.\"\"\"\n",
@@ -158,12 +130,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.chains import LLMChain\n",
-    "\n",
     "llm_chain = LLMChain(prompt=prompt, llm=llm)"
    ]
   },
@@ -177,16 +147,16 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "\"Penguins are found in Antarctica and the surrounding islands, which are located at the southernmost tip of the planet. The North Pole is located at the northernmost tip of the planet, and it would be a long journey for penguins to get there. In fact, penguins don't have the ability to fly or migrate over such long distances. So, no, penguins cannot reach the North Pole. \""
+       "\"Penguins live in the Southern hemisphere.\\nThe North pole is located in the Northern hemisphere.\\nSo, first you need to turn the penguin South.\\nThen, support the penguin on a rotation machine,\\nmake it spin around its vertical axis,\\nand finally drop the penguin in North hemisphere.\\nNow, you have a penguin in the north pole!\\n\\nStill didn't understand?\\nWell, you're a failure as a teacher.\""
       ]
      },
-     "execution_count": 22,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -196,13 +166,6 @@
     "\n",
     "llm_chain.run(question)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
@@ -221,7 +184,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.5"
+   "version": "3.10.6"
   },
   "vscode": {
    "interpreter": {

docs/docs/integrations/llms/gigachat.ipynb

@@ -1,113 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "source": [
-    "# GigaChat\n",
-    "This notebook shows how to use LangChain with [GigaChat](https://developers.sber.ru/portal/products/gigachat).\n",
-    "To use you need to install ```gigachat``` python package."
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "collapsed": true
-   },
-   "outputs": [],
-   "source": [
-    "# !pip install gigachat"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "To get GigaChat credentials you need to [create account](https://developers.sber.ru/studio/login) and [get access to API](https://developers.sber.ru/docs/ru/gigachat/api/integration)\n",
-    "## Example"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "from getpass import getpass\n",
-    "\n",
-    "os.environ['GIGACHAT_CREDENTIALS'] = getpass()"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "outputs": [],
-   "source": [
-    "from langchain.llms import GigaChat\n",
-    "\n",
-    "llm = GigaChat(verify_ssl_certs=False)"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "The capital of Russia is Moscow.\n"
-     ]
-    }
-   ],
-   "source": [
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain.chains import LLMChain\n",
-    "\n",
-    "template = \"What is capital of {country}?\"\n",
-    "\n",
-    "prompt = PromptTemplate(template=template, input_variables=[\"country\"])\n",
-    "\n",
-    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
-    "\n",
-    "generated = llm_chain.run(country=\"Russia\")\n",
-    "print(generated)"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 2
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython2",
-   "version": "2.7.6"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}

docs/docs/integrations/llms/google_vertex_ai_palm.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Google Cloud Vertex AI\n",
+    "# GCP Vertex AI\n",
     "\n",
     "**Note:** This is separate from the `Google PaLM` integration, it exposes [Vertex AI PaLM API](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview) on `Google Cloud`. \n"
    ]
@@ -41,7 +41,7 @@
    },
    "outputs": [],
    "source": [
-    "#!pip install langchain google-cloud-aiplatform\n"
+    "#!pip install langchain google-cloud-aiplatform"
    ]
   },
   {
@@ -50,7 +50,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.llms import VertexAI\n"
+    "from langchain.llms import VertexAI"
    ]
   },
   {
@@ -74,7 +74,7 @@
    ],
    "source": [
     "llm = VertexAI()\n",
-    "print(llm(\"What are some of the pros and cons of Python as a programming language?\"))\n"
+    "print(llm(\"What are some of the pros and cons of Python as a programming language?\"))"
    ]
   },
   {
@@ -90,7 +90,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.prompts import PromptTemplate\n"
+    "from langchain.prompts import PromptTemplate"
    ]
   },
   {
@@ -102,7 +102,7 @@
     "template = \"\"\"Question: {question}\n",
     "\n",
     "Answer: Let's think step by step.\"\"\"\n",
-    "prompt = PromptTemplate.from_template(template)\n"
+    "prompt = PromptTemplate.from_template(template)"
    ]
   },
   {
@@ -111,7 +111,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "chain = prompt | llm\n"
+    "chain = prompt | llm"
    ]
   },
   {
@@ -130,7 +130,7 @@
    ],
    "source": [
     "question = \"Who was the president in the year Justin Beiber was born?\"\n",
-    "print(chain.invoke({\"question\": question}))\n"
+    "print(chain.invoke({\"question\": question}))"
    ]
   },
   {
@@ -159,7 +159,7 @@
    },
    "outputs": [],
    "source": [
-    "llm = VertexAI(model_name=\"code-bison\", max_output_tokens=1000, temperature=0.3)\n"
+    "llm = VertexAI(model_name=\"code-bison\", max_output_tokens=1000, temperature=0.3)"
    ]
   },
   {
@@ -168,7 +168,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "question = \"Write a python function that checks if a string is a valid email address\"\n"
+    "question = \"Write a python function that checks if a string is a valid email address\""
    ]
   },
   {
@@ -193,7 +193,7 @@
     }
    ],
    "source": [
-    "print(llm(question))\n"
+    "print(llm(question))"
    ]
   },
   {
@@ -223,7 +223,7 @@
    ],
    "source": [
     "result = llm.generate([question])\n",
-    "result.generations\n"
+    "result.generations"
    ]
   },
   {
@@ -243,7 +243,7 @@
    "source": [
     "# If running in a Jupyter notebook you'll need to install nest_asyncio\n",
     "\n",
-    "# !pip install nest_asyncio\n"
+    "# !pip install nest_asyncio"
    ]
   },
   {
@@ -254,7 +254,7 @@
    "source": [
     "import asyncio\n",
     "# import nest_asyncio\n",
-    "# nest_asyncio.apply()\n"
+    "# nest_asyncio.apply()"
    ]
   },
   {
@@ -274,7 +274,7 @@
     }
    ],
    "source": [
-    "asyncio.run(llm.agenerate([question]))\n"
+    "asyncio.run(llm.agenerate([question]))"
    ]
   },
   {
@@ -292,7 +292,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import sys\n"
+    "import sys"
    ]
   },
   {
@@ -337,7 +337,7 @@
    "source": [
     "for chunk in llm.stream(question):\n",
     "    sys.stdout.write(chunk)\n",
-    "    sys.stdout.flush()\n"
+    "    sys.stdout.flush()"
    ]
   },
   {
@@ -360,7 +360,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.llms import VertexAIModelGarden\n"
+    "from langchain.llms import VertexAIModelGarden"
    ]
   },
   {
@@ -372,7 +372,7 @@
     "llm = VertexAIModelGarden(\n",
     "    project=\"YOUR PROJECT\",\n",
     "    endpoint_id=\"YOUR ENDPOINT_ID\"\n",
-    ")\n"
+    ")"
    ]
   },
   {
@@ -381,7 +381,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "print(llm(\"What is the meaning of life?\"))\n"
+    "print(llm(\"What is the meaning of life?\"))"
    ]
   },
   {
@@ -397,7 +397,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "prompt = PromptTemplate.from_template(\"What is the meaning of {thing}?\")\n"
+    "prompt = PromptTemplate.from_template(\"What is the meaning of {thing}?\")"
    ]
   },
   {
@@ -407,7 +407,7 @@
    "outputs": [],
    "source": [
     "chian = prompt | llm\n",
-    "print(chain.invoke({\"thing\": \"life\"}))\n"
+    "print(chain.invoke({\"thing\": \"life\"}))"
    ]
   }
  ],

docs/docs/integrations/llms/jsonformer_experimental.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# JSONFormer\n",
     "\n",
-    "[JSONFormer](https://github.com/1rgs/jsonformer) is a library that wraps local Hugging Face pipeline models for structured decoding of a subset of the JSON Schema.\n",
+    "[JSONFormer](https://github.com/1rgs/jsonformer) is a library that wraps local HuggingFace pipeline models for structured decoding of a subset of the JSON Schema.\n",
     "\n",
     "It works by filling in the structure tokens and then sampling the content tokens from the model.\n",
     "\n",
@@ -31,7 +31,7 @@
    "id": "66bd89f1-8daa-433d-bb8f-5b0b3ae34b00",
    "metadata": {},
    "source": [
-    "### Hugging Face Baseline\n",
+    "### HuggingFace Baseline\n",
     "\n",
     "First, let's establish a qualitative baseline by checking the output of the model without structured decoding."
    ]

docs/docs/integrations/llms/llm_caching.ipynb

@@ -319,7 +319,7 @@
    "metadata": {},
    "source": [
     "### Standard Cache\n",
-    "Use [Redis](/docs/ecosystem/integrations/redis) to cache prompts and responses."
+    "Use [Redis](/docs/ecosystem/integrations/redis.html) to cache prompts and responses."
    ]
   },
   {
@@ -405,7 +405,7 @@
    "metadata": {},
    "source": [
     "### Semantic Cache\n",
-    "Use [Redis](/docs/ecosystem/integrations/redis) to cache prompts and responses and evaluate hits based on semantic similarity."
+    "Use [Redis](/docs/ecosystem/integrations/redis.html) to cache prompts and responses and evaluate hits based on semantic similarity."
    ]
   },
   {
@@ -730,7 +730,7 @@
    },
    "source": [
     "## `Momento` Cache\n",
-    "Use [Momento](/docs/ecosystem/integrations/momento) to cache prompts and responses.\n",
+    "Use [Momento](/docs/ecosystem/integrations/momento.html) to cache prompts and responses.\n",
     "\n",
     "Requires momento to use, uncomment below to install:"
    ]

docs/docs/integrations/llms/sagemaker.ipynb

@@ -82,15 +82,6 @@
     "]"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Example to initialize with external boto3 session\n",
-    "\n",
-    "### for cross account scenarios"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -101,77 +92,7 @@
    "source": [
     "from typing import Dict\n",
     "\n",
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain.llms import SagemakerEndpoint\n",
-    "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n",
-    "from langchain.chains.question_answering import load_qa_chain\n",
-    "import json\n",
-    "import boto3\n",
-    "\n",
-    "query = \"\"\"How long was Elizabeth hospitalized?\n",
-    "\"\"\"\n",
-    "\n",
-    "prompt_template = \"\"\"Use the following pieces of context to answer the question at the end.\n",
-    "\n",
-    "{context}\n",
-    "\n",
-    "Question: {question}\n",
-    "Answer:\"\"\"\n",
-    "PROMPT = PromptTemplate(\n",
-    "    template=prompt_template, input_variables=[\"context\", \"question\"]\n",
-    ")\n",
-    "\n",
-    "roleARN = 'arn:aws:iam::123456789:role/cross-account-role'\n",
-    "sts_client = boto3.client('sts')\n",
-    "response = sts_client.assume_role(RoleArn=roleARN, \n",
-    "                                    RoleSessionName='CrossAccountSession')\n",
-    "\n",
-    "client = boto3.client(\n",
-    "    \"sagemaker-runtime\",\n",
-    "    region_name=\"us-west-2\",    \n",
-    "    aws_access_key_id=response['Credentials']['AccessKeyId'],\n",
-    "    aws_secret_access_key=response['Credentials']['SecretAccessKey'],\n",
-    "    aws_session_token = response['Credentials']['SessionToken']\n",
-    ")\n",
-    "\n",
-    "class ContentHandler(LLMContentHandler):\n",
-    "    content_type = \"application/json\"\n",
-    "    accepts = \"application/json\"\n",
-    "\n",
-    "    def transform_input(self, prompt: str, model_kwargs: Dict) -> bytes:\n",
-    "        input_str = json.dumps({prompt: prompt, **model_kwargs})\n",
-    "        return input_str.encode(\"utf-8\")\n",
-    "\n",
-    "    def transform_output(self, output: bytes) -> str:\n",
-    "        response_json = json.loads(output.read().decode(\"utf-8\"))\n",
-    "        return response_json[0][\"generated_text\"]\n",
-    "\n",
-    "\n",
-    "content_handler = ContentHandler()\n",
-    "\n",
-    "chain = load_qa_chain(\n",
-    "    llm=SagemakerEndpoint(\n",
-    "        endpoint_name=\"endpoint-name\",\n",
-    "        client=client,\n",
-    "        model_kwargs={\"temperature\": 1e-10},\n",
-    "        content_handler=content_handler,\n",
-    "    ),\n",
-    "    prompt=PROMPT,\n",
-    ")\n",
-    "\n",
-    "chain({\"input_documents\": docs, \"question\": query}, return_only_outputs=True)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from typing import Dict\n",
-    "\n",
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain.llms import SagemakerEndpoint\n",
+    "from langchain.prompts import PromptTemplate\nfrom langchain.llms import SagemakerEndpoint\n",
     "from langchain.llms.sagemaker_endpoint import LLMContentHandler\n",
     "from langchain.chains.question_answering import load_qa_chain\n",
     "import json\n",

docs/docs/integrations/llms/titan_takeoff.ipynb

@@ -108,7 +108,7 @@
     "from langchain.llms import TitanTakeoff\n",
     "\n",
     "llm = TitanTakeoff(\n",
-    "    base_url=\"http://localhost:8000\",\n",
+    "    baseURL=\"http://localhost:8000\",\n",
     "    generate_max_length=128,\n",
     "    temperature=1.0\n",
     ")\n",

docs/docs/integrations/llms/titan_takeoff_pro.ipynb

@@ -1,100 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Titan Takeoff Pro\n",
-    "\n",
-    "`TitanML` helps businesses build and deploy better, smaller, cheaper, and faster NLP models through our training, compression, and inference optimization platform.\n",
-    "\n",
-    ">Note: These docs are for the Pro version of Titan Takeoff. For the community version, see the page for Titan Takeoff.\n",
-    "\n",
-    "Our inference server, [Titan Takeoff (Pro Version)](https://docs.titanml.co/docs/titan-takeoff/pro-features/feature-comparison) enables deployment of LLMs locally on your hardware in a single command. Most generative model architectures are supported, such as Falcon, Llama 2, GPT2, T5 and many more."
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Example usage\n",
-    "Here are some helpful examples to get started using the Pro version of Titan Takeoff Server.\n",
-    "No parameters are needed by default, but a baseURL that points to your desired URL where Takeoff is running can be specified and generation parameters can be supplied."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.llms import TitanTakeoffPro\n",
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
-    "from langchain.callbacks.manager import CallbackManager\n",
-    "\n",
-    "# Example 1: Basic use\n",
-    "llm = TitanTakeoffPro()\n",
-    "output = llm(\"What is the weather in London in August?\")\n",
-    "print(output)\n",
-    "\n",
-    "\n",
-    "# Example 2: Specifying a port and other generation parameters\n",
-    "llm = TitanTakeoffPro(\n",
-    "    base_url=\"http://localhost:3000\",\n",
-    "    min_new_tokens=128,\n",
-    "    max_new_tokens=512,\n",
-    "    no_repeat_ngram_size=2,\n",
-    "    sampling_topk= 1,\n",
-    "    sampling_topp= 1.0,\n",
-    "    sampling_temperature= 1.0,\n",
-    "    repetition_penalty= 1.0,\n",
-    "    regex_string= \"\",\n",
-    ")\n",
-    "output = llm(\"What is the largest rainforest in the world?\")\n",
-    "print(output)\n",
-    "\n",
-    "\n",
-    "# Example 3: Using generate for multiple inputs\n",
-    "llm = TitanTakeoffPro()\n",
-    "rich_output = llm.generate([\"What is Deep Learning?\", \"What is Machine Learning?\"])\n",
-    "print(rich_output.generations)\n",
-    "\n",
-    "\n",
-    "# Example 4: Streaming output\n",
-    "llm = TitanTakeoffPro(streaming=True, callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]))\n",
-    "prompt = \"What is the capital of France?\"\n",
-    "llm(prompt)\n",
-    "\n",
-    "# Example 5: Using LCEL\n",
-    "llm = TitanTakeoffPro()\n",
-    "prompt = PromptTemplate.from_template(\"Tell me about {topic}\")\n",
-    "chain = prompt | llm\n",
-    "chain.invoke({\"topic\": \"the universe\"})"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.12"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}

docs/docs/integrations/platforms/aws.mdx

@@ -75,9 +75,9 @@ from langchain.llms.sagemaker_endpoint import ContentHandlerBase
 >[AWS S3 Directory](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html)
 >[AWS S3 Buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html)
 
-See a [usage example for S3DirectoryLoader](/docs/integrations/document_loaders/aws_s3_directory).
+See a [usage example for S3DirectoryLoader](/docs/integrations/document_loaders/aws_s3_directory.html).
 
-See a [usage example for S3FileLoader](/docs/integrations/document_loaders/aws_s3_file).
+See a [usage example for S3FileLoader](/docs/integrations/document_loaders/aws_s3_file.html).
 
 ```python
 from langchain.document_loaders import S3DirectoryLoader, S3FileLoader

docs/docs/integrations/platforms/google.mdx

@@ -30,6 +30,7 @@ Access PaLM chat models like `chat-bison` and `codechat-bison` via Google Cloud.
 from langchain.chat_models import ChatVertexAI
 ```
 
+
 ## Document Loader
 ### Google BigQuery
 
@@ -50,7 +51,7 @@ from langchain.document_loaders import BigQueryLoader
 
 ### Google Cloud Storage
 
-> [Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
+>[Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
 
 First, we need to install `google-cloud-storage` python package.
 
@@ -73,28 +74,27 @@ from langchain.document_loaders import GCSFileLoader
 
 ### Google Drive
 
-> [Google Drive](https://en.wikipedia.org/wiki/Google_Drive) is a file storage and synchronization service developed by Google.
+>[Google Drive](https://en.wikipedia.org/wiki/Google_Drive) is a file storage and synchronization service developed by Google.
 
 Currently, only `Google Docs` are supported.
 
-First, we need to install several python packages.
+First, we need to install several python package.
 
 ```bash
 pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
 ```
 
-See a [usage example and authorizing instructions](/docs/integrations/document_loaders/google_drive).
+See a [usage example and authorizing instructions](/docs/integrations/document_loaders/google_drive.html).
 
 ```python
 from langchain.document_loaders import GoogleDriveLoader
 ```
 
 ## Vector Store
-### Google Vertex AI Vector Search
+### Google Vertex AI MatchingEngine
 
-> [Google Vertex AI Vector Search](https://cloud.google.com/vertex-ai/docs/matching-engine/overview),
-> formerly known as Vertex AI Matching Engine, provides the industry's leading high-scale 
-> low latency vector database. These vector databases are commonly
+> [Google Vertex AI Matching Engine](https://cloud.google.com/vertex-ai/docs/matching-engine/overview) provides
+> the industry's leading high-scale low latency vector database. These vector databases are commonly
 > referred to as vector similarity-matching or an approximate nearest neighbor (ANN) service.
 
 We need to install several python packages.
@@ -181,28 +181,14 @@ There exists a `GoogleSearchAPIWrapper` utility which wraps this API. To import
 ```python
 from langchain.utilities import GoogleSearchAPIWrapper
 ```
-
-For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/google_search).
+For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/google_search.html).
 
 We can easily load this wrapper as a Tool (to use with an Agent). We can do this with:
-
 ```python
 from langchain.agents import load_tools
 tools = load_tools(["google-search"])
 ```
 
-### Google Places
-
-See a [usage example](/docs/integrations/tools/google_places).
-
-```
-pip install googlemaps
-```
-
-```python
-from langchain.tools import GooglePlacesTool
-```
-
 ## Document Transformer
 ### Google Document AI
 
@@ -230,40 +216,3 @@ See a [usage example](/docs/integrations/document_transformers/docai).
 from langchain.document_loaders.blob_loaders import Blob
 from langchain.document_loaders.parsers import DocAIParser
 ```
-
-## Chat loaders
-### Gmail
-
-> [Gmail](https://en.wikipedia.org/wiki/Gmail) is a free email service provided by Google.
-
-First, we need to install several python packages.
-
-```bash
-pip install --upgrade google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client
-```
-
-See a [usage example and authorizing instructions](/docs/integrations/chat_loaders/gmail).
-
-```python
-from langchain.chat_loaders.gmail import GMailLoader
-```
-
-## Agents and Toolkits
-### Gmail
-
-See a [usage example and authorizing instructions](/docs/integrations/toolkits/gmail).
-
-```python
-from langchain.agents.agent_toolkits import GmailToolkit
-
-toolkit = GmailToolkit()
-```
-
-### Google Drive
-
-See a [usage example and authorizing instructions](/docs/integrations/toolkits/google_drive).
-
-```python
-from langchain_googledrive.utilities.google_drive import GoogleDriveAPIWrapper
-from langchain_googledrive.tools.google_drive.tool import GoogleDriveSearchTool
-```

docs/docs/integrations/platforms/microsoft.mdx

@@ -1,6 +1,6 @@
 # Microsoft
 
-All functionality related to `Microsoft Azure` and other `Microsoft` products.
+All functionality related to Microsoft Azure
 
 ## LLM
 ### Azure OpenAI
@@ -70,13 +70,13 @@ from langchain.chat_models import AzureChatOpenAI
 pip install azure-storage-blob
 ```
 
-See a [usage example for the Azure Blob Storage](/docs/integrations/document_loaders/azure_blob_storage_container).
+See a [usage example for the Azure Blob Storage](/docs/integrations/document_loaders/azure_blob_storage_container.html).
 
 ```python
 from langchain.document_loaders import AzureBlobStorageContainerLoader
 ```
 
-See a [usage example for the Azure Files](/docs/integrations/document_loaders/azure_blob_storage_file).
+See a [usage example for the Azure Files](/docs/integrations/document_loaders/azure_blob_storage_file.html).
 
 ```python
 from langchain.document_loaders import AzureBlobStorageFileLoader
@@ -161,59 +161,3 @@ See a [usage example](/docs/integrations/retrievers/azure_cognitive_search).
 from langchain.retrievers import AzureCognitiveSearchRetriever
 ```
 
-## Utilities
-
-### Bing Search API
-
-See a [usage example](/docs/integrations/tools/bing_search).
-
-```python
-from langchain.utilities import BingSearchAPIWrapper
-```
-
-## Toolkits
-
-### Azure Cognitive Services
-
-We need to install several python packages.
-
-```bash
-pip install azure-ai-formrecognizer azure-cognitiveservices-speech azure-ai-vision
-```
-
-See a [usage example](/docs/integrations/toolkits/azure_cognitive_services).
-
-```python
-from langchain.agents.agent_toolkits import O365Toolkit
-```
-### Microsoft Office 365 email and calendar
-
-We need to install `O365` python package.
-
-```bash
-pip install O365
-```
-
-
-See a [usage example](/docs/integrations/toolkits/office365).
-
-```python
-from langchain.agents.agent_toolkits import O365Toolkit
-```
-
-### Microsoft Azure PowerBI
-
-We need to install `azure-identity` python package.
-
-```bash
-pip install azure-identity
-```
-
-See a [usage example](/docs/integrations/toolkits/powerbi).
-
-```python
-from langchain.agents.agent_toolkits import PowerBIToolkit
-from langchain.utilities.powerbi import PowerBIDataset
-```
-
-

docs/docs/integrations/providers/airtable.md

@@ -25,4 +25,4 @@ pip install pyairtable
 from langchain.document_loaders import AirtableLoader
 ```
 
-See an [example](/docs/integrations/document_loaders/airtable).
+See an [example](/docs/integrations/document_loaders/airtable.html).

docs/docs/integrations/providers/analyticdb.mdx

@@ -12,4 +12,4 @@ To import this vectorstore:
 from langchain.vectorstores import AnalyticDB
 ```
 
-For a more detailed walkthrough of the AnalyticDB wrapper, see [this notebook](/docs/integrations/vectorstores/analyticdb)
+For a more detailed walkthrough of the AnalyticDB wrapper, see [this notebook](/docs/integrations/vectorstores/analyticdb.html)

docs/docs/integrations/providers/apify.mdx

@@ -32,7 +32,7 @@ You can use the `ApifyWrapper` to run Actors on the Apify platform.
 from langchain.utilities import ApifyWrapper
 ```
 
-For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/apify).
+For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/apify.html).
 
 
 ### Loader
@@ -43,4 +43,4 @@ You can also use our `ApifyDatasetLoader` to get data from Apify dataset.
 from langchain.document_loaders import ApifyDatasetLoader
 ```
 
-For a more detailed walkthrough of this loader, see [this notebook](/docs/integrations/document_loaders/apify_dataset).
+For a more detailed walkthrough of this loader, see [this notebook](/docs/integrations/document_loaders/apify_dataset.html).

docs/docs/integrations/providers/arangodb.mdx

@@ -13,7 +13,7 @@ pip install python-arango
 
 Connect your ArangoDB Database with a chat model to get insights on your data. 
 
-See the notebook example [here](/docs/use_cases/graph/graph_arangodb_qa).
+See the notebook example [here](/docs/use_cases/graph/graph_arangodb_qa.html).
 
 ```python
 from arango import ArangoClient

docs/docs/integrations/providers/argilla.mdx

@@ -22,7 +22,7 @@ If you don't you can refer to [Argilla - 🚀 Quickstart](https://docs.argilla.i
 
 ## Tracking
 
-See a [usage example of `ArgillaCallbackHandler`](/docs/integrations/callbacks/argilla).
+See a [usage example of `ArgillaCallbackHandler`](/docs/integrations/callbacks/argilla.html).
 
 ```python
 from langchain.callbacks import ArgillaCallbackHandler

docs/docs/integrations/providers/chroma.mdx

@@ -18,7 +18,7 @@ whether for semantic search or example selection.
 from langchain.vectorstores import Chroma
 ```
 
-For a more detailed walkthrough of the Chroma wrapper, see [this notebook](/docs/integrations/vectorstores/chroma)
+For a more detailed walkthrough of the Chroma wrapper, see [this notebook](/docs/integrations/vectorstores/chroma.html)
 
 ## Retriever
 

docs/docs/integrations/providers/clarifai.mdx

@@ -25,7 +25,7 @@ from langchain.llms import Clarifai
 llm = Clarifai(pat=CLARIFAI_PAT, user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID)
 ```
 
-For more details, the docs on the Clarifai LLM wrapper provide a [detailed walkthrough](/docs/integrations/llms/clarifai).
+For more details, the docs on the Clarifai LLM wrapper provide a [detailed walkthrough](/docs/integrations/llms/clarifai.html).
 
 
 ### Text Embedding Models
@@ -37,7 +37,7 @@ There is a Clarifai Embedding model in LangChain, which you can access with:
 from langchain.embeddings import ClarifaiEmbeddings
 embeddings = ClarifaiEmbeddings(pat=CLARIFAI_PAT, user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID)
 ```
-For more details, the docs on the Clarifai Embeddings wrapper provide a [detailed walkthrough](/docs/integrations/text_embedding/clarifai).
+For more details, the docs on the Clarifai Embeddings wrapper provide a [detailed walkthrough](/docs/integrations/text_embedding/clarifai.html).
 
 ## Vectorstore
 

docs/docs/integrations/providers/cohere.mdx

@@ -27,7 +27,7 @@ There exists an Cohere Embedding model, which you can access with
 ```python
 from langchain.embeddings import CohereEmbeddings
 ```
-For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/cohere)
+For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/cohere.html)
 
 ## Retriever
 

docs/docs/integrations/providers/comet_tracking.ipynb

@@ -20,7 +20,7 @@
    "source": [
     "In this guide we will demonstrate how to track your Langchain Experiments, Evaluation Metrics, and LLM Sessions with [Comet](https://www.comet.com/site/?utm_source=langchain&utm_medium=referral&utm_campaign=comet_notebook).  \n",
     "\n",
-    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/hwchase17/langchain/blob/master/docs/ecosystem/comet_tracking\">\n",
+    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/hwchase17/langchain/blob/master/docs/ecosystem/comet_tracking.html\">\n",
     "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
     "</a>\n",
     "\n",

docs/docs/integrations/providers/ctransformers.mdx

@@ -54,4 +54,4 @@ llm = CTransformers(model='marella/gpt-2-ggml', config=config)
 
 See [Documentation](https://github.com/marella/ctransformers#config) for a list of available parameters.
 
-For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/ctransformers).
+For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/ctransformers.html).

docs/docs/integrations/providers/dashvector.mdx

@@ -21,4 +21,4 @@ You may import the vectorstore by:
 from langchain.vectorstores import DashVector
 ```
 
-For a detailed walkthrough of the DashVector wrapper, please refer to [this notebook](/docs/integrations/vectorstores/dashvector)
+For a detailed walkthrough of the DashVector wrapper, please refer to [this notebook](/docs/integrations/vectorstores/dashvector.html)

docs/docs/integrations/providers/databricks.md

@@ -33,11 +33,11 @@ See [MLflow AI Gateway](/docs/integrations/providers/mlflow_ai_gateway).
 Databricks as an LLM provider
 -----------------------------
 
-The notebook [Wrap Databricks endpoints as LLMs](/docs/integrations/llms/databricks) illustrates the method to wrap Databricks endpoints as LLMs in LangChain. It supports two types of endpoints: the serving endpoint, which is recommended for both production and development, and the cluster driver proxy app, which is recommended for interactive development. 
+The notebook [Wrap Databricks endpoints as LLMs](/docs/integrations/llms/databricks.html) illustrates the method to wrap Databricks endpoints as LLMs in LangChain. It supports two types of endpoints: the serving endpoint, which is recommended for both production and development, and the cluster driver proxy app, which is recommended for interactive development. 
 
 Databricks endpoints support Dolly, but are also great for hosting models like MPT-7B or any other models from the Hugging Face ecosystem. Databricks endpoints can also be used with proprietary models like OpenAI to provide a governance layer for enterprises.
 
 Databricks Dolly
 ----------------
 
-Databricks’ Dolly is an instruction-following large language model trained on the Databricks machine learning platform that is licensed for commercial use. The model is available on Hugging Face Hub as databricks/dolly-v2-12b. See the notebook [Hugging Face Hub](/docs/integrations/llms/huggingface_hub) for instructions to access it through the Hugging Face Hub integration with LangChain. 
+Databricks’ Dolly is an instruction-following large language model trained on the Databricks machine learning platform that is licensed for commercial use. The model is available on Hugging Face Hub as databricks/dolly-v2-12b. See the notebook [Hugging Face Hub](/docs/integrations/llms/huggingface_hub.html) for instructions to access it through the Hugging Face Hub integration with LangChain. 

docs/docs/integrations/providers/deepinfra.mdx

@@ -10,27 +10,16 @@ It is broken into two parts: installation and setup, and then references to spec
 ## Available Models
 
 DeepInfra provides a range of Open Source LLMs ready for deployment.
-You can list supported models for
-[text-generation](https://deepinfra.com/models?type=text-generation) and
-[embeddings](https://deepinfra.com/models?type=embeddings).
+You can list supported models [here](https://deepinfra.com/models?type=text-generation).
 google/flan\* models can be viewed [here](https://deepinfra.com/models?type=text2text-generation).
 
-You can view a [list of request and response parameters](https://deepinfra.com/meta-llama/Llama-2-70b-chat-hf/api).
+You can view a list of request and response parameters [here](https://deepinfra.com/databricks/dolly-v2-12b#API)
 
 ## Wrappers
 
 ### LLM
 
 There exists an DeepInfra LLM wrapper, which you can access with
-
 ```python
 from langchain.llms import DeepInfra
 ```
-
-### Embeddings
-
-There is also an DeepInfra Embeddings wrapper, you can access with
-
-```python
-from langchain.embeddings import DeepInfraEmbeddings
-```

docs/docs/integrations/providers/dingo.mdx

@@ -1,14 +1,14 @@
-# DingoDB
+# Dingo
 
-This page covers how to use the DingoDB ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific DingoDB wrappers.
+This page covers how to use the Dingo ecosystem within LangChain.
+It is broken into two parts: installation and setup, and then references to specific Dingo wrappers.
 
 ## Installation and Setup
 - Install the Python SDK with `pip install dingodb`
 
 ## VectorStore
 
-There exists a wrapper around DingoDB indexes, allowing you to use it as a vectorstore,
+There exists a wrapper around Dingo indexes, allowing you to use it as a vectorstore,
 whether for semantic search or example selection.
 
 To import this vectorstore:
@@ -16,4 +16,4 @@ To import this vectorstore:
 from langchain.vectorstores import Dingo
 ```
 
-For a more detailed walkthrough of the DingoDB wrapper, see [this notebook](/docs/integrations/vectorstores/dingo)
+For a more detailed walkthrough of the Dingo wrapper, see [this notebook](/docs/integrations/vectorstores/dingo.html)

docs/docs/integrations/providers/epsilla.mdx

@@ -20,4 +20,4 @@ To import this vectorstore:
 from langchain.vectorstores import Epsilla
 ```
 
-For a more detailed walkthrough of the Epsilla wrapper, see [this notebook](/docs/integrations/vectorstores/epsilla)
+For a more detailed walkthrough of the Epsilla wrapper, see [this notebook](/docs/integrations/vectorstores/epsilla.html)

docs/docs/integrations/providers/golden.mdx

@@ -20,7 +20,7 @@ There exists a GoldenQueryAPIWrapper utility which wraps this API. To import thi
 from langchain.utilities.golden_query import GoldenQueryAPIWrapper
 ```
 
-For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/golden_query).
+For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/golden_query.html).
 
 ### Tool
 

docs/docs/integrations/providers/google_document_ai.mdx

@@ -0,0 +1,28 @@
+# Google Document AI
+
+>[Document AI](https://cloud.google.com/document-ai/docs/overview) is a `Google Cloud Platform` 
+> service to transform unstructured data from documents into structured data, making it easier 
+> to understand, analyze, and consume.  
+
+
+## Installation and Setup
+
+You need to set up a [`GCS` bucket and create your own OCR processor](https://cloud.google.com/document-ai/docs/create-processor)  
+The `GCS_OUTPUT_PATH` should be a path to a folder on GCS (starting with `gs://`) 
+and a processor name should look like `projects/PROJECT_NUMBER/locations/LOCATION/processors/PROCESSOR_ID`.
+You can get it either programmatically or copy from the `Prediction endpoint` section of the `Processor details`
+tab in the Google Cloud Console.
+
+```bash
+pip install google-cloud-documentai
+pip install google-cloud-documentai-toolbox
+```
+
+## Document Transformer
+
+See a [usage example](/docs/integrations/document_transformers/docai).
+
+```python
+from langchain.document_loaders.blob_loaders import Blob
+from langchain.document_loaders.parsers import DocAIParser
+```

docs/docs/integrations/providers/google_serper.mdx

@@ -1,4 +1,4 @@
-# Serper - Google Search API
+# Google Serper
 
 This page covers how to use the [Serper](https://serper.dev) Google Search API within LangChain. Serper is a low-cost Google Search API that can be used to add answer box, knowledge graph, and organic results data from Google Search. 
 It is broken into two parts: setup, and then references to the specific Google Serper wrapper.
@@ -59,7 +59,7 @@ So the final answer is: El Palmar, Spain
 'El Palmar, Spain'
 ```
 
-For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/google_serper).
+For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/google_serper.html).
 
 ### Tool
 

docs/docs/integrations/providers/gpt4all.mdx

@@ -45,4 +45,4 @@ model("Once upon a time, ", callbacks=callbacks)
 
 You can find links to model file downloads in the [pyllamacpp](https://github.com/nomic-ai/pyllamacpp) repository.
 
-For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/gpt4all)
+For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/gpt4all.html)

docs/docs/integrations/providers/gradient.mdx

@@ -24,4 +24,4 @@ There exists an Gradient Embedding model, which you can access with
 ```python
 from langchain.embeddings import GradientEmbeddings
 ```
-For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/gradient)
+For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/gradient.html)

docs/docs/integrations/providers/huggingface.mdx

@@ -30,7 +30,7 @@ To use a the wrapper for a model hosted on Hugging Face Hub:
 ```python
 from langchain.llms import HuggingFaceHub
 ```
-For a more detailed walkthrough of the Hugging Face Hub wrapper, see [this notebook](/docs/integrations/llms/huggingface_hub)
+For a more detailed walkthrough of the Hugging Face Hub wrapper, see [this notebook](/docs/integrations/llms/huggingface_hub.html)
 
 
 ### Embeddings

docs/docs/integrations/providers/infino.mdx

@@ -28,7 +28,7 @@ you don't, follow the next steps to start it:
 
 ## Using Infino
 
-See a [usage example of `InfinoCallbackHandler`](/docs/integrations/callbacks/infino).
+See a [usage example of `InfinoCallbackHandler`](/docs/integrations/callbacks/infino.html).
 
 ```python
 from langchain.callbacks import InfinoCallbackHandler

docs/docs/integrations/providers/jina.mdx

@@ -15,7 +15,7 @@ There exists a Jina Embeddings wrapper, which you can access with
 ```python
 from langchain.embeddings import JinaEmbeddings
 ```
-For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/jina)
+For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/jina.html)
 
 ## Deployment
 

docs/docs/integrations/providers/johnsnowlabs.mdx

@@ -1,117 +0,0 @@
-# Johnsnowlabs
-
-Gain access to the [johnsnowlabs](https://www.johnsnowlabs.com/) ecosystem of enterprise NLP libraries
-with over 21.000 enterprise NLP models in over 200 languages with the open source `johnsnowlabs` library.
-For all 24.000+ models, see the [John Snow Labs Model Models Hub](https://nlp.johnsnowlabs.com/models)
-
-## Installation and Setup
-
-
-```bash
-pip install johnsnowlabs
-```
-
-To [install enterprise features](https://nlp.johnsnowlabs.com/docs/en/jsl/install_licensed_quick, run:
-```python
-# for more details see https://nlp.johnsnowlabs.com/docs/en/jsl/install_licensed_quick
-nlp.install()
-```
-
-
-You can embed your queries and documents with either `gpu`,`cpu`,`apple_silicon`,`aarch` based optimized binaries.
-By default cpu binaries are used.
-Once a session is started, you must restart your notebook to switch between GPU or CPU, or changes will not take effect.
-
-## Embed Query with CPU:
-```python
-document = "foo bar"
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert')
-output = embedding.embed_query(document)
-```
-
-
-## Embed Query with GPU:
-
-
-```python
-document = "foo bar"
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert','gpu')
-output = embedding.embed_query(document)
-```
-
-
-
-
-## Embed Query with Apple Silicon (M1,M2,etc..):
-
-```python
-documents = ["foo bar", 'bar foo']
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert','apple_silicon')
-output = embedding.embed_query(document)
-```
-
-
-
-## Embed Query with AARCH:
-
-```python
-documents = ["foo bar", 'bar foo']
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert','aarch')
-output = embedding.embed_query(document)
-```
-
-
-
-
-
-
-## Embed Document with CPU:
-```python
-documents = ["foo bar", 'bar foo']
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert','gpu')
-output = embedding.embed_documents(documents)
-```
-
-
-
-## Embed Document with GPU:
-
-```python
-documents = ["foo bar", 'bar foo']
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert','gpu')
-output = embedding.embed_documents(documents)
-```
-
-
-
-
-
-## Embed Document with Apple Silicon (M1,M2,etc..):
-
-```python
-
-```python
-documents = ["foo bar", 'bar foo']
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert','apple_silicon')
-output = embedding.embed_documents(documents)
-```
-
-
-
-## Embed Document with AARCH:
-
-```python
-
-```python
-documents = ["foo bar", 'bar foo']
-embedding = JohnSnowLabsEmbeddings('embed_sentence.bert','aarch')
-output = embedding.embed_documents(documents)
-```
-
-
-
-
-Models are loaded with [nlp.load](https://nlp.johnsnowlabs.com/docs/en/jsl/load_api) and spark session is started with [nlp.start()](https://nlp.johnsnowlabs.com/docs/en/jsl/start-a-sparksession) under the hood.
-
-
-

docs/docs/integrations/providers/lancedb.mdx

@@ -20,4 +20,4 @@ To import this vectorstore:
 from langchain.vectorstores import LanceDB
 ```
 
-For a more detailed walkthrough of the LanceDB wrapper, see [this notebook](/docs/integrations/vectorstores/lancedb)
+For a more detailed walkthrough of the LanceDB wrapper, see [this notebook](/docs/integrations/vectorstores/lancedb.html)

docs/docs/integrations/providers/llamacpp.mdx

@@ -15,7 +15,7 @@ There exists a LlamaCpp LLM wrapper, which you can access with
 ```python
 from langchain.llms import LlamaCpp
 ```
-For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/llamacpp)
+For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/llamacpp.html)
 
 ### Embeddings
 
@@ -23,4 +23,4 @@ There exists a LlamaCpp Embeddings wrapper, which you can access with
 ```python
 from langchain.embeddings import LlamaCppEmbeddings
 ```
-For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/llamacpp)
+For a more detailed walkthrough of this, see [this notebook](/docs/integrations/text_embedding/llamacpp.html)

docs/docs/integrations/providers/marqo.md

@@ -28,4 +28,4 @@ To import this vectorstore:
 from langchain.vectorstores import Marqo
 ```
 
-For a more detailed walkthrough of the Marqo wrapper and some of its unique features, see [this notebook](/docs/integrations/vectorstores/marqo)
+For a more detailed walkthrough of the Marqo wrapper and some of its unique features, see [this notebook](/docs/integrations/vectorstores/marqo.html)

docs/docs/integrations/providers/milvus.mdx

@@ -22,4 +22,4 @@ To import this vectorstore:
 from langchain.vectorstores import Milvus
 ```
 
-For a more detailed walkthrough of the `Miluvs` wrapper, see [this notebook](/docs/integrations/vectorstores/milvus)
+For a more detailed walkthrough of the `Miluvs` wrapper, see [this notebook](/docs/integrations/vectorstores/milvus.html)

docs/docs/integrations/providers/minimax.mdx

@@ -11,7 +11,7 @@ Get a [Minimax group id](https://api.minimax.chat/user-center/basic-information)
 ## LLM
 
 There exists a Minimax LLM wrapper, which you can access with
-See a [usage example](/docs/modules/model_io/models/llms/integrations/minimax).
+See a [usage example](/docs/modules/model_io/models/llms/integrations/minimax.html).
 
 ```python
 from langchain.llms import Minimax
@@ -19,7 +19,7 @@ from langchain.llms import Minimax
 
 ## Chat Models
 
-See a [usage example](/docs/modules/model_io/models/chat/integrations/minimax)
+See a [usage example](/docs/modules/model_io/models/chat/integrations/minimax.html)
 
 ```python
 from langchain.chat_models import MiniMaxChat

docs/docs/integrations/providers/mlflow_ai_gateway.mdx

@@ -1,9 +1,9 @@
 # MLflow AI Gateway
 
->[The MLflow AI Gateway](https://www.mlflow.org/docs/latest/gateway/index) service is a powerful tool designed to streamline the usage and management of various large 
+>[The MLflow AI Gateway](https://www.mlflow.org/docs/latest/gateway/index.html) service is a powerful tool designed to streamline the usage and management of various large 
 > language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface 
 > that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests.
-> See [the MLflow AI Gateway documentation](https://mlflow.org/docs/latest/gateway/index) for more details.
+> See [the MLflow AI Gateway documentation](https://mlflow.org/docs/latest/gateway/index.html) for more details.
 
 ## Installation and Setup
 
@@ -52,7 +52,7 @@ mlflow gateway start --config-path /path/to/config.yaml
 > This module exports multivariate LangChain models in the langchain flavor and univariate LangChain 
 > models in the pyfunc flavor.
  
-See the [API documentation and examples](https://www.mlflow.org/docs/latest/python_api/mlflow.langchain).
+See the [API documentation and examples](https://www.mlflow.org/docs/latest/python_api/mlflow.langchain.html).
 
 
 

docs/docs/integrations/providers/mlflow_tracking.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# MLflow\n",
     "\n",
-    ">[MLflow](https://www.mlflow.org/docs/latest/what-is-mlflow) is a versatile, expandable, open-source platform for managing workflows and artifacts across the machine learning lifecycle. It has built-in integrations with many popular ML libraries, but can be used with any library, algorithm, or deployment tool. It is designed to be extensible, so you can write plugins to support new workflows, libraries, and tools.\n",
+    ">[MLflow](https://www.mlflow.org/docs/latest/what-is-mlflow.html) is a versatile, expandable, open-source platform for managing workflows and artifacts across the machine learning lifecycle. It has built-in integrations with many popular ML libraries, but can be used with any library, algorithm, or deployment tool. It is designed to be extensible, so you can write plugins to support new workflows, libraries, and tools.\n",
     "\n",
     "This notebook goes over how to track your LangChain experiments into your `MLflow Server`"
    ]

docs/docs/integrations/providers/momento.mdx

@@ -50,10 +50,10 @@ Momento can be used as a distributed memory store for LLMs.
 
 ### Chat Message History Memory
 
-See [this notebook](/docs/integrations/memory/momento_chat_message_history) for a walkthrough of how to use Momento as a memory store for chat message history.
+See [this notebook](/docs/integrations/memory/momento_chat_message_history.html) for a walkthrough of how to use Momento as a memory store for chat message history.
 
 ## Vector Store
 
 Momento Vector Index (MVI) can be used as a vector store.
 
-See [this notebook](/docs/integrations/vectorstores/momento_vector_index) for a walkthrough of how to use MVI as a vector store.
+See [this notebook](/docs/integrations/vectorstores/momento_vector_index.html) for a walkthrough of how to use MVI as a vector store.

docs/docs/integrations/providers/motherduck.mdx

@@ -31,7 +31,7 @@ db = SQLDatabase.from_uri(conn_str)
 db_chain = SQLDatabaseChain.from_llm(OpenAI(temperature=0), db, verbose=True)
 ```
 
-From here, see the [SQL Chain](/docs/use_cases/tabular/sqlite) documentation on how to use.
+From here, see the [SQL Chain](/docs/use_cases/tabular/sqlite.html) documentation on how to use.
 
 
 ## LLMCache

docs/docs/integrations/providers/myscale.mdx

@@ -63,4 +63,4 @@ To import this vectorstore:
 from langchain.vectorstores import MyScale
 ```
 
-For a more detailed walkthrough of the MyScale wrapper, see [this notebook](/docs/integrations/vectorstores/myscale)
+For a more detailed walkthrough of the MyScale wrapper, see [this notebook](/docs/integrations/vectorstores/myscale.html)

docs/docs/integrations/providers/neo4j.mdx

@@ -29,7 +29,7 @@ To import this vectorstore:
 from langchain.vectorstores import Neo4jVector
 ```
 
-For a more detailed walkthrough of the Neo4j vector index wrapper, see [documentation](/docs/integrations/vectorstores/neo4jvector)
+For a more detailed walkthrough of the Neo4j vector index wrapper, see [documentation](/docs/integrations/vectorstores/neo4jvector.html)
 
 ### GraphCypherQAChain
 
@@ -41,7 +41,7 @@ from langchain.graphs import Neo4jGraph
 from langchain.chains import GraphCypherQAChain
 ```
 
-For a more detailed walkthrough of Cypher generating chain, see [documentation](/docs/use_cases/graph/graph_cypher_qa)
+For a more detailed walkthrough of Cypher generating chain, see [documentation](/docs/use_cases/graph/graph_cypher_qa.html)
 
 ### Constructing a knowledge graph from text
 
@@ -55,4 +55,4 @@ from langchain.graphs import Neo4jGraph
 from langchain_experimental.graph_transformers.diffbot import DiffbotGraphTransformer
 ```
 
-For a more detailed walkthrough generating graphs from text, see [documentation](/docs/use_cases/graph/diffbot_graphtransformer)
+For a more detailed walkthrough generating graphs from text, see [documentation](/docs/use_cases/graph/diffbot_graphtransformer.html)

docs/docs/integrations/providers/notion.mdx

@@ -12,14 +12,14 @@ All instructions are in examples below.
 
 We have two different loaders: `NotionDirectoryLoader` and `NotionDBLoader`.
 
-See a [usage example for the NotionDirectoryLoader](/docs/integrations/document_loaders/notion).
+See a [usage example for the NotionDirectoryLoader](/docs/integrations/document_loaders/notion.html).
 
 
 ```python
 from langchain.document_loaders import NotionDirectoryLoader
 ```
 
-See a [usage example for the NotionDBLoader](/docs/integrations/document_loaders/notiondb).
+See a [usage example for the NotionDBLoader](/docs/integrations/document_loaders/notiondb.html).
 
 
 ```python

docs/docs/integrations/providers/openllm.mdx

@@ -67,4 +67,4 @@ llm("What is the difference between a duck and a goose? And why there are so man
 ### Usage
 
 For a more detailed walkthrough of the OpenLLM Wrapper, see the
-[example notebook](/docs/integrations/llms/openllm)
+[example notebook](/docs/integrations/llms/openllm.html)

docs/docs/integrations/providers/opensearch.mdx

@@ -18,4 +18,4 @@ To import this vectorstore:
 from langchain.vectorstores import OpenSearchVectorSearch
 ```
 
-For a more detailed walkthrough of the OpenSearch wrapper, see [this notebook](/docs/integrations/vectorstores/opensearch)
+For a more detailed walkthrough of the OpenSearch wrapper, see [this notebook](/docs/integrations/vectorstores/opensearch.html)

docs/docs/integrations/providers/openweathermap.mdx

@@ -29,7 +29,7 @@ There exists a OpenWeatherMapAPIWrapper utility which wraps this API. To import
 from langchain.utilities.openweathermap import OpenWeatherMapAPIWrapper
 ```
 
-For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/openweathermap).
+For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/openweathermap.html).
 
 ### Tool
 

docs/docs/integrations/providers/pgvector.mdx

@@ -26,4 +26,4 @@ from langchain.vectorstores.pgvector import PGVector
 
 ### Usage
 
-For a more detailed walkthrough of the PGVector Wrapper, see [this notebook](/docs/integrations/vectorstores/pgvector)
+For a more detailed walkthrough of the PGVector Wrapper, see [this notebook](/docs/integrations/vectorstores/pgvector.html)

docs/docs/integrations/providers/pinecone.mdx

@@ -21,4 +21,4 @@ whether for semantic search or example selection.
 from langchain.vectorstores import Pinecone
 ```
 
-For a more detailed walkthrough of the Pinecone vectorstore, see [this notebook](/docs/integrations/vectorstores/pinecone)
+For a more detailed walkthrough of the Pinecone vectorstore, see [this notebook](/docs/integrations/vectorstores/pinecone.html)