Merge branch 'kayvane1-align-search-tools' into vwp/align_search_tools

A minimal example of how to deploy LangChain to Fly.io using Flask.

.devcontainer/Dockerfile

@@ -1,42 +0,0 @@
-# This is a Dockerfile for Developer Container
-
-# Use the Python base image
-ARG VARIANT="3.11-bullseye"
-FROM mcr.microsoft.com/vscode/devcontainers/python:0-${VARIANT} AS langchain-dev-base
-
-USER vscode
-
-# Define the version of Poetry to install (default is 1.4.2)
-# Define the directory of python virtual environment
-ARG PYTHON_VIRTUALENV_HOME=/home/vscode/langchain-py-env \
-    POETRY_VERSION=1.4.2 
-
-ENV POETRY_VIRTUALENVS_IN_PROJECT=false \
-    POETRY_NO_INTERACTION=true 
-
-# Create a Python virtual environment for Poetry and install it
-RUN python3 -m venv ${PYTHON_VIRTUALENV_HOME} && \
-    $PYTHON_VIRTUALENV_HOME/bin/pip install --upgrade pip && \
-    $PYTHON_VIRTUALENV_HOME/bin/pip install poetry==${POETRY_VERSION}
-
-ENV PATH="$PYTHON_VIRTUALENV_HOME/bin:$PATH" \
-    VIRTUAL_ENV=$PYTHON_VIRTUALENV_HOME
-
-# Setup for bash
-RUN poetry completions bash >> /home/vscode/.bash_completion && \
-    echo "export PATH=$PYTHON_VIRTUALENV_HOME/bin:$PATH" >> ~/.bashrc
-
-# Set the working directory for the app
-WORKDIR /workspaces/langchain
-
-# Use a multi-stage build to install dependencies
-FROM langchain-dev-base AS langchain-dev-dependencies
-
-ARG PYTHON_VIRTUALENV_HOME
-
-# Copy only the dependency files for installation
-COPY pyproject.toml poetry.lock poetry.toml ./
-
-# Install the Poetry dependencies (this layer will be cached as long as the dependencies don't change)
-RUN poetry install --no-interaction --no-ansi --with dev,test,docs
-

.devcontainer/devcontainer.json

@@ -1,33 +0,0 @@
-// For format details, see https://aka.ms/devcontainer.json. For config options, see the
-// README at: https://github.com/devcontainers/templates/tree/main/src/docker-existing-dockerfile
-{
-	"dockerComposeFile": "./docker-compose.yaml",
-	"service": "langchain",
-	"workspaceFolder": "/workspaces/langchain",
-	"name": "langchain",
-	"customizations": {
-		"vscode": {
-			"extensions": [   
-				"ms-python.python"
-			],
-			"settings": {
-				"python.defaultInterpreterPath": "/home/vscode/langchain-py-env/bin/python3.11"
-			}
-		}
-		
-	},
-
-	// Features to add to the dev container. More info: https://containers.dev/features.
-	"features": {},
-
-	// Use 'forwardPorts' to make a list of ports inside the container available locally.
-	// "forwardPorts": [],
-
-	// Uncomment the next line to run commands after the container is created.
-	// "postCreateCommand": "cat /etc/os-release",
-
-	// Uncomment to connect as an existing user other than the container default. More info: https://aka.ms/dev-containers-non-root.
-	// "remoteUser": "devcontainer"
-	"remoteUser": "vscode",
-	"overrideCommand": true
-}

.devcontainer/docker-compose.yaml

@@ -1,31 +0,0 @@
-version: '3'
-services:
-  langchain:
-    build:
-      dockerfile: .devcontainer/Dockerfile
-      context: ../ 
-    volumes:
-      - ../:/workspaces/langchain
-    networks:
-      - langchain-network 
-  #   environment:
-  #     MONGO_ROOT_USERNAME: root
-  #     MONGO_ROOT_PASSWORD: example123
-  #   depends_on:
-  #     - mongo   
-  # mongo:
-  #   image: mongo
-  #   restart: unless-stopped
-  #   environment:
-  #     MONGO_INITDB_ROOT_USERNAME: root
-  #     MONGO_INITDB_ROOT_PASSWORD: example123
-  #   ports:
-  #     - "27017:27017"
-  #   networks:
-  #     - langchain-network
-
-networks:
-  langchain-network:
-    driver: bridge
-    
-    

.github/CONTRIBUTING.md

@@ -2,62 +2,60 @@
 
 Hi there! Thank you for even being interested in contributing to LangChain.
 As an open source project in a rapidly developing field, we are extremely open
-to contributions, whether they be in the form of new features, improved infra, better documentation, or bug fixes.
-
-## 🗺️ Guidelines
-
-### 👩‍💻 Contributing Code
+to contributions, whether it be in the form of a new feature, improved infra, or better documentation.
 
 To contribute to this project, please follow a ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow.
 Please do not try to push directly to this repo unless you are maintainer.
 
-Please follow the checked-in pull request template when opening pull requests. Note related issues and tag relevant
-maintainers.
-
-Pull requests cannot land without passing the formatting, linting and testing checks first. See
-[Common Tasks](#-common-tasks) for how to run these checks locally.
-
-It's essential that we maintain great documentation and testing. If you:
-- Fix a bug
-  - Add a relevant unit or integration test when possible. These live in `tests/unit_tests` and `tests/integration_tests`.
-- Make an improvement
-  - Update any affected example notebooks and documentation. These lives in `docs`.
-  - Update unit and integration tests when relevant.
-- Add a feature
-  - Add a demo notebook in `docs/modules`.
-  - Add unit and integration tests.
-
-We're a small, building-oriented team. If there's something you'd like to add or change, opening a pull request is the
-best way to get our attention.
+## 🗺️Contributing Guidelines
 
 ### 🚩GitHub Issues
 
 Our [issues](https://github.com/hwchase17/langchain/issues) page is kept up to date
-with bugs, improvements, and feature requests. 
+with bugs, improvements, and feature requests. There is a taxonomy of labels to help
+with sorting and discovery of issues of interest. These include:
 
-There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help
-organize issues.
+- prompts: related to prompt tooling/infra.
+- llms: related to LLM wrappers/tooling/infra.
+- chains
+- utilities: related to different types of utilities to integrate with (Python, SQL, etc.).
+- agents
+- memory
+- applications: related to example applications to build
 
 If you start working on an issue, please assign it to yourself.
 
-If you are adding an issue, please try to keep it focused on a single, modular bug/improvement/feature.
-If two issues are related, or blocking, please link them rather than combining them.
+If you are adding an issue, please try to keep it focused on a single modular bug/improvement/feature.
+If the two issues are related, or blocking, please link them rather than keep them as one single one.
 
 We will try to keep these issues as up to date as possible, though
 with the rapid rate of develop in this field some may get out of date.
-If you notice this happening, please let us know.
+If you notice this happening, please just let us know.
 
 ### 🙋Getting Help
 
-Our goal is to have the simplest developer setup possible. Should you experience any difficulty getting setup, please
-contact a maintainer! Not only do we want to help get you unblocked, but we also want to make sure that the process is
-smooth for future contributors.
+Although we try to have a developer setup to make it as easy as possible for others to contribute (see below)
+it is possible that some pain point may arise around environment setup, linting, documentation, or other.
+Should that occur, please contact a maintainer! Not only do we want to help get you unblocked,
+but we also want to make sure that the process is smooth for future contributors.
 
 In a similar vein, we do enforce certain linting, formatting, and documentation standards in the codebase.
-If you are finding these difficult (or even just annoying) to work with, feel free to contact a maintainer for help -
-we do not want these to get in the way of getting good code into the codebase.
+If you are finding these difficult (or even just annoying) to work with,
+feel free to contact a maintainer for help - we do not want these to get in the way of getting
+good code into the codebase.
 
-## 🚀 Quick Start
+### 🏭Release process
+
+As of now, LangChain has an ad hoc release process: releases are cut with high frequency by
+a developer and published to [PyPI](https://pypi.org/project/langchain/).
+
+LangChain follows the [semver](https://semver.org/) versioning standard. However, as pre-1.0 software,
+even patch releases may contain [non-backwards-compatible changes](https://semver.org/#spec-item-4).
+
+If your contribution has made its way into a release, we will want to give you credit on Twitter (only if you want though)!
+If you have a Twitter account you would like us to mention, please let us know in the PR or in another manner.
+
+## 🚀Quick Start
 
 This project uses [Poetry](https://python-poetry.org/) as a dependency manager. Check out Poetry's [documentation on how to install it](https://python-poetry.org/docs/#installation) on your system before proceeding.
 
@@ -79,7 +77,7 @@ This will install all requirements for running the package, examples, linting, f
 
 Now, you should be able to run the common tasks in the following section. To double check, run `make test`, all tests should pass. If they don't you may need to pip install additional dependencies, such as `numexpr` and `openapi_schema_pydantic`.
 
-## ✅ Common Tasks
+## ✅Common Tasks
 
 Type `make` for a list of common tasks.
 
@@ -115,37 +113,8 @@ To get a report of current coverage, run the following:
 make coverage
 ```
 
-### Working with Optional Dependencies
-
-Langchain relies heavily on optional dependencies to keep the Langchain package lightweight.
-
-If you're adding a new dependency to Langchain, assume that it will be an optional dependency, and
-that most users won't have it installed.
-
-Users that do not have the dependency installed should be able to **import** your code without
-any side effects (no warnings, no errors, no exceptions). 
-
-To introduce the dependency to the pyproject.toml file correctly, please do the following: 
-
-1. Add the dependency to the main group as an optional dependency
-  ```bash
-  poetry add --optional [package_name]
-  ```
-2. Open pyproject.toml and add the dependency to the `extended_testing` extra
-3. Relock the poetry file to update the extra.
-  ```bash
-  poetry lock --no-update
-  ```
-4. Add a unit test that the very least attempts to import the new code. Ideally the unit
-test makes use of lightweight fixtures to test the logic of the code.
-5. Please use the `@pytest.mark.requires(package_name)` decorator for any tests that require the dependency.
-
 ### Testing
 
-See section about optional dependencies.
-
-#### Unit Tests
-
 Unit tests cover modular logic that does not require calls to outside APIs.
 
 To run unit tests:
@@ -162,20 +131,8 @@ make docker_tests
 
 If you add new logic, please add a unit test.
 
-
-
-#### Integration Tests
-
 Integration tests cover logic that requires making calls to outside APIs (often integration with other services).
 
-**warning** Almost no tests should be integration tests. 
-
-  Tests that require making network connections make it difficult for other
-  developers to test the code.
-
-  Instead favor relying on `responses` library and/or mock.patch to mock
-  requests using small fixtures.
-
 To run integration tests:
 
 ```bash
@@ -231,17 +188,3 @@ Finally, you can build the documentation as outlined below:
 ```bash
 make docs_build
 ```
-
-## 🏭 Release Process
-
-As of now, LangChain has an ad hoc release process: releases are cut with high frequency by
-a developer and published to [PyPI](https://pypi.org/project/langchain/).
-
-LangChain follows the [semver](https://semver.org/) versioning standard. However, as pre-1.0 software,
-even patch releases may contain [non-backwards-compatible changes](https://semver.org/#spec-item-4).
-
-### 🌟 Recognition
-
-If your contribution has made its way into a release, we will want to give you credit on Twitter (only if you want though)!
-If you have a Twitter account you would like us to mention, please let us know in the PR or in another manner.
-

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -1,106 +0,0 @@
-name: "\U0001F41B Bug Report"
-description: Submit a bug report to help us improve LangChain
-labels: ["02 Bug Report"]
-body:
-  - type: markdown
-    attributes:
-      value: >
-        Thank you for taking the time to file a bug report. Before creating a new
-        issue, please make sure to take a few moments to check the issue tracker
-        for existing issues about the bug.
-
-  - type: textarea
-    id: system-info
-    attributes:
-      label: System Info
-      description: Please share your system info with us.
-      placeholder: LangChain version, platform, python version, ...
-    validations:
-      required: true
-
-  - type: textarea
-    id: who-can-help
-    attributes:
-      label: Who can help?
-      description: |
-        Your issue will be replied to more quickly if you can figure out the right person to tag with @
-        If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
-
-        The core maintainers strive to read all issues, but tagging them will help them prioritize.
-
-        Please tag fewer than 3 people.
-
-        @hwchase17 - project lead
-
-        Tracing / Callbacks
-        - @agola11
-
-        Async
-        - @agola11
-
-        DataLoader Abstractions
-        - @eyurtsev
-
-        LLM/Chat Wrappers
-        - @hwchase17
-        - @agola11
-
-        Tools / Toolkits
-        - ...
-
-      placeholder: "@Username ..."
-
-  - type: checkboxes
-    id: information-scripts-examples
-    attributes:
-      label: Information
-      description: "The problem arises when using:"
-      options:
-        - label: "The official example notebooks/scripts"
-        - label: "My own modified scripts"
-
-  - type: checkboxes
-    id: related-components
-    attributes:
-      label: Related Components
-      description: "Select the components related to the issue (if applicable):"
-      options:
-        - label: "LLMs/Chat Models"
-        - label: "Embedding Models"
-        - label: "Prompts / Prompt Templates / Prompt Selectors"
-        - label: "Output Parsers"
-        - label: "Document Loaders"
-        - label: "Vector Stores / Retrievers"
-        - label: "Memory"
-        - label: "Agents / Agent Executors"
-        - label: "Tools / Toolkits"
-        - label: "Chains"
-        - label: "Callbacks/Tracing"
-        - label: "Async"
-
-  - type: textarea
-    id: reproduction
-    validations:
-      required: true
-    attributes:
-      label: Reproduction
-      description: |
-        Please provide a [code sample](https://stackoverflow.com/help/minimal-reproducible-example) that reproduces the problem you ran into. It can be a Colab link or just a code snippet.
-        If you have code snippets, error messages, stack traces please provide them here as well.
-        Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
-        Avoid screenshots when possible, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
-
-      placeholder: |
-        Steps to reproduce the behavior:
-
-          1.
-          2.
-          3.
-
-  - type: textarea
-    id: expected-behavior
-    validations:
-      required: true
-    attributes:
-      label: Expected behavior
-      description: "A clear and concise description of what you would expect to happen."

.github/ISSUE_TEMPLATE/config.yml

@@ -1,6 +0,0 @@
-blank_issues_enabled: true
-version: 2.1
-contact_links:
-  - name: Discord
-    url: https://discord.gg/6adMQxSpJS
-    about: General community discussions

.github/ISSUE_TEMPLATE/documentation.yml

@@ -1,19 +0,0 @@
-name: Documentation
-description: Report an issue related to the LangChain documentation.
-title: "DOC: <Please write a comprehensive title after the 'DOC: ' prefix>"
-labels: [03 - Documentation]
-
-body:
-- type: textarea
-  attributes: 
-    label: "Issue with current documentation:"
-    description: >
-      Please make sure to leave a reference to the document/code you're
-      referring to.
-
-- type: textarea
-  attributes:
-    label: "Idea or request for content:"
-    description: >
-      Please describe as clearly as possible what topics you think are missing
-      from the current documentation.

.github/ISSUE_TEMPLATE/feature-request.yml

@@ -1,30 +0,0 @@
-name: "\U0001F680 Feature request"
-description: Submit a proposal/request for a new LangChain feature
-labels: ["02 Feature Request"]
-body:
-  - type: textarea
-    id: feature-request
-    validations:
-      required: true
-    attributes:
-      label: Feature request
-      description: |
-        A clear and concise description of the feature proposal. Please provide links to any relevant GitHub repos, papers, or other resources if relevant.
-
-  - type: textarea
-    id: motivation
-    validations:
-      required: true
-    attributes:
-      label: Motivation
-      description: |
-        Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too.
-
-  - type: textarea
-    id: contribution
-    validations:
-      required: true
-    attributes:
-      label: Your contribution
-      description: |
-        Is there any way that you could help, e.g. by submitting a PR? Make sure to read the CONTRIBUTING.MD [readme](https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md)

.github/ISSUE_TEMPLATE/other.yml

@@ -1,18 +0,0 @@
-name: Other Issue
-description: Raise an issue that wouldn't be covered by the other templates.
-title: "Issue: <Please write a comprehensive title after the 'Issue: ' prefix>"
-labels: [04 - Other]
-
-body:
-  - type: textarea
-    attributes:
-      label: "Issue you'd like to raise."
-      description: >
-        Please describe the issue you'd like to raise as clearly as possible.
-        Make sure to include any relevant links or references.
-
-  - type: textarea
-    attributes:
-      label: "Suggestion:"
-      description: >
-        Please outline a suggestion to improve the issue here.

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,56 +0,0 @@
-<!--
-Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.
-
-Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.
-
-After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.
-
-Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
--->
-
-<!-- Remove if not applicable -->
-
-Fixes # (issue)
-
-#### Before submitting
-
-<!-- If you're adding a new integration, please include:
-
-1. a test for the integration - favor unit tests that does not rely on network access.
-2. an example notebook showing its use
-
-
-See contribution guidelines for more information on how to write tests, lint
-etc:
-
-https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
--->
-
-#### Who can review?
-
-Tag maintainers/contributors who might be interested:
-
-<!-- For a quicker response, figure out the right person to tag with @
-
-  @hwchase17 - project lead
-
-  Tracing / Callbacks
-  - @agola11
-
-  Async
-  - @agola11
-
-  DataLoaders
-  - @eyurtsev
-
-  Models
-  - @hwchase17
-  - @agola11
-
-  Agents / Tools / Toolkits
-  - @hwchase17
-
-  VectorStores / Retrievers / Memory
-  - @dev2049
-
- -->

.github/actions/poetry_setup/action.yml

@@ -1,76 +0,0 @@
-# An action for setting up poetry install with caching.
-# Using a custom action since the default action does not
-# take poetry install groups into account.
-# Action code from:
-# https://github.com/actions/setup-python/issues/505#issuecomment-1273013236
-name: poetry-install-with-caching
-description: Poetry install with support for caching of dependency groups.
-
-inputs:
-  python-version:
-    description: Python version, supporting MAJOR.MINOR only
-    required: true
-
-  poetry-version:
-    description: Poetry version
-    required: true
-
-  install-command:
-    description: Command run for installing dependencies
-    required: false
-    default: poetry install
-
-  cache-key:
-    description: Cache key to use for manual handling of caching
-    required: true
-
-  working-directory:
-    description: Directory to run install-command in
-    required: false
-    default: ""
-
-runs:
-  using: composite
-  steps:
-    - uses: actions/setup-python@v4
-      name: Setup python $${ inputs.python-version }}
-      with:
-        python-version: ${{ inputs.python-version }}
-
-    - uses: actions/cache@v3
-      id: cache-pip
-      name: Cache Pip ${{ inputs.python-version }}
-      env:
-        SEGMENT_DOWNLOAD_TIMEOUT_MIN: "15"
-      with:
-        path: |
-          ~/.cache/pip
-        key: pip-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}
-
-    - run: pipx install poetry==${{ inputs.poetry-version }} --python python${{ inputs.python-version }}
-      shell: bash
-
-    - name: Check Poetry File
-      shell: bash
-      run: |
-        poetry check
-
-    - name: Check lock file
-      shell: bash
-      run: |
-        poetry lock --check
-
-    - uses: actions/cache@v3
-      id: cache-poetry
-      env:
-        SEGMENT_DOWNLOAD_TIMEOUT_MIN: "15"
-      with:
-        path: |
-          ~/.cache/pypoetry/virtualenvs
-          ~/.cache/pypoetry/cache
-          ~/.cache/pypoetry/artifacts
-        key: poetry-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-poetry-${{ inputs.poetry-version }}-${{ inputs.cache-key }}-${{ hashFiles('poetry.lock') }}
-
-    - run: ${{ inputs.install-command }}
-      working-directory: ${{ inputs.working-directory }}
-      shell: bash

.github/workflows/linkcheck.yml

@@ -4,8 +4,6 @@ on:
   push:
     branches: [master]
   pull_request:
-    paths:
-      - 'docs/**'
 
 env:
   POETRY_VERSION: "1.4.2"

.github/workflows/test.yml

@@ -4,7 +4,6 @@ on:
   push:
     branches: [master]
   pull_request:
-  workflow_dispatch:
 
 env:
   POETRY_VERSION: "1.4.2"
@@ -19,31 +18,17 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
-        test_type:
-          - "core"
-          - "extended"
-    name: Python ${{ matrix.python-version }} ${{ matrix.test_type }}
     steps:
       - uses: actions/checkout@v3
+      - name: Install poetry
+        run: pipx install poetry==$POETRY_VERSION
       - name: Set up Python ${{ matrix.python-version }}
-        uses: "./.github/actions/poetry_setup"
+        uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
-          poetry-version: "1.4.2"
-          cache-key: ${{ matrix.test_type }}
-          install-command: |
-              if [ "${{ matrix.test_type }}" == "core" ]; then
-                echo "Running core tests, installing dependencies with poetry..."
-                poetry install
-              else
-                echo "Running extended tests, installing dependencies with poetry..."
-                poetry install -E extended_testing
-              fi
-      - name: Run ${{matrix.test_type}} tests
+          cache: "poetry"
+      - name: Install dependencies
+        run: poetry install
+      - name: Run unit tests
         run: |
-          if [ "${{ matrix.test_type }}" == "core" ]; then
-            make test
-          else
-            make extended_tests
-          fi
-        shell: bash
+          make test

.gitignore

@@ -1,4 +1,3 @@
-.vs/
 .vscode/
 .idea/
 # Byte-compiled / optimized / DLL files
@@ -145,11 +144,4 @@ wandb/
 /.ruff_cache/
 
 *.pkl
-*.bin
-
-# integration test artifacts
-data_map*
-\[('_type', 'fake'), ('stop', None)]
-
-# Replit files
-*replit*
+*.bin

.readthedocs.yaml

@@ -1,26 +0,0 @@
-# Read the Docs configuration file
-# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
-
-# Required
-version: 2
-
-# Set the version of Python and other tools you might need
-build:
-  os: ubuntu-22.04
-  tools:
-    python: "3.11"
-
-# Build documentation in the docs/ directory with Sphinx
-sphinx:
-   configuration: docs/conf.py
-
-# If using Sphinx, optionally build your docs in additional formats such as PDF
-# formats:
-#    - pdf
-
-# Optionally declare the Python requirements required to build your docs
-python:
-   install:
-   - requirements: docs/requirements.txt
-   - method: pip
-     path: .

Dockerfile

@@ -1,7 +1,5 @@
 # This is a Dockerfile for running unit tests
 
-ARG POETRY_HOME=/opt/poetry
-
 # Use the Python base image
 FROM python:3.11.2-bullseye AS builder
 
@@ -9,7 +7,7 @@ FROM python:3.11.2-bullseye AS builder
 ARG POETRY_VERSION=1.4.2
 
 # Define the directory to install Poetry to (default is /opt/poetry)
-ARG POETRY_HOME
+ARG POETRY_HOME=/opt/poetry
 
 # Create a Python virtual environment for Poetry and install it
 RUN python3 -m venv ${POETRY_HOME} && \
@@ -25,8 +23,6 @@ WORKDIR /app
 # Use a multi-stage build to install dependencies
 FROM builder AS dependencies
 
-ARG POETRY_HOME
-
 # Copy only the dependency files for installation
 COPY pyproject.toml poetry.lock poetry.toml ./
 

Makefile

@@ -1,4 +1,4 @@
-.PHONY: all clean format lint test tests test_watch integration_tests docker_tests help extended_tests
+.PHONY: all clean format lint test tests test_watch integration_tests docker_tests help
 
 all: help
 
@@ -32,16 +32,11 @@ lint lint_diff:
 	poetry run black $(PYTHON_FILES) --check
 	poetry run ruff .
 
-TEST_FILE ?= tests/unit_tests/
-
 test:
-	poetry run pytest --disable-socket --allow-unix-socket $(TEST_FILE)
+	poetry run pytest tests/unit_tests
 
-tests: 
-	poetry run pytest --disable-socket --allow-unix-socket $(TEST_FILE)
-
-extended_tests:
-	poetry run pytest --disable-socket --allow-unix-socket --only-extended tests/unit_tests
+tests:
+	poetry run pytest tests/unit_tests
 
 test_watch:
 	poetry run ptw --now . -- tests/unit_tests
@@ -55,16 +50,13 @@ docker_tests:
 
 help:
 	@echo '----'
-	@echo 'coverage                     - run unit tests and generate coverage report'
-	@echo 'docs_build                   - build the documentation'
-	@echo 'docs_clean                   - clean the documentation build artifacts'
-	@echo 'docs_linkcheck               - run linkchecker on the documentation'
-	@echo 'format                       - run code formatters'
-	@echo 'lint                         - run linters'
-	@echo 'test                         - run unit tests'
-	@echo 'tests                        - run unit tests'
-	@echo 'test TEST_FILE=<test_file>   - run all tests in file'
-	@echo 'extended_tests               - run only extended unit tests'
-	@echo 'test_watch                   - run unit tests in watch mode'
-	@echo 'integration_tests            - run integration tests'
-	@echo 'docker_tests                 - run unit tests in docker'
+	@echo 'coverage            - run unit tests and generate coverage report'
+	@echo 'docs_build          - build the documentation'
+	@echo 'docs_clean          - clean the documentation build artifacts'
+	@echo 'docs_linkcheck      - run linkchecker on the documentation'
+	@echo 'format              - run code formatters'
+	@echo 'lint                - run linters'
+	@echo 'test                - run unit tests'
+	@echo 'test_watch          - run unit tests in watch mode'
+	@echo 'integration_tests   - run integration tests'
+	@echo 'docker_tests        - run unit tests in docker'

README.md

@@ -2,22 +2,7 @@
 
 ⚡ Building applications with LLMs through composability ⚡
 
-[![Release Notes](https://img.shields.io/github/release/hwchase17/langchain)](https://github.com/hwchase17/langchain/releases)
-[![lint](https://github.com/hwchase17/langchain/actions/workflows/lint.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/lint.yml)
-[![test](https://github.com/hwchase17/langchain/actions/workflows/test.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/test.yml)
-[![linkcheck](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml)
-[![Downloads](https://static.pepy.tech/badge/langchain/month)](https://pepy.tech/project/langchain)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![Twitter](https://img.shields.io/twitter/url/https/twitter.com/langchainai.svg?style=social&label=Follow%20%40LangChainAI)](https://twitter.com/langchainai)
-[![](https://dcbadge.vercel.app/api/server/6adMQxSpJS?compact=true&style=flat)](https://discord.gg/6adMQxSpJS)
-[![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/hwchase17/langchain)
-[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hwchase17/langchain)
-[![GitHub star chart](https://img.shields.io/github/stars/hwchase17/langchain?style=social)](https://star-history.com/#hwchase17/langchain)
-[![Dependency Status](https://img.shields.io/librariesio/github/hwchase17/langchain)](https://libraries.io/github/hwchase17/langchain)
-[![Open Issues](https://img.shields.io/github/issues-raw/hwchase17/langchain)](https://github.com/hwchase17/langchain/issues)
-
-
-Looking for the JS/TS version? Check out [LangChain.js](https://github.com/hwchase17/langchainjs).
+[![lint](https://github.com/hwchase17/langchain/actions/workflows/lint.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/lint.yml) [![test](https://github.com/hwchase17/langchain/actions/workflows/test.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/test.yml) [![linkcheck](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml) [![Downloads](https://static.pepy.tech/badge/langchain/month)](https://pepy.tech/project/langchain) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Twitter](https://img.shields.io/twitter/url/https/twitter.com/langchainai.svg?style=social&label=Follow%20%40LangChainAI)](https://twitter.com/langchainai) [![](https://dcbadge.vercel.app/api/server/6adMQxSpJS?compact=true&style=flat)](https://discord.gg/6adMQxSpJS)
 
 **Production Support:** As you move your LangChains into production, we'd love to offer more comprehensive support.
 Please fill out [this form](https://forms.gle/57d8AmXBYp8PP8tZA) and we'll set up a dedicated support Slack channel.

docs/_static/css/custom.css

@@ -13,5 +13,5 @@ pre {
 }
 
 #my-component-root *, #headlessui-portal-root * {
-  z-index: 10000;
+  z-index: 1000000000000;
 }

docs/_static/js/mendablesearch.js

@@ -30,17 +30,18 @@ document.addEventListener('DOMContentLoaded', () => {
     const icon = React.createElement('p', {
       style: { color: '#ffffff', fontSize: '22px',width: '48px', height: '48px', margin: '0px', padding: '0px', display: 'flex', alignItems: 'center', justifyContent: 'center', textAlign: 'center' },
     }, [iconSpan1, iconSpan2]);
+
     
+    
+
     const mendableFloatingButton = React.createElement(
       MendableFloatingButton,
       {
         style: { darkMode: false, accentColor: '#010810' },
         floatingButtonStyle: { color: '#ffffff', backgroundColor: '#010810' },
         anon_key: '82842b36-3ea6-49b2-9fb8-52cfc4bde6bf', // Mendable Search Public ANON key, ok to be public
-        cmdShortcutKey:'j',
         messageSettings: {
           openSourcesInNewTab: false,
-          prettySources: true // Prettify the sources displayed now
         },
         icon: icon,
       }
@@ -51,7 +52,7 @@ document.addEventListener('DOMContentLoaded', () => {
 
   loadScript('https://unpkg.com/react@17/umd/react.production.min.js', () => {
     loadScript('https://unpkg.com/react-dom@17/umd/react-dom.production.min.js', () => {
-      loadScript('https://unpkg.com/@mendable/search@0.0.102/dist/umd/mendable.min.js', initializeMendable);
+      loadScript('https://unpkg.com/@mendable/search@0.0.83/dist/umd/mendable.min.js', initializeMendable);
     });
   });
 });

docs/additional_resources/deploy_llms.rst

@@ -1,137 +0,0 @@
-
-===========================
-Deploying LLMs in Production
-===========================
-
-In today's fast-paced technological landscape, the use of Large Language Models (LLMs) is rapidly expanding. As a result, it's crucial for developers to understand how to effectively deploy these models in production environments. LLM interfaces typically fall into two categories:
-
-- **Case 1: Utilizing External LLM Providers (OpenAI, Anthropic, etc.)**
-    In this scenario, most of the computational burden is handled by the LLM providers, while LangChain simplifies the implementation of business logic around these services. This approach includes features such as prompt templating, chat message generation, caching, vector embedding database creation, preprocessing, etc.
-
-- **Case 2: Self-hosted Open-Source Models**
-    Alternatively, developers can opt to use smaller, yet comparably capable, self-hosted open-source LLM models. This approach can significantly decrease costs, latency, and privacy concerns associated with transferring data to external LLM providers.
-
-Regardless of the framework that forms the backbone of your product, deploying LLM applications comes with its own set of challenges. It's vital to understand the trade-offs and key considerations when evaluating serving frameworks.
-
-Outline
-=======
-
-This guide aims to provide a comprehensive overview of the requirements for deploying LLMs in a production setting, focusing on:
-
-- `Designing a Robust LLM Application Service <#robust>`_
-- `Maintaining Cost-Efficiency <#cost>`_
-- `Ensuring Rapid Iteration <#iteration>`_
-
-Understanding these components is crucial when assessing serving systems. LangChain integrates with several open-source projects designed to tackle these issues, providing a robust framework for productionizing your LLM applications. Some notable frameworks include:
-
-- `Ray Serve <../integrations/ray_serve.html>`_
-- `BentoML <https://github.com/ssheng/BentoChain>`_
-- `Modal <../integrations/modal.html>`_
-
-These links will provide further information on each ecosystem, assisting you in finding the best fit for your LLM deployment needs.
-
-Designing a Robust LLM Application Service
-===========================================
-.. _robust:
-
-When deploying an LLM service in production, it's imperative to provide a seamless user experience free from outages. Achieving 24/7 service availability involves creating and maintaining several sub-systems surrounding your application.
-
-Monitoring
-----------
-
-Monitoring forms an integral part of any system running in a production environment. In the context of LLMs, it is essential to monitor both performance and quality metrics.
-
-**Performance Metrics:** These metrics provide insights into the efficiency and capacity of your model. Here are some key examples:
-
-- Query per second (QPS): This measures the number of queries your model processes in a second, offering insights into its utilization.
-- Latency: This metric quantifies the delay from when your client sends a request to when they receive a response.
-- Tokens Per Second (TPS): This represents the number of tokens your model can generate in a second.
-
-**Quality Metrics:** These metrics are typically customized according to the business use-case. For instance, how does the output of your system compare to a baseline, such as a previous version? Although these metrics can be calculated offline, you need to log the necessary data to use them later.
-
-Fault tolerance
----------------
-
-Your application may encounter errors such as exceptions in your model inference or business logic code, causing failures and disrupting traffic. Other potential issues could arise from the machine running your application, such as unexpected hardware breakdowns or loss of spot-instances during high-demand periods. One way to mitigate these risks is by increasing redundancy through replica scaling and implementing recovery mechanisms for failed replicas. However, model replicas aren't the only potential points of failure. It's essential to build resilience against various failures that could occur at any point in your stack.
-
-
-Zero down time upgrade
-----------------------
-
-System upgrades are often necessary but can result in service disruptions if not handled correctly. One way to prevent downtime during upgrades is by implementing a smooth transition process from the old version to the new one. Ideally, the new version of your LLM service is deployed, and traffic gradually shifts from the old to the new version, maintaining a constant QPS throughout the process.
-
-
-Load balancing
---------------
-
-Load balancing, in simple terms, is a technique to distribute work evenly across multiple computers, servers, or other resources to optimize the utilization of the system, maximize throughput, minimize response time, and avoid overload of any single resource. Think of it as a traffic officer directing cars (requests) to different roads (servers) so that no single road becomes too congested.
-
-There are several strategies for load balancing. For example, one common method is the *Round Robin* strategy, where each request is sent to the next server in line, cycling back to the first when all servers have received a request. This works well when all servers are equally capable. However, if some servers are more powerful than others, you might use a *Weighted Round Robin* or *Least Connections* strategy, where more requests are sent to the more powerful servers, or to those currently handling the fewest active requests. Let's imagine you're running a LLM chain. If your application becomes popular, you could have hundreds or even thousands of users asking questions at the same time. If one server gets too busy (high load), the load balancer would direct new requests to another server that is less busy. This way, all your users get a timely response and the system remains stable.
-
-
-
-Maintaining Cost-Efficiency and Scalability
-============================================
-.. _cost:
-
-Deploying LLM services can be costly, especially when you're handling a large volume of user interactions. Charges by LLM providers are usually based on tokens used, making a chat system inference on these models potentially expensive. However, several strategies can help manage these costs without compromising the quality of the service.
-
-
-Self-hosting models
--------------------
-
-Several smaller and open-source LLMs are emerging to tackle the issue of reliance on LLM providers. Self-hosting allows you to maintain similar quality to LLM provider models while managing costs. The challenge lies in building a reliable, high-performing LLM serving system on your own machines. 
-
-Resource Management and Auto-Scaling
-------------------------------------
-
-Computational logic within your application requires precise resource allocation. For instance, if part of your traffic is served by an OpenAI endpoint and another part by a self-hosted model, it's crucial to allocate suitable resources for each. Auto-scaling—adjusting resource allocation based on traffic—can significantly impact the cost of running your application. This strategy requires a balance between cost and responsiveness, ensuring neither resource over-provisioning nor compromised application responsiveness.
-
-Utilizing Spot Instances
-------------------------
-
-On platforms like AWS, spot instances offer substantial cost savings, typically priced at about a third of on-demand instances. The trade-off is a higher crash rate, necessitating a robust fault-tolerance mechanism for effective use.
-
-Independent Scaling
--------------------
-
-When self-hosting your models, you should consider independent scaling. For example, if you have two translation models, one fine-tuned for French and another for Spanish, incoming requests might necessitate different scaling requirements for each.
-
-Batching requests
------------------
-
-In the context of Large Language Models, batching requests can enhance efficiency by better utilizing your GPU resources. GPUs are inherently parallel processors, designed to handle multiple tasks simultaneously. If you send individual requests to the model, the GPU might not be fully utilized as it's only working on a single task at a time. On the other hand, by batching requests together, you're allowing the GPU to work on multiple tasks at once, maximizing its utilization and improving inference speed. This not only leads to cost savings but can also improve the overall latency of your LLM service.
-
-
-In summary, managing costs while scaling your LLM services requires a strategic approach. Utilizing self-hosting models, managing resources effectively, employing auto-scaling, using spot instances, independently scaling models, and batching requests are key strategies to consider. Open-source libraries such as Ray Serve and BentoML are designed to deal with these complexities. 
-
-
-
-Ensuring Rapid Iteration
-========================
-
-.. _iteration:
-
-The LLM landscape is evolving at an unprecedented pace, with new libraries and model architectures being introduced constantly. Consequently, it's crucial to avoid tying yourself to a solution specific to one particular framework. This is especially relevant in serving, where changes to your infrastructure can be time-consuming, expensive, and risky. Strive for infrastructure that is not locked into any specific machine learning library or framework, but instead offers a general-purpose, scalable serving layer. Here are some aspects where flexibility plays a key role:
-
-Model composition
------------------
-
-Deploying systems like LangChain demands the ability to piece together different models and connect them via logic. Take the example of building a natural language input SQL query engine. Querying an LLM and obtaining the SQL command is only part of the system. You need to extract metadata from the connected database, construct a prompt for the LLM, run the SQL query on an engine, collect and feed back the response to the LLM as the query runs, and present the results to the user. This demonstrates the need to seamlessly integrate various complex components built in Python into a dynamic chain of logical blocks that can be served together.
-
-Cloud providers
----------------
-
-Many hosted solutions are restricted to a single cloud provider, which can limit your options in today's multi-cloud world. Depending on where your other infrastructure components are built, you might prefer to stick with your chosen cloud provider.
-
-
-Infrastructure as Code (IaC)
----------------------------
-
-Rapid iteration also involves the ability to recreate your infrastructure quickly and reliably. This is where Infrastructure as Code (IaC) tools like Terraform, CloudFormation, or Kubernetes YAML files come into play. They allow you to define your infrastructure in code files, which can be version controlled and quickly deployed, enabling faster and more reliable iterations.
-
-
-CI/CD
------
-
-In a fast-paced environment, implementing CI/CD pipelines can significantly speed up the iteration process. They help automate the testing and deployment of your LLM applications, reducing the risk of errors and enabling faster feedback and iteration.

docs/additional_resources/youtube.md

@@ -1,90 +0,0 @@
-# YouTube
-
-This is a collection of `LangChain` videos on `YouTube`.
-
-### ⛓️[Official LangChain YouTube channel](https://www.youtube.com/@LangChain)⛓️
-
-### Introduction to LangChain with Harrison Chase, creator of LangChain
-- [Building the Future with LLMs, `LangChain`, & `Pinecone`](https://youtu.be/nMniwlGyX-c) by [Pinecone](https://www.youtube.com/@pinecone-io)
-- [LangChain and Weaviate with Harrison Chase and Bob van Luijt - Weaviate Podcast #36](https://youtu.be/lhby7Ql7hbk) by [Weaviate • Vector Database](https://www.youtube.com/@Weaviate)
-- [LangChain Demo + Q&A with Harrison Chase](https://youtu.be/zaYTXQFR0_s?t=788) by [Full Stack Deep Learning](https://www.youtube.com/@FullStackDeepLearning)
-- [LangChain Agents: Build Personal Assistants For Your Data (Q&A with Harrison Chase and Mayo Oshin)](https://youtu.be/gVkF8cwfBLI) by [Chat with data](https://www.youtube.com/@chatwithdata)
-- ⛓️ [LangChain "Agents in Production" Webinar](https://youtu.be/k8GNCCs16F4) by [LangChain](https://www.youtube.com/@LangChain)
-
-## Videos (sorted by views)
-
-- [Building AI LLM Apps with LangChain (and more?) - LIVE STREAM](https://www.youtube.com/live/M-2Cj_2fzWI?feature=share) by [Nicholas Renotte](https://www.youtube.com/@NicholasRenotte)
-- [First look - `ChatGPT` + `WolframAlpha` (`GPT-3.5` and Wolfram|Alpha via LangChain by James Weaver)](https://youtu.be/wYGbY811oMo) by [Dr Alan D. Thompson](https://www.youtube.com/@DrAlanDThompson) 
-- [LangChain explained - The hottest new Python framework](https://youtu.be/RoR4XJw8wIc) by [AssemblyAI](https://www.youtube.com/@AssemblyAI)
-- [Chatbot with INFINITE MEMORY using `OpenAI` & `Pinecone` - `GPT-3`, `Embeddings`, `ADA`, `Vector DB`, `Semantic`](https://youtu.be/2xNzB7xq8nk) by [David Shapiro ~ AI](https://www.youtube.com/@DavidShapiroAutomator)
-- [LangChain for LLMs is... basically just an Ansible playbook](https://youtu.be/X51N9C-OhlE) by [David Shapiro ~ AI](https://www.youtube.com/@DavidShapiroAutomator)
-- [Build your own LLM Apps with LangChain & `GPT-Index`](https://youtu.be/-75p09zFUJY) by [1littlecoder](https://www.youtube.com/@1littlecoder)
-- [`BabyAGI` - New System of Autonomous AI Agents with LangChain](https://youtu.be/lg3kJvf1kXo) by [1littlecoder](https://www.youtube.com/@1littlecoder)
-- [Run `BabyAGI` with Langchain Agents (with Python Code)](https://youtu.be/WosPGHPObx8) by [1littlecoder](https://www.youtube.com/@1littlecoder)
-- [How to Use Langchain With `Zapier` | Write and Send Email with GPT-3 | OpenAI API Tutorial](https://youtu.be/p9v2-xEa9A0) by [StarMorph AI](https://www.youtube.com/@starmorph)
-- [Use Your Locally Stored Files To Get Response From GPT - `OpenAI` | Langchain | Python](https://youtu.be/NC1Ni9KS-rk) by [Shweta Lodha](https://www.youtube.com/@shweta-lodha)
-- [`Langchain JS` | How to Use GPT-3, GPT-4 to Reference your own Data | `OpenAI Embeddings` Intro](https://youtu.be/veV2I-NEjaM) by [StarMorph AI](https://www.youtube.com/@starmorph)
-- [The easiest way to work with large language models | Learn LangChain in 10min](https://youtu.be/kmbS6FDQh7c) by [Sophia Yang](https://www.youtube.com/@SophiaYangDS)
-- [4 Autonomous AI Agents: “Westworld” simulation `BabyAGI`, `AutoGPT`, `Camel`, `LangChain`](https://youtu.be/yWbnH6inT_U) by [Sophia Yang](https://www.youtube.com/@SophiaYangDS)
-- [AI CAN SEARCH THE INTERNET? Langchain Agents + OpenAI ChatGPT](https://youtu.be/J-GL0htqda8) by [tylerwhatsgood](https://www.youtube.com/@tylerwhatsgood)
-- [Query Your Data with GPT-4 | Embeddings, Vector Databases | Langchain JS Knowledgebase](https://youtu.be/jRnUPUTkZmU) by [StarMorph AI](https://www.youtube.com/@starmorph)
-- [`Weaviate` + LangChain for LLM apps presented by Erika Cardenas](https://youtu.be/7AGj4Td5Lgw) by [`Weaviate` • Vector Database](https://www.youtube.com/@Weaviate)
-- [Langchain Overview — How to Use Langchain & `ChatGPT`](https://youtu.be/oYVYIq0lOtI) by [Python In Office](https://www.youtube.com/@pythoninoffice6568)
-- [Langchain Overview - How to Use Langchain & `ChatGPT`](https://youtu.be/oYVYIq0lOtI) by [Python In Office](https://www.youtube.com/@pythoninoffice6568)
-- [Custom langchain Agent & Tools with memory. Turn any `Python function` into langchain tool with Gpt 3](https://youtu.be/NIG8lXk0ULg) by [echohive](https://www.youtube.com/@echohive)
-- [LangChain: Run Language Models Locally - `Hugging Face Models`](https://youtu.be/Xxxuw4_iCzw) by [Prompt Engineering](https://www.youtube.com/@engineerprompt)
-- [`ChatGPT` with any `YouTube` video using langchain and `chromadb`](https://youtu.be/TQZfB2bzVwU) by [echohive](https://www.youtube.com/@echohive)
-- [How to Talk to a `PDF` using LangChain and `ChatGPT`](https://youtu.be/v2i1YDtrIwk) by [Automata Learning Lab](https://www.youtube.com/@automatalearninglab)
-- [Langchain Document Loaders Part 1: Unstructured Files](https://youtu.be/O5C0wfsen98) by [Merk](https://www.youtube.com/@merksworld) 
-- [LangChain - Prompt Templates (what all the best prompt engineers use)](https://youtu.be/1aRu8b0XNOQ) by [Nick Daigler](https://www.youtube.com/@nick_daigs)
-- [LangChain. Crear aplicaciones Python impulsadas por GPT](https://youtu.be/DkW_rDndts8) by [Jesús Conde](https://www.youtube.com/@0utKast)
-- [Easiest Way to Use GPT In Your Products | LangChain Basics Tutorial](https://youtu.be/fLy0VenZyGc) by [Rachel Woods](https://www.youtube.com/@therachelwoods)
-- [`BabyAGI` + `GPT-4` Langchain Agent with Internet Access](https://youtu.be/wx1z_hs5P6E) by [tylerwhatsgood](https://www.youtube.com/@tylerwhatsgood)
-- [Learning LLM Agents. How does it actually work? LangChain, AutoGPT & OpenAI](https://youtu.be/mb_YAABSplk) by [Arnoldas Kemeklis](https://www.youtube.com/@processusAI)
-- [Get Started with LangChain in `Node.js`](https://youtu.be/Wxx1KUWJFv4) by [Developers Digest](https://www.youtube.com/@DevelopersDigest)
-- [LangChain + `OpenAI` tutorial: Building a Q&A system w/ own text data](https://youtu.be/DYOU_Z0hAwo) by [Samuel Chan](https://www.youtube.com/@SamuelChan)
-- [Langchain + `Zapier` Agent](https://youtu.be/yribLAb-pxA) by [Merk](https://www.youtube.com/@merksworld)
-- [Connecting the Internet with `ChatGPT` (LLMs) using Langchain And Answers Your Questions](https://youtu.be/9Y0TBC63yZg) by [Kamalraj M M](https://www.youtube.com/@insightbuilder)
-- [Build More Powerful LLM Applications for Business’s with LangChain (Beginners Guide)](https://youtu.be/sp3-WLKEcBg) by[ No Code Blackbox](https://www.youtube.com/@nocodeblackbox)
-- ⛓️ [LangFlow LLM Agent Demo for 🦜🔗LangChain](https://youtu.be/zJxDHaWt-6o) by [Cobus Greyling](https://www.youtube.com/@CobusGreylingZA)
-- ⛓️ [Chatbot Factory: Streamline Python Chatbot Creation with LLMs and Langchain](https://youtu.be/eYer3uzrcuM) by [Finxter](https://www.youtube.com/@CobusGreylingZA)
-- ⛓️ [LangChain Tutorial - ChatGPT mit eigenen Daten](https://youtu.be/0XDLyY90E2c) by [Coding Crashkurse](https://www.youtube.com/@codingcrashkurse6429)
-- ⛓️ [Chat with a `CSV` | LangChain Agents Tutorial (Beginners)](https://youtu.be/tjeti5vXWOU) by [GoDataProf](https://www.youtube.com/@godataprof)
-- ⛓️ [Introdução ao Langchain - #Cortes - Live DataHackers](https://youtu.be/fw8y5VRei5Y) by [Prof. João Gabriel Lima](https://www.youtube.com/@profjoaogabriellima)
-- ⛓️ [LangChain: Level up `ChatGPT` !? | LangChain Tutorial Part 1](https://youtu.be/vxUGx8aZpDE) by [Code Affinity](https://www.youtube.com/@codeaffinitydev)
-- ⛓️ [KI schreibt krasses Youtube Skript 😲😳 | LangChain Tutorial Deutsch](https://youtu.be/QpTiXyK1jus) by [SimpleKI](https://www.youtube.com/@simpleki)
-- ⛓️ [Chat with Audio: Langchain, `Chroma DB`, OpenAI, and `Assembly AI`](https://youtu.be/Kjy7cx1r75g) by [AI Anytime](https://www.youtube.com/@AIAnytime)
-- ⛓️ [QA over documents with Auto vector index selection with Langchain router chains](https://youtu.be/9G05qybShv8) by [echohive](https://www.youtube.com/@echohive)
-- ⛓️ [Build your own custom LLM application with `Bubble.io` & Langchain (No Code & Beginner friendly)](https://youtu.be/O7NhQGu1m6c) by [No Code Blackbox](https://www.youtube.com/@nocodeblackbox)
-- ⛓️ [Simple App to Question Your Docs: Leveraging `Streamlit`, `Hugging Face Spaces`, LangChain, and `Claude`!](https://youtu.be/X4YbNECRr7o) by [Chris Alexiuk](https://www.youtube.com/@chrisalexiuk)
-- ⛓️ [LANGCHAIN AI- `ConstitutionalChainAI` + Databutton AI ASSISTANT Web App](https://youtu.be/5zIU6_rdJCU) by [Avra](https://www.youtube.com/@Avra_b)
-- ⛓️ [LANGCHAIN AI AUTONOMOUS AGENT WEB APP - 👶 `BABY AGI` 🤖 with EMAIL AUTOMATION using `DATABUTTON`](https://youtu.be/cvAwOGfeHgw) by [Avra](https://www.youtube.com/@Avra_b)
-- ⛓️ [The Future of Data Analysis: Using A.I. Models in Data Analysis (LangChain)](https://youtu.be/v_LIcVyg5dk) by [Absent Data](https://www.youtube.com/@absentdata)
-- ⛓️ [Memory in LangChain | Deep dive (python)](https://youtu.be/70lqvTFh_Yg) by [Eden Marco](https://www.youtube.com/@EdenMarco)
-- ⛓️ [9 LangChain UseCases | Beginner's Guide | 2023](https://youtu.be/zS8_qosHNMw) by [Data Science Basics](https://www.youtube.com/@datasciencebasics)
-- ⛓️ [Use Large Language Models in Jupyter Notebook | LangChain | Agents & Indexes](https://youtu.be/JSe11L1a_QQ) by [Abhinaw Tiwari](https://www.youtube.com/@AbhinawTiwariAT)
-- ⛓️ [How to Talk to Your Langchain Agent | `11 Labs` + `Whisper`](https://youtu.be/N4k459Zw2PU) by [VRSEN](https://www.youtube.com/@vrsen)
-- ⛓️ [LangChain Deep Dive: 5 FUN AI App Ideas To Build Quickly and Easily](https://youtu.be/mPYEPzLkeks) by [James NoCode](https://www.youtube.com/@jamesnocode)
-- ⛓️ [BEST OPEN Alternative to OPENAI's EMBEDDINGs for Retrieval QA: LangChain](https://youtu.be/ogEalPMUCSY) by [Prompt Engineering](https://www.youtube.com/@engineerprompt)
-- ⛓️ [LangChain 101: Models](https://youtu.be/T6c_XsyaNSQ) by [Mckay Wrigley](https://www.youtube.com/@realmckaywrigley)
-- ⛓️ [LangChain with JavaScript Tutorial #1 | Setup & Using LLMs](https://youtu.be/W3AoeMrg27o) by [Leon van Zyl](https://www.youtube.com/@leonvanzyl)
-- ⛓️ [LangChain Overview & Tutorial for Beginners: Build Powerful AI Apps Quickly & Easily (ZERO CODE)](https://youtu.be/iI84yym473Q) by [James NoCode](https://www.youtube.com/@jamesnocode)
-- ⛓️ [LangChain In Action: Real-World Use Case With Step-by-Step Tutorial](https://youtu.be/UO699Szp82M) by [Rabbitmetrics](https://www.youtube.com/@rabbitmetrics)
-- ⛓️ [Summarizing and Querying Multiple Papers with LangChain](https://youtu.be/p_MQRWH5Y6k) by [Automata Learning Lab](https://www.youtube.com/@automatalearninglab)
-- ⛓️ [Using Langchain (and `Replit`) through `Tana`, ask `Google`/`Wikipedia`/`Wolfram Alpha` to fill out a table](https://youtu.be/Webau9lEzoI) by [Stian Håklev](https://www.youtube.com/@StianHaklev)
-- ⛓️ [Langchain PDF App (GUI) | Create a ChatGPT For Your `PDF` in Python](https://youtu.be/wUAUdEw5oxM) by [Alejandro AO - Software & Ai](https://www.youtube.com/@alejandro_ao)
-- ⛓️ [Auto-GPT with LangChain 🔥 | Create Your Own Personal AI Assistant](https://youtu.be/imDfPmMKEjM) by [Data Science Basics](https://www.youtube.com/@datasciencebasics)
-- ⛓️ [Create Your OWN Slack AI Assistant with Python & LangChain](https://youtu.be/3jFXRNn2Bu8) by [Dave Ebbelaar](https://www.youtube.com/@daveebbelaar)
-- ⛓️ [How to Create LOCAL Chatbots with GPT4All and LangChain [Full Guide]](https://youtu.be/4p1Fojur8Zw) by [Liam Ottley](https://www.youtube.com/@LiamOttley)
-- ⛓️ [Build a `Multilingual PDF` Search App with LangChain, `Cohere` and `Bubble`](https://youtu.be/hOrtuumOrv8) by [Menlo Park Lab](https://www.youtube.com/@menloparklab)
-- ⛓️ [Building a LangChain Agent (code-free!) Using `Bubble` and `Flowise`](https://youtu.be/jDJIIVWTZDE) by [Menlo Park Lab](https://www.youtube.com/@menloparklab)
-- ⛓️ [Build a LangChain-based Semantic PDF Search App with No-Code Tools Bubble and Flowise](https://youtu.be/s33v5cIeqA4) by [Menlo Park Lab](https://www.youtube.com/@menloparklab)
-- ⛓️ [LangChain Memory Tutorial | Building a ChatGPT Clone in Python](https://youtu.be/Cwq91cj2Pnc) by [Alejandro AO - Software & Ai](https://www.youtube.com/@alejandro_ao)
-- ⛓️ [ChatGPT For Your DATA | Chat with Multiple Documents Using LangChain](https://youtu.be/TeDgIDqQmzs) by [Data Science Basics](https://www.youtube.com/@datasciencebasics)
-- ⛓️ [`Llama Index`: Chat with Documentation using URL Loader](https://youtu.be/XJRoDEctAwA) by [Merk](https://www.youtube.com/@merksworld)
-- ⛓️ [Using OpenAI, LangChain, and `Gradio` to Build Custom GenAI Applications](https://youtu.be/1MsmqMg3yUc) by [David Hundley](https://www.youtube.com/@dkhundley)
-
-
-
----------------------
-⛓ icon marks a new video [last update 2023-05-15]

docs/dependents.md

@@ -1,231 +0,0 @@
-# Dependents
-
-Dependents stats for `hwchase17/langchain`
-
-[![](https://img.shields.io/static/v1?label=Used%20by&message=7484&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
-[![](https://img.shields.io/static/v1?label=Used%20by%20(public)&message=212&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
-[![](https://img.shields.io/static/v1?label=Used%20by%20(private)&message=7272&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
-[![](https://img.shields.io/static/v1?label=Used%20by%20(stars)&message=19095&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
-
-[update: 2023-06-05; only dependent repositories with Stars > 100]
-
-
-| Repository | Stars  |
-| :--------  | -----: |
-|[openai/openai-cookbook](https://github.com/openai/openai-cookbook) | 38024 |
-|[LAION-AI/Open-Assistant](https://github.com/LAION-AI/Open-Assistant) | 33609 |
-|[microsoft/TaskMatrix](https://github.com/microsoft/TaskMatrix) | 33136 |
-|[hpcaitech/ColossalAI](https://github.com/hpcaitech/ColossalAI) | 30032 |
-|[imartinez/privateGPT](https://github.com/imartinez/privateGPT) | 28094 |
-|[reworkd/AgentGPT](https://github.com/reworkd/AgentGPT) | 23430 |
-|[openai/chatgpt-retrieval-plugin](https://github.com/openai/chatgpt-retrieval-plugin) | 17942 |
-|[jerryjliu/llama_index](https://github.com/jerryjliu/llama_index) | 16697 |
-|[mindsdb/mindsdb](https://github.com/mindsdb/mindsdb) | 16410 |
-|[mlflow/mlflow](https://github.com/mlflow/mlflow) | 14517 |
-|[GaiZhenbiao/ChuanhuChatGPT](https://github.com/GaiZhenbiao/ChuanhuChatGPT) | 10793 |
-|[databrickslabs/dolly](https://github.com/databrickslabs/dolly) | 10155 |
-|[openai/evals](https://github.com/openai/evals) | 10076 |
-|[AIGC-Audio/AudioGPT](https://github.com/AIGC-Audio/AudioGPT) | 8619 |
-|[logspace-ai/langflow](https://github.com/logspace-ai/langflow) | 8211 |
-|[imClumsyPanda/langchain-ChatGLM](https://github.com/imClumsyPanda/langchain-ChatGLM) | 8154 |
-|[PromtEngineer/localGPT](https://github.com/PromtEngineer/localGPT) | 6853 |
-|[StanGirard/quivr](https://github.com/StanGirard/quivr) | 6830 |
-|[PipedreamHQ/pipedream](https://github.com/PipedreamHQ/pipedream) | 6520 |
-|[go-skynet/LocalAI](https://github.com/go-skynet/LocalAI) | 6018 |
-|[arc53/DocsGPT](https://github.com/arc53/DocsGPT) | 5643 |
-|[e2b-dev/e2b](https://github.com/e2b-dev/e2b) | 5075 |
-|[langgenius/dify](https://github.com/langgenius/dify) | 4281 |
-|[nsarrazin/serge](https://github.com/nsarrazin/serge) | 4228 |
-|[zauberzeug/nicegui](https://github.com/zauberzeug/nicegui) | 4084 |
-|[madawei2699/myGPTReader](https://github.com/madawei2699/myGPTReader) | 4039 |
-|[wenda-LLM/wenda](https://github.com/wenda-LLM/wenda) | 3871 |
-|[GreyDGL/PentestGPT](https://github.com/GreyDGL/PentestGPT) | 3837 |
-|[zilliztech/GPTCache](https://github.com/zilliztech/GPTCache) | 3625 |
-|[csunny/DB-GPT](https://github.com/csunny/DB-GPT) | 3545 |
-|[gkamradt/langchain-tutorials](https://github.com/gkamradt/langchain-tutorials) | 3404 |
-|[mmabrouk/chatgpt-wrapper](https://github.com/mmabrouk/chatgpt-wrapper) | 3303 |
-|[postgresml/postgresml](https://github.com/postgresml/postgresml) | 3052 |
-|[marqo-ai/marqo](https://github.com/marqo-ai/marqo) | 3014 |
-|[MineDojo/Voyager](https://github.com/MineDojo/Voyager) | 2945 |
-|[PrefectHQ/marvin](https://github.com/PrefectHQ/marvin) | 2761 |
-|[project-baize/baize-chatbot](https://github.com/project-baize/baize-chatbot) | 2673 |
-|[hwchase17/chat-langchain](https://github.com/hwchase17/chat-langchain) | 2589 |
-|[whitead/paper-qa](https://github.com/whitead/paper-qa) | 2572 |
-|[Azure-Samples/azure-search-openai-demo](https://github.com/Azure-Samples/azure-search-openai-demo) | 2366 |
-|[GerevAI/gerev](https://github.com/GerevAI/gerev) | 2330 |
-|[OpenGVLab/InternGPT](https://github.com/OpenGVLab/InternGPT) | 2289 |
-|[ParisNeo/gpt4all-ui](https://github.com/ParisNeo/gpt4all-ui) | 2159 |
-|[OpenBMB/BMTools](https://github.com/OpenBMB/BMTools) | 2158 |
-|[guangzhengli/ChatFiles](https://github.com/guangzhengli/ChatFiles) | 2005 |
-|[h2oai/h2ogpt](https://github.com/h2oai/h2ogpt) | 1939 |
-|[Farama-Foundation/PettingZoo](https://github.com/Farama-Foundation/PettingZoo) | 1845 |
-|[OpenGVLab/Ask-Anything](https://github.com/OpenGVLab/Ask-Anything) | 1749 |
-|[IntelligenzaArtificiale/Free-Auto-GPT](https://github.com/IntelligenzaArtificiale/Free-Auto-GPT) | 1740 |
-|[Unstructured-IO/unstructured](https://github.com/Unstructured-IO/unstructured) | 1628 |
-|[hwchase17/notion-qa](https://github.com/hwchase17/notion-qa) | 1607 |
-|[NVIDIA/NeMo-Guardrails](https://github.com/NVIDIA/NeMo-Guardrails) | 1544 |
-|[SamurAIGPT/privateGPT](https://github.com/SamurAIGPT/privateGPT) | 1543 |
-|[paulpierre/RasaGPT](https://github.com/paulpierre/RasaGPT) | 1526 |
-|[yanqiangmiffy/Chinese-LangChain](https://github.com/yanqiangmiffy/Chinese-LangChain) | 1485 |
-|[Kav-K/GPTDiscord](https://github.com/Kav-K/GPTDiscord) | 1402 |
-|[vocodedev/vocode-python](https://github.com/vocodedev/vocode-python) | 1387 |
-|[Chainlit/chainlit](https://github.com/Chainlit/chainlit) | 1336 |
-|[lunasec-io/lunasec](https://github.com/lunasec-io/lunasec) | 1323 |
-|[psychic-api/psychic](https://github.com/psychic-api/psychic) | 1248 |
-|[agiresearch/OpenAGI](https://github.com/agiresearch/OpenAGI) | 1208 |
-|[jina-ai/thinkgpt](https://github.com/jina-ai/thinkgpt) | 1193 |
-|[thomas-yanxin/LangChain-ChatGLM-Webui](https://github.com/thomas-yanxin/LangChain-ChatGLM-Webui) | 1182 |
-|[ttengwang/Caption-Anything](https://github.com/ttengwang/Caption-Anything) | 1137 |
-|[jina-ai/dev-gpt](https://github.com/jina-ai/dev-gpt) | 1135 |
-|[greshake/llm-security](https://github.com/greshake/llm-security) | 1086 |
-|[keephq/keep](https://github.com/keephq/keep) | 1063 |
-|[juncongmoo/chatllama](https://github.com/juncongmoo/chatllama) | 1037 |
-|[richardyc/Chrome-GPT](https://github.com/richardyc/Chrome-GPT) | 1035 |
-|[visual-openllm/visual-openllm](https://github.com/visual-openllm/visual-openllm) | 997 |
-|[mmz-001/knowledge_gpt](https://github.com/mmz-001/knowledge_gpt) | 995 |
-|[jina-ai/langchain-serve](https://github.com/jina-ai/langchain-serve) | 949 |
-|[irgolic/AutoPR](https://github.com/irgolic/AutoPR) | 936 |
-|[microsoft/X-Decoder](https://github.com/microsoft/X-Decoder) | 908 |
-|[poe-platform/api-bot-tutorial](https://github.com/poe-platform/api-bot-tutorial) | 902 |
-|[peterw/Chat-with-Github-Repo](https://github.com/peterw/Chat-with-Github-Repo) | 875 |
-|[cirediatpl/FigmaChain](https://github.com/cirediatpl/FigmaChain) | 822 |
-|[homanp/superagent](https://github.com/homanp/superagent) | 806 |
-|[seanpixel/Teenage-AGI](https://github.com/seanpixel/Teenage-AGI) | 800 |
-|[chatarena/chatarena](https://github.com/chatarena/chatarena) | 796 |
-|[hashintel/hash](https://github.com/hashintel/hash) | 795 |
-|[SamurAIGPT/Camel-AutoGPT](https://github.com/SamurAIGPT/Camel-AutoGPT) | 786 |
-|[rlancemartin/auto-evaluator](https://github.com/rlancemartin/auto-evaluator) | 770 |
-|[corca-ai/EVAL](https://github.com/corca-ai/EVAL) | 769 |
-|[101dotxyz/GPTeam](https://github.com/101dotxyz/GPTeam) | 755 |
-|[noahshinn024/reflexion](https://github.com/noahshinn024/reflexion) | 706 |
-|[eyurtsev/kor](https://github.com/eyurtsev/kor) | 695 |
-|[cheshire-cat-ai/core](https://github.com/cheshire-cat-ai/core) | 681 |
-|[e-johnstonn/BriefGPT](https://github.com/e-johnstonn/BriefGPT) | 656 |
-|[run-llama/llama-lab](https://github.com/run-llama/llama-lab) | 635 |
-|[griptape-ai/griptape](https://github.com/griptape-ai/griptape) | 583 |
-|[namuan/dr-doc-search](https://github.com/namuan/dr-doc-search) | 555 |
-|[getmetal/motorhead](https://github.com/getmetal/motorhead) | 550 |
-|[kreneskyp/ix](https://github.com/kreneskyp/ix) | 543 |
-|[hwchase17/chat-your-data](https://github.com/hwchase17/chat-your-data) | 510 |
-|[Anil-matcha/ChatPDF](https://github.com/Anil-matcha/ChatPDF) | 501 |
-|[whyiyhw/chatgpt-wechat](https://github.com/whyiyhw/chatgpt-wechat) | 497 |
-|[SamurAIGPT/ChatGPT-Developer-Plugins](https://github.com/SamurAIGPT/ChatGPT-Developer-Plugins) | 496 |
-|[microsoft/PodcastCopilot](https://github.com/microsoft/PodcastCopilot) | 492 |
-|[debanjum/khoj](https://github.com/debanjum/khoj) | 485 |
-|[akshata29/chatpdf](https://github.com/akshata29/chatpdf) | 485 |
-|[langchain-ai/langchain-aiplugin](https://github.com/langchain-ai/langchain-aiplugin) | 462 |
-|[jina-ai/agentchain](https://github.com/jina-ai/agentchain) | 460 |
-|[alexanderatallah/window.ai](https://github.com/alexanderatallah/window.ai) | 457 |
-|[yeagerai/yeagerai-agent](https://github.com/yeagerai/yeagerai-agent) | 451 |
-|[mckaywrigley/repo-chat](https://github.com/mckaywrigley/repo-chat) | 446 |
-|[michaelthwan/searchGPT](https://github.com/michaelthwan/searchGPT) | 446 |
-|[mpaepper/content-chatbot](https://github.com/mpaepper/content-chatbot) | 441 |
-|[freddyaboulton/gradio-tools](https://github.com/freddyaboulton/gradio-tools) | 439 |
-|[ruoccofabrizio/azure-open-ai-embeddings-qna](https://github.com/ruoccofabrizio/azure-open-ai-embeddings-qna) | 429 |
-|[StevenGrove/GPT4Tools](https://github.com/StevenGrove/GPT4Tools) | 422 |
-|[jonra1993/fastapi-alembic-sqlmodel-async](https://github.com/jonra1993/fastapi-alembic-sqlmodel-async) | 407 |
-|[msoedov/langcorn](https://github.com/msoedov/langcorn) | 405 |
-|[amosjyng/langchain-visualizer](https://github.com/amosjyng/langchain-visualizer) | 395 |
-|[ajndkr/lanarky](https://github.com/ajndkr/lanarky) | 384 |
-|[mtenenholtz/chat-twitter](https://github.com/mtenenholtz/chat-twitter) | 376 |
-|[steamship-core/steamship-langchain](https://github.com/steamship-core/steamship-langchain) | 371 |
-|[langchain-ai/auto-evaluator](https://github.com/langchain-ai/auto-evaluator) | 365 |
-|[xuwenhao/geektime-ai-course](https://github.com/xuwenhao/geektime-ai-course) | 358 |
-|[continuum-llms/chatgpt-memory](https://github.com/continuum-llms/chatgpt-memory) | 357 |
-|[opentensor/bittensor](https://github.com/opentensor/bittensor) | 347 |
-|[showlab/VLog](https://github.com/showlab/VLog) | 345 |
-|[daodao97/chatdoc](https://github.com/daodao97/chatdoc) | 345 |
-|[logan-markewich/llama_index_starter_pack](https://github.com/logan-markewich/llama_index_starter_pack) | 332 |
-|[poe-platform/poe-protocol](https://github.com/poe-platform/poe-protocol) | 320 |
-|[explosion/spacy-llm](https://github.com/explosion/spacy-llm) | 312 |
-|[andylokandy/gpt-4-search](https://github.com/andylokandy/gpt-4-search) | 311 |
-|[alejandro-ao/langchain-ask-pdf](https://github.com/alejandro-ao/langchain-ask-pdf) | 310 |
-|[jupyterlab/jupyter-ai](https://github.com/jupyterlab/jupyter-ai) | 294 |
-|[BlackHC/llm-strategy](https://github.com/BlackHC/llm-strategy) | 283 |
-|[itamargol/openai](https://github.com/itamargol/openai) | 281 |
-|[momegas/megabots](https://github.com/momegas/megabots) | 279 |
-|[personoids/personoids-lite](https://github.com/personoids/personoids-lite) | 277 |
-|[yvann-hub/Robby-chatbot](https://github.com/yvann-hub/Robby-chatbot) | 267 |
-|[Anil-matcha/Website-to-Chatbot](https://github.com/Anil-matcha/Website-to-Chatbot) | 266 |
-|[Cheems-Seminar/grounded-segment-any-parts](https://github.com/Cheems-Seminar/grounded-segment-any-parts) | 260 |
-|[sullivan-sean/chat-langchainjs](https://github.com/sullivan-sean/chat-langchainjs) | 248 |
-|[bborn/howdoi.ai](https://github.com/bborn/howdoi.ai) | 245 |
-|[daveebbelaar/langchain-experiments](https://github.com/daveebbelaar/langchain-experiments) | 240 |
-|[MagnivOrg/prompt-layer-library](https://github.com/MagnivOrg/prompt-layer-library) | 237 |
-|[ur-whitelab/exmol](https://github.com/ur-whitelab/exmol) | 234 |
-|[conceptofmind/toolformer](https://github.com/conceptofmind/toolformer) | 234 |
-|[recalign/RecAlign](https://github.com/recalign/RecAlign) | 226 |
-|[OpenBMB/AgentVerse](https://github.com/OpenBMB/AgentVerse) | 220 |
-|[alvarosevilla95/autolang](https://github.com/alvarosevilla95/autolang) | 219 |
-|[JohnSnowLabs/nlptest](https://github.com/JohnSnowLabs/nlptest) | 216 |
-|[kaleido-lab/dolphin](https://github.com/kaleido-lab/dolphin) | 215 |
-|[truera/trulens](https://github.com/truera/trulens) | 208 |
-|[NimbleBoxAI/ChainFury](https://github.com/NimbleBoxAI/ChainFury) | 208 |
-|[airobotlab/KoChatGPT](https://github.com/airobotlab/KoChatGPT) | 207 |
-|[monarch-initiative/ontogpt](https://github.com/monarch-initiative/ontogpt) | 200 |
-|[paolorechia/learn-langchain](https://github.com/paolorechia/learn-langchain) | 195 |
-|[shaman-ai/agent-actors](https://github.com/shaman-ai/agent-actors) | 185 |
-|[Haste171/langchain-chatbot](https://github.com/Haste171/langchain-chatbot) | 184 |
-|[plchld/InsightFlow](https://github.com/plchld/InsightFlow) | 182 |
-|[su77ungr/CASALIOY](https://github.com/su77ungr/CASALIOY) | 180 |
-|[jbrukh/gpt-jargon](https://github.com/jbrukh/gpt-jargon) | 177 |
-|[benthecoder/ClassGPT](https://github.com/benthecoder/ClassGPT) | 174 |
-|[billxbf/ReWOO](https://github.com/billxbf/ReWOO) | 170 |
-|[filip-michalsky/SalesGPT](https://github.com/filip-michalsky/SalesGPT) | 168 |
-|[hwchase17/langchain-streamlit-template](https://github.com/hwchase17/langchain-streamlit-template) | 168 |
-|[radi-cho/datasetGPT](https://github.com/radi-cho/datasetGPT) | 164 |
-|[hardbyte/qabot](https://github.com/hardbyte/qabot) | 164 |
-|[gia-guar/JARVIS-ChatGPT](https://github.com/gia-guar/JARVIS-ChatGPT) | 158 |
-|[plastic-labs/tutor-gpt](https://github.com/plastic-labs/tutor-gpt) | 154 |
-|[yasyf/compress-gpt](https://github.com/yasyf/compress-gpt) | 154 |
-|[fengyuli-dev/multimedia-gpt](https://github.com/fengyuli-dev/multimedia-gpt) | 154 |
-|[ethanyanjiali/minChatGPT](https://github.com/ethanyanjiali/minChatGPT) | 153 |
-|[hwchase17/chroma-langchain](https://github.com/hwchase17/chroma-langchain) | 153 |
-|[edreisMD/plugnplai](https://github.com/edreisMD/plugnplai) | 148 |
-|[chakkaradeep/pyCodeAGI](https://github.com/chakkaradeep/pyCodeAGI) | 145 |
-|[ccurme/yolopandas](https://github.com/ccurme/yolopandas) | 145 |
-|[shamspias/customizable-gpt-chatbot](https://github.com/shamspias/customizable-gpt-chatbot) | 144 |
-|[realminchoi/babyagi-ui](https://github.com/realminchoi/babyagi-ui) | 143 |
-|[PradipNichite/Youtube-Tutorials](https://github.com/PradipNichite/Youtube-Tutorials) | 140 |
-|[gustavz/DataChad](https://github.com/gustavz/DataChad) | 140 |
-|[Klingefjord/chatgpt-telegram](https://github.com/Klingefjord/chatgpt-telegram) | 140 |
-|[Jaseci-Labs/jaseci](https://github.com/Jaseci-Labs/jaseci) | 139 |
-|[handrew/browserpilot](https://github.com/handrew/browserpilot) | 137 |
-|[jmpaz/promptlib](https://github.com/jmpaz/promptlib) | 137 |
-|[SamPink/dev-gpt](https://github.com/SamPink/dev-gpt) | 135 |
-|[menloparklab/langchain-cohere-qdrant-doc-retrieval](https://github.com/menloparklab/langchain-cohere-qdrant-doc-retrieval) | 135 |
-|[hirokidaichi/wanna](https://github.com/hirokidaichi/wanna) | 135 |
-|[steamship-core/vercel-examples](https://github.com/steamship-core/vercel-examples) | 134 |
-|[pablomarin/GPT-Azure-Search-Engine](https://github.com/pablomarin/GPT-Azure-Search-Engine) | 133 |
-|[ibiscp/LLM-IMDB](https://github.com/ibiscp/LLM-IMDB) | 133 |
-|[shauryr/S2QA](https://github.com/shauryr/S2QA) | 133 |
-|[jerlendds/osintbuddy](https://github.com/jerlendds/osintbuddy) | 132 |
-|[yuanjie-ai/ChatLLM](https://github.com/yuanjie-ai/ChatLLM) | 132 |
-|[yasyf/summ](https://github.com/yasyf/summ) | 132 |
-|[WongSaang/chatgpt-ui-server](https://github.com/WongSaang/chatgpt-ui-server) | 130 |
-|[peterw/StoryStorm](https://github.com/peterw/StoryStorm) | 127 |
-|[Teahouse-Studios/akari-bot](https://github.com/Teahouse-Studios/akari-bot) | 126 |
-|[vaibkumr/prompt-optimizer](https://github.com/vaibkumr/prompt-optimizer) | 125 |
-|[preset-io/promptimize](https://github.com/preset-io/promptimize) | 124 |
-|[homanp/vercel-langchain](https://github.com/homanp/vercel-langchain) | 124 |
-|[petehunt/langchain-github-bot](https://github.com/petehunt/langchain-github-bot) | 123 |
-|[eunomia-bpf/GPTtrace](https://github.com/eunomia-bpf/GPTtrace) | 118 |
-|[nicknochnack/LangchainDocuments](https://github.com/nicknochnack/LangchainDocuments) | 116 |
-|[jiran214/GPT-vup](https://github.com/jiran214/GPT-vup) | 112 |
-|[rsaryev/talk-codebase](https://github.com/rsaryev/talk-codebase) | 112 |
-|[zenml-io/zenml-projects](https://github.com/zenml-io/zenml-projects) | 112 |
-|[microsoft/azure-openai-in-a-day-workshop](https://github.com/microsoft/azure-openai-in-a-day-workshop) | 112 |
-|[davila7/file-gpt](https://github.com/davila7/file-gpt) | 112 |
-|[prof-frink-lab/slangchain](https://github.com/prof-frink-lab/slangchain) | 111 |
-|[aurelio-labs/arxiv-bot](https://github.com/aurelio-labs/arxiv-bot) | 110 |
-|[fixie-ai/fixie-examples](https://github.com/fixie-ai/fixie-examples) | 108 |
-|[miaoshouai/miaoshouai-assistant](https://github.com/miaoshouai/miaoshouai-assistant) | 105 |
-|[flurb18/AgentOoba](https://github.com/flurb18/AgentOoba) | 103 |
-|[solana-labs/chatgpt-plugin](https://github.com/solana-labs/chatgpt-plugin) | 102 |
-|[Significant-Gravitas/Auto-GPT-Benchmarks](https://github.com/Significant-Gravitas/Auto-GPT-Benchmarks) | 102 |
-|[kaarthik108/snowChat](https://github.com/kaarthik108/snowChat) | 100 |
-
-
-_Generated by [github-dependents-info](https://github.com/nvuillam/github-dependents-info)_
-
-`github-dependents-info --repo hwchase17/langchain --markdownfile dependents.md --minstars 100 --sort stars`

docs/deployments.md

@@ -1,15 +1,14 @@
 # Deployments
 
-So, you've created a really cool chain - now what? How do you deploy it and make it easily shareable with the world?
+So you've made a really cool chain - now what? How do you deploy it and make it easily sharable with the world?
 
-This section covers several options for that. Note that these options are meant for quick deployment of prototypes and demos, not for production systems. If you need help with the deployment of a production system, please contact us directly.
+This section covers several options for that.
+Note that these are meant as quick deployment options for prototypes and demos, and not for production systems.
+If you are looking for help with deployment of a production system, please contact us directly.
 
-What follows is a list of template GitHub repositories designed to be easily forked and modified to use your chain. This list is far from exhaustive, and we are EXTREMELY open to contributions here.
-
-## [Anyscale](https://www.anyscale.com/model-serving)
-
-Anyscale is a unified compute platform that makes it easy to develop, deploy, and manage scalable LLM applications in production using Ray.
-With Anyscale you can scale the most challenging LLM-based workloads and both develop and deploy LLM-based apps on a single compute platform.
+What follows is a list of template GitHub repositories aimed that are intended to be
+very easy to fork and modify to use your chain.
+This is far from an exhaustive list of options, and we are EXTREMELY open to contributions here.
 
 ## [Streamlit](https://github.com/hwchase17/langchain-streamlit-template)
 
@@ -24,12 +23,6 @@ It implements a chatbot interface, with a "Bring-Your-Own-Token" approach (nice
 It also contains instructions for how to deploy this app on the Hugging Face platform.
 This is heavily influenced by James Weaver's [excellent examples](https://huggingface.co/JavaFXpert).
 
-## [Chainlit](https://github.com/Chainlit/cookbook)
-
-This repo is a cookbook explaining how to visualize and deploy LangChain agents with Chainlit.
-You create ChatGPT-like UIs with Chainlit. Some of the key features include intermediary steps visualisation, element management & display (images, text, carousel, etc.) as well as cloud deployment.
-Chainlit [doc](https://docs.chainlit.io/langchain) on the integration with LangChain
-
 ## [Beam](https://github.com/slai-labs/get-beam/tree/main/examples/langchain-question-answering)
 
 This repo serves as a template for how deploy a LangChain with [Beam](https://beam.cloud).
@@ -40,14 +33,6 @@ It implements a Question Answering app and contains instructions for deploying t
 
 A minimal example on how to run LangChain on Vercel using Flask.
 
-## [FastAPI + Vercel](https://github.com/msoedov/langcorn)
-
-A minimal example on how to run LangChain on Vercel using FastAPI and LangCorn/Uvicorn.
-
-## [Kinsta](https://github.com/kinsta/hello-world-langchain)
-
-A minimal example on how to deploy LangChain to [Kinsta](https://kinsta.com) using Flask.
-
 ## [Fly.io](https://github.com/fly-apps/hello-fly-langchain)
 
 A minimal example of how to deploy LangChain to [Fly.io](https://fly.io/) using Flask.
@@ -62,11 +47,12 @@ A minimal example on how to deploy LangChain to Google Cloud Run.
 
 ## [SteamShip](https://github.com/steamship-core/steamship-langchain/)
 
-This repository contains LangChain adapters for Steamship, enabling LangChain developers to rapidly deploy their apps on Steamship. This includes: production-ready endpoints, horizontal scaling across dependencies, persistent storage of app state, multi-tenancy support, etc.
+This repository contains LangChain adapters for Steamship, enabling LangChain developers to rapidly deploy their apps on Steamship.
+This includes: production ready endpoints, horizontal scaling across dependencies, persistant storage of app state, multi-tenancy support, etc.
 
 ## [Langchain-serve](https://github.com/jina-ai/langchain-serve)
 
-This repository allows users to serve local chains and agents as RESTful, gRPC, or WebSocket APIs, thanks to [Jina](https://docs.jina.ai/). Deploy your chains & agents with ease and enjoy independent scaling, serverless and autoscaling APIs, as well as a Streamlit playground on Jina AI Cloud.
+This repository allows users to serve local chains and agents as RESTful, gRPC, or Websocket APIs thanks to [Jina](https://docs.jina.ai/). Deploy your chains & agents with ease and enjoy independent scaling, serverless and autoscaling APIs, as well as a Streamlit playground on Jina AI Cloud.
 
 ## [BentoML](https://github.com/ssheng/BentoChain)
 
@@ -74,4 +60,4 @@ This repository provides an example of how to deploy a LangChain application wit
 
 ## [Databutton](https://databutton.com/home?new-data-app=true)
 
-These templates serve as examples of how to build, deploy, and share LangChain applications using Databutton. You can create user interfaces with Streamlit, automate tasks by scheduling Python code, and store files and data in the built-in store. Examples include a Chatbot interface with conversational memory, a Personal search engine, and a starter template for LangChain apps. Deploying and sharing is just one click away.
+These templates serve as examples of how to build, deploy, and share LangChain applications using Databutton. You can create user interfaces with Streamlit, automate tasks by scheduling Python code, and store files and data in the built-in store. Examples include Chatbot interface with conversational memory, Personal search engine, and a starter template for LangChain apps. Deploying and sharing is one click. 

docs/ecosystem.rst

@@ -1,13 +1,12 @@
-Integrations
+LangChain Ecosystem
 ===================
 
-LangChain integrates with many LLMs, systems, and products.
+Guides for how other companies/products can be used with LangChain
 
-Integrations by Module
---------------------------------
-
-| Integrations grouped by the core LangChain module they map to:
+Groups
+----------
 
+LangChain provides integration with many LLMs and systems:
 
 - `LLM Providers <./modules/models/llms/integrations.html>`_
 - `Chat Model Providers <./modules/models/chat/integrations.html>`_
@@ -19,21 +18,12 @@ Integrations by Module
 - `Tool Providers <./modules/agents/tools.html>`_
 - `Toolkit Integrations <./modules/agents/toolkits.html>`_
 
-
-Dependencies
-----------------
-
-| LangChain depends on `several hungered Python packages <https://github.com/hwchase17/langchain/network/dependencies>`_.
-
-
-All Integrations
--------------------------------------------
-
-| A comprehensive list of LLMs, systems, and products integrated with LangChain:
+Companies / Products
+----------
 
 
 .. toctree::
    :maxdepth: 1
    :glob:
 
-   integrations/*
+   ecosystem/*

docs/ecosystem/aim_tracking.ipynb

@@ -61,6 +61,7 @@
     "from datetime import datetime\n",
     "\n",
     "from langchain.llms import OpenAI\n",
+    "from langchain.callbacks.base import CallbackManager\n",
     "from langchain.callbacks import AimCallbackHandler, StdOutCallbackHandler"
    ]
   },
@@ -108,8 +109,8 @@
     "    experiment_name=\"scenario 1: OpenAI LLM\",\n",
     ")\n",
     "\n",
-    "callbacks = [StdOutCallbackHandler(), aim_callback]\n",
-    "llm = OpenAI(temperature=0, callbacks=callbacks)"
+    "manager = CallbackManager([StdOutCallbackHandler(), aim_callback])\n",
+    "llm = OpenAI(temperature=0, callback_manager=manager, verbose=True)"
    ]
   },
   {
@@ -176,7 +177,7 @@
     "Title: {title}\n",
     "Playwright: This is a synopsis for the above play:\"\"\"\n",
     "prompt_template = PromptTemplate(input_variables=[\"title\"], template=template)\n",
-    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=callbacks)\n",
+    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callback_manager=manager)\n",
     "\n",
     "test_prompts = [\n",
     "    {\"title\": \"documentary about good video games that push the boundary of game design\"},\n",
@@ -248,12 +249,13 @@
    ],
    "source": [
     "# scenario 3 - Agent with Tools\n",
-    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callbacks=callbacks)\n",
+    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callback_manager=manager)\n",
     "agent = initialize_agent(\n",
     "    tools,\n",
     "    llm,\n",
     "    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
-    "    callbacks=callbacks,\n",
+    "    callback_manager=manager,\n",
+    "    verbose=True,\n",
     ")\n",
     "agent.run(\n",
     "    \"Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?\"\n",

docs/ecosystem/baseten.md

@@ -1,25 +0,0 @@
-# Baseten
-
-Learn how to use LangChain with models deployed on Baseten.
-
-## Installation and setup
-
-- Create a [Baseten](https://baseten.co) account and [API key](https://docs.baseten.co/settings/api-keys).
-- Install the Baseten Python client with `pip install baseten`
-- Use your API key to authenticate with `baseten login`
-
-## Invoking a model
-
-Baseten integrates with LangChain through the LLM module, which provides a standardized and interoperable interface for models that are deployed on your Baseten workspace.
-
-You can deploy foundation models like WizardLM and Alpaca with one click from the [Baseten model library](https://app.baseten.co/explore/) or if you have your own model, [deploy it with this tutorial](https://docs.baseten.co/deploying-models/deploy).
-
-In this example, we'll work with WizardLM. [Deploy WizardLM here](https://app.baseten.co/explore/wizardlm) and follow along with the deployed [model's version ID](https://docs.baseten.co/managing-models/manage).
-
-```python
-from langchain.llms import Baseten
-
-wizardlm = Baseten(model="MODEL_VERSION_ID", verbose=True)
-
-wizardlm("What is the difference between a Wizard and a Sorcerer?")
-```

docs/ecosystem/chroma.md

@@ -1,29 +1,20 @@
 # Chroma
 
->[Chroma](https://docs.trychroma.com/getting-started) is a database for building AI applications with embeddings.
+This page covers how to use the Chroma ecosystem within LangChain.
+It is broken into two parts: installation and setup, and then references to specific Chroma wrappers.
 
 ## Installation and Setup
+- Install the Python package with `pip install chromadb`
+## Wrappers
 
-```bash
-pip install chromadb
-```
-
-
-## VectorStore
+### VectorStore
 
 There exists a wrapper around Chroma vector databases, allowing you to use it as a vectorstore,
 whether for semantic search or example selection.
 
+To import this vectorstore:
 ```python
 from langchain.vectorstores import Chroma
 ```
 
 For a more detailed walkthrough of the Chroma wrapper, see [this notebook](../modules/indexes/vectorstores/getting_started.ipynb)
-
-## Retriever
-
-See a [usage example](../modules/indexes/retrievers/examples/chroma_self_query.ipynb).
-
-```python
-from langchain.retrievers import SelfQueryRetriever
-```

docs/ecosystem/clearml_tracking.ipynb

@@ -1,22 +1,13 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# ClearML\n",
+    "# ClearML Integration\n",
     "\n",
-    "> [ClearML](https://github.com/allegroai/clearml) is a ML/DL development and production suite, it contains 5 main modules:\n",
-    "> - `Experiment Manager` - Automagical experiment tracking, environments and results\n",
-    "> - `MLOps` - Orchestration, Automation & Pipelines solution for ML/DL jobs (K8s / Cloud / bare-metal)\n",
-    "> - `Data-Management` - Fully differentiable data management & version control solution on top of object-storage (S3 / GS / Azure / NAS)\n",
-    "> - `Model-Serving` - cloud-ready Scalable model serving solution!\n",
-    "    Deploy new model endpoints in under 5 minutes\n",
-    "    Includes optimized GPU serving support backed by Nvidia-Triton\n",
-    "    with out-of-the-box Model Monitoring\n",
-    "> - `Fire Reports` - Create and share rich MarkDown documents supporting embeddable online content\n",
-    "\n",
-    "In order to properly keep track of your langchain experiments and their results, you can enable the `ClearML` integration. We use the `ClearML Experiment Manager` that neatly tracks and organizes all your experiment runs.\n",
+    "In order to properly keep track of your langchain experiments and their results, you can enable the ClearML integration. ClearML is an experiment manager that neatly tracks and organizes all your experiment runs.\n",
     "\n",
     "<a target=\"_blank\" href=\"https://colab.research.google.com/github/hwchase17/langchain/blob/master/docs/ecosystem/clearml_tracking.ipynb\">\n",
     "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
@@ -24,32 +15,11 @@
    ]
   },
   {
-   "cell_type": "markdown",
-   "metadata": {
-    "tags": []
-   },
-   "source": [
-    "## Installation and Setup"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install clearml\n",
-    "!pip install pandas\n",
-    "!pip install textstat\n",
-    "!pip install spacy\n",
-    "!python -m spacy download en_core_web_sm"
-   ]
-  },
-  {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Getting API Credentials\n",
+    "## Getting API Credentials\n",
     "\n",
     "We'll be using quite some APIs in this notebook, here is a list and where to get them:\n",
     "\n",
@@ -73,21 +43,24 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Callbacks"
+    "## Setting Up"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {
-    "tags": []
-   },
+   "execution_count": null,
+   "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.callbacks import ClearMLCallbackHandler"
+    "!pip install clearml\n",
+    "!pip install pandas\n",
+    "!pip install textstat\n",
+    "!pip install spacy\n",
+    "!python -m spacy download en_core_web_sm"
    ]
   },
   {
@@ -105,7 +78,8 @@
    ],
    "source": [
     "from datetime import datetime\n",
-    "from langchain.callbacks import StdOutCallbackHandler\n",
+    "from langchain.callbacks import ClearMLCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.callbacks.base import CallbackManager\n",
     "from langchain.llms import OpenAI\n",
     "\n",
     "# Setup and use the ClearML Callback\n",
@@ -119,16 +93,17 @@
     "    complexity_metrics=True,\n",
     "    stream_logs=True\n",
     ")\n",
-    "callbacks = [StdOutCallbackHandler(), clearml_callback]\n",
+    "manager = CallbackManager([StdOutCallbackHandler(), clearml_callback])\n",
     "# Get the OpenAI model ready to go\n",
-    "llm = OpenAI(temperature=0, callbacks=callbacks)"
+    "llm = OpenAI(temperature=0, callback_manager=manager, verbose=True)"
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Scenario 1: Just an LLM\n",
+    "## Scenario 1: Just an LLM\n",
     "\n",
     "First, let's just run a single LLM a few times and capture the resulting prompt-answer conversation in ClearML"
    ]
@@ -370,6 +345,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -381,10 +357,11 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Scenario 2: Creating an agent with tools\n",
+    "## Scenario 2: Creating an agent with tools\n",
     "\n",
     "To show a more advanced workflow, let's create an agent with access to tools. The way ClearML tracks the results is not different though, only the table will look slightly different as there are other types of actions taken when compared to the earlier, simpler example.\n",
     "\n",
@@ -546,12 +523,13 @@
     "from langchain.agents import AgentType\n",
     "\n",
     "# SCENARIO 2 - Agent with Tools\n",
-    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callbacks=callbacks)\n",
+    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callback_manager=manager)\n",
     "agent = initialize_agent(\n",
     "    tools,\n",
     "    llm,\n",
     "    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
-    "    callbacks=callbacks,\n",
+    "    callback_manager=manager,\n",
+    "    verbose=True,\n",
     ")\n",
     "agent.run(\n",
     "    \"Who is the wife of the person who sang summer of 69?\"\n",
@@ -560,10 +538,11 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Tips and Next Steps\n",
+    "## Tips and Next Steps\n",
     "\n",
     "- Make sure you always use a unique `name` argument for the `clearml_callback.flush_tracker` function. If not, the model parameters used for a run will override the previous run!\n",
     "\n",
@@ -582,7 +561,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": ".venv",
    "language": "python",
    "name": "python3"
   },
@@ -596,8 +575,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.10.9"
   },
+  "orig_nbformat": 4,
   "vscode": {
    "interpreter": {
     "hash": "a53ebf4a859167383b364e7e7521d0add3c2dbbdecce4edf676e8c4634ff3fbb"
@@ -605,5 +585,5 @@
   }
  },
  "nbformat": 4,
- "nbformat_minor": 4
+ "nbformat_minor": 2
 }

docs/ecosystem/cohere.md

@@ -0,0 +1,25 @@
+# Cohere
+
+This page covers how to use the Cohere ecosystem within LangChain.
+It is broken into two parts: installation and setup, and then references to specific Cohere wrappers.
+
+## Installation and Setup
+- Install the Python SDK with `pip install cohere`
+- Get an Cohere api key and set it as an environment variable (`COHERE_API_KEY`)
+
+## Wrappers
+
+### LLM
+
+There exists an Cohere LLM wrapper, which you can access with 
+```python
+from langchain.llms import Cohere
+```
+
+### Embeddings
+
+There exists an Cohere Embeddings wrapper, which you can access with 
+```python
+from langchain.embeddings import CohereEmbeddings
+```
+For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/cohere.ipynb)

docs/ecosystem/comet_tracking.ipynb

@@ -121,6 +121,7 @@
     "from datetime import datetime\n",
     "\n",
     "from langchain.callbacks import CometCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.callbacks.base import CallbackManager\n",
     "from langchain.llms import OpenAI\n",
     "\n",
     "comet_callback = CometCallbackHandler(\n",
@@ -130,8 +131,8 @@
     "    tags=[\"llm\"],\n",
     "    visualizations=[\"dep\"],\n",
     ")\n",
-    "callbacks = [StdOutCallbackHandler(), comet_callback]\n",
-    "llm = OpenAI(temperature=0.9, callbacks=callbacks, verbose=True)\n",
+    "manager = CallbackManager([StdOutCallbackHandler(), comet_callback])\n",
+    "llm = OpenAI(temperature=0.9, callback_manager=manager, verbose=True)\n",
     "\n",
     "llm_result = llm.generate([\"Tell me a joke\", \"Tell me a poem\", \"Tell me a fact\"] * 3)\n",
     "print(\"LLM result\", llm_result)\n",
@@ -152,6 +153,7 @@
    "outputs": [],
    "source": [
     "from langchain.callbacks import CometCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.callbacks.base import CallbackManager\n",
     "from langchain.chains import LLMChain\n",
     "from langchain.llms import OpenAI\n",
     "from langchain.prompts import PromptTemplate\n",
@@ -162,14 +164,15 @@
     "    stream_logs=True,\n",
     "    tags=[\"synopsis-chain\"],\n",
     ")\n",
-    "callbacks = [StdOutCallbackHandler(), comet_callback]\n",
-    "llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
+    "manager = CallbackManager([StdOutCallbackHandler(), comet_callback])\n",
+    "\n",
+    "llm = OpenAI(temperature=0.9, callback_manager=manager, verbose=True)\n",
     "\n",
     "template = \"\"\"You are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
     "Title: {title}\n",
     "Playwright: This is a synopsis for the above play:\"\"\"\n",
     "prompt_template = PromptTemplate(input_variables=[\"title\"], template=template)\n",
-    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=callbacks)\n",
+    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callback_manager=manager)\n",
     "\n",
     "test_prompts = [{\"title\": \"Documentary about Bigfoot in Paris\"}]\n",
     "print(synopsis_chain.apply(test_prompts))\n",
@@ -191,6 +194,7 @@
    "source": [
     "from langchain.agents import initialize_agent, load_tools\n",
     "from langchain.callbacks import CometCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.callbacks.base import CallbackManager\n",
     "from langchain.llms import OpenAI\n",
     "\n",
     "comet_callback = CometCallbackHandler(\n",
@@ -199,15 +203,15 @@
     "    stream_logs=True,\n",
     "    tags=[\"agent\"],\n",
     ")\n",
-    "callbacks = [StdOutCallbackHandler(), comet_callback]\n",
-    "llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
+    "manager = CallbackManager([StdOutCallbackHandler(), comet_callback])\n",
+    "llm = OpenAI(temperature=0.9, callback_manager=manager, verbose=True)\n",
     "\n",
-    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callbacks=callbacks)\n",
+    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callback_manager=manager)\n",
     "agent = initialize_agent(\n",
     "    tools,\n",
     "    llm,\n",
     "    agent=\"zero-shot-react-description\",\n",
-    "    callbacks=callbacks,\n",
+    "    callback_manager=manager,\n",
     "    verbose=True,\n",
     ")\n",
     "agent.run(\n",
@@ -251,6 +255,7 @@
     "from rouge_score import rouge_scorer\n",
     "\n",
     "from langchain.callbacks import CometCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.callbacks.base import CallbackManager\n",
     "from langchain.chains import LLMChain\n",
     "from langchain.llms import OpenAI\n",
     "from langchain.prompts import PromptTemplate\n",
@@ -293,10 +298,10 @@
     "    tags=[\"custom_metrics\"],\n",
     "    custom_metrics=rouge_score.compute_metric,\n",
     ")\n",
-    "callbacks = [StdOutCallbackHandler(), comet_callback]\n",
-    "llm = OpenAI(temperature=0.9)\n",
+    "manager = CallbackManager([StdOutCallbackHandler(), comet_callback])\n",
+    "llm = OpenAI(temperature=0.9, callback_manager=manager, verbose=True)\n",
     "\n",
-    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template)\n",
+    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callback_manager=manager)\n",
     "\n",
     "test_prompts = [\n",
     "    {\n",
@@ -318,7 +323,7 @@
     "                 \"\"\"\n",
     "    }\n",
     "]\n",
-    "print(synopsis_chain.apply(test_prompts, callbacks=callbacks))\n",
+    "print(synopsis_chain.apply(test_prompts))\n",
     "comet_callback.flush_tracker(synopsis_chain, finish=True)"
    ]
   }

docs/ecosystem/databerry.md

@@ -0,0 +1,25 @@
+# Databerry
+
+This page covers how to use the [Databerry](https://databerry.ai) within LangChain.
+
+## What is Databerry?
+
+Databerry is an [open source](https://github.com/gmpetrov/databerry) document retrievial platform that helps to connect your personal data with Large Language Models.
+
+![Databerry](../_static/DataberryDashboard.png)
+
+## Quick start
+
+Retrieving documents stored in Databerry from LangChain is very easy!
+
+```python
+from langchain.retrievers import DataberryRetriever
+
+retriever = DataberryRetriever(
+    datastore_url="https://api.databerry.ai/query/clg1xg2h80000l708dymr0fxc",
+    # api_key="DATABERRY_API_KEY", # optional if datastore is public
+    # top_k=10 # optional
+)
+
+docs = retriever.get_relevant_documents("What's Databerry?")
+```

docs/ecosystem/deepinfra.md

@@ -7,14 +7,6 @@ It is broken into two parts: installation and setup, and then references to spec
 - Get your DeepInfra api key from this link [here](https://deepinfra.com/).
 - Get an DeepInfra api key and set it as an environment variable (`DEEPINFRA_API_TOKEN`)
 
-## Available Models
-
-DeepInfra provides a range of Open Source LLMs ready for deployment.
-You can list supported models [here](https://deepinfra.com/models?type=text-generation).
-google/flan\* models can be viewed [here](https://deepinfra.com/models?type=text2text-generation).
-
-You can view a list of request and response parameters [here](https://deepinfra.com/databricks/dolly-v2-12b#API)
-
 ## Wrappers
 
 ### LLM

docs/ecosystem/google_search.md

@@ -1,4 +1,4 @@
-# Google Search
+# Google Search Wrapper
 
 This page covers how to use the Google Search API within LangChain.
 It is broken into two parts: installation and setup, and then references to the specific Google Search wrapper.

docs/ecosystem/google_serper.md

@@ -1,4 +1,4 @@
-# Google Serper
+# Google Serper Wrapper
 
 This page covers how to use the [Serper](https://serper.dev) Google Search API within LangChain. Serper is a low-cost Google Search API that can be used to add answer box, knowledge graph, and organic results data from Google Search. 
 It is broken into two parts: setup, and then references to the specific Google Serper wrapper.

docs/ecosystem/gpt4all.md

@@ -3,7 +3,6 @@
 This page covers how to use the `GPT4All` wrapper within LangChain. The tutorial is divided into two parts: installation and setup, followed by usage with an example.
 
 ## Installation and Setup
-
 - Install the Python package with `pip install pyllamacpp`
 - Download a [GPT4All model](https://github.com/nomic-ai/pyllamacpp#supported-model) and place it in your desired directory
 
@@ -29,16 +28,16 @@ To stream the model's predictions, add in a CallbackManager.
 
 ```python
 from langchain.llms import GPT4All
+from langchain.callbacks.base import CallbackManager
 from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
-
 # There are many CallbackHandlers supported, such as
 # from langchain.callbacks.streamlit import StreamlitCallbackHandler
 
-callbacks = [StreamingStdOutCallbackHandler()]
-model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8)
+callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
+model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8, callback_handler=callback_handler, verbose=True)
 
 # Generate text. Tokens are streamed through the callback manager.
-model("Once upon a time, ", callbacks=callbacks)
+model("Once upon a time, ")
 ```
 
 ## Model File

docs/ecosystem/huggingface.md

@@ -47,7 +47,7 @@ To use a the wrapper for a model hosted on Hugging Face Hub:
 ```python
 from langchain.embeddings import HuggingFaceHubEmbeddings
 ```
-For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/huggingface_hub.ipynb)
+For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/huggingfacehub.ipynb)
 
 ### Tokenizer
 

docs/ecosystem/modelscope.md

@@ -1,20 +0,0 @@
-# ModelScope
-
-This page covers how to use the modelscope ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific modelscope wrappers.
-
-## Installation and Setup
-
-* Install the Python SDK with `pip install modelscope`
-
-## Wrappers
-
-### Embeddings
-
-There exists a modelscope Embeddings wrapper, which you can access with 
-
-```python
-from langchain.embeddings import ModelScopeEmbeddings
-```
-
-For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/modelscope_hub.ipynb)

docs/ecosystem/openai.md

@@ -0,0 +1,55 @@
+# OpenAI
+
+This page covers how to use the OpenAI ecosystem within LangChain.
+It is broken into two parts: installation and setup, and then references to specific OpenAI wrappers.
+
+## Installation and Setup
+- Install the Python SDK with `pip install openai`
+- Get an OpenAI api key and set it as an environment variable (`OPENAI_API_KEY`)
+- If you want to use OpenAI's tokenizer (only available for Python 3.9+), install it with `pip install tiktoken`
+
+## Wrappers
+
+### LLM
+
+There exists an OpenAI LLM wrapper, which you can access with 
+```python
+from langchain.llms import OpenAI
+```
+
+If you are using a model hosted on Azure, you should use different wrapper for that:
+```python
+from langchain.llms import AzureOpenAI
+```
+For a more detailed walkthrough of the Azure wrapper, see [this notebook](../modules/models/llms/integrations/azure_openai_example.ipynb)
+
+
+
+### Embeddings
+
+There exists an OpenAI Embeddings wrapper, which you can access with 
+```python
+from langchain.embeddings import OpenAIEmbeddings
+```
+For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/openai.ipynb)
+
+
+### Tokenizer
+
+There are several places you can use the `tiktoken` tokenizer. By default, it is used to count tokens
+for OpenAI LLMs.
+
+You can also use it to count tokens when splitting documents with 
+```python
+from langchain.text_splitter import CharacterTextSplitter
+CharacterTextSplitter.from_tiktoken_encoder(...)
+```
+For a more detailed walkthrough of this, see [this notebook](../modules/indexes/text_splitters/examples/tiktoken.ipynb)
+
+### Moderation
+You can also access the OpenAI content moderation endpoint with
+
+```python
+from langchain.chains import OpenAIModerationChain
+```
+For a more detailed walkthrough of this, see [this notebook](../modules/chains/examples/moderation.ipynb)

docs/ecosystem/pinecone.md

@@ -4,19 +4,17 @@ This page covers how to use the Pinecone ecosystem within LangChain.
 It is broken into two parts: installation and setup, and then references to specific Pinecone wrappers.
 
 ## Installation and Setup
-Install the Python SDK:
-```bash
-pip install pinecone-client
-```
+- Install the Python SDK with `pip install pinecone-client`
+## Wrappers
 
-
-## Vectorstore
+### VectorStore
 
 There exists a wrapper around Pinecone indexes, allowing you to use it as a vectorstore,
 whether for semantic search or example selection.
 
+To import this vectorstore:
 ```python
 from langchain.vectorstores import Pinecone
 ```
 
-For a more detailed walkthrough of the Pinecone vectorstore, see [this notebook](../modules/indexes/vectorstores/examples/pinecone.ipynb)
+For a more detailed walkthrough of the Pinecone wrapper, see [this notebook](../modules/indexes/vectorstores/examples/pinecone.ipynb)

docs/ecosystem/predictionguard.md

@@ -0,0 +1,56 @@
+# Prediction Guard
+
+This page covers how to use the Prediction Guard ecosystem within LangChain.
+It is broken into two parts: installation and setup, and then references to specific Prediction Guard wrappers.
+
+## Installation and Setup
+- Install the Python SDK with `pip install predictionguard`
+- Get an Prediction Guard access token (as described [here](https://docs.predictionguard.com/)) and set it as an environment variable (`PREDICTIONGUARD_TOKEN`)
+
+## LLM Wrapper
+
+There exists a Prediction Guard LLM wrapper, which you can access with 
+```python
+from langchain.llms import PredictionGuard
+```
+
+You can provide the name of your Prediction Guard "proxy" as an argument when initializing the LLM:
+```python
+pgllm = PredictionGuard(name="your-text-gen-proxy")
+```
+
+Alternatively, you can use Prediction Guard's default proxy for SOTA LLMs:
+```python
+pgllm = PredictionGuard(name="default-text-gen")
+```
+
+You can also provide your access token directly as an argument:
+```python
+pgllm = PredictionGuard(name="default-text-gen", token="<your access token>")
+```
+
+## Example usage
+
+Basic usage of the LLM wrapper:
+```python
+from langchain.llms import PredictionGuard
+
+pgllm = PredictionGuard(name="default-text-gen")
+pgllm("Tell me a joke")
+```
+
+Basic LLM Chaining with the Prediction Guard wrapper:
+```python
+from langchain import PromptTemplate, LLMChain
+from langchain.llms import PredictionGuard
+
+template = """Question: {question}
+
+Answer: Let's think step by step."""
+prompt = PromptTemplate(template=template, input_variables=["question"])
+llm_chain = LLMChain(prompt=prompt, llm=PredictionGuard(name="default-text-gen"), verbose=True)
+
+question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"
+
+llm_chain.predict(question=question)
+```

docs/ecosystem/promptlayer.md

@@ -1,35 +1,31 @@
 # PromptLayer
 
->[PromptLayer](https://docs.promptlayer.com/what-is-promptlayer/wxpF9EZkUwvdkwvVE9XEvC/how-promptlayer-works/dvgGSxNe6nB1jj8mUVbG8r) 
-> is a devtool that allows you to track, manage, and share your GPT prompt engineering. 
-> It acts as a middleware between your code and OpenAI's python library, recording all your API requests 
-> and saving relevant metadata for easy exploration and search in the [PromptLayer](https://www.promptlayer.com) dashboard.
+This page covers how to use [PromptLayer](https://www.promptlayer.com) within LangChain.
+It is broken into two parts: installation and setup, and then references to specific PromptLayer wrappers.
 
 ## Installation and Setup
 
-- Install the `promptlayer` python library 
-```bash
-pip install promptlayer
-```
+If you want to work with PromptLayer:
+- Install the promptlayer python library `pip install promptlayer`
 - Create a PromptLayer account
 - Create an api token and set it as an environment variable (`PROMPTLAYER_API_KEY`)
 
+## Wrappers
 
-## LLM
+### LLM
 
+There exists an PromptLayer OpenAI LLM wrapper, which you can access with
 ```python
 from langchain.llms import PromptLayerOpenAI
 ```
 
-### Example
-
-To tag your requests, use the argument `pl_tags` when instantiating the LLM
+To tag your requests, use the argument `pl_tags` when instanializing the LLM
 ```python
 from langchain.llms import PromptLayerOpenAI
 llm = PromptLayerOpenAI(pl_tags=["langchain-requests", "chatbot"])
 ```
 
-To get the PromptLayer request id, use the argument `return_pl_id` when instantiating the LLM
+To get the PromptLayer request id, use the argument `return_pl_id` when instanializing the LLM
 ```python
 from langchain.llms import PromptLayerOpenAI
 llm = PromptLayerOpenAI(return_pl_id=True)
@@ -46,14 +42,8 @@ You can use the PromptLayer request ID to add a prompt, score, or other metadata
 
 This LLM is identical to the [OpenAI LLM](./openai.md), except that
 - all your requests will be logged to your PromptLayer account
-- you can add `pl_tags` when instantiating to tag your requests on PromptLayer
-- you can add `return_pl_id` when instantiating to return a PromptLayer request id to use [while tracking requests](https://magniv.notion.site/Track-4deee1b1f7a34c1680d085f82567dab9).
+- you can add `pl_tags` when instantializing to tag your requests on PromptLayer
+- you can add `return_pl_id` when instantializing to return a PromptLayer request id to use [while tracking requests](https://magniv.notion.site/Track-4deee1b1f7a34c1680d085f82567dab9).
 
-## Chat Model
-
-```python
-from langchain.chat_models import PromptLayerChatOpenAI
-```
-
-See a [usage example](../modules/models/chat/integrations/promptlayer_chatopenai.ipynb).
 
+PromptLayer also provides native wrappers for [`PromptLayerChatOpenAI`](../modules/models/chat/integrations/promptlayer_chatopenai.ipynb) and `PromptLayerOpenAIChat`

docs/ecosystem/unstructured.md

@@ -1,16 +1,15 @@
 # Unstructured
 
->The `unstructured` package from
+This page covers how to use the [`unstructured`](https://github.com/Unstructured-IO/unstructured)
+ecosystem within LangChain. The `unstructured` package from
 [Unstructured.IO](https://www.unstructured.io/) extracts clean text from raw source documents like
 PDFs and Word documents.
-This page covers how to use the [`unstructured`](https://github.com/Unstructured-IO/unstructured)
-ecosystem within LangChain.
+
+
+This page is broken into two parts: installation and setup, and then references to specific
+`unstructured` wrappers.
 
 ## Installation and Setup
-
-If you are using a loader that runs locally, use the following steps to get `unstructured` and
-its dependencies running locally.
-
 - Install the Python SDK with `pip install "unstructured[local-inference]"`
 - Install the following system dependencies if they are not already available on your system.
   Depending on what document types you're parsing, you may not need all of these.
@@ -19,15 +18,12 @@ its dependencies running locally.
     - `tesseract-ocr`(images and PDFs)
     - `libreoffice` (MS Office docs)
     - `pandoc` (EPUBs)
-
-If you want to get up and running with less set up, you can
-simply run `pip install unstructured` and use `UnstructuredAPIFileLoader` or
-`UnstructuredAPIFileIOLoader`. That will process your document using the hosted Unstructured API.
-Note that currently (as of 1 May 2023) the Unstructured API is open, but it will soon require
-an API. The [Unstructured documentation page](https://unstructured-io.github.io/) will have
-instructions on how to generate an API key once they're available. Check out the instructions
-[here](https://github.com/Unstructured-IO/unstructured-api#dizzy-instructions-for-using-the-docker-image)
-if you'd like to self-host the Unstructured API or run it locally.
+- If you are parsing PDFs using the `"hi_res"` strategy, run the following to install the `detectron2` model, which
+  `unstructured` uses for layout detection:
+    - `pip install "detectron2@git+https://github.com/facebookresearch/detectron2.git@e2ce8dc#egg=detectron2"`
+    - If `detectron2` is not installed, `unstructured` will fallback to processing PDFs
+      using the `"fast"` strategy, which uses `pdfminer` directly and doesn't require
+      `detectron2`.
 
 ## Wrappers
 

docs/ecosystem/wandb_tracking.ipynb

@@ -1,7 +1,6 @@
 {
  "cells": [
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -9,15 +8,9 @@
     "\n",
     "This notebook goes over how to track your LangChain experiments into one centralized Weights and Biases dashboard. To learn more about prompt engineering and the callback please refer to this Report which explains both alongside the resultant dashboards you can expect to see.\n",
     "\n",
+    "Run in Colab: https://colab.research.google.com/drive/1DXH4beT4HFaRKy_Vm4PoxhXVDRf7Ym8L?usp=sharing\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/drive/1DXH4beT4HFaRKy_Vm4PoxhXVDRf7Ym8L?usp=sharing\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
-    "\n",
-    "\n",
-    "[View Report](https://wandb.ai/a-sh0ts/langchain_callback_demo/reports/Prompt-Engineering-LLMs-with-LangChain-and-W-B--VmlldzozNjk1NTUw#👋-how-to-build-a-callback-in-langchain-for-better-prompt-engineering\n",
-    ") \n",
-    "\n",
-    "\n",
-    "**Note**: _the `WandbCallbackHandler` is being deprecated in favour of the `WandbTracer`_ . In future please use the `WandbTracer` as it is more flexible and allows for more granular logging. To know more about the `WandbTracer` refer to the [agent_with_wandb_tracing.ipynb](https://python.langchain.com/en/latest/integrations/agent_with_wandb_tracing.html) notebook or use the following [colab notebook](http://wandb.me/prompts-quickstart). To know more about Weights & Biases Prompts refer to the following [prompts documentation](https://docs.wandb.ai/guides/prompts)."
+    "View Report: https://wandb.ai/a-sh0ts/langchain_callback_demo/reports/Prompt-Engineering-LLMs-with-LangChain-and-W-B--VmlldzozNjk1NTUw#👋-how-to-build-a-callback-in-langchain-for-better-prompt-engineering"
    ]
   },
   {
@@ -57,11 +50,11 @@
    "source": [
     "from datetime import datetime\n",
     "from langchain.callbacks import WandbCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.callbacks.base import CallbackManager\n",
     "from langchain.llms import OpenAI"
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -83,7 +76,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {
     "id": "cxBFfZR8d9FC"
@@ -99,7 +91,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -205,12 +196,11 @@
     "    name=\"llm\",\n",
     "    tags=[\"test\"],\n",
     ")\n",
-    "callbacks = [StdOutCallbackHandler(), wandb_callback]\n",
-    "llm = OpenAI(temperature=0, callbacks=callbacks)"
+    "manager = CallbackManager([StdOutCallbackHandler(), wandb_callback])\n",
+    "llm = OpenAI(temperature=0, callback_manager=manager, verbose=True)"
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {
     "id": "Q-65jwrDeK6w"
@@ -228,7 +218,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -495,7 +484,7 @@
     "Title: {title}\n",
     "Playwright: This is a synopsis for the above play:\"\"\"\n",
     "prompt_template = PromptTemplate(input_variables=[\"title\"], template=template)\n",
-    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=callbacks)\n",
+    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callback_manager=manager)\n",
     "\n",
     "test_prompts = [\n",
     "    {\n",
@@ -588,15 +577,16 @@
    ],
    "source": [
     "# SCENARIO 3 - Agent with Tools\n",
-    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm)\n",
+    "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm, callback_manager=manager)\n",
     "agent = initialize_agent(\n",
     "    tools,\n",
     "    llm,\n",
     "    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
+    "    callback_manager=manager,\n",
+    "    verbose=True,\n",
     ")\n",
     "agent.run(\n",
-    "    \"Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?\",\n",
-    "    callbacks=callbacks,\n",
+    "    \"Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?\"\n",
     ")\n",
     "wandb_callback.flush_tracker(agent, reset=False, finish=True)"
    ]

docs/ecosystem/wolfram_alpha.md

@@ -1,17 +1,12 @@
-# Wolfram Alpha
+# Wolfram Alpha Wrapper
 
->[WolframAlpha](https://en.wikipedia.org/wiki/WolframAlpha) is an answer engine developed by `Wolfram Research`. 
-> It answers factual queries by computing answers from externally sourced data.
-
-This page covers how to use the `Wolfram Alpha API` within LangChain.
+This page covers how to use the Wolfram Alpha API within LangChain.
+It is broken into two parts: installation and setup, and then references to specific Wolfram Alpha wrappers.
 
 ## Installation and Setup
-- Install requirements with 
-```bash
-pip install wolframalpha
-```
+- Install requirements with `pip install wolframalpha`
 - Go to wolfram alpha and sign up for a developer account [here](https://developer.wolframalpha.com/)
-- Create an app and get your `APP ID`
+- Create an app and get your APP ID
 - Set your APP ID as an environment variable `WOLFRAM_ALPHA_APPID`
 
 

docs/ecosystem/zilliz.md

@@ -0,0 +1,21 @@
+# Zilliz
+
+This page covers how to use the Zilliz Cloud ecosystem within LangChain.
+Zilliz uses the Milvus integration. 
+It is broken into two parts: installation and setup, and then references to specific Milvus wrappers.
+
+## Installation and Setup
+- Install the Python SDK with `pip install pymilvus`
+## Wrappers
+
+### VectorStore
+
+There exists a wrapper around Zilliz indexes, allowing you to use it as a vectorstore,
+whether for semantic search or example selection.
+
+To import this vectorstore:
+```python
+from langchain.vectorstores import Milvus
+```
+
+For a more detailed walkthrough of the Miluvs wrapper, see [this notebook](../modules/indexes/vectorstores/examples/zilliz.ipynb)

docs/gallery.rst

@@ -0,0 +1,346 @@
+LangChain Gallery
+=================
+
+Lots of people have built some pretty awesome stuff with LangChain.
+This is a collection of our favorites.
+If you see any other demos that you think we should highlight, be sure to let us know!
+
+
+Open Source
+-----------
+
+.. panels::
+    :body: text-center
+
+    ---
+
+    .. link-button:: https://github.com/bborn/howdoi.ai
+        :type: url
+        :text: HowDoI.ai
+        :classes: stretched-link btn-lg
+
+    +++
+
+    This is an experiment in building a large-language-model-backed chatbot. It can hold a conversation, remember previous comments/questions, 
+    and answer all types of queries (history, web search, movie data, weather, news, and more).
+
+    ---
+
+    .. link-button:: https://colab.research.google.com/drive/1sKSTjt9cPstl_WMZ86JsgEqFG-aSAwkn?usp=sharing
+        :type: url
+        :text: YouTube Transcription QA with Sources
+        :classes: stretched-link btn-lg
+
+    +++
+
+    An end-to-end example of doing question answering on YouTube transcripts, returning the timestamps as sources to legitimize the answer.
+
+    ---
+
+    .. link-button:: https://github.com/normandmickey/MrsStax
+        :type: url
+        :text: QA Slack Bot
+        :classes: stretched-link btn-lg
+
+    +++
+
+    This application is a Slack Bot that uses Langchain and OpenAI's GPT3 language model to provide domain specific answers. You provide the documents.
+
+    ---
+
+    .. link-button:: https://github.com/OpenBioLink/ThoughtSource
+        :type: url
+        :text: ThoughtSource
+        :classes: stretched-link btn-lg
+
+    +++
+
+    A central, open resource and community around data and tools related to chain-of-thought reasoning in large language models.
+
+    ---
+
+    .. link-button:: https://github.com/blackhc/llm-strategy
+        :type: url
+        :text: LLM Strategy
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    This Python package adds a decorator llm_strategy that connects to an LLM (such as OpenAI’s GPT-3) and uses the LLM to "implement" abstract methods in interface classes. It does this by forwarding requests to the LLM and converting the responses back to Python data using Python's @dataclasses.
+
+    ---
+
+    .. link-button:: https://github.com/JohnNay/llm-lobbyist
+        :type: url
+        :text: Zero-Shot Corporate Lobbyist
+        :classes: stretched-link btn-lg
+
+    +++
+
+    A notebook showing how to use GPT to help with the work of a corporate lobbyist.
+
+    ---
+
+    .. link-button:: https://dagster.io/blog/chatgpt-langchain
+        :type: url
+        :text: Dagster Documentation ChatBot
+        :classes: stretched-link btn-lg
+
+    +++
+
+    A jupyter notebook demonstrating how you could create a semantic search engine on documents in one of your Google Folders
+
+    ---
+
+    .. link-button:: https://github.com/venuv/langchain_semantic_search
+        :type: url
+        :text: Google Folder Semantic Search
+        :classes: stretched-link btn-lg
+
+    +++
+
+    Build a GitHub support bot with GPT3, LangChain, and Python.
+
+    ---
+
+    .. link-button:: https://huggingface.co/spaces/team7/talk_with_wind
+        :type: url
+        :text: Talk With Wind
+        :classes: stretched-link btn-lg
+
+    +++
+
+    Record sounds of anything (birds, wind, fire, train station) and chat with it.
+
+    ---
+
+    .. link-button:: https://huggingface.co/spaces/JavaFXpert/Chat-GPT-LangChain
+        :type: url
+        :text: ChatGPT LangChain
+        :classes: stretched-link btn-lg
+
+    +++
+
+    This simple application demonstrates a conversational agent implemented with OpenAI GPT-3.5 and LangChain. When necessary, it leverages tools for complex math, searching the internet, and accessing news and weather.
+
+    ---
+
+    .. link-button:: https://huggingface.co/spaces/JavaFXpert/gpt-math-techniques
+        :type: url
+        :text: GPT Math Techniques
+        :classes: stretched-link btn-lg
+
+    +++
+
+    A Hugging Face spaces project showing off the benefits of using PAL for math problems.
+
+    ---
+
+    .. link-button:: https://colab.research.google.com/drive/1xt2IsFPGYMEQdoJFNgWNAjWGxa60VXdV
+        :type: url
+        :text: GPT Political Compass
+        :classes: stretched-link btn-lg
+
+    +++
+
+    Measure the political compass of GPT.
+
+    ---
+
+    .. link-button:: https://github.com/hwchase17/notion-qa
+        :type: url
+        :text: Notion Database Question-Answering Bot
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    Open source GitHub project shows how to use LangChain to create a chatbot that can answer questions about an arbitrary Notion database.
+
+    ---
+
+    .. link-button:: https://github.com/jerryjliu/llama_index
+        :type: url
+        :text: LlamaIndex
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    LlamaIndex (formerly GPT Index) is a project consisting of a set of data structures that are created using GPT-3 and can be traversed using GPT-3 in order to answer queries.
+
+    ---
+
+    .. link-button:: https://github.com/JavaFXpert/llm-grovers-search-party
+        :type: url
+        :text: Grover's Algorithm
+        :classes: stretched-link btn-lg
+
+    +++
+
+    Leveraging Qiskit, OpenAI and LangChain to demonstrate Grover's algorithm
+
+    ---
+
+    .. link-button:: https://huggingface.co/spaces/rituthombre/QNim
+        :type: url
+        :text: QNimGPT
+        :classes: stretched-link btn-lg
+
+    +++
+
+    A chat UI to play Nim, where a player can select an opponent, either a quantum computer or an AI
+
+    ---
+
+    .. link-button:: https://colab.research.google.com/drive/19WTIWC3prw5LDMHmRMvqNV2loD9FHls6?usp=sharing
+        :type: url
+        :text: ReAct TextWorld
+        :classes: stretched-link btn-lg
+
+    +++
+
+    Leveraging the ReActTextWorldAgent to play TextWorld with an LLM!
+
+    ---
+
+    .. link-button:: https://github.com/jagilley/fact-checker
+        :type: url
+        :text: Fact Checker
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    This repo is a simple demonstration of using LangChain to do fact-checking with prompt chaining.
+    
+    ---
+
+    .. link-button:: https://github.com/arc53/docsgpt
+        :type: url
+        :text: DocsGPT
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    Answer questions about the documentation of any project    
+
+Misc. Colab Notebooks
+~~~~~~~~~~~~~~~~~~~~~
+
+.. panels::
+    :body: text-center
+
+    ---
+
+    .. link-button:: https://colab.research.google.com/drive/1AAyEdTz-Z6ShKvewbt1ZHUICqak0MiwR?usp=sharing
+        :type: url
+        :text: Wolfram Alpha in Conversational Agent
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    Give ChatGPT a WolframAlpha neural implant
+    
+    ---
+
+    .. link-button:: https://colab.research.google.com/drive/1UsCLcPy8q5PMNQ5ytgrAAAHa124dzLJg?usp=sharing
+        :type: url
+        :text: Tool Updates in Agents
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    Agent improvements (6th Jan 2023)
+    
+    ---
+
+    .. link-button:: https://colab.research.google.com/drive/1UsCLcPy8q5PMNQ5ytgrAAAHa124dzLJg?usp=sharing
+        :type: url
+        :text: Conversational Agent with Tools (Langchain AGI)
+        :classes: stretched-link btn-lg
+    
+    +++
+
+    Langchain AGI (23rd Dec 2022)    
+
+Proprietary
+-----------
+
+.. panels::
+    :body: text-center
+
+    ---
+
+    .. link-button:: https://twitter.com/sjwhitmore/status/1580593217153531908?s=20&t=neQvtZZTlp623U3LZwz3bQ
+        :type: url
+        :text: Daimon
+        :classes: stretched-link btn-lg
+
+    +++
+
+    A chat-based AI personal assistant with long-term memory about you.
+
+    ---
+
+    .. link-button:: https://anysummary.app
+        :type: url
+        :text: Summarize any file with AI
+        :classes: stretched-link btn-lg
+
+    +++
+
+    Summarize not only long docs, interview audio or video files quickly, but also entire websites and YouTube videos. Share or download your generated summaries to collaborate with others, or revisit them at any time! Bonus: `@anysummary <https://twitter.com/anysummary>`_ on Twitter will also summarize any thread it is tagged in.
+    
+    ---
+
+    .. link-button:: https://twitter.com/dory111111/status/1608406234646052870?s=20&t=XYlrbKM0ornJsrtGa0br-g
+        :type: url
+        :text: AI Assisted SQL Query Generator
+        :classes: stretched-link btn-lg
+
+    +++
+
+    An app to write SQL using natural language, and execute against real DB.
+
+    ---
+
+    .. link-button:: https://twitter.com/krrish_dh/status/1581028925618106368?s=20&t=neQvtZZTlp623U3LZwz3bQ
+        :type: url
+        :text: Clerkie
+        :classes: stretched-link btn-lg
+
+    +++
+
+    Stack Tracing QA Bot to help debug complex stack tracing (especially the ones that go multi-function/file deep).
+
+    ---
+
+    .. link-button:: https://twitter.com/Raza_Habib496/status/1596880140490838017?s=20&t=6MqEQYWfSqmJwsKahjCVOA
+        :type: url
+        :text: Sales Email Writer
+        :classes: stretched-link btn-lg
+
+    +++
+
+    By Raza Habib, this demo utilizes LangChain + SerpAPI + HumanLoop to write sales emails. Give it a company name and a person, this application will use Google Search (via SerpAPI) to get more information on the company and the person, and then write them a sales message.
+
+    ---
+
+    .. link-button:: https://twitter.com/chillzaza_/status/1592961099384905730?s=20&t=EhU8jl0KyCPJ7vE9Rnz-cQ
+        :type: url
+        :text: Question-Answering on a Web Browser
+        :classes: stretched-link btn-lg
+
+    +++
+
+    By Zahid Khawaja, this demo utilizes question answering to answer questions about a given website. A followup added this for `YouTube videos <https://twitter.com/chillzaza_/status/1593739682013220865?s=20&t=EhU8jl0KyCPJ7vE9Rnz-cQ>`_, and then another followup added it for `Wikipedia <https://twitter.com/chillzaza_/status/1594847151238037505?s=20&t=EhU8jl0KyCPJ7vE9Rnz-cQ>`_.
+
+    ---
+
+    .. link-button:: https://mynd.so
+        :type: url
+        :text: Mynd
+        :classes: stretched-link btn-lg
+
+    +++
+
+    A journaling app for self-care that uses AI to uncover insights and patterns over time.
+    

docs/getting_started/concepts.md

@@ -1,75 +0,0 @@
-# Concepts
-
-These are concepts and terminology commonly used when developing LLM applications.
-It contains reference to external papers or sources where the concept was first introduced,
-as well as to places in LangChain where the concept is used.
-
-## Chain of Thought
-
-`Chain of Thought (CoT)` is a prompting technique used to encourage the model to generate a series of intermediate reasoning steps.
-A less formal way to induce this behavior is to include “Let’s think step-by-step” in the prompt.
-
-- [Chain-of-Thought Paper](https://arxiv.org/pdf/2201.11903.pdf)
-- [Step-by-Step Paper](https://arxiv.org/abs/2112.00114)
-
-## Action Plan Generation
-
-`Action Plan Generation` is a prompting technique that uses a language model to generate actions to take.
-The results of these actions can then be fed back into the language model to generate a subsequent action.
-
-- [WebGPT Paper](https://arxiv.org/pdf/2112.09332.pdf)
-- [SayCan Paper](https://say-can.github.io/assets/palm_saycan.pdf)
-
-## ReAct
-
-`ReAct` is a prompting technique that combines Chain-of-Thought prompting with action plan generation.
-This induces the model to think about what action to take, then take it.
-
-- [Paper](https://arxiv.org/pdf/2210.03629.pdf)
-- [LangChain Example](../modules/agents/agents/examples/react.ipynb)
-
-## Self-ask
-
-`Self-ask` is a prompting method that builds on top of chain-of-thought prompting.
-In this method, the model explicitly asks itself follow-up questions, which are then answered by an external search engine.
-
-- [Paper](https://ofir.io/self-ask.pdf)
-- [LangChain Example](../modules/agents/agents/examples/self_ask_with_search.ipynb)
-
-## Prompt Chaining
-
-`Prompt Chaining` is combining multiple LLM calls, with the output of one-step being the input to the next.
-
-- [PromptChainer Paper](https://arxiv.org/pdf/2203.06566.pdf)
-- [Language Model Cascades](https://arxiv.org/abs/2207.10342)
-- [ICE Primer Book](https://primer.ought.org/)
-- [Socratic Models](https://socraticmodels.github.io/)
-
-## Memetic Proxy
-
-`Memetic Proxy` is encouraging the LLM
-to respond in a certain way framing the discussion in a context that the model knows of and that 
-will result in that type of response.
-For example, as a conversation between a student and a teacher.
-
-- [Paper](https://arxiv.org/pdf/2102.07350.pdf)
-
-## Self Consistency
-
-`Self Consistency` is a decoding strategy that samples a diverse set of reasoning paths and then selects the most consistent answer.
-Is most effective when combined with Chain-of-thought prompting.
-
-- [Paper](https://arxiv.org/pdf/2203.11171.pdf)
-
-## Inception
-
-`Inception` is also called `First Person Instruction`.
-It is encouraging the model to think a certain way by including the start of the model’s response in the prompt.
-
-- [Example](https://twitter.com/goodside/status/1583262455207460865?s=20&t=8Hz7XBnK1OF8siQrxxCIGQ)
-
-## MemPrompt
-
-`MemPrompt` maintains a memory of errors and user feedback, and uses them to prevent repetition of mistakes.
-
-- [Paper](https://memprompt.com/)

docs/getting_started/getting_started.md

@@ -37,12 +37,6 @@ import os
 os.environ["OPENAI_API_KEY"] = "..."
 ```
 
-If you want to set the API key dynamically, you can use the openai_api_key parameter when initiating OpenAI class—for instance, each user's API key.
-
-```python
-from langchain.llms import OpenAI
-llm = OpenAI(openai_api_key="OPENAI_API_KEY")
-```
 
 ## Building a Language Model Application: LLMs
 
@@ -178,9 +172,9 @@ In order to load agents, you should understand the following concepts:
 - LLM: The language model powering the agent.
 - Agent: The agent to use. This should be a string that references a support agent class. Because this notebook focuses on the simplest, highest level API, this only covers using the standard supported agents. If you want to implement a custom agent, see the documentation for custom agents (coming soon).
 
-**Agents**: For a list of supported agents and their specifications, see [here](../modules/agents/getting_started.ipynb).
+**Agents**: For a list of supported agents and their specifications, see [here](../modules/agents/agents.md).
 
-**Tools**: For a list of predefined tools and their specifications, see [here](../modules/agents/tools/getting_started.md).
+**Tools**: For a list of predefined tools and their specifications, see [here](../modules/agents/tools.md).
 
 For this example, you will also need to install the SerpAPI Python package.
 
@@ -322,7 +316,7 @@ You can also pass in multiple messages for OpenAI's gpt-3.5-turbo and gpt-4 mode
 ```python
 messages = [
     SystemMessage(content="You are a helpful assistant that translates English to French."),
-    HumanMessage(content="I love programming.")
+    HumanMessage(content="Translate this sentence from English to French. I love programming.")
 ]
 chat(messages)
 # -> AIMessage(content="J'aime programmer.", additional_kwargs={})
@@ -333,29 +327,29 @@ You can go one step further and generate completions for multiple sets of messag
 batch_messages = [
     [
         SystemMessage(content="You are a helpful assistant that translates English to French."),
-        HumanMessage(content="I love programming.")
+        HumanMessage(content="Translate this sentence from English to French. I love programming.")
     ],
     [
         SystemMessage(content="You are a helpful assistant that translates English to French."),
-        HumanMessage(content="I love artificial intelligence.")
+        HumanMessage(content="Translate this sentence from English to French. I love artificial intelligence.")
     ],
 ]
 result = chat.generate(batch_messages)
 result
-# -> LLMResult(generations=[[ChatGeneration(text="J'aime programmer.", generation_info=None, message=AIMessage(content="J'aime programmer.", additional_kwargs={}))], [ChatGeneration(text="J'aime l'intelligence artificielle.", generation_info=None, message=AIMessage(content="J'aime l'intelligence artificielle.", additional_kwargs={}))]], llm_output={'token_usage': {'prompt_tokens': 57, 'completion_tokens': 20, 'total_tokens': 77}})
+# -> LLMResult(generations=[[ChatGeneration(text="J'aime programmer.", generation_info=None, message=AIMessage(content="J'aime programmer.", additional_kwargs={}))], [ChatGeneration(text="J'aime l'intelligence artificielle.", generation_info=None, message=AIMessage(content="J'aime l'intelligence artificielle.", additional_kwargs={}))]], llm_output={'token_usage': {'prompt_tokens': 71, 'completion_tokens': 18, 'total_tokens': 89}})
 ```
 
 You can recover things like token usage from this LLMResult:
 ```
 result.llm_output['token_usage']
-# -> {'prompt_tokens': 57, 'completion_tokens': 20, 'total_tokens': 77}
+# -> {'prompt_tokens': 71, 'completion_tokens': 18, 'total_tokens': 89}
 ```
 
 
 ## Chat Prompt Templates
 Similar to LLMs, you can make use of templating by using a `MessagePromptTemplate`. You can build a `ChatPromptTemplate` from one or more `MessagePromptTemplate`s. You can use `ChatPromptTemplate`'s `format_prompt` -- this returns a `PromptValue`, which you can convert to a string or `Message` object, depending on whether you want to use the formatted value as input to an llm or chat model.
 
-For convenience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:
+For convience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:
 
 ```python
 from langchain.chat_models import ChatOpenAI
@@ -367,9 +361,9 @@ from langchain.prompts.chat import (
 
 chat = ChatOpenAI(temperature=0)
 
-template = "You are a helpful assistant that translates {input_language} to {output_language}."
+template="You are a helpful assistant that translates {input_language} to {output_language}."
 system_message_prompt = SystemMessagePromptTemplate.from_template(template)
-human_template = "{text}"
+human_template="{text}"
 human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
 
 chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])
@@ -393,9 +387,9 @@ from langchain.prompts.chat import (
 
 chat = ChatOpenAI(temperature=0)
 
-template = "You are a helpful assistant that translates {input_language} to {output_language}."
+template="You are a helpful assistant that translates {input_language} to {output_language}."
 system_message_prompt = SystemMessagePromptTemplate.from_template(template)
-human_template = "{text}"
+human_template="{text}"
 human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
 chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])
 

docs/getting_started/tutorials.md

@@ -1,113 +0,0 @@
-# Tutorials
-
-⛓ icon marks a new addition [last update 2023-05-15]
-
-### DeepLearning.AI course
-⛓[LangChain for LLM Application Development](https://learn.deeplearning.ai/langchain) by Harrison Chase presented by [Andrew Ng](https://en.wikipedia.org/wiki/Andrew_Ng)
-
-### Handbook
-[LangChain AI Handbook](https://www.pinecone.io/learn/langchain/) By **James Briggs** and **Francisco Ingham**
-
-### Tutorials
-[LangChain Tutorials](https://www.youtube.com/watch?v=FuqdVNB_8c0&list=PL9V0lbeJ69brU-ojMpU1Y7Ic58Tap0Cw6) by [Edrick](https://www.youtube.com/@edrickdch):
-- ⛓ [LangChain, Chroma DB, OpenAI Beginner Guide | ChatGPT with your PDF](https://youtu.be/FuqdVNB_8c0)
-- ⛓ [LangChain 101: The Complete Beginner's Guide](https://youtu.be/P3MAbZ2eMUI) 
-
-[LangChain Crash Course: Build an AutoGPT app in 25 minutes](https://youtu.be/MlK6SIjcjE8) by [Nicholas Renotte](https://www.youtube.com/@NicholasRenotte)
-
-
-[LangChain Crash Course - Build apps with language models](https://youtu.be/LbT1yp6quS8) by [Patrick Loeber](https://www.youtube.com/@patloeber)
-
-
-[LangChain Explained in 13 Minutes | QuickStart Tutorial for Beginners](https://youtu.be/aywZrzNaKjs) by [Rabbitmetrics](https://www.youtube.com/@rabbitmetrics)
-
-
-###
-[LangChain for Gen AI and LLMs](https://www.youtube.com/playlist?list=PLIUOU7oqGTLieV9uTIFMm6_4PXg-hlN6F) by [James Briggs](https://www.youtube.com/@jamesbriggs):
-- #1 [Getting Started with `GPT-3` vs. Open Source LLMs](https://youtu.be/nE2skSRWTTs)
-- #2 [Prompt Templates for `GPT 3.5` and other LLMs](https://youtu.be/RflBcK0oDH0)
-- #3 [LLM Chains using `GPT 3.5` and other LLMs](https://youtu.be/S8j9Tk0lZHU)
-- #4 [Chatbot Memory for `Chat-GPT`, `Davinci` + other LLMs](https://youtu.be/X05uK0TZozM)
-- #5 [Chat with OpenAI in LangChain](https://youtu.be/CnAgB3A5OlU)
-- ⛓ #6 [Fixing LLM Hallucinations with Retrieval Augmentation in LangChain](https://youtu.be/kvdVduIJsc8)
-- ⛓ #7 [LangChain Agents Deep Dive with GPT 3.5](https://youtu.be/jSP-gSEyVeI)
-- ⛓ #8 [Create Custom Tools for Chatbots in LangChain](https://youtu.be/q-HNphrWsDE)
-- ⛓ #9 [Build Conversational Agents with Vector DBs](https://youtu.be/H6bCqqw9xyI)
-
-
-###
-[LangChain 101](https://www.youtube.com/playlist?list=PLqZXAkvF1bPNQER9mLmDbntNfSpzdDIU5) by [Data Independent](https://www.youtube.com/@DataIndependent):
-- [What Is LangChain? - LangChain + `ChatGPT` Overview](https://youtu.be/_v_fgW2SkkQ)
-- [Quickstart Guide](https://youtu.be/kYRB-vJFy38)
-- [Beginner Guide To 7 Essential Concepts](https://youtu.be/2xxziIWmaSA)
-- [`OpenAI` + `Wolfram Alpha`](https://youtu.be/UijbzCIJ99g)
-- [Ask Questions On Your Custom (or Private) Files](https://youtu.be/EnT-ZTrcPrg)
-- [Connect `Google Drive Files` To `OpenAI`](https://youtu.be/IqqHqDcXLww)
-- [`YouTube Transcripts` + `OpenAI`](https://youtu.be/pNcQ5XXMgH4)
-- [Question A 300 Page Book (w/ `OpenAI` + `Pinecone`)](https://youtu.be/h0DHDp1FbmQ)
-- [Workaround `OpenAI's` Token Limit With Chain Types](https://youtu.be/f9_BWhCI4Zo)
-- [Build Your Own OpenAI + LangChain Web App in 23 Minutes](https://youtu.be/U_eV8wfMkXU)
-- [Working With The New `ChatGPT API`](https://youtu.be/e9P7FLi5Zy8)
-- [OpenAI + LangChain Wrote Me 100 Custom Sales Emails](https://youtu.be/y1pyAQM-3Bo)
-- [Structured Output From `OpenAI` (Clean Dirty Data)](https://youtu.be/KwAXfey-xQk)
-- [Connect `OpenAI` To +5,000 Tools (LangChain + `Zapier`)](https://youtu.be/7tNm0yiDigU)
-- [Use LLMs To Extract Data From Text (Expert Mode)](https://youtu.be/xZzvwR9jdPA)
-- ⛓ [Extract Insights From Interview Transcripts Using LLMs](https://youtu.be/shkMOHwJ4SM)
-- ⛓ [5 Levels Of LLM Summarizing: Novice to Expert](https://youtu.be/qaPMdcCqtWk)
-
-
-###
-[LangChain How to and guides](https://www.youtube.com/playlist?list=PL8motc6AQftk1Bs42EW45kwYbyJ4jOdiZ) by [Sam Witteveen](https://www.youtube.com/@samwitteveenai):
-- [LangChain Basics - LLMs & PromptTemplates with Colab](https://youtu.be/J_0qvRt4LNk)
-- [LangChain Basics - Tools and Chains](https://youtu.be/hI2BY7yl_Ac)
-- [`ChatGPT API` Announcement & Code Walkthrough with LangChain](https://youtu.be/phHqvLHCwH4)
-- [Conversations with Memory (explanation & code walkthrough)](https://youtu.be/X550Zbz_ROE)
-- [Chat with `Flan20B`](https://youtu.be/VW5LBavIfY4)
-- [Using `Hugging Face Models` locally (code walkthrough)](https://youtu.be/Kn7SX2Mx_Jk)
-- [`PAL` : Program-aided Language Models with LangChain code](https://youtu.be/dy7-LvDu-3s)
-- [Building a Summarization System with LangChain and `GPT-3` - Part 1](https://youtu.be/LNq_2s_H01Y)
-- [Building a Summarization System with LangChain and `GPT-3` - Part 2](https://youtu.be/d-yeHDLgKHw)
-- [Microsoft's `Visual ChatGPT` using LangChain](https://youtu.be/7YEiEyfPF5U)
-- [LangChain Agents - Joining Tools and Chains with Decisions](https://youtu.be/ziu87EXZVUE)
-- [Comparing LLMs with LangChain](https://youtu.be/rFNG0MIEuW0)
-- [Using `Constitutional AI` in LangChain](https://youtu.be/uoVqNFDwpX4)
-- [Talking to `Alpaca` with LangChain - Creating an Alpaca Chatbot](https://youtu.be/v6sF8Ed3nTE)
-- [Talk to your `CSV` & `Excel` with LangChain](https://youtu.be/xQ3mZhw69bc)
-- [`BabyAGI`: Discover the Power of Task-Driven Autonomous Agents!](https://youtu.be/QBcDLSE2ERA)
-- [Improve your `BabyAGI` with LangChain](https://youtu.be/DRgPyOXZ-oE)
-- ⛓ [Master `PDF` Chat with LangChain - Your essential guide to queries on documents](https://youtu.be/ZzgUqFtxgXI)
-- ⛓ [Using LangChain with `DuckDuckGO` `Wikipedia` & `PythonREPL` Tools](https://youtu.be/KerHlb8nuVc)
-- ⛓ [Building Custom Tools and Agents with LangChain (gpt-3.5-turbo)](https://youtu.be/biS8G8x8DdA)
-- ⛓ [LangChain Retrieval QA Over Multiple Files with `ChromaDB`](https://youtu.be/3yPBVii7Ct0)
-- ⛓ [LangChain Retrieval QA with Instructor Embeddings & `ChromaDB` for PDFs](https://youtu.be/cFCGUjc33aU)
-- ⛓ [LangChain + Retrieval Local LLMs for Retrieval QA - No OpenAI!!!](https://youtu.be/9ISVjh8mdlA)
-
-
-###
-[LangChain](https://www.youtube.com/playlist?list=PLVEEucA9MYhOu89CX8H3MBZqayTbcCTMr) by [Prompt Engineering](https://www.youtube.com/@engineerprompt):
-- [LangChain Crash Course — All You Need to Know to Build Powerful Apps with LLMs](https://youtu.be/5-fc4Tlgmro)
-- [Working with MULTIPLE `PDF` Files in LangChain: `ChatGPT` for your Data](https://youtu.be/s5LhRdh5fu4)
-- [`ChatGPT` for YOUR OWN `PDF` files with LangChain](https://youtu.be/TLf90ipMzfE)
-- [Talk to YOUR DATA without OpenAI APIs: LangChain](https://youtu.be/wrD-fZvT6UI)
-- ⛓️ [CHATGPT For WEBSITES: Custom ChatBOT](https://youtu.be/RBnuhhmD21U)
-
-
-###
-LangChain by [Chat with data](https://www.youtube.com/@chatwithdata)
-- [LangChain Beginner's Tutorial for `Typescript`/`Javascript`](https://youtu.be/bH722QgRlhQ)
-- [`GPT-4` Tutorial: How to Chat With Multiple `PDF` Files (~1000 pages of Tesla's 10-K Annual Reports)](https://youtu.be/Ix9WIZpArm0)
-- [`GPT-4` & LangChain Tutorial: How to Chat With A 56-Page `PDF` Document (w/`Pinecone`)](https://youtu.be/ih9PBGVVOO4)
-- ⛓ [LangChain & Supabase Tutorial: How to Build a ChatGPT Chatbot For Your Website](https://youtu.be/R2FMzcsmQY8)
-
-
-###
-[Get SH\*T Done with Prompt Engineering and LangChain](https://www.youtube.com/watch?v=muXbPpG_ys4&list=PLEJK-H61Xlwzm5FYLDdKt_6yibO33zoMW) by [Venelin Valkov](https://www.youtube.com/@venelin_valkov)
-- [Getting Started with LangChain: Load Custom Data, Run OpenAI Models, Embeddings and `ChatGPT`](https://www.youtube.com/watch?v=muXbPpG_ys4)
-- [Loaders, Indexes & Vectorstores in LangChain: Question Answering on `PDF` files with `ChatGPT`](https://www.youtube.com/watch?v=FQnvfR8Dmr0)
-- [LangChain Models: `ChatGPT`, `Flan Alpaca`, `OpenAI Embeddings`, Prompt Templates & Streaming](https://www.youtube.com/watch?v=zy6LiK5F5-s)
-- [LangChain Chains: Use `ChatGPT` to Build Conversational Agents, Summaries and Q&A on Text With LLMs](https://www.youtube.com/watch?v=h1tJZQPcimM)
-- [Analyze Custom CSV Data with `GPT-4` using Langchain](https://www.youtube.com/watch?v=Ew3sGdX8at4)
-- ⛓ [Build ChatGPT Chatbots with LangChain Memory: Understanding and Implementing Memory in Conversations](https://youtu.be/CyuUlf54wTs)
-
----------------------
-⛓ icon marks a new addition [last update 2023-05-15]

docs/glossary.md

@@ -0,0 +1,90 @@
+# Glossary
+
+This is a collection of terminology commonly used when developing LLM applications.
+It contains reference to external papers or sources where the concept was first introduced,
+as well as to places in LangChain where the concept is used.
+
+## Chain of Thought Prompting
+
+A prompting technique used to encourage the model to generate a series of intermediate reasoning steps.
+A less formal way to induce this behavior is to include “Let’s think step-by-step” in the prompt.
+
+Resources:
+
+- [Chain-of-Thought Paper](https://arxiv.org/pdf/2201.11903.pdf)
+- [Step-by-Step Paper](https://arxiv.org/abs/2112.00114)
+
+## Action Plan Generation
+
+A prompt usage that uses a language model to generate actions to take.
+The results of these actions can then be fed back into the language model to generate a subsequent action.
+
+Resources:
+
+- [WebGPT Paper](https://arxiv.org/pdf/2112.09332.pdf)
+- [SayCan Paper](https://say-can.github.io/assets/palm_saycan.pdf)
+
+## ReAct Prompting
+
+A prompting technique that combines Chain-of-Thought prompting with action plan generation.
+This induces the to model to think about what action to take, then take it.
+
+Resources:
+
+- [Paper](https://arxiv.org/pdf/2210.03629.pdf)
+- [LangChain Example](modules/agents/agents/examples/react.ipynb)
+
+## Self-ask
+
+A prompting method that builds on top of chain-of-thought prompting.
+In this method, the model explicitly asks itself follow-up questions, which are then answered by an external search engine.
+
+Resources:
+
+- [Paper](https://ofir.io/self-ask.pdf)
+- [LangChain Example](modules/agents/agents/examples/self_ask_with_search.ipynb)
+
+## Prompt Chaining
+
+Combining multiple LLM calls together, with the output of one-step being the input to the next.
+
+Resources:
+
+- [PromptChainer Paper](https://arxiv.org/pdf/2203.06566.pdf)
+- [Language Model Cascades](https://arxiv.org/abs/2207.10342)
+- [ICE Primer Book](https://primer.ought.org/)
+- [Socratic Models](https://socraticmodels.github.io/)
+
+## Memetic Proxy
+
+Encouraging the LLM to respond in a certain way framing the discussion in a context that the model knows of and that will result in that type of response. For example, as a conversation between a student and a teacher.
+
+Resources:
+
+- [Paper](https://arxiv.org/pdf/2102.07350.pdf)
+
+## Self Consistency
+
+A decoding strategy that samples a diverse set of reasoning paths and then selects the most consistent answer.
+Is most effective when combined with Chain-of-thought prompting.
+
+Resources:
+
+- [Paper](https://arxiv.org/pdf/2203.11171.pdf)
+
+## Inception
+
+Also called “First Person Instruction”.
+Encouraging the model to think a certain way by including the start of the model’s response in the prompt.
+
+Resources:
+
+- [Example](https://twitter.com/goodside/status/1583262455207460865?s=20&t=8Hz7XBnK1OF8siQrxxCIGQ)
+
+## MemPrompt
+
+MemPrompt maintains a memory of errors and user feedback, and uses them to prevent repetition of mistakes.
+
+Resources:
+
+- [Paper](https://memprompt.com/)

docs/index.rst

@@ -1,63 +1,49 @@
 Welcome to LangChain
 ==========================
 
-| **LangChain** is a framework for developing applications powered by language models. We believe that the most powerful and differentiated applications will not only call out to a language model, but will also be:
-1. *Data-aware*: connect a language model to other sources of data
-2. *Agentic*: allow a language model to interact with its environment
+LangChain is a framework for developing applications powered by language models. We believe that the most powerful and differentiated applications will not only call out to a language model via an API, but will also:
 
-| The LangChain framework is designed around these principles.
+- *Be data-aware*: connect a language model to other sources of data
+- *Be agentic*: allow a language model to interact with its environment
 
-| This is the Python specific portion of the documentation. For a purely conceptual guide to LangChain, see `here <https://docs.langchain.com/docs/>`_. For the JavaScript documentation, see `here <https://js.langchain.com/docs/>`_.
+The LangChain framework is designed with the above principles in mind.
+
+This is the Python specific portion of the documentation. For a purely conceptual guide to LangChain, see `here <https://docs.langchain.com/docs/>`_. For the JavaScript documentation, see `here <https://js.langchain.com/docs/>`_.
 
 Getting Started
 ----------------
 
-| How to get started using LangChain to create an Language Model application.
+Checkout the below guide for a walkthrough of how to get started using LangChain to create an Language Model application.
 
-- `Quickstart Guide <./getting_started/getting_started.html>`_
-
-| Concepts and terminology.
-
-- `Concepts and terminology <./getting_started/concepts.html>`_
-
-| Tutorials created by community experts and presented on YouTube.
-
-- `Tutorials <./getting_started/tutorials.html>`_
+- `Getting Started Documentation <./getting_started/getting_started.html>`_
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :caption: Getting Started
    :name: getting_started
    :hidden:
 
    getting_started/getting_started.md
-   getting_started/concepts.md
-   getting_started/tutorials.md
-
 
 Modules
 -----------
 
-| These modules are the core abstractions which we view as the building blocks of any LLM-powered application.
-For each module LangChain provides standard, extendable interfaces. LangChain also provides external integrations and even end-to-end implementations for off-the-shelf use.
+There are several main modules that LangChain provides support for.
+For each module we provide some examples to get started, how-to guides, reference docs, and conceptual guides.
+These modules are, in increasing order of complexity:
 
-| The docs for each module contain quickstart examples, how-to guides, reference docs, and conceptual guides.
+- `Models <./modules/models.html>`_: The various model types and model integrations LangChain supports.
 
-| The modules are (from least to most complex):
+- `Prompts <./modules/prompts.html>`_: This includes prompt management, prompt optimization, and prompt serialization.
 
-- `Models <./modules/models.html>`_: Supported model types and integrations.
+- `Memory <./modules/memory.html>`_: Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.
 
-- `Prompts <./modules/prompts.html>`_: Prompt management, optimization, and serialization.
+- `Indexes <./modules/indexes.html>`_: Language models are often more powerful when combined with your own text data - this module covers best practices for doing exactly that.
 
-- `Memory <./modules/memory.html>`_: Memory refers to state that is persisted between calls of a chain/agent.
+- `Chains <./modules/chains.html>`_: Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.
 
-- `Indexes <./modules/indexes.html>`_: Language models become much more powerful when combined with application-specific data - this module contains interfaces and integrations for loading, querying and updating external data.
+- `Agents <./modules/agents.html>`_: Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.
 
-- `Chains <./modules/chains.html>`_: Chains are structured sequences of calls (to an LLM or to a different utility).
-
-- `Agents <./modules/agents.html>`_: An agent is a Chain in which an LLM, given a high-level directive and a set of tools, repeatedly decides an action, executes the action and observes the outcome until the high-level directive is complete.
-
-- `Callbacks <./modules/callbacks/getting_started.html>`_: Callbacks let you log and stream the intermediate steps of any chain, making it easy to observe, debug, and evaluate the internals of an application.
 
 .. toctree::
    :maxdepth: 1
@@ -67,38 +53,37 @@ For each module LangChain provides standard, extendable interfaces. LangChain al
 
    ./modules/models.rst
    ./modules/prompts.rst
-   ./modules/memory.md
    ./modules/indexes.md
+   ./modules/memory.md
    ./modules/chains.md
    ./modules/agents.md
-   ./modules/callbacks/getting_started.ipynb
 
 Use Cases
 ----------
 
-| Best practices and built-in implementations for common LangChain use cases:
+The above modules can be used in a variety of ways. LangChain also provides guidance and assistance in this. Below are some of the common use cases LangChain supports.
 
-- `Autonomous Agents <./use_cases/autonomous_agents.html>`_: Autonomous agents are long-running agents that take many steps in an attempt to accomplish an objective. Examples include AutoGPT and BabyAGI.
+- `Autonomous Agents <./use_cases/autonomous_agents.html>`_: Autonomous agents are long running agents that take many steps in an attempt to accomplish an objective. Examples include AutoGPT and BabyAGI.
 
-- `Agent Simulations <./use_cases/agent_simulations.html>`_: Putting agents in a sandbox and observing how they interact with each other and react to events can be an effective way to evaluate their long-range reasoning and planning abilities.
+- `Agent Simulations <./use_cases/agent_simulations.html>`_: Putting agents in a sandbox and observing how they interact with each other or to events can be an interesting way to observe their long-term memory abilities.
 
-- `Personal Assistants <./use_cases/personal_assistants.html>`_: One of the primary LangChain use cases. Personal assistants need to take actions, remember interactions, and have knowledge about your data.
+- `Personal Assistants <./use_cases/personal_assistants.html>`_: The main LangChain use case. Personal assistants need to take actions, remember interactions, and have knowledge about your data.
 
-- `Question Answering <./use_cases/question_answering.html>`_: Another common LangChain use case. Answering questions over specific documents, only utilizing the information in those documents to construct an answer.
+- `Question Answering <./use_cases/question_answering.html>`_: The second big LangChain use case. Answering questions over specific documents, only utilizing the information in those documents to construct an answer.
 
-- `Chatbots <./use_cases/chatbots.html>`_: Language models love to chat, making this a very natural use of them.
+- `Chatbots <./use_cases/chatbots.html>`_: Since language models are good at producing text, that makes them ideal for creating chatbots.
 
-- `Querying Tabular Data <./use_cases/tabular.html>`_: Recommended reading if you want to use language models to query structured data (CSVs, SQL, dataframes, etc).
+- `Querying Tabular Data <./use_cases/tabular.html>`_: If you want to understand how to use LLMs to query data that is stored in a tabular format (csvs, SQL, dataframes, etc) you should read this page.
 
-- `Code Understanding <./use_cases/code.html>`_: Recommended reading if you want to use language models to analyze code.
+- `Code Understanding <./use_cases/code.html>`_: If you want to understand how to use LLMs to query source code from github, you should read this page.
 
-- `Interacting with APIs <./use_cases/apis.html>`_: Enabling language models to interact with APIs is extremely powerful. It gives them access to up-to-date information and allows them to take actions.
+- `Interacting with APIs <./use_cases/apis.html>`_: Enabling LLMs to interact with APIs is extremely powerful in order to give them more up-to-date information and allow them to take actions.
 
 - `Extraction <./use_cases/extraction.html>`_: Extract structured information from text.
 
-- `Summarization <./use_cases/summarization.html>`_: Compressing longer documents. A type of Data-Augmented Generation.
+- `Summarization <./use_cases/summarization.html>`_: Summarizing longer documents into shorter, more condensed chunks of information. A type of Data Augmented Generation.
 
-- `Evaluation <./use_cases/evaluation.html>`_: Generative models are hard to evaluate with traditional metrics. One promising approach is to use language models themselves to do the evaluation.
+- `Evaluation <./use_cases/evaluation.html>`_: Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.
 
 
 .. toctree::
@@ -107,29 +92,26 @@ Use Cases
    :name: use_cases
    :hidden:
 
+   ./use_cases/personal_assistants.md
    ./use_cases/autonomous_agents.md
    ./use_cases/agent_simulations.md
-   ./use_cases/personal_assistants.md
    ./use_cases/question_answering.md
    ./use_cases/chatbots.md
    ./use_cases/tabular.rst
    ./use_cases/code.md
    ./use_cases/apis.md
-   ./use_cases/extraction.md
    ./use_cases/summarization.md
+   ./use_cases/extraction.md
    ./use_cases/evaluation.rst
 
 
 Reference Docs
 ---------------
 
-| Full documentation on all methods, classes, installation methods, and integration setups for LangChain.
+All of LangChain's reference documentation, in one place. Full documentation on all methods, classes, installation methods, and integration setups for LangChain.
 
 
-- `LangChain Installation <./reference/installation.html>`_
-
 - `Reference Documentation <./reference.html>`_
-
 .. toctree::
    :maxdepth: 1
    :caption: Reference
@@ -137,54 +119,47 @@ Reference Docs
    :hidden:
 
    ./reference/installation.md
+   ./reference/integrations.md
    ./reference.rst
 
 
-Ecosystem
-------------
+LangChain Ecosystem
+-------------------
 
-| LangChain integrates a lot of different LLMs, systems, and products.
-| From the other side, many systems and products depend on LangChain.
-| It creates a vibrant and thriving ecosystem.
-
-
-- `Integrations <./integrations.html>`_: Guides for how other products can be used with LangChain.
-
-- `Dependents <./dependents.html>`_: List of repositories that use LangChain.
-
-- `Deployments <./ecosystem/deployments.html>`_: A collection of instructions, code snippets, and template repositories for deploying LangChain apps.
+Guides for how other companies/products can be used with LangChain
 
+- `LangChain Ecosystem <./ecosystem.html>`_
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
    :glob:
    :caption: Ecosystem
    :name: ecosystem
    :hidden:
 
-   ./integrations.rst
-   ./dependents.md
-   ./ecosystem/deployments.md
+   ./ecosystem.rst
 
 
 Additional Resources
 ---------------------
 
-| Additional resources we think may be useful as you develop your application!
+Additional collection of resources we think may be useful as you develop your application!
 
 - `LangChainHub <https://github.com/hwchase17/langchain-hub>`_: The LangChainHub is a place to share and explore other prompts, chains, and agents.
 
-- `Gallery <https://github.com/kyrolabs/awesome-langchain>`_: A collection of great projects that use Langchain, compiled by the folks at `Kyrolabs <https://kyrolabs.com>`_. Useful for finding inspiration and example implementations.
+- `Glossary <./glossary.html>`_: A glossary of all related terms, papers, methods, etc. Whether implemented in LangChain or not!
 
-- `Deploying LLMs in Production <./additional_resources/deploy_llms.html>`_: A collection of best practices and tutorials for deploying LLMs in production.
+- `Gallery <./gallery.html>`_: A collection of our favorite projects that use LangChain. Useful for finding inspiration or seeing how things were done in other applications.
 
-- `Tracing <./additional_resources/tracing.html>`_: A guide on using tracing in LangChain to visualize the execution of chains and agents.
+- `Deployments <./deployments.html>`_: A collection of instructions, code snippets, and template repositories for deploying LangChain apps.
 
-- `Model Laboratory <./additional_resources/model_laboratory.html>`_: Experimenting with different prompts, models, and chains is a big part of developing the best possible application. The ModelLaboratory makes it easy to do so.
+- `Tracing <./tracing.html>`_: A guide on using tracing in LangChain to visualize the execution of chains and agents.
+
+- `Model Laboratory <./model_laboratory.html>`_: Experimenting with different prompts, models, and chains is a big part of developing the best possible application. The ModelLaboratory makes it easy to do so.
 
 - `Discord <https://discord.gg/6adMQxSpJS>`_: Join us on our Discord to discuss all things LangChain!
 
-- `YouTube <./additional_resources/youtube.html>`_: A collection of the LangChain tutorials and videos.
+- `YouTube <./youtube.html>`_: A collection of the LangChain tutorials and videos.
 
 - `Production Support <https://forms.gle/57d8AmXBYp8PP8tZA>`_: As you move your LangChains into production, we'd love to offer more comprehensive support. Please fill out this form and we'll set up a dedicated support Slack channel.
 
@@ -196,11 +171,11 @@ Additional Resources
    :hidden:
 
    LangChainHub <https://github.com/hwchase17/langchain-hub>
-   ./additional_resources/deployments.md
-   ./additional_resources/deploy_llms.rst
-   Gallery <https://github.com/kyrolabs/awesome-langchain>
-   ./additional_resources/tracing.md
-   ./additional_resources/model_laboratory.ipynb
+   ./glossary.md
+   ./gallery.rst
+   ./deployments.md
+   ./tracing.md
+   ./use_cases/model_laboratory.ipynb
    Discord <https://discord.gg/6adMQxSpJS>
-   ./additional_resources/youtube.md
+   ./youtube.md
    Production Support <https://forms.gle/57d8AmXBYp8PP8tZA>

docs/integrations/airbyte.md

@@ -1,29 +0,0 @@
-# Airbyte
-
->[Airbyte](https://github.com/airbytehq/airbyte) is a data integration platform for ELT pipelines from APIs, 
-> databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.
-
-## Installation and Setup
-
-This instruction shows how to load any source from `Airbyte` into a local `JSON` file that can be read in as a document.
-
-**Prerequisites:**
-Have `docker desktop` installed.
-
-**Steps:**
-1. Clone Airbyte from GitHub - `git clone https://github.com/airbytehq/airbyte.git`.
-2. Switch into Airbyte directory - `cd airbyte`.
-3. Start Airbyte - `docker compose up`.
-4. In your browser, just visit http://localhost:8000. You will be asked for a username and password. By default, that's username `airbyte` and password `password`.
-5. Setup any source you wish.
-6. Set destination as Local JSON, with specified destination path - lets say `/json_data`. Set up a manual sync.
-7. Run the connection.
-8. To see what files are created, navigate to: `file:///tmp/airbyte_local/`.
-
-## Document Loader
-
-See a [usage example](../modules/indexes/document_loaders/examples/airbyte_json.ipynb).
-
-```python
-from langchain.document_loaders import AirbyteJSONLoader
-```

docs/integrations/aleph_alpha.md

@@ -1,36 +0,0 @@
-# Aleph Alpha
-
->[Aleph Alpha](https://docs.aleph-alpha.com/) was founded in 2019 with the mission to research and build the foundational technology for an era of strong AI. The team of international scientists, engineers, and innovators researches, develops, and deploys transformative AI like large language and multimodal models and runs the fastest European commercial AI cluster.
-
->[The Luminous series](https://docs.aleph-alpha.com/docs/introduction/luminous/) is a family of large language models.
-
-## Installation and Setup
-
-```bash
-pip install aleph-alpha-client
-```
-
-You have to create a new token. Please, see [instructions](https://docs.aleph-alpha.com/docs/account/#create-a-new-token).
-
-```python
-from getpass import getpass
-
-ALEPH_ALPHA_API_KEY = getpass()
-```
-
-
-## LLM
-
-See a [usage example](../modules/models/llms/integrations/aleph_alpha.ipynb).
-
-```python
-from langchain.llms import AlephAlpha
-```
-
-## Text Embedding Models
-
-See a [usage example](../modules/models/text_embedding/examples/aleph_alpha.ipynb).
-
-```python
-from langchain.embeddings import AlephAlphaSymmetricSemanticEmbedding, AlephAlphaAsymmetricSemanticEmbedding
-```

docs/integrations/amazon_bedrock.md

@@ -1,24 +0,0 @@
-# Amazon Bedrock
-
->[Amazon Bedrock](https://aws.amazon.com/bedrock/) is a fully managed service that makes FMs from leading AI startups and Amazon available via an API, so you can choose from a wide range of FMs to find the model that is best suited for your use case.
-
-## Installation and Setup
-
-```bash
-pip install boto3
-```
-
-## LLM
-
-See a [usage example](../modules/models/llms/integrations/bedrock.ipynb).
-
-```python
-from langchain import Bedrock
-```
-
-## Text Embedding Models
-
-See a [usage example](../modules/models/text_embedding/examples/amazon_bedrock.ipynb).
-```python
-from langchain.embeddings import BedrockEmbeddings
-```

docs/integrations/annoy.md

@@ -1,18 +0,0 @@
-# Annoy
-
-> [Annoy](https://github.com/spotify/annoy) (`Approximate Nearest Neighbors Oh Yeah`) is a C++ library with Python bindings to search for points in space that are close to a given query point. It also creates large read-only file-based data structures that are mmapped into memory so that many processes may share the same data. 
-## Installation and Setup
-
-
-```bash
-pip install annoy
-```
-
-
-## Vectorstore
-
-See a [usage example](../modules/indexes/vectorstores/examples/annoy.ipynb).
-
-```python
-from langchain.vectorstores import Annoy
-```

docs/integrations/anthropic.md

@@ -1,26 +0,0 @@
-# Anthropic
-
->[Anthropic](https://en.wikipedia.org/wiki/Anthropic) is an American artificial intelligence (AI) startup and 
-> public-benefit corporation, founded by former members of OpenAI. `Anthropic` specializes in developing general AI 
-> systems and language models, with a company ethos of responsible AI usage.
-> `Anthropic` develops a chatbot, named `Claude`. Similar to `ChatGPT`, `Claude` uses a messaging 
-> interface where users can submit questions or requests and receive highly detailed and relevant responses.
- 
-## Installation and Setup
-
-
-```bash
-pip install anthropic
-```
-
-See the [setup documentation](https://console.anthropic.com/docs/access).
-
-
-
-## Chat Models
-
-See a [usage example](../modules/models/chat/integrations/anthropic.ipynb)
-
-```python
-from langchain.chat_models import ChatAnthropic
-```

docs/integrations/anyscale.md

@@ -1,17 +0,0 @@
-# Anyscale
-
-This page covers how to use the Anyscale ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Anyscale wrappers.
-
-## Installation and Setup
-- Get an Anyscale Service URL, route and API key and set them as environment variables (`ANYSCALE_SERVICE_URL`,`ANYSCALE_SERVICE_ROUTE`, `ANYSCALE_SERVICE_TOKEN`). 
-- Please see [the Anyscale docs](https://docs.anyscale.com/productionize/services-v2/get-started) for more details.
-
-## Wrappers
-
-### LLM
-
-There exists an Anyscale LLM wrapper, which you can access with 
-```python
-from langchain.llms import Anyscale
-```

docs/integrations/argilla.md

@@ -1,29 +0,0 @@
-# Argilla
-
-![Argilla - Open-source data platform for LLMs](https://argilla.io/og.png)
-
->[Argilla](https://argilla.io/) is an open-source data curation platform for LLMs.
-> Using Argilla, everyone can build robust language models through faster data curation 
-> using both human and machine feedback. We provide support for each step in the MLOps cycle, 
-> from data labeling to model monitoring.
-
-## Installation and Setup
-
-First, you'll need to install the  `argilla` Python package as follows:
-
-```bash
-pip install argilla --upgrade
-```
-
-If you already have an Argilla Server running, then you're good to go; but if
-you don't, follow the next steps to install it.
-
-If you don't you can refer to [Argilla - 🚀 Quickstart](https://docs.argilla.io/en/latest/getting_started/quickstart.html#Running-Argilla-Quickstart) to deploy Argilla either on HuggingFace Spaces, locally, or on a server.
-
-## Tracking
-
-See a [usage example of `ArgillaCallbackHandler`](../modules/callbacks/examples/examples/argilla.ipynb).
-
-```python
-from langchain.callbacks import ArgillaCallbackHandler
-```

docs/integrations/arxiv.md

@@ -1,36 +0,0 @@
-# Arxiv
-
->[arXiv](https://arxiv.org/) is an open-access archive for 2 million scholarly articles in the fields of physics, 
-> mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and 
-> systems science, and economics.
-
-
-## Installation and Setup
-
-First, you need to install `arxiv` python package.
-
-```bash
-pip install arxiv
-```
-
-Second, you need to install `PyMuPDF` python package which transforms PDF files downloaded from the `arxiv.org` site into the text format.
-
-```bash
-pip install pymupdf
-```
-
-## Document Loader
-
-See a [usage example](../modules/indexes/document_loaders/examples/arxiv.ipynb).
-
-```python
-from langchain.document_loaders import ArxivLoader
-```
-
-## Retriever
-
-See a [usage example](../modules/indexes/retrievers/examples/arxiv.ipynb).
-
-```python
-from langchain.retrievers import ArxivRetriever
-```

docs/integrations/awadb.md

@@ -1,21 +0,0 @@
-# AwaDB
-
->[AwaDB](https://github.com/awa-ai/awadb) is an AI Native database for the search and storage of embedding vectors used by LLM Applications.
-
-## Installation and Setup
-
-```bash
-pip install awadb
-```
-
-
-## VectorStore
-
-There exists a wrapper around AwaDB vector databases, allowing you to use it as a vectorstore,
-whether for semantic search or example selection.
-
-```python
-from langchain.vectorstores import AwaDB
-```
-
-For a more detailed walkthrough of the AwaDB wrapper, see [this notebook](../modules/indexes/vectorstores/examples/awadb.ipynb)

docs/integrations/aws_s3.md

@@ -1,25 +0,0 @@
-# AWS S3 Directory
-
->[Amazon Simple Storage Service (Amazon S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html) is an object storage service.
-
->[AWS S3 Directory](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html)
-
->[AWS S3 Buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html)
-
-
-## Installation and Setup
-
-```bash
-pip install boto3
-```
-
-
-## Document Loader
-
-See a [usage example for S3DirectoryLoader](../modules/indexes/document_loaders/examples/aws_s3_directory.ipynb).
-
-See a [usage example for S3FileLoader](../modules/indexes/document_loaders/examples/aws_s3_file.ipynb).
-
-```python
-from langchain.document_loaders import S3DirectoryLoader, S3FileLoader
-```

docs/integrations/azlyrics.md

@@ -1,16 +0,0 @@
-# AZLyrics
-
->[AZLyrics](https://www.azlyrics.com/) is a large, legal, every day growing collection of lyrics.
-
-## Installation and Setup
-
-There isn't any special setup for it.
-
-
-## Document Loader
-
-See a [usage example](../modules/indexes/document_loaders/examples/azlyrics.ipynb).
-
-```python
-from langchain.document_loaders import AZLyricsLoader
-```

docs/integrations/azure_blob_storage.md

@@ -1,36 +0,0 @@
-# Azure Blob Storage
-
->[Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) is Microsoft's object storage solution for the cloud. Blob Storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn't adhere to a particular data model or definition, such as text or binary data.
-
->[Azure Files](https://learn.microsoft.com/en-us/azure/storage/files/storage-files-introduction) offers fully managed
-> file shares in the cloud that are accessible via the industry standard Server Message Block (`SMB`) protocol, 
-> Network File System (`NFS`) protocol, and `Azure Files REST API`. `Azure Files` are based on the `Azure Blob Storage`.
-
-`Azure Blob Storage` is designed for:
-- Serving images or documents directly to a browser.
-- Storing files for distributed access.
-- Streaming video and audio.
-- Writing to log files.
-- Storing data for backup and restore, disaster recovery, and archiving.
-- Storing data for analysis by an on-premises or Azure-hosted service.
-
-## Installation and Setup
-
-```bash
-pip install azure-storage-blob
-```
-
-
-## Document Loader
-
-See a [usage example for the Azure Blob Storage](../modules/indexes/document_loaders/examples/azure_blob_storage_container.ipynb).
-
-```python
-from langchain.document_loaders import AzureBlobStorageContainerLoader
-```
-
-See a [usage example for the Azure Files](../modules/indexes/document_loaders/examples/azure_blob_storage_file.ipynb).
-
-```python
-from langchain.document_loaders import AzureBlobStorageFileLoader
-```

docs/integrations/azure_cognitive_search_.md

@@ -1,24 +0,0 @@
-# Azure Cognitive Search
-
->[Azure Cognitive Search](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search) (formerly known as `Azure Search`) is a cloud search service that gives developers infrastructure, APIs, and tools for building a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.
-
->Search is foundational to any app that surfaces text to users, where common scenarios include catalog or document search, online retail apps, or data exploration over proprietary content. When you create a search service, you'll work with the following capabilities:
->- A search engine for full text search over a search index containing user-owned content
->- Rich indexing, with lexical analysis and optional AI enrichment for content extraction and transformation
->- Rich query syntax for text search, fuzzy search, autocomplete, geo-search and more
->- Programmability through REST APIs and client libraries in Azure SDKs
->- Azure integration at the data layer, machine learning layer, and AI (Cognitive Services)
-
-
-## Installation and Setup
-
-See [set up instructions](https://learn.microsoft.com/en-us/azure/search/search-create-service-portal).
-
-
-## Retriever
-
-See a [usage example](../modules/indexes/retrievers/examples/azure_cognitive_search.ipynb).
-
-```python
-from langchain.retrievers import AzureCognitiveSearchRetriever
-```

docs/integrations/azure_openai.md

@@ -1,50 +0,0 @@
-# Azure OpenAI
-
->[Microsoft Azure](https://en.wikipedia.org/wiki/Microsoft_Azure), often referred to as `Azure` is a cloud computing platform run by `Microsoft`, which offers access, management, and development of applications and services through global data centers. It provides a range of capabilities, including software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). `Microsoft Azure` supports many programming languages, tools, and frameworks, including Microsoft-specific and third-party software and systems.
-
-
->[Azure OpenAI](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/) is an `Azure` service with powerful language models from `OpenAI` including the `GPT-3`, `Codex` and `Embeddings model` series for content generation, summarization, semantic search, and natural language to code translation.
-
-
-## Installation and Setup
-
-```bash
-pip install openai
-pip install tiktoken
-```
-
-
-Set the environment variables to get access to the `Azure OpenAI` service.
-
-```python
-import os
-
-os.environ["OPENAI_API_TYPE"] = "azure"
-os.environ["OPENAI_API_BASE"] = "https://<your-endpoint.openai.azure.com/"
-os.environ["OPENAI_API_KEY"] = "your AzureOpenAI key"
-os.environ["OPENAI_API_VERSION"] = "2023-03-15-preview"
-```
-
-## LLM
-
-See a [usage example](../modules/models/llms/integrations/azure_openai_example.ipynb).
-
-```python
-from langchain.llms import AzureOpenAI
-```
-
-## Text Embedding Models
-
-See a [usage example](../modules/models/text_embedding/examples/azureopenai.ipynb)
-
-```python
-from langchain.embeddings import OpenAIEmbeddings
-```
-
-## Chat Models
-
-See a [usage example](../modules/models/chat/integrations/azure_chat_openai.ipynb)
-
-```python
-from langchain.chat_models import AzureChatOpenAI
-```

docs/integrations/beam.md

@@ -1,93 +0,0 @@
-# Beam
-
->[Beam](https://docs.beam.cloud/introduction) makes it easy to run code on GPUs, deploy scalable web APIs, 
-> schedule cron jobs, and run massively parallel workloads — without managing any infrastructure.
- 
-
-## Installation and Setup
-
-- [Create an account](https://www.beam.cloud/)
-- Install the Beam CLI with `curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh`
-- Register API keys with `beam configure`
-- Set environment variables (`BEAM_CLIENT_ID`) and (`BEAM_CLIENT_SECRET`)
-- Install the Beam SDK:
-```bash
-pip install beam-sdk
-```
-
-## LLM
-
-
-```python
-from langchain.llms.beam import Beam
-```
-
-### Example of the Beam app
-
-This is the environment you’ll be developing against once you start the app.
-It's also used to define the maximum response length from the model.
-```python
-llm = Beam(model_name="gpt2",
-           name="langchain-gpt2-test",
-           cpu=8,
-           memory="32Gi",
-           gpu="A10G",
-           python_version="python3.8",
-           python_packages=[
-               "diffusers[torch]>=0.10",
-               "transformers",
-               "torch",
-               "pillow",
-               "accelerate",
-               "safetensors",
-               "xformers",],
-           max_length="50",
-           verbose=False)
-```
-
-### Deploy the Beam app
-
-Once defined, you can deploy your Beam app by calling your model's `_deploy()` method.
-
-```python
-llm._deploy()
-```
-
-### Call the Beam app
-
-Once a beam model is deployed, it can be called by calling your model's `_call()` method.
-This returns the GPT2 text response to your prompt.
-
-```python
-response = llm._call("Running machine learning on a remote GPU")
-```
-
-An example script which deploys the model and calls it would be:
-
-```python
-from langchain.llms.beam import Beam
-import time
-
-llm = Beam(model_name="gpt2",
-           name="langchain-gpt2-test",
-           cpu=8,
-           memory="32Gi",
-           gpu="A10G",
-           python_version="python3.8",
-           python_packages=[
-               "diffusers[torch]>=0.10",
-               "transformers",
-               "torch",
-               "pillow",
-               "accelerate",
-               "safetensors",
-               "xformers",],
-           max_length="50",
-           verbose=False)
-
-llm._deploy()
-
-response = llm._call("Running machine learning on a remote GPU")
-
-print(response)
-```