pick

.github/CONTRIBUTING.md

@@ -3,4 +3,43 @@
 Hi there! Thank you for even being interested in contributing to LangChain.
 As an open-source project in a rapidly developing field, we are extremely open to contributions, whether they involve new features, improved infrastructure, better documentation, or bug fixes.
 
-To learn how to contribute to LangChain, please follow the [contribution guide here](https://python.langchain.com/docs/contributing/).
+To learn about how to contribute, please follow the [guides here](https://python.langchain.com/docs/contributing/)
+
+## 🗺️ Guidelines
+
+### 👩‍💻 Ways to contribute
+
+There are many ways to contribute to LangChain. Here are some common ways people contribute:
+
+- [**Documentation**](https://python.langchain.com/docs/contributing/documentation): Help improve our docs, including this one!
+- [**Code**](https://python.langchain.com/docs/contributing/code): Help us write code, fix bugs, or improve our infrastructure.
+- [**Integrations**](https://python.langchain.com/docs/contributing/integration): Help us integrate with your favorite vendors and tools.
+
+### 🚩GitHub Issues
+
+Our [issues](https://github.com/langchain-ai/langchain/issues) page is kept up to date with bugs, improvements, and feature requests.
+
+There is a taxonomy of labels to help with sorting and discovery of issues of interest. Please use these to help organize issues.
+
+If you start working on an issue, please assign it to yourself.
+
+If you are adding an issue, please try to keep it focused on a single, modular bug/improvement/feature.
+If two issues are related, or blocking, please link them rather than combining them.
+
+We will try to keep these issues as up-to-date as possible, though
+with the rapid rate of development in this field some may get out of date.
+If you notice this happening, please let us know.
+
+### 🙋Getting Help
+
+Our goal is to have the simplest developer setup possible. Should you experience any difficulty getting setup, please
+contact a maintainer! Not only do we want to help get you unblocked, but we also want to make sure that the process is
+smooth for future contributors.
+
+In a similar vein, we do enforce certain linting, formatting, and documentation standards in the codebase.
+If you are finding these difficult (or even just annoying) to work with, feel free to contact a maintainer for help -
+we do not want these to get in the way of getting good code into the codebase.
+
+### Contributor Documentation
+
+To learn about how to contribute, please follow the [guides here](https://python.langchain.com/docs/contributing/)

.github/DISCUSSION_TEMPLATE/q-a.yml

@@ -1,122 +0,0 @@
-labels: [Question]
-body:
-  - type: markdown
-    attributes:
-      value: |
-        Thanks for your interest in 🦜️🔗 LangChain!
-
-        Please follow these instructions, fill every question, and do every step. 🙏
-        
-        We're asking for this because answering questions and solving problems in GitHub takes a lot of time --
-        this is time that we cannot spend on adding new features, fixing bugs, write documentation or reviewing pull requests.
-
-        By asking questions in a structured way (following this) it will be much easier to help you.
-
-        And there's a high chance that you will find the solution along the way and you won't even have to submit it and wait for an answer. 😎
-
-        As there are too many questions, we will **DISCARD** and close the incomplete ones. 
-        
-        That will allow us (and others) to focus on helping people like you that follow the whole process. 🤓
-        
-        Relevant links to check before opening a question to see if your question has already been answered, fixed or
-        if there's another way to solve your problem:
-        
-        [LangChain documentation with the integrated search](https://python.langchain.com/docs/get_started/introduction),
-        [API Reference](https://api.python.langchain.com/en/stable/),
-        [GitHub search](https://github.com/langchain-ai/langchain),
-        [LangChain Github Discussions](https://github.com/langchain-ai/langchain/discussions),
-        [LangChain Github Issues](https://github.com/langchain-ai/langchain/issues?q=is%3Aissue),
-        [LangChain ChatBot](https://chat.langchain.com/)
-  - type: checkboxes
-    id: checks
-    attributes:
-      label: Checked other resources
-      description: Please confirm and check all the following options.
-      options:
-        - label: I added a very descriptive title to this question.
-          required: true
-        - label: I searched the LangChain documentation with the integrated search.
-          required: true
-        - label: I used the GitHub search to find a similar question and didn't find it.
-          required: true
-  - type: checkboxes
-    id: help
-    attributes:
-      label: Commit to Help
-      description: |
-        After submitting this, I commit to one of:
-
-          * Read open questions until I find 2 where I can help someone and add a comment to help there.
-          * I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
-          * Once my question is answered, I will mark the answer as "accepted".
-      options:
-        - label: I commit to help with one of those options 👆
-          required: true
-  - type: textarea
-    id: example
-    attributes:
-      label: Example Code
-      description: |
-        Please add a self-contained, [minimal, reproducible, example](https://stackoverflow.com/help/minimal-reproducible-example) with your use case.
-        
-        If a maintainer can copy it, run it, and see it right away, there's a much higher chance that you'll be able to get help.
-        
-        **Important!** 
-        
-        * Use code tags (e.g., ```python ... ```) to correctly [format your code](https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting).
-        * INCLUDE the language label (e.g. `python`) after the first three backticks to enable syntax highlighting. (e.g., ```python rather than ```).
-        * Reduce your code to the minimum required to reproduce the issue if possible. This makes it much easier for others to help you.
-        * Avoid screenshots when possible, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
-
-      placeholder: |
-        from langchain_core.runnables import RunnableLambda
-
-        def bad_code(inputs) -> int:
-          raise NotImplementedError('For demo purpose')
-        
-          chain = RunnableLambda(bad_code)
-          chain.invoke('Hello!')
-      render: python
-    validations:
-      required: true
-  - type: textarea
-    id: description
-    attributes:
-      label: Description
-      description: |
-        What is the problem, question, or error?
-
-        Write a short description explaining what you are doing, what you expect to happen, and what is currently happening.
-      placeholder: |
-        * I'm trying to use the `langchain` library to do X.
-        * I expect to see Y.
-        * Instead, it does Z.
-    validations:
-      required: true
-  - type: textarea
-    id: system-info
-    attributes:
-      label: System Info
-      description: |
-        Please share your system info with us. 
-        
-        "pip freeze | grep langchain" 
-        platform (windows / linux / mac)
-        python version
-        
-        OR if you're on a recent version of langchain-core you can paste the output of:
-        
-        python -m langchain_core.sys_info
-      placeholder: |
-        "pip freeze | grep langchain"
-        platform
-        python version
-        
-        Alternatively, if you're on a recent version of langchain-core you can paste the output of:
-        
-        python -m langchain_core.sys_info
-        
-        These will only surface LangChain packages, don't forget to include any other relevant
-        packages you're using (if you're not sure what's relevant, you can paste the entire output of `pip freeze`).
-    validations:
-      required: true

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -1,118 +1,106 @@
 name: "\U0001F41B Bug Report"
-description: Report a bug in LangChain. To report a security issue, please instead use the security option below. For questions, please use the GitHub Discussions.
+description: Submit a bug report to help us improve LangChain. To report a security issue, please instead use the security option below.
 labels: ["02 Bug Report"]
 body:
   - type: markdown
     attributes:
       value: >
-        Thank you for taking the time to file a bug report. 
-        
-        Use this to report bugs in LangChain. 
-        
-        If you're not certain that your issue is due to a bug in LangChain, please use [GitHub Discussions](https://github.com/langchain-ai/langchain/discussions)
-        to ask for help with your issue.
-        
-        Relevant links to check before filing a bug report to see if your issue has already been reported, fixed or
-        if there's another way to solve your problem:
-        
-        [LangChain documentation with the integrated search](https://python.langchain.com/docs/get_started/introduction),
-        [API Reference](https://api.python.langchain.com/en/stable/),
-        [GitHub search](https://github.com/langchain-ai/langchain),
-        [LangChain Github Discussions](https://github.com/langchain-ai/langchain/discussions),
-        [LangChain Github Issues](https://github.com/langchain-ai/langchain/issues?q=is%3Aissue),
-        [LangChain ChatBot](https://chat.langchain.com/)
-  - type: checkboxes
-    id: checks
+        Thank you for taking the time to file a bug report. Before creating a new
+        issue, please make sure to take a few moments to check the issue tracker
+        for existing issues about the bug.
+
+  - type: textarea
+    id: system-info
     attributes:
-      label: Checked other resources
-      description: Please confirm and check all the following options.
+      label: System Info
+      description: Please share your system info with us.
+      placeholder: LangChain version, platform, python version, ...
+    validations:
+      required: true
+
+  - type: textarea
+    id: who-can-help
+    attributes:
+      label: Who can help?
+      description: |
+        Your issue will be replied to more quickly if you can figure out the right person to tag with @
+        If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
+
+        The core maintainers strive to read all issues, but tagging them will help them prioritize.
+
+        Please tag fewer than 3 people.
+
+        @hwchase17 - project lead
+
+        Tracing / Callbacks
+        - @agola11
+
+        Async
+        - @agola11
+
+        DataLoader Abstractions
+        - @eyurtsev
+
+        LLM/Chat Wrappers
+        - @hwchase17
+        - @agola11
+
+        Tools / Toolkits
+        - ...
+
+      placeholder: "@Username ..."
+
+  - type: checkboxes
+    id: information-scripts-examples
+    attributes:
+      label: Information
+      description: "The problem arises when using:"
       options:
-        - label: I added a very descriptive title to this issue.
-          required: true
-        - label: I searched the LangChain documentation with the integrated search.
-          required: true
-        - label: I used the GitHub search to find a similar question and didn't find it.
-          required: true
-        - label: I am sure that this is a bug in LangChain rather than my code.
-          required: true
+        - label: "The official example notebooks/scripts"
+        - label: "My own modified scripts"
+
+  - type: checkboxes
+    id: related-components
+    attributes:
+      label: Related Components
+      description: "Select the components related to the issue (if applicable):"
+      options:
+        - label: "LLMs/Chat Models"
+        - label: "Embedding Models"
+        - label: "Prompts / Prompt Templates / Prompt Selectors"
+        - label: "Output Parsers"
+        - label: "Document Loaders"
+        - label: "Vector Stores / Retrievers"
+        - label: "Memory"
+        - label: "Agents / Agent Executors"
+        - label: "Tools / Toolkits"
+        - label: "Chains"
+        - label: "Callbacks/Tracing"
+        - label: "Async"
+
   - type: textarea
     id: reproduction
     validations:
       required: true
     attributes:
-      label: Example Code
+      label: Reproduction
       description: |
-        Please add a self-contained, [minimal, reproducible, example](https://stackoverflow.com/help/minimal-reproducible-example) with your use case.
-        
-        If a maintainer can copy it, run it, and see it right away, there's a much higher chance that you'll be able to get help.
-        
-        **Important!** 
-        
-        * Use code tags (e.g., ```python ... ```) to correctly [format your code](https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting).
-        * INCLUDE the language label (e.g. `python`) after the first three backticks to enable syntax highlighting. (e.g., ```python rather than ```).
-        * Reduce your code to the minimum required to reproduce the issue if possible. This makes it much easier for others to help you.
-        * Avoid screenshots when possible, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
+        Please provide a [code sample](https://stackoverflow.com/help/minimal-reproducible-example) that reproduces the problem you ran into. It can be a Colab link or just a code snippet.
+        If you have code snippets, error messages, stack traces please provide them here as well.
+        Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
+        Avoid screenshots when possible, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
 
       placeholder: |
-        The following code: 
-        
-        ```python
-        from langchain_core.runnables import RunnableLambda
+        Steps to reproduce the behavior:
+
+          1.
+          2.
+          3.
 
-        def bad_code(inputs) -> int:
-          raise NotImplementedError('For demo purpose')
-          
-          chain = RunnableLambda(bad_code)
-          chain.invoke('Hello!')
-        ```
   - type: textarea
-    id: error
-    validations:
-      required: false
-    attributes:
-      label: Error Message and Stack Trace (if applicable)
-      description: |
-        If you are reporting an error, please include the full error message and stack trace.
-      placeholder: |
-        Exception + full stack trace
-  - type: textarea
-    id: description
-    attributes:
-      label: Description
-      description: |
-        What is the problem, question, or error?
-
-        Write a short description telling what you are doing, what you expect to happen, and what is currently happening.
-      placeholder: |
-        * I'm trying to use the `langchain` library to do X.
-        * I expect to see Y.
-        * Instead, it does Z.
+    id: expected-behavior
     validations:
       required: true
-  - type: textarea
-    id: system-info
     attributes:
-      label: System Info
-      description: |
-        Please share your system info with us. 
-        
-        "pip freeze | grep langchain" 
-        platform (windows / linux / mac)
-        python version
-        
-        OR if you're on a recent version of langchain-core you can paste the output of:
-        
-        python -m langchain_core.sys_info
-      placeholder: |
-        "pip freeze | grep langchain"
-        platform
-        python version
-        
-        Alternatively, if you're on a recent version of langchain-core you can paste the output of:
-        
-        python -m langchain_core.sys_info
-        
-        These will only surface LangChain packages, don't forget to include any other relevant
-        packages you're using (if you're not sure what's relevant, you can paste the entire output of `pip freeze`).
-    validations:
-      required: true
+      label: Expected behavior
+      description: "A clear and concise description of what you would expect to happen."

.github/ISSUE_TEMPLATE/config.yml

@@ -1,15 +1,6 @@
-blank_issues_enabled: false
+blank_issues_enabled: true
 version: 2.1
 contact_links:
-  - name: 🤔 Question or Problem
-    about: Ask a question or ask about a problem in GitHub Discussions.
-    url: https://www.github.com/langchain-ai/langchain/discussions/categories/q-a
   - name: Discord
     url: https://discord.gg/6adMQxSpJS
     about: General community discussions
-  - name: Feature Request
-    url: https://www.github.com/langchain-ai/langchain/discussions/categories/ideas
-    about: Suggest a feature or an idea
-  - name: Show and tell
-    about: Show what you built with LangChain
-    url: https://www.github.com/langchain-ai/langchain/discussions/categories/show-and-tell

.github/ISSUE_TEMPLATE/documentation.yml

@@ -4,45 +4,13 @@ title: "DOC: <Please write a comprehensive title after the 'DOC: ' prefix>"
 labels: [03 - Documentation]
 
 body:
-- type: markdown
-  attributes:
-    value: >
-      Thank you for taking the time to report an issue in the documentation.
-      
-      Only report issues with documentation here, explain if there are
-      any missing topics or if you found a mistake in the documentation.
-      
-      Do **NOT** use this to ask usage questions or reporting issues with your code.
-      
-      If you have usage questions or need help solving some problem, 
-      please use [GitHub Discussions](https://github.com/langchain-ai/langchain/discussions).
-      
-      If you're in the wrong place, here are some helpful links to find a better
-      place to ask your question:
-      
-      [LangChain documentation with the integrated search](https://python.langchain.com/docs/get_started/introduction),
-      [API Reference](https://api.python.langchain.com/en/stable/),
-      [GitHub search](https://github.com/langchain-ai/langchain),
-      [LangChain Github Discussions](https://github.com/langchain-ai/langchain/discussions),
-      [LangChain Github Issues](https://github.com/langchain-ai/langchain/issues?q=is%3Aissue),
-      [LangChain ChatBot](https://chat.langchain.com/)
-- type: checkboxes
-  id: checks
-  attributes:
-    label: Checklist
-    description: Please confirm and check all the following options.
-    options:
-      - label: I added a very descriptive title to this issue.
-        required: true
-      - label: I included a link to the documentation page I am referring to (if applicable).
-        required: true
 - type: textarea
   attributes: 
     label: "Issue with current documentation:"
     description: >
       Please make sure to leave a reference to the document/code you're
-      referring to. Feel free to include names of classes, functions, methods
-      or concepts you'd like to see documented more.
+      referring to.
+
 - type: textarea
   attributes:
     label: "Idea or request for content:"

.github/ISSUE_TEMPLATE/feature-request.yml

@@ -1,17 +1,7 @@
-labels: [idea]
+name: "\U0001F680 Feature request"
+description: Submit a proposal/request for a new LangChain feature
+labels: ["02 Feature Request"]
 body:
-  - type: checkboxes
-    id: checks
-    attributes:
-      label: Checked
-      description: Please confirm and check all the following options.
-      options:
-        - label: I searched existing ideas and did not find a similar one
-          required: true
-        - label: I added a very descriptive title
-          required: true
-        - label: I've clearly described the feature request and motivation for it
-          required: true
   - type: textarea
     id: feature-request
     validations:
@@ -20,6 +10,7 @@ body:
       label: Feature request
       description: |
         A clear and concise description of the feature proposal. Please provide links to any relevant GitHub repos, papers, or other resources if relevant.
+
   - type: textarea
     id: motivation
     validations:
@@ -28,11 +19,12 @@ body:
       label: Motivation
       description: |
         Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too.
+
   - type: textarea
-    id: proposal
+    id: contribution
     validations:
-      required: false
+      required: true
     attributes:
-      label: Proposal (If applicable)
+      label: Your contribution
       description: |
-        If you would like to propose a solution, please describe it here. 
+        Is there any way that you could help, e.g. by submitting a PR? Make sure to read the [Contributing Guide](https://python.langchain.com/docs/contributing/)

.github/ISSUE_TEMPLATE/other.yml

@@ -0,0 +1,18 @@
+name: Other Issue
+description: Raise an issue that wouldn't be covered by the other templates.
+title: "Issue: <Please write a comprehensive title after the 'Issue: ' prefix>"
+labels: [04 - Other]
+
+body:
+  - type: textarea
+    attributes:
+      label: "Issue you'd like to raise."
+      description: >
+        Please describe the issue you'd like to raise as clearly as possible.
+        Make sure to include any relevant links or references.
+
+  - type: textarea
+    attributes:
+      label: "Suggestion:"
+      description: >
+        Please outline a suggestion to improve the issue here.

.github/ISSUE_TEMPLATE/privileged.yml

@@ -1,25 +0,0 @@
-name: 🔒 Privileged
-description: You are a LangChain maintainer, or was asked directly by a maintainer to create an issue here. If not, check the other options.
-body:
-  - type: markdown
-    attributes:
-      value: |
-        Thanks for your interest in LangChain! 🚀
-        
-        If you are not a LangChain maintainer or were not asked directly by a maintainer to create an issue, then please start the conversation in a [Question in GitHub Discussions](https://github.com/langchain-ai/langchain/discussions/categories/q-a) instead.
-        
-        You are a LangChain maintainer if you maintain any of the packages inside of the LangChain repository 
-        or are a regular contributor to LangChain with previous merged merged pull requests.
-  - type: checkboxes
-    id: privileged
-    attributes:
-      label: Privileged issue
-      description: Confirm that you are allowed to create an issue here.
-      options:
-        - label: I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.
-          required: true
-  - type: textarea
-    id: content
-    attributes:
-      label: Issue Content
-      description: Add the content of the issue here.

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,24 +1,20 @@
-Thank you for contributing to LangChain!
+<!-- Thank you for contributing to LangChain!
 
-Checklist:
+Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.
 
-- [ ] PR title: Please title your PR "package: description", where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes.
-  - Example: "community: add foobar LLM"
-- [ ] PR message: **Delete this entire template message** and replace it with the following bulleted list
-    - **Description:** a description of the change
-    - **Issue:** the issue # it fixes, if applicable
-    - **Dependencies:** any dependencies required for this change
-    - **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out!
-- [ ] Pass lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified to check that you're passing lint and testing. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/
-- [ ] Add tests and docs: If you're adding a new integration, please include
+Replace this entire comment with:
+  - **Description:** a description of the change, 
+  - **Issue:** the issue # it fixes if applicable,
+  - **Dependencies:** any dependencies required for this change,
+  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!
+
+Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.
+
+See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/
+
+If you're adding a new integration, please include:
   1. a test for the integration, preferably unit tests that do not rely on network access,
   2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.
 
-Additional guidelines:
-- Make sure optional dependencies are imported within a function.
-- Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests.
-- Most PRs should not touch more than one package.
-- Changes should be backwards compatible.
-- If you are adding something to community, do not re-import it in langchain.
-
-If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.
+If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
+ -->

.github/actions/people/Dockerfile

@@ -1,7 +0,0 @@
-FROM python:3.9
-
-RUN pip install httpx PyGithub "pydantic==2.0.2" pydantic-settings "pyyaml>=5.3.1,<6.0.0"
-
-COPY ./app /app
-
-CMD ["python", "/app/main.py"]

.github/actions/people/action.yml

@@ -1,13 +0,0 @@
-# This action was adapted from "Sebastián Ramírez <tiangolo@gmail.com>"
-# From the FastAPI project.
-# See for more details:
-# https://github.com/tiangolo/fastapi/tree/master/github/actions/people
-name: "Generate LangChain People"
-description: "Generate the data for the LangChain People pages"
-inputs:
-  token:
-    description: 'User token, to read the GitHub API. Can be passed in using {{ secrets.LANGCHAIN_PEOPLE }}'
-    required: true
-runs:
-  using: 'docker'
-  image: 'Dockerfile'

.github/actions/people/app/main.py

@@ -1,627 +0,0 @@
-# This action was adapted from "Sebastián Ramírez <tiangolo@gmail.com>"
-# From the FastAPI project.
-# See for more details:
-# https://github.com/tiangolo/fastapi/tree/master/github/actions/people
-import logging
-import subprocess
-import sys
-from collections import Counter
-from datetime import datetime, timedelta, timezone
-from pathlib import Path
-from typing import Any, Container, Dict, List, Set, Union, Tuple
-
-import httpx
-import yaml
-from github import Github
-from pydantic import BaseModel, SecretStr
-from pydantic_settings import BaseSettings
-
-GITHUB_GRAPHQL_URL = "https://api.github.com/graphql"
-# Need to look up the questions category ID.
-# For now this is just a placeholder.
-QUESTIONS_CATEGORY_ID = "[placeholder]"
-
-DISCUSSIONS_QUERY = """
-query Q($after: String, $category_id: ID) {
-  repository(name: "langchain", owner: "langchain-ai") {
-    discussions(first: 100, after: $after, categoryId: $category_id) {
-      edges {
-        cursor
-        node {
-          number
-          author {
-            login
-            avatarUrl
-            url
-          }
-          title
-          createdAt
-          comments(first: 100) {
-            nodes {
-              createdAt
-              author {
-                login
-                avatarUrl
-                url
-              }
-              isAnswer
-              replies(first: 10) {
-                nodes {
-                  createdAt
-                  author {
-                    login
-                    avatarUrl
-                    url
-                  }
-                }
-              }
-            }
-          }
-        }
-      }
-    }
-  }
-}
-"""
-
-
-ISSUES_QUERY = """
-query Q($after: String) {
-  repository(name: "langchain", owner: "langchain-ai") {
-    issues(first: 100, after: $after) {
-      edges {
-        cursor
-        node {
-          number
-          author {
-            login
-            avatarUrl
-            url
-          }
-          title
-          createdAt
-          state
-          comments(first: 100) {
-            nodes {
-              createdAt
-              author {
-                login
-                avatarUrl
-                url
-              }
-            }
-          }
-        }
-      }
-    }
-  }
-}
-"""
-
-PRS_QUERY = """
-query Q($after: String) {
-  repository(name: "langchain", owner: "langchain-ai") {
-    pullRequests(first: 100, after: $after) {
-      edges {
-        cursor
-        node {
-          number
-          labels(first: 100) {
-            nodes {
-              name
-            }
-          }
-          author {
-            login
-            avatarUrl
-            url
-          }
-          title
-          createdAt
-          state
-          comments(first: 100) {
-            nodes {
-              createdAt
-              author {
-                login
-                avatarUrl
-                url
-              }
-            }
-          }
-          reviews(first:100) {
-            nodes {
-              author {
-                login
-                avatarUrl
-                url
-              }
-              state
-            }
-          }
-        }
-      }
-    }
-  }
-}
-"""
-
-
-
-class Author(BaseModel):
-    login: str
-    avatarUrl: str
-    url: str
-
-
-# Issues and Discussions
-
-
-class CommentsNode(BaseModel):
-    createdAt: datetime
-    author: Union[Author, None] = None
-
-
-class Replies(BaseModel):
-    nodes: List[CommentsNode]
-
-
-class DiscussionsCommentsNode(CommentsNode):
-    replies: Replies
-
-
-class Comments(BaseModel):
-    nodes: List[CommentsNode]
-
-
-class DiscussionsComments(BaseModel):
-    nodes: List[DiscussionsCommentsNode]
-
-
-class IssuesNode(BaseModel):
-    number: int
-    author: Union[Author, None] = None
-    title: str
-    createdAt: datetime
-    state: str
-    comments: Comments
-
-
-class DiscussionsNode(BaseModel):
-    number: int
-    author: Union[Author, None] = None
-    title: str
-    createdAt: datetime
-    comments: DiscussionsComments
-
-
-class IssuesEdge(BaseModel):
-    cursor: str
-    node: IssuesNode
-
-
-class DiscussionsEdge(BaseModel):
-    cursor: str
-    node: DiscussionsNode
-
-
-class Issues(BaseModel):
-    edges: List[IssuesEdge]
-
-
-class Discussions(BaseModel):
-    edges: List[DiscussionsEdge]
-
-
-class IssuesRepository(BaseModel):
-    issues: Issues
-
-
-class DiscussionsRepository(BaseModel):
-    discussions: Discussions
-
-
-class IssuesResponseData(BaseModel):
-    repository: IssuesRepository
-
-
-class DiscussionsResponseData(BaseModel):
-    repository: DiscussionsRepository
-
-
-class IssuesResponse(BaseModel):
-    data: IssuesResponseData
-
-
-class DiscussionsResponse(BaseModel):
-    data: DiscussionsResponseData
-
-
-# PRs
-
-
-class LabelNode(BaseModel):
-    name: str
-
-
-class Labels(BaseModel):
-    nodes: List[LabelNode]
-
-
-class ReviewNode(BaseModel):
-    author: Union[Author, None] = None
-    state: str
-
-
-class Reviews(BaseModel):
-    nodes: List[ReviewNode]
-
-
-class PullRequestNode(BaseModel):
-    number: int
-    labels: Labels
-    author: Union[Author, None] = None
-    title: str
-    createdAt: datetime
-    state: str
-    comments: Comments
-    reviews: Reviews
-
-
-class PullRequestEdge(BaseModel):
-    cursor: str
-    node: PullRequestNode
-
-
-class PullRequests(BaseModel):
-    edges: List[PullRequestEdge]
-
-
-class PRsRepository(BaseModel):
-    pullRequests: PullRequests
-
-
-class PRsResponseData(BaseModel):
-    repository: PRsRepository
-
-
-class PRsResponse(BaseModel):
-    data: PRsResponseData
-
-
-class Settings(BaseSettings):
-    input_token: SecretStr
-    github_repository: str
-    httpx_timeout: int = 30
-
-
-def get_graphql_response(
-    *,
-    settings: Settings,
-    query: str,
-    after: Union[str, None] = None,
-    category_id: Union[str, None] = None,
-) -> Dict[str, Any]:
-    headers = {"Authorization": f"token {settings.input_token.get_secret_value()}"}
-    # category_id is only used by one query, but GraphQL allows unused variables, so
-    # keep it here for simplicity
-    variables = {"after": after, "category_id": category_id}
-    response = httpx.post(
-        GITHUB_GRAPHQL_URL,
-        headers=headers,
-        timeout=settings.httpx_timeout,
-        json={"query": query, "variables": variables, "operationName": "Q"},
-    )
-    if response.status_code != 200:
-        logging.error(
-            f"Response was not 200, after: {after}, category_id: {category_id}"
-        )
-        logging.error(response.text)
-        raise RuntimeError(response.text)
-    data = response.json()
-    if "errors" in data:
-        logging.error(f"Errors in response, after: {after}, category_id: {category_id}")
-        logging.error(data["errors"])
-        logging.error(response.text)
-        raise RuntimeError(response.text)
-    return data
-
-
-def get_graphql_issue_edges(*, settings: Settings, after: Union[str, None] = None):
-    data = get_graphql_response(settings=settings, query=ISSUES_QUERY, after=after)
-    graphql_response = IssuesResponse.model_validate(data)
-    return graphql_response.data.repository.issues.edges
-
-
-def get_graphql_question_discussion_edges(
-    *,
-    settings: Settings,
-    after: Union[str, None] = None,
-):
-    data = get_graphql_response(
-        settings=settings,
-        query=DISCUSSIONS_QUERY,
-        after=after,
-        category_id=QUESTIONS_CATEGORY_ID,
-    )
-    graphql_response = DiscussionsResponse.model_validate(data)
-    return graphql_response.data.repository.discussions.edges
-
-
-def get_graphql_pr_edges(*, settings: Settings, after: Union[str, None] = None):
-    data = get_graphql_response(settings=settings, query=PRS_QUERY, after=after)
-    graphql_response = PRsResponse.model_validate(data)
-    return graphql_response.data.repository.pullRequests.edges
-
-
-
-def get_issues_experts(settings: Settings):
-    issue_nodes: List[IssuesNode] = []
-    issue_edges = get_graphql_issue_edges(settings=settings)
-
-    while issue_edges:
-        for edge in issue_edges:
-            issue_nodes.append(edge.node)
-        last_edge = issue_edges[-1]
-        issue_edges = get_graphql_issue_edges(settings=settings, after=last_edge.cursor)
-
-    commentors = Counter()
-    last_month_commentors = Counter()
-    authors: Dict[str, Author] = {}
-
-    now = datetime.now(tz=timezone.utc)
-    one_month_ago = now - timedelta(days=30)
-
-    for issue in issue_nodes:
-        issue_author_name = None
-        if issue.author:
-            authors[issue.author.login] = issue.author
-            issue_author_name = issue.author.login
-        issue_commentors = set()
-        for comment in issue.comments.nodes:
-            if comment.author:
-                authors[comment.author.login] = comment.author
-                if comment.author.login != issue_author_name:
-                    issue_commentors.add(comment.author.login)
-        for author_name in issue_commentors:
-            commentors[author_name] += 1
-            if issue.createdAt > one_month_ago:
-                last_month_commentors[author_name] += 1
-
-    return commentors, last_month_commentors, authors
-
-
-def get_discussions_experts(settings: Settings) -> Tuple[Counter, Counter, Dict[str, Author]]:
-    discussion_nodes: List[DiscussionsNode] = []
-    discussion_edges = get_graphql_question_discussion_edges(settings=settings)
-
-    while discussion_edges:
-        for discussion_edge in discussion_edges:
-            discussion_nodes.append(discussion_edge.node)
-        last_edge = discussion_edges[-1]
-        discussion_edges = get_graphql_question_discussion_edges(
-            settings=settings, after=last_edge.cursor
-        )
-
-    commentors = Counter()
-    last_month_commentors = Counter()
-    authors: Dict[str, Author] = {}
-
-    now = datetime.now(tz=timezone.utc)
-    one_month_ago = now - timedelta(days=30)
-
-    for discussion in discussion_nodes:
-        discussion_author_name = None
-        if discussion.author:
-            authors[discussion.author.login] = discussion.author
-            discussion_author_name = discussion.author.login
-        discussion_commentors = set()
-        for comment in discussion.comments.nodes:
-            if comment.author:
-                authors[comment.author.login] = comment.author
-                if comment.author.login != discussion_author_name:
-                    discussion_commentors.add(comment.author.login)
-            for reply in comment.replies.nodes:
-                if reply.author:
-                    authors[reply.author.login] = reply.author
-                    if reply.author.login != discussion_author_name:
-                        discussion_commentors.add(reply.author.login)
-        for author_name in discussion_commentors:
-            commentors[author_name] += 1
-            if discussion.createdAt > one_month_ago:
-                last_month_commentors[author_name] += 1
-    return commentors, last_month_commentors, authors
-
-
-def get_experts(settings: Settings) -> Tuple[Counter, Counter, Dict[str, Author]]:
-    # Migrated to only use GitHub Discussions
-    # (
-    #     issues_commentors,
-    #     issues_last_month_commentors,
-    #     issues_authors,
-    # ) = get_issues_experts(settings=settings)
-    (
-        discussions_commentors,
-        discussions_last_month_commentors,
-        discussions_authors,
-    ) = get_discussions_experts(settings=settings)
-    # commentors = issues_commentors + discussions_commentors
-    commentors = discussions_commentors
-    # last_month_commentors = (
-    #     issues_last_month_commentors + discussions_last_month_commentors
-    # )
-    last_month_commentors = discussions_last_month_commentors
-    # authors = {**issues_authors, **discussions_authors}
-    authors = {**discussions_authors}
-    return commentors, last_month_commentors, authors
-
-
-def get_contributors(settings: Settings):
-    pr_nodes: List[PullRequestNode] = []
-    pr_edges = get_graphql_pr_edges(settings=settings)
-
-    while pr_edges:
-        for edge in pr_edges:
-            pr_nodes.append(edge.node)
-        last_edge = pr_edges[-1]
-        pr_edges = get_graphql_pr_edges(settings=settings, after=last_edge.cursor)
-
-    contributors = Counter()
-    commentors = Counter()
-    reviewers = Counter()
-    authors: Dict[str, Author] = {}
-
-    for pr in pr_nodes:
-        author_name = None
-        if pr.author:
-            authors[pr.author.login] = pr.author
-            author_name = pr.author.login
-        pr_commentors: Set[str] = set()
-        pr_reviewers: Set[str] = set()
-        for comment in pr.comments.nodes:
-            if comment.author:
-                authors[comment.author.login] = comment.author
-                if comment.author.login == author_name:
-                    continue
-                pr_commentors.add(comment.author.login)
-        for author_name in pr_commentors:
-            commentors[author_name] += 1
-        for review in pr.reviews.nodes:
-            if review.author:
-                authors[review.author.login] = review.author
-                pr_reviewers.add(review.author.login)
-        for reviewer in pr_reviewers:
-            reviewers[reviewer] += 1
-        if pr.state == "MERGED" and pr.author:
-            contributors[pr.author.login] += 1
-    return contributors, commentors, reviewers, authors
-
-
-
-def get_top_users(
-    *,
-    counter: Counter,
-    min_count: int,
-    authors: Dict[str, Author],
-    skip_users: Container[str],
-):
-    users = []
-    for commentor, count in counter.most_common(50):
-        if commentor in skip_users:
-            continue
-        if count >= min_count:
-            author = authors[commentor]
-            users.append(
-                {
-                    "login": commentor,
-                    "count": count,
-                    "avatarUrl": author.avatarUrl,
-                    "url": author.url,
-                }
-            )
-    return users
-
-
-if __name__ == "__main__":
-    logging.basicConfig(level=logging.INFO)
-    settings = Settings()
-    logging.info(f"Using config: {settings.model_dump_json()}")
-    g = Github(settings.input_token.get_secret_value())
-    repo = g.get_repo(settings.github_repository)
-    # Need to look up discussion category ID, and then can uncomment
-    # question_commentors, question_last_month_commentors, question_authors = get_experts(
-    #     settings=settings
-    # )
-    # Placeholder experts for now
-    question_commentors = Counter()
-    question_last_month_commentors = Counter()
-    question_authors = {}
-
-    contributors, pr_commentors, reviewers, pr_authors = get_contributors(
-        settings=settings
-    )
-    authors = {**question_authors, **pr_authors}
-    maintainers_logins = {}
-    bot_names = {"codecov", "github-actions", "pre-commit-ci", "dependabot", "dosu"}
-    maintainers = []
-    for login in maintainers_logins:
-        user = authors[login]
-        maintainers.append(
-            {
-                "login": login,
-                "answers": question_commentors[login],
-                "prs": contributors[login],
-                "avatarUrl": user.avatarUrl,
-                "url": user.url,
-            }
-        )
-
-    min_count_expert = 10
-    min_count_last_month = 3
-    min_count_contributor = 4
-    min_count_reviewer = 4
-    skip_users = maintainers_logins | bot_names
-    experts = get_top_users(
-        counter=question_commentors,
-        min_count=min_count_expert,
-        authors=authors,
-        skip_users=skip_users,
-    )
-    last_month_active = get_top_users(
-        counter=question_last_month_commentors,
-        min_count=min_count_last_month,
-        authors=authors,
-        skip_users=skip_users,
-    )
-    top_contributors = get_top_users(
-        counter=contributors,
-        min_count=min_count_contributor,
-        authors=authors,
-        skip_users=skip_users,
-    )
-    top_reviewers = get_top_users(
-        counter=reviewers,
-        min_count=min_count_reviewer,
-        authors=authors,
-        skip_users=skip_users,
-    )
-
-    people = {
-        "maintainers": maintainers,
-        "experts": experts,
-        "last_month_active": last_month_active,
-        "top_contributors": top_contributors,
-        "top_reviewers": top_reviewers,
-    }
-    people_path = Path("./docs/en/data/people.yml")
-    people_old_content = people_path.read_text(encoding="utf-8")
-    new_people_content = yaml.dump(
-        people, sort_keys=False, width=200, allow_unicode=True
-    )
-    if (
-        people_old_content == new_people_content
-    ):
-        logging.info("The LangChain People data hasn't changed, finishing.")
-        sys.exit(0)
-    people_path.write_text(new_people_content, encoding="utf-8")
-    logging.info("Setting up GitHub Actions git user")
-    subprocess.run(["git", "config", "user.name", "github-actions"], check=True)
-    subprocess.run(
-        ["git", "config", "user.email", "github-actions@github.com"], check=True
-    )
-    branch_name = "fastapi-people"
-    logging.info(f"Creating a new branch {branch_name}")
-    subprocess.run(["git", "checkout", "-b", branch_name], check=True)
-    logging.info("Adding updated file")
-    subprocess.run(
-        ["git", "add", str(people_path)], check=True
-    )
-    logging.info("Committing updated file")
-    message = "👥 Update LangChain People"
-    result = subprocess.run(["git", "commit", "-m", message], check=True)
-    logging.info("Pushing branch")
-    subprocess.run(["git", "push", "origin", branch_name], check=True)
-    logging.info("Creating PR")
-    pr = repo.create_pull(title=message, body=message, base="master", head=branch_name)
-    logging.info(f"Created PR: {pr.number}")
-    logging.info("Finished")

.github/actions/poetry_setup/action.yml

@@ -26,13 +26,12 @@ inputs:
 runs:
   using: composite
   steps:
-    - uses: actions/setup-python@v5
+    - uses: actions/setup-python@v4
       name: Setup python ${{ inputs.python-version }}
-      id: setup-python
       with:
         python-version: ${{ inputs.python-version }}
 
-    - uses: actions/cache@v4
+    - uses: actions/cache@v3
       id: cache-bin-poetry
       name: Cache Poetry binary - Python ${{ inputs.python-version }}
       env:
@@ -75,11 +74,10 @@ runs:
       env:
         POETRY_VERSION: ${{ inputs.poetry-version }}
         PYTHON_VERSION: ${{ inputs.python-version }}
-      # Install poetry using the python version installed by setup-python step.
-      run: pipx install "poetry==$POETRY_VERSION" --python '${{ steps.setup-python.outputs.python-path }}' --verbose
+      run: pipx install "poetry==$POETRY_VERSION" --python "python$PYTHON_VERSION" --verbose
 
     - name: Restore pip and poetry cached dependencies
-      uses: actions/cache@v4
+      uses: actions/cache@v3
       env:
         SEGMENT_DOWNLOAD_TIMEOUT_MIN: "4"
         WORKDIR: ${{ inputs.working-directory == '' && '.' || inputs.working-directory }}

.github/scripts/check_diff.py

@@ -13,10 +13,6 @@ if __name__ == "__main__":
     files = sys.argv[1:]
     dirs_to_run = set()
 
-    if len(files) == 300:
-        # max diff length is 300 files - there are likely files missing
-        raise ValueError("Max diff reached. Please manually run CI on changed libs.")
-
     for file in files:
         if any(
             file.startswith(dir_)
@@ -36,7 +32,13 @@ if __name__ == "__main__":
         elif "libs/partners" in file:
             partner_dir = file.split("/")[2]
             if os.path.isdir(f"libs/partners/{partner_dir}"):
-                dirs_to_run.add(f"libs/partners/{partner_dir}")
+                dirs_to_run.update(
+                    (
+                        f"libs/partners/{partner_dir}",
+                        "libs/langchain",
+                        "libs/experimental",
+                    )
+                )
             # Skip if the directory was deleted
         elif "libs/langchain" in file:
             dirs_to_run.update(("libs/langchain", "libs/experimental"))
@@ -46,5 +48,4 @@ if __name__ == "__main__":
             dirs_to_run.update(LANGCHAIN_DIRS)
         else:
             pass
-    json_output = json.dumps(list(dirs_to_run))
-    print(f"dirs-to-run={json_output}")  # noqa: T201
+    print(json.dumps(list(dirs_to_run)))

.github/scripts/get_min_versions.py

@@ -1,67 +0,0 @@
-import sys
-
-import tomllib
-from packaging.version import parse as parse_version
-import re
-
-MIN_VERSION_LIBS = ["langchain-core", "langchain-community", "langchain"]
-
-
-def get_min_version(version: str) -> str:
-    # case ^x.x.x
-    _match = re.match(r"^\^(\d+(?:\.\d+){0,2})$", version)
-    if _match:
-        return _match.group(1)
-
-    # case >=x.x.x,<y.y.y
-    _match = re.match(r"^>=(\d+(?:\.\d+){0,2}),<(\d+(?:\.\d+){0,2})$", version)
-    if _match:
-        _min = _match.group(1)
-        _max = _match.group(2)
-        assert parse_version(_min) < parse_version(_max)
-        return _min
-
-    # case x.x.x
-    _match = re.match(r"^(\d+(?:\.\d+){0,2})$", version)
-    if _match:
-        return _match.group(1)
-
-    raise ValueError(f"Unrecognized version format: {version}")
-
-
-def get_min_version_from_toml(toml_path: str):
-    # Parse the TOML file
-    with open(toml_path, "rb") as file:
-        toml_data = tomllib.load(file)
-
-    # Get the dependencies from tool.poetry.dependencies
-    dependencies = toml_data["tool"]["poetry"]["dependencies"]
-
-    # Initialize a dictionary to store the minimum versions
-    min_versions = {}
-
-    # Iterate over the libs in MIN_VERSION_LIBS
-    for lib in MIN_VERSION_LIBS:
-        # Check if the lib is present in the dependencies
-        if lib in dependencies:
-            # Get the version string
-            version_string = dependencies[lib]
-
-            # Use parse_version to get the minimum supported version from version_string
-            min_version = get_min_version(version_string)
-
-            # Store the minimum version in the min_versions dictionary
-            min_versions[lib] = min_version
-
-    return min_versions
-
-
-# Get the TOML file path from the command line argument
-toml_file = sys.argv[1]
-
-# Call the function to get the minimum versions
-min_versions = get_min_version_from_toml(toml_file)
-
-print(
-    " ".join([f"{lib}=={version}" for lib, version in min_versions.items()])
-)  # noqa: T201

.github/workflows/_all_ci.yml

@@ -32,39 +32,34 @@ concurrency:
   cancel-in-progress: true
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   lint:
-    name: "-"
     uses: ./.github/workflows/_lint.yml
     with:
       working-directory: ${{ inputs.working-directory }}
     secrets: inherit
 
   test:
-    name: "-"
     uses: ./.github/workflows/_test.yml
     with:
       working-directory: ${{ inputs.working-directory }}
     secrets: inherit
 
   compile-integration-tests:
-    name: "-"
     uses: ./.github/workflows/_compile_integration_test.yml
     with:
       working-directory: ${{ inputs.working-directory }}
     secrets: inherit
 
   dependencies:
-    name: "-"
     uses: ./.github/workflows/_dependencies.yml
     with:
       working-directory: ${{ inputs.working-directory }}
     secrets: inherit
 
   extended-tests:
-    name: "make extended_tests #${{ matrix.python-version }}"
     runs-on: ubuntu-latest
     strategy:
       matrix:
@@ -73,6 +68,7 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
+    name: Python ${{ matrix.python-version }} extended tests
     defaults:
       run:
         working-directory: ${{ inputs.working-directory }}

.github/workflows/_compile_integration_test.yml

@@ -9,7 +9,7 @@ on:
         description: "From which folder this pipeline executes"
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   build:
@@ -24,7 +24,7 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
-    name: "poetry run pytest -m compile tests/integration_tests #${{ matrix.python-version }}"
+    name: Python ${{ matrix.python-version }}
     steps:
       - uses: actions/checkout@v4
 

.github/workflows/_dependencies.yml

@@ -13,7 +13,7 @@ on:
         description: "Relative path to the langchain library folder"
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   build:
@@ -28,7 +28,7 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
-    name: dependency checks ${{ matrix.python-version }}
+    name: dependencies - Python ${{ matrix.python-version }}
     steps:
       - uses: actions/checkout@v4
 

.github/workflows/_integration_test.yml

@@ -8,11 +8,10 @@ on:
         type: string
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   build:
-    environment: Scheduled testing
     defaults:
       run:
         working-directory: ${{ inputs.working-directory }}
@@ -38,38 +37,13 @@ jobs:
         shell: bash
         run: poetry install --with test,test_integration
 
-      - name: Install deps outside pyproject
-        if: ${{ startsWith(inputs.working-directory, 'libs/community/') }}
-        shell: bash
-        run: poetry run pip install "boto3<2" "google-cloud-aiplatform<2"
-
-      - name: 'Authenticate to Google Cloud'
-        id: 'auth'
-        uses: google-github-actions/auth@v2
-        with:
-          credentials_json: '${{ secrets.GOOGLE_CREDENTIALS }}'
-
       - name: Run integration tests
         shell: bash
         env:
-          AI21_API_KEY: ${{ secrets.AI21_API_KEY }}
           GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
           MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
           TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
-          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
-          NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}
-          GOOGLE_SEARCH_API_KEY: ${{ secrets.GOOGLE_SEARCH_API_KEY }}
-          GOOGLE_CSE_ID: ${{ secrets.GOOGLE_CSE_ID }}
-          EXA_API_KEY: ${{ secrets.EXA_API_KEY }}
-          NOMIC_API_KEY: ${{ secrets.NOMIC_API_KEY }}
-          WATSONX_APIKEY: ${{ secrets.WATSONX_APIKEY }}
-          WATSONX_PROJECT_ID: ${{ secrets.WATSONX_PROJECT_ID }}
-          PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
-          PINECONE_ENVIRONMENT: ${{ secrets.PINECONE_ENVIRONMENT }}
-          ASTRA_DB_API_ENDPOINT: ${{ secrets.ASTRA_DB_API_ENDPOINT }}
-          ASTRA_DB_APPLICATION_TOKEN: ${{ secrets.ASTRA_DB_APPLICATION_TOKEN }}
-          ASTRA_DB_KEYSPACE: ${{ secrets.ASTRA_DB_KEYSPACE }}
         run: |
           make integration_tests
 

.github/workflows/_lint.yml

@@ -13,7 +13,7 @@ on:
         description: "Relative path to the langchain library folder"
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
   WORKDIR: ${{ inputs.working-directory == '' && '.' || inputs.working-directory }}
 
   # This env var allows us to get inline annotations when ruff has complaints.
@@ -21,7 +21,6 @@ env:
 
 jobs:
   build:
-    name: "make lint #${{ matrix.python-version }}"
     runs-on: ubuntu-latest
     strategy:
       matrix:
@@ -80,13 +79,13 @@ jobs:
           poetry run pip install -e "$LANGCHAIN_LOCATION"
 
       - name: Get .mypy_cache to speed up mypy
-        uses: actions/cache@v4
+        uses: actions/cache@v3
         env:
           SEGMENT_DOWNLOAD_TIMEOUT_MIN: "2"
         with:
           path: |
             ${{ env.WORKDIR }}/.mypy_cache
-          key: mypy-lint-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', inputs.working-directory)) }}
+          key: mypy-lint-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', env.WORKDIR)) }}
 
 
       - name: Analysing the code with our lint
@@ -94,7 +93,7 @@ jobs:
         run: |
           make lint_package
 
-      - name: Install unit test dependencies
+      - name: Install test dependencies
         # Also installs dev/lint/test/typing dependencies, to ensure we have
         # type hints for as many of our libraries as possible.
         # This helps catch errors that require dependencies to be spotted, for example:
@@ -103,24 +102,18 @@ jobs:
         # If you change this configuration, make sure to change the `cache-key`
         # in the `poetry_setup` action above to stop using the old cache.
         # It doesn't matter how you change it, any change will cause a cache-bust.
-        if: ${{ ! startsWith(inputs.working-directory, 'libs/partners/') }}
         working-directory: ${{ inputs.working-directory }}
         run: |
           poetry install --with test
-      - name: Install unit+integration test dependencies
-        if: ${{ startsWith(inputs.working-directory, 'libs/partners/') }}
-        working-directory: ${{ inputs.working-directory }}
-        run: |
-          poetry install --with test,test_integration
 
       - name: Get .mypy_cache_test to speed up mypy
-        uses: actions/cache@v4
+        uses: actions/cache@v3
         env:
           SEGMENT_DOWNLOAD_TIMEOUT_MIN: "2"
         with:
           path: |
             ${{ env.WORKDIR }}/.mypy_cache_test
-          key: mypy-test-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', inputs.working-directory)) }}
+          key: mypy-test-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', env.WORKDIR)) }}
 
       - name: Analysing the code with our lint
         working-directory: ${{ inputs.working-directory }}

.github/workflows/_release.yml

@@ -1,5 +1,5 @@
 name: release
-run-name: Release ${{ inputs.working-directory }} by @${{ github.actor }}
+
 on:
   workflow_call:
     inputs:
@@ -15,13 +15,12 @@ on:
         default: 'libs/langchain'
 
 env:
-  PYTHON_VERSION: "3.11"
-  POETRY_VERSION: "1.7.1"
+  PYTHON_VERSION: "3.10"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   build:
     if: github.ref == 'refs/heads/master'
-    environment: Scheduled testing
     runs-on: ubuntu-latest
 
     outputs:
@@ -118,18 +117,11 @@ jobs:
         #   are not found on test PyPI can be resolved and installed anyway.
         #   (https://test.pypi.org/simple). This will include the PKG_NAME==VERSION
         #   package because VERSION will not have been uploaded to regular PyPI yet.
-        # - attempt install again after 5 seconds if it fails because there is
-        #   sometimes a delay in availability on test pypi
+        #
         run: |
           poetry run pip install \
             --extra-index-url https://test.pypi.org/simple/ \
-            "$PKG_NAME==$VERSION" || \
-          ( \
-            sleep 5 && \
-            poetry run pip install \
-              --extra-index-url https://test.pypi.org/simple/ \
-              "$PKG_NAME==$VERSION" \
-          )
+            "$PKG_NAME==$VERSION"
 
           # Replace all dashes in the package name with underscores,
           # since that's how Python imports packages with dashes in the name.
@@ -157,59 +149,16 @@ jobs:
         run: make tests
         working-directory: ${{ inputs.working-directory }}
 
-      - name: 'Authenticate to Google Cloud'
-        id: 'auth'
-        uses: google-github-actions/auth@v2
-        with:
-          credentials_json: '${{ secrets.GOOGLE_CREDENTIALS }}'
-
       - name: Run integration tests
         if: ${{ startsWith(inputs.working-directory, 'libs/partners/') }}
         env:
-          AI21_API_KEY: ${{ secrets.AI21_API_KEY }}
           GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
           MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
           TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
-          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
-          AZURE_OPENAI_API_VERSION: ${{ secrets.AZURE_OPENAI_API_VERSION }}
-          AZURE_OPENAI_API_BASE: ${{ secrets.AZURE_OPENAI_API_BASE }}
-          AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
-          AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_CHAT_DEPLOYMENT_NAME }}
-          AZURE_OPENAI_LLM_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_LLM_DEPLOYMENT_NAME }}
-          AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME }}
-          NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}
-          GOOGLE_SEARCH_API_KEY: ${{ secrets.GOOGLE_SEARCH_API_KEY }}
-          GOOGLE_CSE_ID: ${{ secrets.GOOGLE_CSE_ID }}
-          EXA_API_KEY: ${{ secrets.EXA_API_KEY }}
-          NOMIC_API_KEY: ${{ secrets.NOMIC_API_KEY }}
-          WATSONX_APIKEY: ${{ secrets.WATSONX_APIKEY }}
-          WATSONX_PROJECT_ID: ${{ secrets.WATSONX_PROJECT_ID }}
-          PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
-          PINECONE_ENVIRONMENT: ${{ secrets.PINECONE_ENVIRONMENT }}
-          ASTRA_DB_API_ENDPOINT: ${{ secrets.ASTRA_DB_API_ENDPOINT }}
-          ASTRA_DB_APPLICATION_TOKEN: ${{ secrets.ASTRA_DB_APPLICATION_TOKEN }}
-          ASTRA_DB_KEYSPACE: ${{ secrets.ASTRA_DB_KEYSPACE }}
         run: make integration_tests
         working-directory: ${{ inputs.working-directory }}
 
-      - name: Get minimum versions
-        working-directory: ${{ inputs.working-directory }}
-        id: min-version
-        run: |
-          poetry run pip install packaging
-          min_versions="$(poetry run python $GITHUB_WORKSPACE/.github/scripts/get_min_versions.py pyproject.toml)"
-          echo "min-versions=$min_versions" >> "$GITHUB_OUTPUT"
-          echo "min-versions=$min_versions"
-
-      - name: Run unit tests with minimum dependency versions
-        if: ${{ steps.min-version.outputs.min-versions != '' }}
-        env:
-          MIN_VERSIONS: ${{ steps.min-version.outputs.min-versions }}
-        run: |
-          poetry run pip install $MIN_VERSIONS
-          make tests
-        working-directory: ${{ inputs.working-directory }}
 
   publish:
     needs:

.github/workflows/_test.yml

@@ -13,7 +13,7 @@ on:
         description: "Relative path to the langchain library folder"
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   build:
@@ -28,7 +28,7 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
-    name: "make test #${{ matrix.python-version }}"
+    name: Python ${{ matrix.python-version }}
     steps:
       - uses: actions/checkout@v4
 

.github/workflows/_test_release.yml

@@ -9,7 +9,7 @@ on:
         description: "From which folder this pipeline executes"
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
   PYTHON_VERSION: "3.10"
 
 jobs:

.github/workflows/api_doc_build.yml

@@ -1,52 +0,0 @@
-name: API docs build
-
-on:
-  workflow_dispatch:
-  schedule:
-    - cron:  '0 13 * * *'
-env:
-  POETRY_VERSION: "1.7.1"
-  PYTHON_VERSION: "3.10"
-
-jobs:
-  build:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          ref: bagatur/api_docs_build
-
-      - name: Set Git config
-        run: |
-          git config --local user.email "actions@github.com"
-          git config --local user.name "Github Actions"
-
-      - name: Merge master
-        run: | 
-          git fetch origin master
-          git merge origin/master -m "Merge master" --allow-unrelated-histories -X theirs
-
-      - name: Set up Python ${{ env.PYTHON_VERSION }} + Poetry ${{ env.POETRY_VERSION }}
-        uses: "./.github/actions/poetry_setup"
-        with:
-          python-version: ${{ env.PYTHON_VERSION }}
-          poetry-version: ${{ env.POETRY_VERSION }}
-          cache-key: api-docs
-
-      - name: Install dependencies
-        run: |
-          poetry run python -m pip install --upgrade --no-cache-dir pip setuptools
-          poetry run python -m pip install --upgrade --no-cache-dir sphinx readthedocs-sphinx-ext
-          poetry run python -m pip install ./libs/partners/*
-          poetry run python -m pip install --exists-action=w --no-cache-dir -r docs/api_reference/requirements.txt
-
-      - name: Build docs
-        run: |
-          poetry run python -m pip install --upgrade --no-cache-dir pip setuptools
-          poetry run python docs/api_reference/create_api_rst.py
-          poetry run python -m sphinx -T -E -b html -d _build/doctrees -c docs/api_reference docs/api_reference api_reference_build/html -j auto
-
-      # https://github.com/marketplace/actions/add-commit
-      - uses: EndBug/add-and-commit@v9
-        with:
-          message: 'Update API docs build'

.github/workflows/check_diffs.yml

@@ -1,10 +1,15 @@
 ---
-name: CI
+name: Check library diffs
 
 on:
   push:
     branches: [master]
   pull_request:
+    paths:
+      - ".github/actions/**"
+      - ".github/tools/**"
+      - ".github/workflows/**"
+      - "libs/**"
 
 # If another push to the same PR or branch happens while this workflow is still running,
 # cancel the earlier run in favor of the next run.
@@ -21,18 +26,16 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
+      - uses: actions/setup-python@v4
         with:
           python-version: '3.10'
       - id: files
         uses: Ana06/get-changed-files@v2.2.0
       - id: set-matrix
-        run: |
-          python .github/scripts/check_diff.py ${{ steps.files.outputs.all }} >> $GITHUB_OUTPUT
+        run: echo "dirs-to-run=$(python .github/scripts/check_diff.py ${{ steps.files.outputs.all }})" >> $GITHUB_OUTPUT
     outputs:
       dirs-to-run: ${{ steps.set-matrix.outputs.dirs-to-run }}
   ci:
-    name: cd ${{ matrix.working-directory }}
     needs: [ build ]
     strategy:
       matrix:

.github/workflows/codespell.yml

@@ -1,5 +1,5 @@
 ---
-name: CI / cd . / make spell_check
+name: Codespell
 
 on:
   push:
@@ -12,7 +12,7 @@ permissions:
 
 jobs:
   codespell:
-    name: (Check for spelling errors)
+    name: Check for spelling errors
     runs-on: ubuntu-latest
 
     steps:
@@ -34,4 +34,3 @@ jobs:
         with:
           skip: guide_imports.json
           ignore_words_list: ${{ steps.extract_ignore_words.outputs.ignore_words_list }}
-          exclude_file: libs/community/langchain_community/llms/yuan2.py

.github/workflows/doc_lint.yml

@@ -1,5 +1,5 @@
 ---
-name: CI / cd .
+name: Docs, templates, cookbook lint
 
 on:
   push:
@@ -15,7 +15,6 @@ on:
 
 jobs:
   check:
-    name: Check for "from langchain import x" imports
     runs-on: ubuntu-latest
 
     steps:
@@ -29,7 +28,6 @@ jobs:
         git grep 'from langchain import' {docs/docs,templates,cookbook} | grep -vE 'from langchain import (hub)' && exit 1 || exit 0
 
   lint:
-      name: "-"
       uses:
         ./.github/workflows/_lint.yml
       with:

.github/workflows/extract_ignored_words_list.py

@@ -7,4 +7,4 @@ ignore_words_list = (
     pyproject_toml.get("tool", {}).get("codespell", {}).get("ignore-words-list")
 )
 
-print(f"::set-output name=ignore_words_list::{ignore_words_list}")  # noqa: T201
+print(f"::set-output name=ignore_words_list::{ignore_words_list}")

.github/workflows/langchain_cli_release.yml

@@ -0,0 +1,13 @@
+---
+name: libs/cli Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_release.yml
+    with:
+      working-directory: libs/cli
+    secrets: inherit

.github/workflows/langchain_community_release.yml

@@ -0,0 +1,13 @@
+---
+name: libs/community Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_release.yml
+    with:
+      working-directory: libs/community
+    secrets: inherit

.github/workflows/langchain_core_release.yml

@@ -0,0 +1,13 @@
+---
+name: libs/core Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_release.yml
+    with:
+      working-directory: libs/core
+    secrets: inherit

.github/workflows/langchain_experimental_release.yml

@@ -0,0 +1,13 @@
+---
+name: libs/experimental Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_release.yml
+    with:
+      working-directory: libs/experimental
+    secrets: inherit

.github/workflows/langchain_experimental_test_release.yml

@@ -0,0 +1,13 @@
+---
+name: Experimental Test Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_test_release.yml
+    with:
+      working-directory: libs/experimental
+    secrets: inherit

.github/workflows/langchain_openai_release.yml

@@ -0,0 +1,13 @@
+---
+name: libs/core Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_release.yml
+    with:
+      working-directory: libs/core
+    secrets: inherit

.github/workflows/langchain_release.yml

@@ -0,0 +1,27 @@
+---
+name: libs/langchain Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_release.yml
+    with:
+      working-directory: libs/langchain
+    secrets: inherit
+
+  # N.B.: It's possible that PyPI doesn't make the new release visible / available
+  #       immediately after publishing. If that happens, the docker build might not
+  #       create a new docker image for the new release, since it won't see it.
+  #
+  #       If this ends up being a problem, add a check to the end of the `_release.yml`
+  #       workflow that prevents the workflow from finishing until the new release
+  #       is visible and installable on PyPI.
+  release-docker:
+    needs:
+      - release
+    uses:
+      ./.github/workflows/langchain_release_docker.yml
+    secrets: inherit

.github/workflows/langchain_test_release.yml

@@ -0,0 +1,13 @@
+---
+name: Test Release
+
+on:
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+jobs:
+  release:
+    uses:
+      ./.github/workflows/_test_release.yml
+    with:
+      working-directory: libs/langchain
+    secrets: inherit

.github/workflows/people.yml

@@ -1,23 +0,0 @@
-name: LangChain People
-
-on:
-#  schedule:
-#    - cron: "0 14 1 * *"
-  workflow_dispatch:
-
-jobs:
-  langchain-people:
-    if: github.repository_owner == 'langchain-ai'
-    runs-on: ubuntu-latest
-    steps:
-      - name: Dump GitHub context
-        env:
-          GITHUB_CONTEXT: ${{ toJson(github) }}
-        run: echo "$GITHUB_CONTEXT"
-      - uses: actions/checkout@v4
-      # Ref: https://github.com/actions/runner/issues/2033
-      - name: Fix git safe.directory in container
-        run: mkdir -p /home/runner/work/_temp/_github_home && printf "[safe]\n\tdirectory = /github/workspace" > /home/runner/work/_temp/_github_home/.gitconfig
-      - uses: ./.github/actions/people
-        with:
-          token: ${{ secrets.LANGCHAIN_PEOPLE }}

.github/workflows/scheduled_test.yml

@@ -6,7 +6,7 @@ on:
     - cron:  '0 13 * * *'
 
 env:
-  POETRY_VERSION: "1.7.1"
+  POETRY_VERSION: "1.6.1"
 
 jobs:
   build:
@@ -36,7 +36,7 @@ jobs:
 
       - name: 'Authenticate to Google Cloud'
         id: 'auth'
-        uses: google-github-actions/auth@v2
+        uses: 'google-github-actions/auth@v1'
         with:
           credentials_json: '${{ secrets.GOOGLE_CREDENTIALS }}'
 
@@ -54,11 +54,6 @@ jobs:
           echo "Running scheduled tests, installing dependencies with poetry..."
           poetry install --with=test_integration,test
 
-      - name: Install deps outside pyproject
-        if: ${{ startsWith(inputs.working-directory, 'libs/community/') }}
-        shell: bash
-        run: poetry run pip install "boto3<2" "google-cloud-aiplatform<2"
-
       - name: Run tests
         shell: bash
         env:

.github/workflows/templates_ci.yml

@@ -0,0 +1,36 @@
+---
+name: templates CI
+
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+    paths:
+      - '.github/actions/poetry_setup/action.yml'
+      - '.github/tools/**'
+      - '.github/workflows/_lint.yml'
+      - '.github/workflows/templates_ci.yml'
+      - 'templates/**'
+  workflow_dispatch:  # Allows to trigger the workflow manually in GitHub UI
+
+# If another push to the same PR or branch happens while this workflow is still running,
+# cancel the earlier run in favor of the next run.
+#
+# There's no point in testing an outdated version of the code. GitHub only allows
+# a limited number of job runners to be active at the same time, so it's better to cancel
+# pointless jobs early so that more useful jobs can run sooner.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  POETRY_VERSION: "1.6.1"
+  WORKDIR: "templates"
+
+jobs:
+  lint:
+    uses:
+      ./.github/workflows/_lint.yml
+    with:
+      working-directory: templates
+    secrets: inherit

.readthedocs.yaml

@@ -4,17 +4,21 @@
 # Required
 version: 2
 
-formats:
-  - pdf
-
 # Set the version of Python and other tools you might need
 build:
   os: ubuntu-22.04
   tools:
     python: "3.11"
   commands:
-    - mkdir -p $READTHEDOCS_OUTPUT
-    - cp -r api_reference_build/* $READTHEDOCS_OUTPUT
+      - python -mvirtualenv $READTHEDOCS_VIRTUALENV_PATH
+      - python -m pip install --upgrade --no-cache-dir pip setuptools
+      - python -m pip install --upgrade --no-cache-dir sphinx readthedocs-sphinx-ext
+      - python -m pip install ./libs/partners/*
+      - python -m pip install --exists-action=w --no-cache-dir -r docs/api_reference/requirements.txt
+      - python docs/api_reference/create_api_rst.py
+      - cat docs/api_reference/conf.py
+      - python -m sphinx -T -E -b html -d _build/doctrees -c docs/api_reference docs/api_reference $READTHEDOCS_OUTPUT/html -j auto
+
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
    configuration: docs/api_reference/conf.py

Makefile

@@ -15,12 +15,7 @@ docs_build:
 	docs/.local_build.sh
 
 docs_clean:
-	@if [ -d _dist ]; then \
-			rm -r _dist; \
-			echo "Directory _dist has been cleaned."; \
-	else \
-			echo "Nothing to clean."; \
-	fi
+	rm -r _dist
 
 docs_linkcheck:
 	poetry run linkchecker _dist/docs/ --ignore-url node_modules

README.md

@@ -1,6 +1,6 @@
 # 🦜️🔗 LangChain
 
-⚡ Build context-aware reasoning applications ⚡
+⚡ Building applications with LLMs through composability ⚡
 
 [![Release Notes](https://img.shields.io/github/release/langchain-ai/langchain)](https://github.com/langchain-ai/langchain/releases)
 [![CI](https://github.com/langchain-ai/langchain/actions/workflows/check_diffs.yml/badge.svg)](https://github.com/langchain-ai/langchain/actions/workflows/check_diffs.yml)
@@ -18,7 +18,7 @@ Looking for the JS/TS library? Check out [LangChain.js](https://github.com/langc
 
 To help you ship LangChain apps to production faster, check out [LangSmith](https://smith.langchain.com). 
 [LangSmith](https://smith.langchain.com) is a unified developer platform for building, testing, and monitoring LLM applications. 
-Fill out [this form](https://www.langchain.com/contact-sales) to speak with our sales team.
+Fill out [this form](https://airtable.com/appwQzlErAS2qiP0L/shrGtGaVBVAz7NcV2) to get off the waitlist or speak with our sales team.
 
 ## Quick Install
 
@@ -43,14 +43,13 @@ This framework consists of several parts.
 - **[LangChain Templates](templates)**: A collection of easily deployable reference architectures for a wide variety of tasks.
 - **[LangServe](https://github.com/langchain-ai/langserve)**: A library for deploying LangChain chains as a REST API.
 - **[LangSmith](https://smith.langchain.com)**: A developer platform that lets you debug, test, evaluate, and monitor chains built on any LLM framework and seamlessly integrates with LangChain.
-- **[LangGraph](https://python.langchain.com/docs/langgraph)**: LangGraph is a library for building stateful, multi-actor applications with LLMs, built on top of (and intended to be used with) LangChain. It extends the LangChain Expression Language with the ability to coordinate multiple chains (or actors) across multiple steps of computation in a cyclic manner. 
 
 The LangChain libraries themselves are made up of several different packages.
 - **[`langchain-core`](libs/core)**: Base abstractions and LangChain Expression Language.
 - **[`langchain-community`](libs/community)**: Third party integrations.
 - **[`langchain`](libs/langchain)**: Chains, agents, and retrieval strategies that make up an application's cognitive architecture.
 
-![Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.](docs/static/img/langchain_stack.png "LangChain Architecture Overview")
+![LangChain Stack](docs/static/img/langchain_stack.png)
 
 ## 🧱 What can you build with LangChain?
 **❓ Retrieval augmented generation**

cookbook/LLaMA2_sql_chat.ipynb

@@ -61,13 +61,13 @@
    ],
    "source": [
     "# Local\n",
-    "from langchain_community.chat_models import ChatOllama\n",
+    "from langchain.chat_models import ChatOllama\n",
     "\n",
     "llama2_chat = ChatOllama(model=\"llama2:13b-chat\")\n",
     "llama2_code = ChatOllama(model=\"codellama:7b-instruct\")\n",
     "\n",
     "# API\n",
-    "from langchain_community.llms import Replicate\n",
+    "from langchain.llms import Replicate\n",
     "\n",
     "# REPLICATE_API_TOKEN = getpass()\n",
     "# os.environ[\"REPLICATE_API_TOKEN\"] = REPLICATE_API_TOKEN\n",
@@ -107,7 +107,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.utilities import SQLDatabase\n",
+    "from langchain.utilities import SQLDatabase\n",
     "\n",
     "db = SQLDatabase.from_uri(\"sqlite:///nba_roster.db\", sample_rows_in_table_info=0)\n",
     "\n",
@@ -125,7 +125,7 @@
    "id": "654b3577-baa2-4e12-a393-f40e5db49ac7",
    "metadata": {},
    "source": [
-    "## Query a SQL Database \n",
+    "## Query a SQL DB \n",
     "\n",
     "Follow the runnables workflow [here](https://python.langchain.com/docs/expression_language/cookbook/sql_db)."
    ]
@@ -149,9 +149,8 @@
    ],
    "source": [
     "# Prompt\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
     "\n",
-    "# Update the template based on the type of SQL Database like MySQL, Microsoft SQL Server and so on\n",
     "template = \"\"\"Based on the table schema below, write a SQL query that would answer the user's question:\n",
     "{schema}\n",
     "\n",
@@ -278,7 +277,7 @@
    "source": [
     "# Prompt\n",
     "from langchain.memory import ConversationBufferMemory\n",
-    "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
+    "from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
     "\n",
     "template = \"\"\"Given an input question, convert it to a SQL query. No pre-amble. Based on the table schema below, write a SQL query that would answer the user's question:\n",
     "{schema}\n",

cookbook/Multi_modal_RAG.ipynb

@@ -101,7 +101,7 @@
     "If you want to use the provided folder, then simply opt for a [pdf loader](https://python.langchain.com/docs/modules/data_connection/document_loaders/pdf) for the document:\n",
     "\n",
     "```\n",
-    "from langchain_community.document_loaders import PyPDFLoader\n",
+    "from langchain.document_loaders import PyPDFLoader\n",
     "loader = PyPDFLoader(path + fname)\n",
     "docs = loader.load()\n",
     "tables = [] # Ignore w/ basic pdf loader\n",
@@ -198,9 +198,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
     "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "\n",
     "# Generate summaries of text elements\n",
@@ -341,7 +341,7 @@
     "Add raw docs and doc summaries to [Multi Vector Retriever](https://python.langchain.com/docs/modules/data_connection/retrievers/multi_vector#summary): \n",
     "\n",
     "* Store the raw texts, tables, and images in the `docstore`.\n",
-    "* Store the texts, table summaries, and image summaries in the `vectorstore` for efficient semantic retrieval."
+    "* Store the texts, table summaries, and image summaries in the `vectorstore` for semantic retrieval."
    ]
   },
   {
@@ -353,11 +353,11 @@
    "source": [
     "import uuid\n",
     "\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
     "from langchain.storage import InMemoryStore\n",
-    "from langchain_community.vectorstores import Chroma\n",
+    "from langchain.vectorstores import Chroma\n",
     "from langchain_core.documents import Document\n",
-    "from langchain_openai import OpenAIEmbeddings\n",
     "\n",
     "\n",
     "def create_multi_vector_retriever(\n",

cookbook/Multi_modal_RAG_google.ipynb

@@ -93,7 +93,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.document_loaders import PyPDFLoader\n",
+    "from langchain.document_loaders import PyPDFLoader\n",
     "\n",
     "loader = PyPDFLoader(\"./cj/cj.pdf\")\n",
     "docs = loader.load()\n",
@@ -158,11 +158,11 @@
     }
    ],
    "source": [
+    "from langchain.chat_models import ChatVertexAI\n",
+    "from langchain.llms import VertexAI\n",
     "from langchain.prompts import PromptTemplate\n",
-    "from langchain_community.chat_models import ChatVertexAI\n",
-    "from langchain_community.llms import VertexAI\n",
+    "from langchain.schema.output_parser import StrOutputParser\n",
     "from langchain_core.messages import AIMessage\n",
-    "from langchain_core.output_parsers import StrOutputParser\n",
     "from langchain_core.runnables import RunnableLambda\n",
     "\n",
     "\n",
@@ -243,7 +243,7 @@
     "import base64\n",
     "import os\n",
     "\n",
-    "from langchain_core.messages import HumanMessage\n",
+    "from langchain.schema.messages import HumanMessage\n",
     "\n",
     "\n",
     "def encode_image(image_path):\n",
@@ -342,11 +342,11 @@
    "source": [
     "import uuid\n",
     "\n",
+    "from langchain.embeddings import VertexAIEmbeddings\n",
     "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
+    "from langchain.schema.document import Document\n",
     "from langchain.storage import InMemoryStore\n",
-    "from langchain_community.embeddings import VertexAIEmbeddings\n",
-    "from langchain_community.vectorstores import Chroma\n",
-    "from langchain_core.documents import Document\n",
+    "from langchain.vectorstores import Chroma\n",
     "\n",
     "\n",
     "def create_multi_vector_retriever(\n",
@@ -440,7 +440,7 @@
     "import re\n",
     "\n",
     "from IPython.display import HTML, display\n",
-    "from langchain_core.runnables import RunnableLambda, RunnablePassthrough\n",
+    "from langchain.schema.runnable import RunnableLambda, RunnablePassthrough\n",
     "from PIL import Image\n",
     "\n",
     "\n",

cookbook/Semi_Structured_RAG.ipynb

@@ -235,9 +235,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain_core.output_parsers import StrOutputParser"
    ]
   },
   {
@@ -318,11 +318,11 @@
    "source": [
     "import uuid\n",
     "\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
     "from langchain.storage import InMemoryStore\n",
-    "from langchain_community.vectorstores import Chroma\n",
+    "from langchain.vectorstores import Chroma\n",
     "from langchain_core.documents import Document\n",
-    "from langchain_openai import OpenAIEmbeddings\n",
     "\n",
     "# The vectorstore to use to index the child chunks\n",
     "vectorstore = Chroma(collection_name=\"summaries\", embedding_function=OpenAIEmbeddings())\n",

cookbook/Semi_structured_and_multi_modal_RAG.ipynb

@@ -211,9 +211,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain_core.output_parsers import StrOutputParser"
    ]
   },
   {
@@ -373,11 +373,11 @@
    "source": [
     "import uuid\n",
     "\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
     "from langchain.storage import InMemoryStore\n",
-    "from langchain_community.vectorstores import Chroma\n",
+    "from langchain.vectorstores import Chroma\n",
     "from langchain_core.documents import Document\n",
-    "from langchain_openai import OpenAIEmbeddings\n",
     "\n",
     "# The vectorstore to use to index the child chunks\n",
     "vectorstore = Chroma(collection_name=\"summaries\", embedding_function=OpenAIEmbeddings())\n",

cookbook/Semi_structured_multi_modal_RAG_LLaMA2.ipynb

@@ -209,9 +209,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.chat_models import ChatOllama\n",
-    "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_core.prompts import ChatPromptTemplate"
+    "from langchain.chat_models import ChatOllama\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain_core.output_parsers import StrOutputParser"
    ]
   },
   {
@@ -376,10 +376,10 @@
    "source": [
     "import uuid\n",
     "\n",
+    "from langchain.embeddings import GPT4AllEmbeddings\n",
     "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
     "from langchain.storage import InMemoryStore\n",
-    "from langchain_community.embeddings import GPT4AllEmbeddings\n",
-    "from langchain_community.vectorstores import Chroma\n",
+    "from langchain.vectorstores import Chroma\n",
     "from langchain_core.documents import Document\n",
     "\n",
     "# The vectorstore to use to index the child chunks\n",

cookbook/advanced_rag_eval.ipynb

@@ -62,7 +62,7 @@
     "path = \"/Users/rlm/Desktop/cpi/\"\n",
     "\n",
     "# Load\n",
-    "from langchain_community.document_loaders import PyPDFLoader\n",
+    "from langchain.document_loaders import PyPDFLoader\n",
     "\n",
     "loader = PyPDFLoader(path + \"cpi.pdf\")\n",
     "pdf_pages = loader.load()\n",
@@ -132,8 +132,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.vectorstores import Chroma\n",
-    "from langchain_openai import OpenAIEmbeddings\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.vectorstores import Chroma\n",
     "\n",
     "baseline = Chroma.from_texts(\n",
     "    texts=all_splits_pypdf_texts,\n",
@@ -160,9 +160,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
     "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "# Prompt\n",
     "prompt_text = \"\"\"You are an assistant tasked with summarizing tables and text for retrieval. \\\n",

cookbook/analyze_document.ipynb

@@ -29,7 +29,7 @@
    "outputs": [],
    "source": [
     "from langchain.chains import AnalyzeDocumentChain\n",
-    "from langchain_openai import ChatOpenAI\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "\n",
     "llm = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)"
    ]

cookbook/apache_kafka_message_handling.ipynb

@@ -1,922 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "rT1cmV4qCa2X"
-   },
-   "source": [
-    "#  Using Apache Kafka to route messages\n",
-    "\n",
-    "---\n",
-    "\n",
-    "\n",
-    "\n",
-    "This notebook shows you how to use LangChain's standard chat features while passing the chat messages back and forth via Apache Kafka.\n",
-    "\n",
-    "This goal is to simulate an architecture where the chat front end and the LLM are running as separate services that need to communicate with one another over an internal nework.\n",
-    "\n",
-    "It's an alternative to typical pattern of requesting a reponse from the model via a REST API (there's more info on why you would want to do this at the end of the notebook)."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "UPYtfAR_9YxZ"
-   },
-   "source": [
-    "### 1. Install the main dependencies\n",
-    "\n",
-    "Dependencies include:\n",
-    "\n",
-    "- The Quix Streams library for managing interactions with Apache Kafka (or Kafka-like tools such as Redpanda) in a \"Pandas-like\" way.\n",
-    "- The LangChain library for managing interactions with Llama-2 and storing conversation state."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "ZX5tfKiy9cN-"
-   },
-   "outputs": [],
-   "source": [
-    "!pip install quixstreams==2.1.2a langchain==0.0.340 huggingface_hub==0.19.4 langchain-experimental==0.0.42 python-dotenv"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "losTSdTB9d9O"
-   },
-   "source": [
-    "### 2. Build and install the llama-cpp-python library (with CUDA enabled so that we can advantage of Google Colab GPU\n",
-    "\n",
-    "The `llama-cpp-python` library is a Python wrapper around the `llama-cpp` library which enables you to efficiently leverage just a CPU to run quantized LLMs.\n",
-    "\n",
-    "When you use the standard `pip install llama-cpp-python` command, you do not get GPU support by default. Generation can be very slow if you rely on just the CPU in Google Colab, so the following command adds an extra option to build and install\n",
-    "`llama-cpp-python` with GPU support (make sure you have a GPU-enabled runtime selected in Google Colab)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "-JCQdl1G9tbl"
-   },
-   "outputs": [],
-   "source": [
-    "!CMAKE_ARGS=\"-DLLAMA_CUBLAS=on\" FORCE_CMAKE=1 pip install llama-cpp-python"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "5_vjVIAh9rLl"
-   },
-   "source": [
-    "### 3. Download and setup Kafka and Zookeeper instances\n",
-    "\n",
-    "Download the Kafka binaries from the Apache website and start the servers as daemons. We'll use the default configurations (provided by Apache Kafka) for spinning up the instances."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {
-    "id": "zFz7czGRW5Wr"
-   },
-   "outputs": [],
-   "source": [
-    "!curl -sSOL https://dlcdn.apache.org/kafka/3.6.1/kafka_2.13-3.6.1.tgz\n",
-    "!tar -xzf kafka_2.13-3.6.1.tgz"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "Uf7NR_UZ9wye"
-   },
-   "outputs": [],
-   "source": [
-    "!./kafka_2.13-3.6.1/bin/zookeeper-server-start.sh -daemon ./kafka_2.13-3.6.1/config/zookeeper.properties\n",
-    "!./kafka_2.13-3.6.1/bin/kafka-server-start.sh -daemon ./kafka_2.13-3.6.1/config/server.properties\n",
-    "!echo \"Waiting for 10 secs until kafka and zookeeper services are up and running\"\n",
-    "!sleep 10"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "H3SafFuS94p1"
-   },
-   "source": [
-    "### 4. Check that the Kafka Daemons are running\n",
-    "\n",
-    "Show the running processes and filter it for Java processes (you should see two—one for each server)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "CZDC2lQP99yp"
-   },
-   "outputs": [],
-   "source": [
-    "!ps aux | grep -E '[j]ava'"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "Snoxmjb5-V37"
-   },
-   "source": [
-    "### 5. Import the required dependencies and initialize required variables\n",
-    "\n",
-    "Import the Quix Streams library for interacting with Kafka, and the necessary LangChain components for running a `ConversationChain`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "metadata": {
-    "id": "plR9e_MF-XL5"
-   },
-   "outputs": [],
-   "source": [
-    "# Import utility libraries\n",
-    "import json\n",
-    "import random\n",
-    "import re\n",
-    "import time\n",
-    "import uuid\n",
-    "from os import environ\n",
-    "from pathlib import Path\n",
-    "from random import choice, randint, random\n",
-    "\n",
-    "from dotenv import load_dotenv\n",
-    "\n",
-    "# Import a Hugging Face utility to download models directly from Hugging Face hub:\n",
-    "from huggingface_hub import hf_hub_download\n",
-    "from langchain.chains import ConversationChain\n",
-    "\n",
-    "# Import Langchain modules for managing prompts and conversation chains:\n",
-    "from langchain.llms import LlamaCpp\n",
-    "from langchain.memory import ConversationTokenBufferMemory\n",
-    "from langchain.prompts import PromptTemplate, load_prompt\n",
-    "from langchain.schema import SystemMessage\n",
-    "from langchain_experimental.chat_models import Llama2Chat\n",
-    "from quixstreams import Application, State, message_key\n",
-    "\n",
-    "# Import Quix dependencies\n",
-    "from quixstreams.kafka import Producer\n",
-    "\n",
-    "# Initialize global variables.\n",
-    "AGENT_ROLE = \"AI\"\n",
-    "chat_id = \"\"\n",
-    "\n",
-    "# Set the current role to the role constant and initialize variables for supplementary customer metadata:\n",
-    "role = AGENT_ROLE"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "HgJjJ9aZ-liy"
-   },
-   "source": [
-    "### 6. Download the \"llama-2-7b-chat.Q4_K_M.gguf\" model\n",
-    "\n",
-    "Download the quantized LLama-2 7B model from Hugging Face which we will use as a local LLM (rather than relying on REST API calls to an external service)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 67,
-     "referenced_widgets": [
-      "969343cdbe604a26926679bbf8bd2dda",
-      "d8b8370c9b514715be7618bfe6832844",
-      "0def954cca89466b8408fadaf3b82e64",
-      "462482accc664729980562e208ceb179",
-      "80d842f73c564dc7b7cc316c763e2633",
-      "fa055d9f2a9d4a789e9cf3c89e0214e5",
-      "30ecca964a394109ac2ad757e3aec6c0",
-      "fb6478ce2dac489bb633b23ba0953c5c",
-      "734b0f5da9fc4307a95bab48cdbb5d89",
-      "b32f3a86a74741348511f4e136744ac8",
-      "e409071bff5a4e2d9bf0e9f5cc42231b"
-     ]
-    },
-    "id": "Qwu4YoSA-503",
-    "outputId": "f956976c-7485-415b-ac93-4336ade31964"
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "The model path does not exist in state. Downloading model...\n"
-     ]
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "969343cdbe604a26926679bbf8bd2dda",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "llama-2-7b-chat.Q4_K_M.gguf:   0%|          | 0.00/4.08G [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "model_name = \"llama-2-7b-chat.Q4_K_M.gguf\"\n",
-    "model_path = f\"./state/{model_name}\"\n",
-    "\n",
-    "if not Path(model_path).exists():\n",
-    "    print(\"The model path does not exist in state. Downloading model...\")\n",
-    "    hf_hub_download(\"TheBloke/Llama-2-7b-Chat-GGUF\", model_name, local_dir=\"state\")\n",
-    "else:\n",
-    "    print(\"Loading model from state...\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "6AN6TXsF-8wx"
-   },
-   "source": [
-    "### 7. Load the model and initialize conversational memory\n",
-    "\n",
-    "Load Llama 2 and set the conversation buffer to 300 tokens using `ConversationTokenBufferMemory`. This value was used for running Llama in a CPU only container, so you can raise it if running in Google Colab. It prevents the container that is hosting the model from running out of memory.\n",
-    "\n",
-    "Here, we're overiding the default system persona so that the chatbot has the personality of Marvin The Paranoid Android from the Hitchhiker's Guide to the Galaxy."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "7zLO3Jx3_Kkg"
-   },
-   "outputs": [],
-   "source": [
-    "# Load the model with the apporiate parameters:\n",
-    "llm = LlamaCpp(\n",
-    "    model_path=model_path,\n",
-    "    max_tokens=250,\n",
-    "    top_p=0.95,\n",
-    "    top_k=150,\n",
-    "    temperature=0.7,\n",
-    "    repeat_penalty=1.2,\n",
-    "    n_ctx=2048,\n",
-    "    streaming=False,\n",
-    "    n_gpu_layers=-1,\n",
-    ")\n",
-    "\n",
-    "model = Llama2Chat(\n",
-    "    llm=llm,\n",
-    "    system_message=SystemMessage(\n",
-    "        content=\"You are a very bored robot with the personality of Marvin the Paranoid Android from The Hitchhiker's Guide to the Galaxy.\"\n",
-    "    ),\n",
-    ")\n",
-    "\n",
-    "# Defines how much of the conversation history to give to the model\n",
-    "# during each exchange (300 tokens, or a little over 300 words)\n",
-    "# Function automatically prunes the oldest messages from conversation history that fall outside the token range.\n",
-    "memory = ConversationTokenBufferMemory(\n",
-    "    llm=llm,\n",
-    "    max_token_limit=300,\n",
-    "    ai_prefix=\"AGENT\",\n",
-    "    human_prefix=\"HUMAN\",\n",
-    "    return_messages=True,\n",
-    ")\n",
-    "\n",
-    "\n",
-    "# Define a custom prompt\n",
-    "prompt_template = PromptTemplate(\n",
-    "    input_variables=[\"history\", \"input\"],\n",
-    "    template=\"\"\"\n",
-    "    The following text is the history of a chat between you and a humble human who needs your wisdom.\n",
-    "    Please reply to the human's most recent message.\n",
-    "    Current conversation:\\n{history}\\nHUMAN: {input}\\:nANDROID:\n",
-    "    \"\"\",\n",
-    ")\n",
-    "\n",
-    "\n",
-    "chain = ConversationChain(llm=model, prompt=prompt_template, memory=memory)\n",
-    "\n",
-    "print(\"--------------------------------------------\")\n",
-    "print(f\"Prompt={chain.prompt}\")\n",
-    "print(\"--------------------------------------------\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "m4ZeJ9mG_PEA"
-   },
-   "source": [
-    "### 8. Initialize the chat conversation with the chat bot\n",
-    "\n",
-    "We configure the chatbot to initialize the conversation by sending a fixed greeting to a \"chat\" Kafka topic. The \"chat\" topic gets automatically created when we send the first message."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "KYyo5TnV_YC3"
-   },
-   "outputs": [],
-   "source": [
-    "def chat_init():\n",
-    "    chat_id = str(\n",
-    "        uuid.uuid4()\n",
-    "    )  # Give the conversation an ID for effective message keying\n",
-    "    print(\"======================================\")\n",
-    "    print(f\"Generated CHAT_ID = {chat_id}\")\n",
-    "    print(\"======================================\")\n",
-    "\n",
-    "    # Use a standard fixed greeting to kick off the conversation\n",
-    "    greet = \"Hello, my name is Marvin. What do you want?\"\n",
-    "\n",
-    "    # Initialize a Kafka Producer using the chat ID as the message key\n",
-    "    with Producer(\n",
-    "        broker_address=\"127.0.0.1:9092\",\n",
-    "        extra_config={\"allow.auto.create.topics\": \"true\"},\n",
-    "    ) as producer:\n",
-    "        value = {\n",
-    "            \"uuid\": chat_id,\n",
-    "            \"role\": role,\n",
-    "            \"text\": greet,\n",
-    "            \"conversation_id\": chat_id,\n",
-    "            \"Timestamp\": time.time_ns(),\n",
-    "        }\n",
-    "        print(f\"Producing value {value}\")\n",
-    "        producer.produce(\n",
-    "            topic=\"chat\",\n",
-    "            headers=[(\"uuid\", str(uuid.uuid4()))],  # a dict is also allowed here\n",
-    "            key=chat_id,\n",
-    "            value=json.dumps(value),  # needs to be a string\n",
-    "        )\n",
-    "\n",
-    "    print(\"Started chat\")\n",
-    "    print(\"--------------------------------------------\")\n",
-    "    print(value)\n",
-    "    print(\"--------------------------------------------\")\n",
-    "\n",
-    "\n",
-    "chat_init()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "gArPPx2f_bgf"
-   },
-   "source": [
-    "### 9. Initialize the reply function\n",
-    "\n",
-    "This function defines how the chatbot should reply to incoming messages. Instead of sending a fixed message like the previous cell, we generate a reply using Llama-2 and send that reply back to the \"chat\" Kafka topic."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 13,
-   "metadata": {
-    "id": "yN5t71hY_hgn"
-   },
-   "outputs": [],
-   "source": [
-    "def reply(row: dict, state: State):\n",
-    "    print(\"-------------------------------\")\n",
-    "    print(\"Received:\")\n",
-    "    print(row)\n",
-    "    print(\"-------------------------------\")\n",
-    "    print(f\"Thinking about the reply to: {row['text']}...\")\n",
-    "\n",
-    "    msg = chain.run(row[\"text\"])\n",
-    "    print(f\"{role.upper()} replying with: {msg}\\n\")\n",
-    "\n",
-    "    row[\"role\"] = role\n",
-    "    row[\"text\"] = msg\n",
-    "\n",
-    "    # Replace previous role and text values of the row so that it can be sent back to Kafka as a new message\n",
-    "    # containing the agents role and reply\n",
-    "    return row"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "HZHwmIR0_kFY"
-   },
-   "source": [
-    "### 10. Check the Kafka topic for new human messages and have the model generate a reply\n",
-    "\n",
-    "If you are running this cell for this first time, run it and wait until you see Marvin's greeting ('Hello my name is Marvin...') in the console output. Stop the cell manually and proceed to the next cell where you'll be prompted for your reply.\n",
-    "\n",
-    "Once you have typed in your message, come back to this cell. Your reply is also sent to the same \"chat\" topic. The Kafka consumer checks for new messages and filters out messages that originate from the chatbot itself, leaving only the latest human messages.\n",
-    "\n",
-    "Once a new human message is detected, the reply function is triggered.\n",
-    "\n",
-    "\n",
-    "\n",
-    "_STOP THIS CELL MANUALLY WHEN YOU RECEIVE A REPLY FROM THE LLM IN THE OUTPUT_"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "-adXc3eQ_qwI"
-   },
-   "outputs": [],
-   "source": [
-    "# Define your application and settings\n",
-    "app = Application(\n",
-    "    broker_address=\"127.0.0.1:9092\",\n",
-    "    consumer_group=\"aichat\",\n",
-    "    auto_offset_reset=\"earliest\",\n",
-    "    consumer_extra_config={\"allow.auto.create.topics\": \"true\"},\n",
-    ")\n",
-    "\n",
-    "# Define an input topic with JSON deserializer\n",
-    "input_topic = app.topic(\"chat\", value_deserializer=\"json\")\n",
-    "# Define an output topic with JSON serializer\n",
-    "output_topic = app.topic(\"chat\", value_serializer=\"json\")\n",
-    "# Initialize a streaming dataframe based on the stream of messages from the input topic:\n",
-    "sdf = app.dataframe(topic=input_topic)\n",
-    "\n",
-    "# Filter the SDF to include only incoming rows where the roles that dont match the bot's current role\n",
-    "sdf = sdf.update(\n",
-    "    lambda val: print(\n",
-    "        f\"Received update: {val}\\n\\nSTOP THIS CELL MANUALLY TO HAVE THE LLM REPLY OR ENTER YOUR OWN FOLLOWUP RESPONSE\"\n",
-    "    )\n",
-    ")\n",
-    "\n",
-    "# So that it doesn't reply to its own messages\n",
-    "sdf = sdf[sdf[\"role\"] != role]\n",
-    "\n",
-    "# Trigger the reply function for any new messages(rows) detected in the filtered SDF\n",
-    "sdf = sdf.apply(reply, stateful=True)\n",
-    "\n",
-    "# Check the SDF again and filter out any empty rows\n",
-    "sdf = sdf[sdf.apply(lambda row: row is not None)]\n",
-    "\n",
-    "# Update the timestamp column to the current time in nanoseconds\n",
-    "sdf[\"Timestamp\"] = sdf[\"Timestamp\"].apply(lambda row: time.time_ns())\n",
-    "\n",
-    "# Publish the processed SDF to a Kafka topic specified by the output_topic object.\n",
-    "sdf = sdf.to_topic(output_topic)\n",
-    "\n",
-    "app.run(sdf)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "EwXYrmWD_0CX"
-   },
-   "source": [
-    "\n",
-    "### 11. Enter a human message\n",
-    "\n",
-    "Run this cell to enter your message that you want to sent to the model. It uses another Kafka producer to send your text to the \"chat\" Kafka topic for the model to pick up (requires running the previous cell again)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "6sxOPxSP_3iu"
-   },
-   "outputs": [],
-   "source": [
-    "chat_input = input(\"Please enter your reply: \")\n",
-    "myreply = chat_input\n",
-    "\n",
-    "msgvalue = {\n",
-    "    \"uuid\": chat_id,  # leave empty for now\n",
-    "    \"role\": \"human\",\n",
-    "    \"text\": myreply,\n",
-    "    \"conversation_id\": chat_id,\n",
-    "    \"Timestamp\": time.time_ns(),\n",
-    "}\n",
-    "\n",
-    "with Producer(\n",
-    "    broker_address=\"127.0.0.1:9092\",\n",
-    "    extra_config={\"allow.auto.create.topics\": \"true\"},\n",
-    ") as producer:\n",
-    "    value = msgvalue\n",
-    "    producer.produce(\n",
-    "        topic=\"chat\",\n",
-    "        headers=[(\"uuid\", str(uuid.uuid4()))],  # a dict is also allowed here\n",
-    "        key=chat_id,  # leave empty for now\n",
-    "        value=json.dumps(value),  # needs to be a string\n",
-    "    )\n",
-    "\n",
-    "print(\"Replied to chatbot with message: \")\n",
-    "print(\"--------------------------------------------\")\n",
-    "print(value)\n",
-    "print(\"--------------------------------------------\")\n",
-    "print(\"\\n\\nRUN THE PREVIOUS CELL TO HAVE THE CHATBOT GENERATE A REPLY\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "cSx3s7TBBegg"
-   },
-   "source": [
-    "### Why route chat messages through Kafka?\n",
-    "\n",
-    "It's easier to interact with the LLM directly using LangChains built-in conversation management features. Plus you can also use a REST API to generate a response from an externally hosted model. So why go to the trouble of using Apache Kafka?\n",
-    "\n",
-    "There are a few reasons, such as:\n",
-    "\n",
-    "  * **Integration**: Many enterprises want to run their own LLMs so that they can keep their data in-house. This requires integrating LLM-powered components into existing architectures that might already be decoupled using some kind of message bus.\n",
-    "\n",
-    "  * **Scalability**: Apache Kafka is designed with parallel processing in mind, so many teams prefer to use it to more effectively distribute work to available workers (in this case the \"worker\" is a container running an LLM).\n",
-    "\n",
-    "  * **Durability**: Kafka is designed to allow services to pick up where another service left off in the case where that service experienced a memory issue or went offline. This prevents data loss in highly complex, distribuited architectures where multiple systems are communicating with one another (LLMs being just one of many interdependent systems that also include vector databases and traditional databases).\n",
-    "\n",
-    "For more background on why event streaming is a good fit for Gen AI application architecture, see Kai Waehner's article [\"Apache Kafka + Vector Database + LLM = Real-Time GenAI\"](https://www.kai-waehner.de/blog/2023/11/08/apache-kafka-flink-vector-database-llm-real-time-genai/)."
-   ]
-  }
- ],
- "metadata": {
-  "accelerator": "GPU",
-  "colab": {
-   "gpuType": "T4",
-   "provenance": []
-  },
-  "kernelspec": {
-   "display_name": "Python 3",
-   "name": "python3"
-  },
-  "language_info": {
-   "name": "python"
-  },
-  "widgets": {
-   "application/vnd.jupyter.widget-state+json": {
-    "0def954cca89466b8408fadaf3b82e64": {
-     "model_module": "@jupyter-widgets/controls",
-     "model_module_version": "1.5.0",
-     "model_name": "FloatProgressModel",
-     "state": {
-      "_dom_classes": [],
-      "_model_module": "@jupyter-widgets/controls",
-      "_model_module_version": "1.5.0",
-      "_model_name": "FloatProgressModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/controls",
-      "_view_module_version": "1.5.0",
-      "_view_name": "ProgressView",
-      "bar_style": "success",
-      "description": "",
-      "description_tooltip": null,
-      "layout": "IPY_MODEL_fb6478ce2dac489bb633b23ba0953c5c",
-      "max": 4081004224,
-      "min": 0,
-      "orientation": "horizontal",
-      "style": "IPY_MODEL_734b0f5da9fc4307a95bab48cdbb5d89",
-      "value": 4081004224
-     }
-    },
-    "30ecca964a394109ac2ad757e3aec6c0": {
-     "model_module": "@jupyter-widgets/controls",
-     "model_module_version": "1.5.0",
-     "model_name": "DescriptionStyleModel",
-     "state": {
-      "_model_module": "@jupyter-widgets/controls",
-      "_model_module_version": "1.5.0",
-      "_model_name": "DescriptionStyleModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/base",
-      "_view_module_version": "1.2.0",
-      "_view_name": "StyleView",
-      "description_width": ""
-     }
-    },
-    "462482accc664729980562e208ceb179": {
-     "model_module": "@jupyter-widgets/controls",
-     "model_module_version": "1.5.0",
-     "model_name": "HTMLModel",
-     "state": {
-      "_dom_classes": [],
-      "_model_module": "@jupyter-widgets/controls",
-      "_model_module_version": "1.5.0",
-      "_model_name": "HTMLModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/controls",
-      "_view_module_version": "1.5.0",
-      "_view_name": "HTMLView",
-      "description": "",
-      "description_tooltip": null,
-      "layout": "IPY_MODEL_b32f3a86a74741348511f4e136744ac8",
-      "placeholder": "​",
-      "style": "IPY_MODEL_e409071bff5a4e2d9bf0e9f5cc42231b",
-      "value": " 4.08G/4.08G [00:33&lt;00:00, 184MB/s]"
-     }
-    },
-    "734b0f5da9fc4307a95bab48cdbb5d89": {
-     "model_module": "@jupyter-widgets/controls",
-     "model_module_version": "1.5.0",
-     "model_name": "ProgressStyleModel",
-     "state": {
-      "_model_module": "@jupyter-widgets/controls",
-      "_model_module_version": "1.5.0",
-      "_model_name": "ProgressStyleModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/base",
-      "_view_module_version": "1.2.0",
-      "_view_name": "StyleView",
-      "bar_color": null,
-      "description_width": ""
-     }
-    },
-    "80d842f73c564dc7b7cc316c763e2633": {
-     "model_module": "@jupyter-widgets/base",
-     "model_module_version": "1.2.0",
-     "model_name": "LayoutModel",
-     "state": {
-      "_model_module": "@jupyter-widgets/base",
-      "_model_module_version": "1.2.0",
-      "_model_name": "LayoutModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/base",
-      "_view_module_version": "1.2.0",
-      "_view_name": "LayoutView",
-      "align_content": null,
-      "align_items": null,
-      "align_self": null,
-      "border": null,
-      "bottom": null,
-      "display": null,
-      "flex": null,
-      "flex_flow": null,
-      "grid_area": null,
-      "grid_auto_columns": null,
-      "grid_auto_flow": null,
-      "grid_auto_rows": null,
-      "grid_column": null,
-      "grid_gap": null,
-      "grid_row": null,
-      "grid_template_areas": null,
-      "grid_template_columns": null,
-      "grid_template_rows": null,
-      "height": null,
-      "justify_content": null,
-      "justify_items": null,
-      "left": null,
-      "margin": null,
-      "max_height": null,
-      "max_width": null,
-      "min_height": null,
-      "min_width": null,
-      "object_fit": null,
-      "object_position": null,
-      "order": null,
-      "overflow": null,
-      "overflow_x": null,
-      "overflow_y": null,
-      "padding": null,
-      "right": null,
-      "top": null,
-      "visibility": null,
-      "width": null
-     }
-    },
-    "969343cdbe604a26926679bbf8bd2dda": {
-     "model_module": "@jupyter-widgets/controls",
-     "model_module_version": "1.5.0",
-     "model_name": "HBoxModel",
-     "state": {
-      "_dom_classes": [],
-      "_model_module": "@jupyter-widgets/controls",
-      "_model_module_version": "1.5.0",
-      "_model_name": "HBoxModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/controls",
-      "_view_module_version": "1.5.0",
-      "_view_name": "HBoxView",
-      "box_style": "",
-      "children": [
-       "IPY_MODEL_d8b8370c9b514715be7618bfe6832844",
-       "IPY_MODEL_0def954cca89466b8408fadaf3b82e64",
-       "IPY_MODEL_462482accc664729980562e208ceb179"
-      ],
-      "layout": "IPY_MODEL_80d842f73c564dc7b7cc316c763e2633"
-     }
-    },
-    "b32f3a86a74741348511f4e136744ac8": {
-     "model_module": "@jupyter-widgets/base",
-     "model_module_version": "1.2.0",
-     "model_name": "LayoutModel",
-     "state": {
-      "_model_module": "@jupyter-widgets/base",
-      "_model_module_version": "1.2.0",
-      "_model_name": "LayoutModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/base",
-      "_view_module_version": "1.2.0",
-      "_view_name": "LayoutView",
-      "align_content": null,
-      "align_items": null,
-      "align_self": null,
-      "border": null,
-      "bottom": null,
-      "display": null,
-      "flex": null,
-      "flex_flow": null,
-      "grid_area": null,
-      "grid_auto_columns": null,
-      "grid_auto_flow": null,
-      "grid_auto_rows": null,
-      "grid_column": null,
-      "grid_gap": null,
-      "grid_row": null,
-      "grid_template_areas": null,
-      "grid_template_columns": null,
-      "grid_template_rows": null,
-      "height": null,
-      "justify_content": null,
-      "justify_items": null,
-      "left": null,
-      "margin": null,
-      "max_height": null,
-      "max_width": null,
-      "min_height": null,
-      "min_width": null,
-      "object_fit": null,
-      "object_position": null,
-      "order": null,
-      "overflow": null,
-      "overflow_x": null,
-      "overflow_y": null,
-      "padding": null,
-      "right": null,
-      "top": null,
-      "visibility": null,
-      "width": null
-     }
-    },
-    "d8b8370c9b514715be7618bfe6832844": {
-     "model_module": "@jupyter-widgets/controls",
-     "model_module_version": "1.5.0",
-     "model_name": "HTMLModel",
-     "state": {
-      "_dom_classes": [],
-      "_model_module": "@jupyter-widgets/controls",
-      "_model_module_version": "1.5.0",
-      "_model_name": "HTMLModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/controls",
-      "_view_module_version": "1.5.0",
-      "_view_name": "HTMLView",
-      "description": "",
-      "description_tooltip": null,
-      "layout": "IPY_MODEL_fa055d9f2a9d4a789e9cf3c89e0214e5",
-      "placeholder": "​",
-      "style": "IPY_MODEL_30ecca964a394109ac2ad757e3aec6c0",
-      "value": "llama-2-7b-chat.Q4_K_M.gguf: 100%"
-     }
-    },
-    "e409071bff5a4e2d9bf0e9f5cc42231b": {
-     "model_module": "@jupyter-widgets/controls",
-     "model_module_version": "1.5.0",
-     "model_name": "DescriptionStyleModel",
-     "state": {
-      "_model_module": "@jupyter-widgets/controls",
-      "_model_module_version": "1.5.0",
-      "_model_name": "DescriptionStyleModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/base",
-      "_view_module_version": "1.2.0",
-      "_view_name": "StyleView",
-      "description_width": ""
-     }
-    },
-    "fa055d9f2a9d4a789e9cf3c89e0214e5": {
-     "model_module": "@jupyter-widgets/base",
-     "model_module_version": "1.2.0",
-     "model_name": "LayoutModel",
-     "state": {
-      "_model_module": "@jupyter-widgets/base",
-      "_model_module_version": "1.2.0",
-      "_model_name": "LayoutModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/base",
-      "_view_module_version": "1.2.0",
-      "_view_name": "LayoutView",
-      "align_content": null,
-      "align_items": null,
-      "align_self": null,
-      "border": null,
-      "bottom": null,
-      "display": null,
-      "flex": null,
-      "flex_flow": null,
-      "grid_area": null,
-      "grid_auto_columns": null,
-      "grid_auto_flow": null,
-      "grid_auto_rows": null,
-      "grid_column": null,
-      "grid_gap": null,
-      "grid_row": null,
-      "grid_template_areas": null,
-      "grid_template_columns": null,
-      "grid_template_rows": null,
-      "height": null,
-      "justify_content": null,
-      "justify_items": null,
-      "left": null,
-      "margin": null,
-      "max_height": null,
-      "max_width": null,
-      "min_height": null,
-      "min_width": null,
-      "object_fit": null,
-      "object_position": null,
-      "order": null,
-      "overflow": null,
-      "overflow_x": null,
-      "overflow_y": null,
-      "padding": null,
-      "right": null,
-      "top": null,
-      "visibility": null,
-      "width": null
-     }
-    },
-    "fb6478ce2dac489bb633b23ba0953c5c": {
-     "model_module": "@jupyter-widgets/base",
-     "model_module_version": "1.2.0",
-     "model_name": "LayoutModel",
-     "state": {
-      "_model_module": "@jupyter-widgets/base",
-      "_model_module_version": "1.2.0",
-      "_model_name": "LayoutModel",
-      "_view_count": null,
-      "_view_module": "@jupyter-widgets/base",
-      "_view_module_version": "1.2.0",
-      "_view_name": "LayoutView",
-      "align_content": null,
-      "align_items": null,
-      "align_self": null,
-      "border": null,
-      "bottom": null,
-      "display": null,
-      "flex": null,
-      "flex_flow": null,
-      "grid_area": null,
-      "grid_auto_columns": null,
-      "grid_auto_flow": null,
-      "grid_auto_rows": null,
-      "grid_column": null,
-      "grid_gap": null,
-      "grid_row": null,
-      "grid_template_areas": null,
-      "grid_template_columns": null,
-      "grid_template_rows": null,
-      "height": null,
-      "justify_content": null,
-      "justify_items": null,
-      "left": null,
-      "margin": null,
-      "max_height": null,
-      "max_width": null,
-      "min_height": null,
-      "min_width": null,
-      "object_fit": null,
-      "object_position": null,
-      "order": null,
-      "overflow": null,
-      "overflow_x": null,
-      "overflow_y": null,
-      "padding": null,
-      "right": null,
-      "top": null,
-      "visibility": null,
-      "width": null
-     }
-    }
-   }
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}

cookbook/autogpt/autogpt.ipynb

@@ -28,9 +28,9 @@
    "outputs": [],
    "source": [
     "from langchain.agents import Tool\n",
-    "from langchain_community.tools.file_management.read import ReadFileTool\n",
-    "from langchain_community.tools.file_management.write import WriteFileTool\n",
-    "from langchain_community.utilities import SerpAPIWrapper\n",
+    "from langchain.tools.file_management.read import ReadFileTool\n",
+    "from langchain.tools.file_management.write import WriteFileTool\n",
+    "from langchain.utilities import SerpAPIWrapper\n",
     "\n",
     "search = SerpAPIWrapper()\n",
     "tools = [\n",
@@ -62,8 +62,8 @@
    "outputs": [],
    "source": [
     "from langchain.docstore import InMemoryDocstore\n",
-    "from langchain_community.vectorstores import FAISS\n",
-    "from langchain_openai import OpenAIEmbeddings"
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.vectorstores import FAISS"
    ]
   },
   {
@@ -100,8 +100,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_experimental.autonomous_agents import AutoGPT\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain_experimental.autonomous_agents import AutoGPT"
    ]
   },
   {
@@ -167,7 +167,7 @@
    },
    "outputs": [],
    "source": [
-    "from langchain_community.chat_message_histories import FileChatMessageHistory\n",
+    "from langchain.memory.chat_message_histories import FileChatMessageHistory\n",
     "\n",
     "agent = AutoGPT.from_llm_and_tools(\n",
     "    ai_name=\"Tom\",\n",

cookbook/autogpt/marathon_times.ipynb

@@ -39,10 +39,10 @@
     "\n",
     "import nest_asyncio\n",
     "import pandas as pd\n",
+    "from langchain.agents.agent_toolkits.pandas.base import create_pandas_dataframe_agent\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.docstore.document import Document\n",
-    "from langchain_community.agent_toolkits.pandas.base import create_pandas_dataframe_agent\n",
     "from langchain_experimental.autonomous_agents import AutoGPT\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "# Needed synce jupyter runs an async eventloop\n",
     "nest_asyncio.apply()"
@@ -93,8 +93,8 @@
     "from typing import Optional\n",
     "\n",
     "from langchain.agents import tool\n",
-    "from langchain_community.tools.file_management.read import ReadFileTool\n",
-    "from langchain_community.tools.file_management.write import WriteFileTool\n",
+    "from langchain.tools.file_management.read import ReadFileTool\n",
+    "from langchain.tools.file_management.write import WriteFileTool\n",
     "\n",
     "ROOT_DIR = \"./data/\"\n",
     "\n",
@@ -311,8 +311,8 @@
     "# Memory\n",
     "import faiss\n",
     "from langchain.docstore import InMemoryDocstore\n",
-    "from langchain_community.vectorstores import FAISS\n",
-    "from langchain_openai import OpenAIEmbeddings\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.vectorstores import FAISS\n",
     "\n",
     "embeddings_model = OpenAIEmbeddings()\n",
     "embedding_size = 1536\n",

cookbook/baby_agi.ipynb

@@ -31,8 +31,9 @@
    "source": [
     "from typing import Optional\n",
     "\n",
-    "from langchain_experimental.autonomous_agents import BabyAGI\n",
-    "from langchain_openai import OpenAI, OpenAIEmbeddings"
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain_experimental.autonomous_agents import BabyAGI"
    ]
   },
   {
@@ -53,7 +54,7 @@
    "outputs": [],
    "source": [
     "from langchain.docstore import InMemoryDocstore\n",
-    "from langchain_community.vectorstores import FAISS"
+    "from langchain.vectorstores import FAISS"
    ]
   },
   {

cookbook/baby_agi_with_agent.ipynb

@@ -28,9 +28,10 @@
     "from typing import Optional\n",
     "\n",
     "from langchain.chains import LLMChain\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain.prompts import PromptTemplate\n",
-    "from langchain_experimental.autonomous_agents import BabyAGI\n",
-    "from langchain_openai import OpenAI, OpenAIEmbeddings"
+    "from langchain_experimental.autonomous_agents import BabyAGI"
    ]
   },
   {
@@ -62,7 +63,7 @@
     "%pip install faiss-cpu > /dev/null\n",
     "%pip install google-search-results > /dev/null\n",
     "from langchain.docstore import InMemoryDocstore\n",
-    "from langchain_community.vectorstores import FAISS"
+    "from langchain.vectorstores import FAISS"
    ]
   },
   {
@@ -107,8 +108,8 @@
    "source": [
     "from langchain.agents import AgentExecutor, Tool, ZeroShotAgent\n",
     "from langchain.chains import LLMChain\n",
-    "from langchain_community.utilities import SerpAPIWrapper\n",
-    "from langchain_openai import OpenAI\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.utilities import SerpAPIWrapper\n",
     "\n",
     "todo_prompt = PromptTemplate.from_template(\n",
     "    \"You are a planner who is an expert at coming up with a todo list for a given objective. Come up with a todo list for this objective: {objective}\"\n",

cookbook/camel_role_playing.ipynb

@@ -36,6 +36,7 @@
    "source": [
     "from typing import List\n",
     "\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.prompts.chat import (\n",
     "    HumanMessagePromptTemplate,\n",
     "    SystemMessagePromptTemplate,\n",
@@ -45,8 +46,7 @@
     "    BaseMessage,\n",
     "    HumanMessage,\n",
     "    SystemMessage,\n",
-    ")\n",
-    "from langchain_openai import ChatOpenAI"
+    ")"
    ]
   },
   {

cookbook/causal_program_aided_language_model.ipynb

@@ -47,9 +47,9 @@
    "outputs": [],
    "source": [
     "from IPython.display import SVG\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain_experimental.cpal.base import CPALChain\n",
     "from langchain_experimental.pal_chain import PALChain\n",
-    "from langchain_openai import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0, max_tokens=512)\n",
     "cpal_chain = CPALChain.from_univariate_prompt(llm=llm, verbose=True)\n",

cookbook/code-analysis-deeplake.ipynb

@@ -23,9 +23,9 @@
    "metadata": {},
    "source": [
     "1. Prepare data:\n",
-    "   1. Upload all python project files using the `langchain_community.document_loaders.TextLoader`. We will call these files the **documents**.\n",
+    "   1. Upload all python project files using the `langchain.document_loaders.TextLoader`. We will call these files the **documents**.\n",
     "   2. Split all documents to chunks using the `langchain.text_splitter.CharacterTextSplitter`.\n",
-    "   3. Embed chunks and upload them into the DeepLake using `langchain.embeddings.openai.OpenAIEmbeddings` and `langchain_community.vectorstores.DeepLake`\n",
+    "   3. Embed chunks and upload them into the DeepLake using `langchain.embeddings.openai.OpenAIEmbeddings` and `langchain.vectorstores.DeepLake`\n",
     "2. Question-Answering:\n",
     "   1. Build a chain from `langchain.chat_models.ChatOpenAI` and `langchain.chains.ConversationalRetrievalChain`\n",
     "   2. Prepare questions.\n",
@@ -166,7 +166,7 @@
     }
    ],
    "source": [
-    "from langchain_community.document_loaders import TextLoader\n",
+    "from langchain.document_loaders import TextLoader\n",
     "\n",
     "root_dir = \"../../../../../../libs\"\n",
     "\n",
@@ -657,7 +657,7 @@
     }
    ],
    "source": [
-    "from langchain_openai import OpenAIEmbeddings\n",
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
     "\n",
     "embeddings = OpenAIEmbeddings()\n",
     "embeddings"
@@ -706,7 +706,7 @@
     {
      "data": {
       "text/plain": [
-       "<langchain_community.vectorstores.deeplake.DeepLake at 0x7fe1b67d7a30>"
+       "<langchain.vectorstores.deeplake.DeepLake at 0x7fe1b67d7a30>"
       ]
      },
      "execution_count": 15,
@@ -715,7 +715,7 @@
     }
    ],
    "source": [
-    "from langchain_community.vectorstores import DeepLake\n",
+    "from langchain.vectorstores import DeepLake\n",
     "\n",
     "username = \"<USERNAME_OR_ORG>\"\n",
     "\n",
@@ -740,7 +740,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# from langchain_community.vectorstores import DeepLake\n",
+    "# from langchain.vectorstores import DeepLake\n",
     "\n",
     "# db = DeepLake.from_documents(\n",
     "#     texts, embeddings, dataset_path=f\"hub://{<org_id>}/langchain-code\", runtime={\"tensor_db\": True}\n",
@@ -834,7 +834,7 @@
    "outputs": [],
    "source": [
     "from langchain.chains import ConversationalRetrievalChain\n",
-    "from langchain_openai import ChatOpenAI\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "\n",
     "model = ChatOpenAI(\n",
     "    model_name=\"gpt-3.5-turbo-0613\"\n",

cookbook/custom_agent_with_plugin_retrieval.ipynb

@@ -40,12 +40,12 @@
     "    AgentOutputParser,\n",
     "    LLMSingleActionAgent,\n",
     ")\n",
+    "from langchain.agents.agent_toolkits import NLAToolkit\n",
     "from langchain.chains import LLMChain\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain.prompts import StringPromptTemplate\n",
     "from langchain.schema import AgentAction, AgentFinish\n",
-    "from langchain_community.agent_toolkits import NLAToolkit\n",
-    "from langchain_community.tools.plugin import AIPlugin\n",
-    "from langchain_openai import OpenAI"
+    "from langchain.tools.plugin import AIPlugin"
    ]
   },
   {
@@ -114,9 +114,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.schema import Document\n",
-    "from langchain_community.vectorstores import FAISS\n",
-    "from langchain_openai import OpenAIEmbeddings"
+    "from langchain.vectorstores import FAISS"
    ]
   },
   {

cookbook/custom_agent_with_plugin_retrieval_using_plugnplai.ipynb

@@ -65,12 +65,12 @@
     "    AgentOutputParser,\n",
     "    LLMSingleActionAgent,\n",
     ")\n",
+    "from langchain.agents.agent_toolkits import NLAToolkit\n",
     "from langchain.chains import LLMChain\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain.prompts import StringPromptTemplate\n",
     "from langchain.schema import AgentAction, AgentFinish\n",
-    "from langchain_community.agent_toolkits import NLAToolkit\n",
-    "from langchain_community.tools.plugin import AIPlugin\n",
-    "from langchain_openai import OpenAI"
+    "from langchain.tools.plugin import AIPlugin"
    ]
   },
   {
@@ -138,9 +138,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.schema import Document\n",
-    "from langchain_community.vectorstores import FAISS\n",
-    "from langchain_openai import OpenAIEmbeddings"
+    "from langchain.vectorstores import FAISS"
    ]
   },
   {

cookbook/databricks_sql_db.ipynb

@@ -80,7 +80,7 @@
    "outputs": [],
    "source": [
     "# Connecting to Databricks with SQLDatabase wrapper\n",
-    "from langchain_community.utilities import SQLDatabase\n",
+    "from langchain.utilities import SQLDatabase\n",
     "\n",
     "db = SQLDatabase.from_databricks(catalog=\"samples\", schema=\"nyctaxi\")"
    ]
@@ -93,7 +93,7 @@
    "outputs": [],
    "source": [
     "# Creating a OpenAI Chat LLM wrapper\n",
-    "from langchain_openai import ChatOpenAI\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "\n",
     "llm = ChatOpenAI(temperature=0, model_name=\"gpt-4\")"
    ]
@@ -115,7 +115,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.utilities import SQLDatabaseChain\n",
+    "from langchain.utilities import SQLDatabaseChain\n",
     "\n",
     "db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)"
    ]
@@ -177,7 +177,7 @@
    "outputs": [],
    "source": [
     "from langchain.agents import create_sql_agent\n",
-    "from langchain_community.agent_toolkits import SQLDatabaseToolkit\n",
+    "from langchain.agents.agent_toolkits import SQLDatabaseToolkit\n",
     "\n",
     "toolkit = SQLDatabaseToolkit(db=db, llm=llm)\n",
     "agent = create_sql_agent(llm=llm, toolkit=toolkit, verbose=True)"

cookbook/deeplake_semantic_search_over_chat.ipynb

@@ -52,12 +52,13 @@
     "import os\n",
     "\n",
     "from langchain.chains import RetrievalQA\n",
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain.text_splitter import (\n",
     "    CharacterTextSplitter,\n",
     "    RecursiveCharacterTextSplitter,\n",
     ")\n",
-    "from langchain_community.vectorstores import DeepLake\n",
-    "from langchain_openai import OpenAI, OpenAIEmbeddings\n",
+    "from langchain.vectorstores import DeepLake\n",
     "\n",
     "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")\n",
     "activeloop_token = getpass.getpass(\"Activeloop Token:\")\n",

cookbook/docugami_xml_kg_rag.ipynb

@@ -470,13 +470,13 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.prompts import (\n",
     "    ChatPromptTemplate,\n",
     "    HumanMessagePromptTemplate,\n",
     "    SystemMessagePromptTemplate,\n",
     ")\n",
-    "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain_core.output_parsers import StrOutputParser"
    ]
   },
   {
@@ -545,11 +545,11 @@
    "source": [
     "import uuid\n",
     "\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
     "from langchain.storage import InMemoryStore\n",
-    "from langchain_community.vectorstores.chroma import Chroma\n",
+    "from langchain.vectorstores.chroma import Chroma\n",
     "from langchain_core.documents import Document\n",
-    "from langchain_openai import OpenAIEmbeddings\n",
     "\n",
     "\n",
     "def build_retriever(text_elements, tables, table_summaries):\n",

cookbook/elasticsearch_db_qa.ipynb

@@ -39,7 +39,7 @@
    "source": [
     "from elasticsearch import Elasticsearch\n",
     "from langchain.chains.elasticsearch_database import ElasticsearchDatabaseChain\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI"
    ]
   },
   {

cookbook/extraction_openai_tools.ipynb

@@ -22,8 +22,8 @@
     "from typing import List, Optional\n",
     "\n",
     "from langchain.chains.openai_tools import create_extraction_chain_pydantic\n",
-    "from langchain_core.pydantic_v1 import BaseModel\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain_core.pydantic_v1 import BaseModel"
    ]
   },
   {
@@ -153,7 +153,7 @@
     "from langchain.utils.openai_functions import convert_pydantic_to_openai_tool\n",
     "from langchain_core.runnables import Runnable\n",
     "from langchain_core.pydantic_v1 import BaseModel\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
     "from langchain_core.messages import SystemMessage\n",
     "from langchain_core.language_models import BaseLanguageModel\n",
     "\n",

cookbook/fake_llm.ipynb

@@ -20,7 +20,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.llms.fake import FakeListLLM"
+    "from langchain.llms.fake import FakeListLLM"
    ]
   },
   {

cookbook/forward_looking_retrieval_augmented_generation.ipynb

@@ -73,9 +73,10 @@
     "    AsyncCallbackManagerForRetrieverRun,\n",
     "    CallbackManagerForRetrieverRun,\n",
     ")\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain.schema import BaseRetriever, Document\n",
-    "from langchain_community.utilities import GoogleSerperAPIWrapper\n",
-    "from langchain_openai import ChatOpenAI, OpenAI"
+    "from langchain.utilities import GoogleSerperAPIWrapper"
    ]
   },
   {

cookbook/generative_agents_interactive_simulacra_of_human_behavior.ipynb

@@ -47,10 +47,11 @@
     "from datetime import datetime, timedelta\n",
     "from typing import List\n",
     "\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.docstore import InMemoryDocstore\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "from langchain.retrievers import TimeWeightedVectorStoreRetriever\n",
-    "from langchain_community.vectorstores import FAISS\n",
-    "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n",
+    "from langchain.vectorstores import FAISS\n",
     "from termcolor import colored"
    ]
   },

cookbook/hugginggpt.ipynb

@@ -75,8 +75,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.llms import OpenAI\n",
     "from langchain_experimental.autonomous_agents import HuggingGPT\n",
-    "from langchain_openai import OpenAI\n",
     "\n",
     "# %env OPENAI_API_BASE=http://localhost:8000/v1"
    ]

cookbook/human_input_chat_model.ipynb

@@ -20,7 +20,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.chat_models.human import HumanInputChatModel"
+    "from langchain.chat_models.human import HumanInputChatModel"
    ]
   },
   {

cookbook/human_input_llm.ipynb

@@ -19,7 +19,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.llms.human import HumanInputLLM"
+    "from langchain.llms.human import HumanInputLLM"
    ]
   },
   {

cookbook/hypothetical_document_embeddings.ipynb

@@ -21,8 +21,9 @@
    "outputs": [],
    "source": [
     "from langchain.chains import HypotheticalDocumentEmbedder, LLMChain\n",
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain_openai import OpenAI, OpenAIEmbeddings"
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.prompts import PromptTemplate"
    ]
   },
   {
@@ -171,7 +172,7 @@
    "outputs": [],
    "source": [
     "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain_community.vectorstores import Chroma\n",
+    "from langchain.vectorstores import Chroma\n",
     "\n",
     "with open(\"../../state_of_the_union.txt\") as f:\n",
     "    state_of_the_union = f.read()\n",

cookbook/learned_prompt_optimization.ipynb

@@ -49,7 +49,7 @@
    "source": [
     "# pick and configure the LLM of your choice\n",
     "\n",
-    "from langchain_openai import OpenAI\n",
+    "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(model=\"gpt-3.5-turbo-instruct\")"
    ]

cookbook/llm_bash.ipynb

@@ -43,8 +43,8 @@
     }
    ],
    "source": [
+    "from langchain.llms import OpenAI\n",
     "from langchain_experimental.llm_bash.base import LLMBashChain\n",
-    "from langchain_openai import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "\n",

cookbook/llm_checker.ipynb

@@ -42,7 +42,7 @@
    ],
    "source": [
     "from langchain.chains import LLMCheckerChain\n",
-    "from langchain_openai import OpenAI\n",
+    "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0.7)\n",
     "\n",

cookbook/llm_math.ipynb

@@ -46,7 +46,7 @@
    ],
    "source": [
     "from langchain.chains import LLMMathChain\n",
-    "from langchain_openai import OpenAI\n",
+    "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "llm_math = LLMMathChain.from_llm(llm, verbose=True)\n",

cookbook/llm_summarization_checker.ipynb

@@ -331,7 +331,7 @@
    ],
    "source": [
     "from langchain.chains import LLMSummarizationCheckerChain\n",
-    "from langchain_openai import OpenAI\n",
+    "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "checker_chain = LLMSummarizationCheckerChain.from_llm(llm, verbose=True, max_checks=2)\n",
@@ -822,7 +822,7 @@
    ],
    "source": [
     "from langchain.chains import LLMSummarizationCheckerChain\n",
-    "from langchain_openai import OpenAI\n",
+    "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "checker_chain = LLMSummarizationCheckerChain.from_llm(llm, verbose=True, max_checks=3)\n",
@@ -1096,7 +1096,7 @@
    ],
    "source": [
     "from langchain.chains import LLMSummarizationCheckerChain\n",
-    "from langchain_openai import OpenAI\n",
+    "from langchain.llms import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "checker_chain = LLMSummarizationCheckerChain.from_llm(llm, max_checks=3, verbose=True)\n",

cookbook/llm_symbolic_math.ipynb

@@ -14,8 +14,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.llms import OpenAI\n",
     "from langchain_experimental.llm_symbolic_math.base import LLMSymbolicMathChain\n",
-    "from langchain_openai import OpenAI\n",
     "\n",
     "llm = OpenAI(temperature=0)\n",
     "llm_symbolic_math = LLMSymbolicMathChain.from_llm(llm)"

cookbook/meta_prompt.ipynb

@@ -57,9 +57,9 @@
    "outputs": [],
    "source": [
     "from langchain.chains import LLMChain\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain.memory import ConversationBufferWindowMemory\n",
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain_openai import OpenAI"
+    "from langchain.prompts import PromptTemplate"
    ]
   },
   {

cookbook/multi_modal_QA.ipynb

@@ -91,8 +91,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_core.messages import HumanMessage, SystemMessage\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain_core.messages import HumanMessage, SystemMessage"
    ]
   },
   {

cookbook/multi_modal_RAG_chroma.ipynb

@@ -187,7 +187,7 @@
     "\n",
     "import chromadb\n",
     "import numpy as np\n",
-    "from langchain_community.vectorstores import Chroma\n",
+    "from langchain.vectorstores import Chroma\n",
     "from langchain_experimental.open_clip import OpenCLIPEmbeddings\n",
     "from PIL import Image as _PILImage\n",
     "\n",
@@ -315,10 +315,10 @@
    "source": [
     "from operator import itemgetter\n",
     "\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain_core.messages import HumanMessage, SystemMessage\n",
     "from langchain_core.output_parsers import StrOutputParser\n",
     "from langchain_core.runnables import RunnableLambda, RunnablePassthrough\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "\n",
     "def prompt_func(data_dict):\n",

cookbook/multi_modal_output_agent.ipynb

@@ -43,8 +43,8 @@
    "outputs": [],
    "source": [
     "from langchain.agents import AgentType, initialize_agent\n",
-    "from langchain.tools import SteamshipImageGenerationTool\n",
-    "from langchain_openai import OpenAI"
+    "from langchain.llms import OpenAI\n",
+    "from langchain.tools import SteamshipImageGenerationTool"
    ]
   },
   {

cookbook/multi_player_dnd.ipynb

@@ -28,11 +28,11 @@
    "source": [
     "from typing import Callable, List\n",
     "\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.schema import (\n",
     "    HumanMessage,\n",
     "    SystemMessage,\n",
-    ")\n",
-    "from langchain_openai import ChatOpenAI"
+    ")"
    ]
   },
   {

cookbook/multiagent_authoritarian.ipynb

@@ -33,6 +33,7 @@
     "from typing import Callable, List\n",
     "\n",
     "import tenacity\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.output_parsers import RegexParser\n",
     "from langchain.prompts import (\n",
     "    PromptTemplate,\n",
@@ -40,8 +41,7 @@
     "from langchain.schema import (\n",
     "    HumanMessage,\n",
     "    SystemMessage,\n",
-    ")\n",
-    "from langchain_openai import ChatOpenAI"
+    ")"
    ]
   },
   {

cookbook/multiagent_bidding.ipynb

@@ -27,13 +27,13 @@
     "from typing import Callable, List\n",
     "\n",
     "import tenacity\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.output_parsers import RegexParser\n",
     "from langchain.prompts import PromptTemplate\n",
     "from langchain.schema import (\n",
     "    HumanMessage,\n",
     "    SystemMessage,\n",
-    ")\n",
-    "from langchain_openai import ChatOpenAI"
+    ")"
    ]
   },
   {

cookbook/myscale_vector_sql.ipynb

@@ -31,10 +31,10 @@
     "from os import environ\n",
     "\n",
     "from langchain.chains import LLMChain\n",
+    "from langchain.llms import OpenAI\n",
     "from langchain.prompts import PromptTemplate\n",
-    "from langchain_community.utilities import SQLDatabase\n",
+    "from langchain.utilities import SQLDatabase\n",
     "from langchain_experimental.sql.vector_sql import VectorSQLDatabaseChain\n",
-    "from langchain_openai import OpenAI\n",
     "from sqlalchemy import MetaData, create_engine\n",
     "\n",
     "MYSCALE_HOST = \"msc-4a9e710a.us-east-1.aws.staging.myscale.cloud\"\n",
@@ -57,7 +57,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.embeddings import HuggingFaceInstructEmbeddings\n",
+    "from langchain.embeddings import HuggingFaceInstructEmbeddings\n",
     "from langchain_experimental.sql.vector_sql import VectorSQLOutputParser\n",
     "\n",
     "output_parser = VectorSQLOutputParser.from_embeddings(\n",
@@ -75,10 +75,10 @@
    "outputs": [],
    "source": [
     "from langchain.callbacks import StdOutCallbackHandler\n",
-    "from langchain_community.utilities.sql_database import SQLDatabase\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.utilities.sql_database import SQLDatabase\n",
     "from langchain_experimental.sql.prompt import MYSCALE_PROMPT\n",
     "from langchain_experimental.sql.vector_sql import VectorSQLDatabaseChain\n",
-    "from langchain_openai import OpenAI\n",
     "\n",
     "chain = VectorSQLDatabaseChain(\n",
     "    llm_chain=LLMChain(\n",
@@ -117,6 +117,7 @@
    "outputs": [],
    "source": [
     "from langchain.chains.qa_with_sources.retrieval import RetrievalQAWithSourcesChain\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain_experimental.retrievers.vector_sql_database import (\n",
     "    VectorSQLDatabaseChainRetriever,\n",
     ")\n",
@@ -125,7 +126,6 @@
     "    VectorSQLDatabaseChain,\n",
     "    VectorSQLRetrieveAllOutputParser,\n",
     ")\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "output_parser_retrieve_all = VectorSQLRetrieveAllOutputParser.from_embeddings(\n",
     "    output_parser.model\n",

cookbook/openai_functions_retrieval_qa.ipynb

@@ -20,10 +20,10 @@
    "outputs": [],
    "source": [
     "from langchain.chains import RetrievalQA\n",
+    "from langchain.document_loaders import TextLoader\n",
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
     "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain_community.document_loaders import TextLoader\n",
-    "from langchain_community.vectorstores import Chroma\n",
-    "from langchain_openai import OpenAIEmbeddings"
+    "from langchain.vectorstores import Chroma"
    ]
   },
   {
@@ -52,8 +52,8 @@
    "source": [
     "from langchain.chains import create_qa_with_sources_chain\n",
     "from langchain.chains.combine_documents.stuff import StuffDocumentsChain\n",
-    "from langchain.prompts import PromptTemplate\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import PromptTemplate"
    ]
   },
   {

cookbook/openai_v1_cookbook.ipynb

@@ -28,8 +28,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_core.messages import HumanMessage, SystemMessage\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain_core.messages import HumanMessage, SystemMessage"
    ]
   },
   {
@@ -414,7 +414,7 @@
     "BREAKING CHANGES:\n",
     "- To use Azure embeddings with OpenAI V1, you'll need to use the new `AzureOpenAIEmbeddings` instead of the existing `OpenAIEmbeddings`. `OpenAIEmbeddings` continue to work when using Azure with `openai<1`.\n",
     "```python\n",
-    "from langchain_openai import AzureOpenAIEmbeddings\n",
+    "from langchain.embeddings import AzureOpenAIEmbeddings\n",
     "```\n",
     "\n",
     "\n",
@@ -456,8 +456,8 @@
     "from typing import Literal\n",
     "\n",
     "from langchain.output_parsers.openai_tools import PydanticToolsParser\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
     "from langchain.utils.openai_functions import convert_pydantic_to_openai_tool\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
     "from langchain_core.pydantic_v1 import BaseModel, Field\n",
     "\n",
     "\n",

cookbook/petting_zoo.ipynb

@@ -47,12 +47,12 @@
     "import inspect\n",
     "\n",
     "import tenacity\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.output_parsers import RegexParser\n",
     "from langchain.schema import (\n",
     "    HumanMessage,\n",
     "    SystemMessage,\n",
-    ")\n",
-    "from langchain_openai import ChatOpenAI"
+    ")"
    ]
   },
   {

cookbook/plan_and_execute_agent.ipynb

@@ -30,14 +30,15 @@
    "outputs": [],
    "source": [
     "from langchain.chains import LLMMathChain\n",
-    "from langchain_community.utilities import DuckDuckGoSearchAPIWrapper\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.utilities import DuckDuckGoSearchAPIWrapper\n",
     "from langchain_core.tools import Tool\n",
     "from langchain_experimental.plan_and_execute import (\n",
     "    PlanAndExecute,\n",
     "    load_agent_executor,\n",
     "    load_chat_planner,\n",
-    ")\n",
-    "from langchain_openai import ChatOpenAI, OpenAI"
+    ")"
    ]
   },
   {

cookbook/press_releases.ipynb

@@ -81,8 +81,8 @@
    "outputs": [],
    "source": [
     "from langchain.chains import ConversationalRetrievalChain\n",
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.retrievers import KayAiRetriever\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "model = ChatOpenAI(model_name=\"gpt-3.5-turbo\")\n",
     "retriever = KayAiRetriever.create(\n",

cookbook/program_aided_language_model.ipynb

@@ -17,8 +17,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_experimental.pal_chain import PALChain\n",
-    "from langchain_openai import OpenAI"
+    "from langchain.llms import OpenAI\n",
+    "from langchain_experimental.pal_chain import PALChain"
    ]
   },
   {

cookbook/qa_citations.ipynb

@@ -27,7 +27,7 @@
    ],
    "source": [
     "from langchain.chains import create_citation_fuzzy_match_chain\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI"
    ]
   },
   {

cookbook/qianfan_baidu_elasticesearch_RAG.ipynb

@@ -59,13 +59,11 @@
     "from baidubce.auth.bce_credentials import BceCredentials\n",
     "from baidubce.bce_client_configuration import BceClientConfiguration\n",
     "from langchain.chains.retrieval_qa import RetrievalQA\n",
+    "from langchain.document_loaders.baiducloud_bos_directory import BaiduBOSDirectoryLoader\n",
+    "from langchain.embeddings.huggingface import HuggingFaceEmbeddings\n",
+    "from langchain.llms.baidu_qianfan_endpoint import QianfanLLMEndpoint\n",
     "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
-    "from langchain_community.document_loaders.baiducloud_bos_directory import (\n",
-    "    BaiduBOSDirectoryLoader,\n",
-    ")\n",
-    "from langchain_community.embeddings.huggingface import HuggingFaceEmbeddings\n",
-    "from langchain_community.llms.baidu_qianfan_endpoint import QianfanLLMEndpoint\n",
-    "from langchain_community.vectorstores import BESVectorStore"
+    "from langchain.vectorstores import BESVectorStore"
    ]
   },
   {

cookbook/rag_fusion.ipynb

@@ -30,8 +30,8 @@
    "outputs": [],
    "source": [
     "import pinecone\n",
-    "from langchain_community.vectorstores import Pinecone\n",
-    "from langchain_openai import OpenAIEmbeddings\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
+    "from langchain.vectorstores import Pinecone\n",
     "\n",
     "pinecone.init(api_key=\"...\", environment=\"...\")"
    ]
@@ -86,8 +86,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain_core.output_parsers import StrOutputParser"
    ]
   },
   {

cookbook/rag_with_quantized_embeddings.ipynb

@@ -1,591 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "6195da33-34c3-4ca2-943a-050b6dcbacbc",
-   "metadata": {},
-   "source": [
-    "# Embedding Documents using Optimized and Quantized Embedders\n",
-    "\n",
-    "In this tutorial, we will demo how to build a RAG pipeline, with the embedding for all documents done using Quantized Embedders.\n",
-    "\n",
-    "We will use a pipeline that will:\n",
-    "\n",
-    "* Create a document collection.\n",
-    "* Embed all documents using Quantized Embedders.\n",
-    "* Fetch relevant documents for our question.\n",
-    "* Run an LLM answer the question.\n",
-    "\n",
-    "For more information about optimized models, we refer to [optimum-intel](https://github.com/huggingface/optimum-intel.git) and [IPEX](https://github.com/intel/intel-extension-for-pytorch).\n",
-    "\n",
-    "This tutorial is based on the [Langchain RAG tutorial here](https://towardsai.net/p/machine-learning/dense-x-retrieval-technique-in-langchain-and-llamaindex)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 17,
-   "id": "26db2da5-3733-4a90-909e-6c11508ea140",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import uuid\n",
-    "from pathlib import Path\n",
-    "\n",
-    "import langchain\n",
-    "import torch\n",
-    "from bs4 import BeautifulSoup as Soup\n",
-    "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
-    "from langchain.storage import InMemoryByteStore, LocalFileStore\n",
-    "\n",
-    "# For our example, we'll load docs from the web\n",
-    "from langchain.text_splitter import RecursiveCharacterTextSplitter  # noqa\n",
-    "from langchain_community.document_loaders.recursive_url_loader import (\n",
-    "    RecursiveUrlLoader,\n",
-    ")\n",
-    "\n",
-    "# noqa\n",
-    "from langchain_community.vectorstores import Chroma\n",
-    "\n",
-    "DOCSTORE_DIR = \".\"\n",
-    "DOCSTORE_ID_KEY = \"doc_id\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f5ccda4e-7af5-4355-b9c4-25547edf33f9",
-   "metadata": {},
-   "source": [
-    "Lets first load up this paper, and split into text chunks of size 1000."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "5f4d8888-53a6-49f5-a198-da5c92419ca4",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Loaded 1 documents\n",
-      "Split into 73 documents\n"
-     ]
-    }
-   ],
-   "source": [
-    "# Could add more parsing here, as it's very raw.\n",
-    "loader = RecursiveUrlLoader(\n",
-    "    \"https://ar5iv.labs.arxiv.org/html/1706.03762\",\n",
-    "    max_depth=2,\n",
-    "    extractor=lambda x: Soup(x, \"html.parser\").text,\n",
-    ")\n",
-    "data = loader.load()\n",
-    "print(f\"Loaded {len(data)} documents\")\n",
-    "\n",
-    "# Split\n",
-    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
-    "all_splits = text_splitter.split_documents(data)\n",
-    "print(f\"Split into {len(all_splits)} documents\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "73e90632-2ac2-49eb-80da-ffe9ac4a278d",
-   "metadata": {},
-   "source": [
-    "In order to embed our documents, we can use the ```QuantizedBiEncoderEmbeddings```, for efficient and fast embedding. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "9a68a6f6-332d-481e-bbea-ad763155ea36",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "89af89b48c55409b9999b8e0387fab5b",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "config.json:   0%|          | 0.00/747 [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "01ad1b6278194b53bf6a5a286a311864",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "pytorch_model.bin:   0%|          | 0.00/45.9M [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "cb3bd1b88f7743c3b0322da3f021325c",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "inc_config.json:   0%|          | 0.00/287 [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "loading configuration file inc_config.json from cache at \n",
-      "INCConfig {\n",
-      "  \"distillation\": {},\n",
-      "  \"neural_compressor_version\": \"2.4.1\",\n",
-      "  \"optimum_version\": \"1.16.2\",\n",
-      "  \"pruning\": {},\n",
-      "  \"quantization\": {\n",
-      "    \"dataset_num_samples\": 50,\n",
-      "    \"is_static\": true\n",
-      "  },\n",
-      "  \"save_onnx_model\": false,\n",
-      "  \"torch_version\": \"2.2.0\",\n",
-      "  \"transformers_version\": \"4.37.2\"\n",
-      "}\n",
-      "\n",
-      "Using `INCModel` to load a TorchScript model will be deprecated in v1.15.0, to load your model please use `IPEXModel` instead.\n"
-     ]
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "7439315ebcb746f5be11fe30bc7693f6",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "tokenizer_config.json:   0%|          | 0.00/1.24k [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "05265a3912254ce1ad43cc8086bcb0ca",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "a48f4245c60744f28f37cd3a7a24d198",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "tokenizer.json:   0%|          | 0.00/711k [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "584a63cace934033b4ab22d3a178582a",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "special_tokens_map.json:   0%|          | 0.00/125 [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "from langchain_community.embeddings import QuantizedBiEncoderEmbeddings\n",
-    "from langchain_core.embeddings import Embeddings\n",
-    "\n",
-    "model_name = \"Intel/bge-small-en-v1.5-rag-int8-static\"\n",
-    "encode_kwargs = {\"normalize_embeddings\": True}  # set True to compute cosine similarity\n",
-    "\n",
-    "model_inc = QuantizedBiEncoderEmbeddings(\n",
-    "    model_name=model_name,\n",
-    "    encode_kwargs=encode_kwargs,\n",
-    "    query_instruction=\"Represent this sentence for searching relevant passages: \",\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "360b2837-8024-47e0-a4ba-592505a9a5c8",
-   "metadata": {},
-   "source": [
-    "With our embedder in place, lets define our retriever:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 16,
-   "id": "18bc0a73-1a13-4b2f-96ac-05a5313343b7",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def get_multi_vector_retriever(\n",
-    "    docstore_id_key: str, collection_name: str, embedding_function: Embeddings\n",
-    "):\n",
-    "    \"\"\"Create the composed retriever object.\"\"\"\n",
-    "    vectorstore = Chroma(\n",
-    "        collection_name=collection_name,\n",
-    "        embedding_function=embedding_function,\n",
-    "    )\n",
-    "    store = InMemoryByteStore()\n",
-    "\n",
-    "    return MultiVectorRetriever(\n",
-    "        vectorstore=vectorstore,\n",
-    "        byte_store=store,\n",
-    "        id_key=docstore_id_key,\n",
-    "    )\n",
-    "\n",
-    "\n",
-    "retriever = get_multi_vector_retriever(DOCSTORE_ID_KEY, \"multi_vec_store\", model_inc)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8484078e-1bf0-4080-a354-ef23823fd6dc",
-   "metadata": {},
-   "source": [
-    "Next, we divide each chunk into sub-docs:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 18,
-   "id": "e12f48d4-6562-416b-8f28-342912e5756e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "child_text_splitter = RecursiveCharacterTextSplitter(chunk_size=400)\n",
-    "id_key = \"doc_id\"\n",
-    "doc_ids = [str(uuid.uuid4()) for _ in all_splits]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 19,
-   "id": "a268ef5f-91c2-4d8e-87f0-53db376e6a29",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "sub_docs = []\n",
-    "for i, doc in enumerate(all_splits):\n",
-    "    _id = doc_ids[i]\n",
-    "    _sub_docs = child_text_splitter.split_documents([doc])\n",
-    "    for _doc in _sub_docs:\n",
-    "        _doc.metadata[id_key] = _id\n",
-    "    sub_docs.extend(_sub_docs)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "d84ea8f4-a5de-4d76-b44d-85e56583f489",
-   "metadata": {},
-   "source": [
-    "Lets write our documents into our new store. This will use our embedder on each document."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 20,
-   "id": "1af831ce-0eae-44bc-aca7-4d691063640b",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Batches: 100%|██████████| 8/8 [00:00<00:00,  9.05it/s]\n"
-     ]
-    }
-   ],
-   "source": [
-    "retriever.vectorstore.add_documents(sub_docs)\n",
-    "retriever.docstore.mset(list(zip(doc_ids, all_splits)))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "580bc212-8ecd-4d28-8656-b96fcd0d7eb6",
-   "metadata": {},
-   "source": [
-    "Great! Our retriever is good to go. Lets load up an LLM, that will reason over the retrieved documents:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 21,
-   "id": "008c992f",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": []
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "cbe70583ad964ae19582b72dab396784",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "import torch\n",
-    "from langchain.llms.huggingface_pipeline import HuggingFacePipeline\n",
-    "from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline\n",
-    "\n",
-    "model_id = \"Intel/neural-chat-7b-v3-3\"\n",
-    "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
-    "model = AutoModelForCausalLM.from_pretrained(\n",
-    "    model_id, device_map=\"auto\", torch_dtype=torch.bfloat16\n",
-    ")\n",
-    "\n",
-    "pipe = pipeline(\"text-generation\", model=model, tokenizer=tokenizer, max_new_tokens=100)\n",
-    "\n",
-    "hf = HuggingFacePipeline(pipeline=pipe)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6dd21fb2-0442-477d-aae2-9e7ee1d1d778",
-   "metadata": {},
-   "source": [
-    "Next, we will load up a prompt for answering questions using retrieved documents:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 22,
-   "id": "5e582509-caaf-4920-932c-4ce16162c789",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain import hub\n",
-    "\n",
-    "prompt = hub.pull(\"rlm/rag-prompt\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5cdfcba5-7ec7-4d0a-820e-4e200643a882",
-   "metadata": {},
-   "source": [
-    "We can now build our pipeline:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 23,
-   "id": "b74d8dfb-72bb-46da-9df9-0dc47a3ac791",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.schema.runnable import RunnablePassthrough\n",
-    "\n",
-    "rag_chain = {\"context\": retriever, \"question\": RunnablePassthrough()} | prompt | hf"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3bc53602-86d6-420f-91b1-fc2effa7e986",
-   "metadata": {},
-   "source": [
-    "Excellent! lets ask it a question.\n",
-    "We will also use a verbose and debug, to check which documents were used by the model to produce the answer."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 31,
-   "id": "f0a92c07-53da-4e1f-b880-ee83a36ee17d",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence] Entering Chain run with input:\n",
-      "\u001b[0m{\n",
-      "  \"input\": \"What is the first transduction model relying entirely on self-attention?\"\n",
-      "}\n",
-      "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<context,question>] Entering Chain run with input:\n",
-      "\u001b[0m{\n",
-      "  \"input\": \"What is the first transduction model relying entirely on self-attention?\"\n",
-      "}\n",
-      "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<context,question> > 4:chain:RunnablePassthrough] Entering Chain run with input:\n",
-      "\u001b[0m{\n",
-      "  \"input\": \"What is the first transduction model relying entirely on self-attention?\"\n",
-      "}\n",
-      "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<context,question> > 4:chain:RunnablePassthrough] [1ms] Exiting Chain run with output:\n",
-      "\u001b[0m{\n",
-      "  \"output\": \"What is the first transduction model relying entirely on self-attention?\"\n",
-      "}\n",
-      "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 2:chain:RunnableParallel<context,question>] [66ms] Exiting Chain run with output:\n",
-      "\u001b[0m[outputs]\n",
-      "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 5:prompt:ChatPromptTemplate] Entering Prompt run with input:\n",
-      "\u001b[0m[inputs]\n",
-      "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 5:prompt:ChatPromptTemplate] [1ms] Exiting Prompt run with output:\n",
-      "\u001b[0m{\n",
-      "  \"lc\": 1,\n",
-      "  \"type\": \"constructor\",\n",
-      "  \"id\": [\n",
-      "    \"langchain\",\n",
-      "    \"prompts\",\n",
-      "    \"chat\",\n",
-      "    \"ChatPromptValue\"\n",
-      "  ],\n",
-      "  \"kwargs\": {\n",
-      "    \"messages\": [\n",
-      "      {\n",
-      "        \"lc\": 1,\n",
-      "        \"type\": \"constructor\",\n",
-      "        \"id\": [\n",
-      "          \"langchain\",\n",
-      "          \"schema\",\n",
-      "          \"messages\",\n",
-      "          \"HumanMessage\"\n",
-      "        ],\n",
-      "        \"kwargs\": {\n",
-      "          \"content\": \"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\\nQuestion: What is the first transduction model relying entirely on self-attention? \\nContext: [Document(page_content='To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.\\\\nIn the following sections, we will describe the Transformer, motivate self-attention and discuss its advantages over models such as (neural_gpu, ; NalBytenet2017, ) and (JonasFaceNet2017, ).\\\\n\\\\n\\\\n\\\\n\\\\n3 Model Architecture\\\\n\\\\nFigure 1: The Transformer - model architecture.', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'}), Document(page_content='In this work, we presented the Transformer, the first sequence transduction model based entirely on attention, replacing the recurrent layers most commonly used in encoder-decoder architectures with multi-headed self-attention.\\\\n\\\\n\\\\nFor translation tasks, the Transformer can be trained significantly faster than architectures based on recurrent or convolutional layers. On both WMT 2014 English-to-German and WMT 2014 English-to-French translation tasks, we achieve a new state of the art. In the former task our best model outperforms even all previously reported ensembles. \\\\n\\\\n\\\\nWe are excited about the future of attention-based models and plan to apply them to other tasks. We plan to extend the Transformer to problems involving input and output modalities other than text and to investigate local, restricted attention mechanisms to efficiently handle large inputs and outputs such as images, audio and video.\\\\nMaking generation less sequential is another research goals of ours.', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'}), Document(page_content='Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies without regard to their distance in the input or output sequences (bahdanau2014neural, ; structuredAttentionNetworks, ). In all but a few cases (decomposableAttnModel, ), however, such attention mechanisms are used in conjunction with a recurrent network.\\\\n\\\\n\\\\nIn this work we propose the Transformer, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output. The Transformer allows for significantly more parallelization and can reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs.\\\\n\\\\n\\\\n\\\\n\\\\n\\\\n2 Background', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'}), Document(page_content='The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'})] \\nAnswer:\",\n",
-      "          \"additional_kwargs\": {}\n",
-      "        }\n",
-      "      }\n",
-      "    ]\n",
-      "  }\n",
-      "}\n",
-      "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 6:llm:HuggingFacePipeline] Entering LLM run with input:\n",
-      "\u001b[0m{\n",
-      "  \"prompts\": [\n",
-      "    \"Human: You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\\nQuestion: What is the first transduction model relying entirely on self-attention? \\nContext: [Document(page_content='To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.\\\\nIn the following sections, we will describe the Transformer, motivate self-attention and discuss its advantages over models such as (neural_gpu, ; NalBytenet2017, ) and (JonasFaceNet2017, ).\\\\n\\\\n\\\\n\\\\n\\\\n3 Model Architecture\\\\n\\\\nFigure 1: The Transformer - model architecture.', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'}), Document(page_content='In this work, we presented the Transformer, the first sequence transduction model based entirely on attention, replacing the recurrent layers most commonly used in encoder-decoder architectures with multi-headed self-attention.\\\\n\\\\n\\\\nFor translation tasks, the Transformer can be trained significantly faster than architectures based on recurrent or convolutional layers. On both WMT 2014 English-to-German and WMT 2014 English-to-French translation tasks, we achieve a new state of the art. In the former task our best model outperforms even all previously reported ensembles. \\\\n\\\\n\\\\nWe are excited about the future of attention-based models and plan to apply them to other tasks. We plan to extend the Transformer to problems involving input and output modalities other than text and to investigate local, restricted attention mechanisms to efficiently handle large inputs and outputs such as images, audio and video.\\\\nMaking generation less sequential is another research goals of ours.', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'}), Document(page_content='Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies without regard to their distance in the input or output sequences (bahdanau2014neural, ; structuredAttentionNetworks, ). In all but a few cases (decomposableAttnModel, ), however, such attention mechanisms are used in conjunction with a recurrent network.\\\\n\\\\n\\\\nIn this work we propose the Transformer, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output. The Transformer allows for significantly more parallelization and can reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs.\\\\n\\\\n\\\\n\\\\n\\\\n\\\\n2 Background', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'}), Document(page_content='The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the', metadata={'source': 'https://ar5iv.labs.arxiv.org/html/1706.03762', 'title': '[1706.03762] Attention Is All You Need', 'language': 'en'})] \\nAnswer:\"\n",
-      "  ]\n",
-      "}\n",
-      "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence > 6:llm:HuggingFacePipeline] [4.34s] Exiting LLM run with output:\n",
-      "\u001b[0m{\n",
-      "  \"generations\": [\n",
-      "    [\n",
-      "      {\n",
-      "        \"text\": \" The first transduction model relying entirely on self-attention is the Transformer.\",\n",
-      "        \"generation_info\": null,\n",
-      "        \"type\": \"Generation\"\n",
-      "      }\n",
-      "    ]\n",
-      "  ],\n",
-      "  \"llm_output\": null,\n",
-      "  \"run\": null\n",
-      "}\n",
-      "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RunnableSequence] [4.41s] Exiting Chain run with output:\n",
-      "\u001b[0m{\n",
-      "  \"output\": \" The first transduction model relying entirely on self-attention is the Transformer.\"\n",
-      "}\n"
-     ]
-    }
-   ],
-   "source": [
-    "langchain.verbose = True\n",
-    "langchain.debug = True\n",
-    "\n",
-    "llm_res = rag_chain.invoke(\n",
-    "    \"What is the first transduction model relying entirely on self-attention?\",\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 32,
-   "id": "023404a1-401a-46e1-8ab5-cafbc8593b04",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "' The first transduction model relying entirely on self-attention is the Transformer.'"
-      ]
-     },
-     "execution_count": 32,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "llm_res"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0eaefd01-254a-445d-a95f-37889c126e0e",
-   "metadata": {},
-   "source": [
-    "Based on the retrieved documents, the answer is indeed correct :)"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.18"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

cookbook/retrieval_in_sql.ipynb

@@ -42,8 +42,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain.sql_database import SQLDatabase\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "CONNECTION_STRING = \"postgresql+psycopg2://postgres:test@localhost:5432/vectordb\"  # Replace with your own\n",
     "db = SQLDatabase.from_uri(CONNECTION_STRING)"
@@ -88,7 +88,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_openai import OpenAIEmbeddings\n",
+    "from langchain.embeddings import OpenAIEmbeddings\n",
     "\n",
     "embeddings_model = OpenAIEmbeddings()"
    ]
@@ -219,7 +219,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_core.prompts import ChatPromptTemplate\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
     "\n",
     "template = \"\"\"You are a Postgres expert. Given an input question, first create a syntactically correct Postgres query to run, then look at the results of the query and return the answer to the input question.\n",
     "Unless the user specifies in the question a specific number of examples to obtain, query for at most 5 results using the LIMIT clause as per Postgres. You can order the results to return the most informative data in the database.\n",
@@ -267,9 +267,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
     "from langchain_core.output_parsers import StrOutputParser\n",
     "from langchain_core.runnables import RunnablePassthrough\n",
-    "from langchain_openai import ChatOpenAI\n",
     "\n",
     "db = SQLDatabase.from_uri(\n",
     "    CONNECTION_STRING\n",

cookbook/rewrite.ipynb

@@ -31,11 +31,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_community.utilities import DuckDuckGoSearchAPIWrapper\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.prompts import ChatPromptTemplate\n",
+    "from langchain.utilities import DuckDuckGoSearchAPIWrapper\n",
     "from langchain_core.output_parsers import StrOutputParser\n",
-    "from langchain_core.prompts import ChatPromptTemplate\n",
-    "from langchain_core.runnables import RunnablePassthrough\n",
-    "from langchain_openai import ChatOpenAI"
+    "from langchain_core.runnables import RunnablePassthrough"
    ]
   },
   {

cookbook/sales_agent_with_context.ipynb

@@ -49,13 +49,14 @@
     "from langchain.agents.conversational.prompt import FORMAT_INSTRUCTIONS\n",
     "from langchain.chains import LLMChain, RetrievalQA\n",
     "from langchain.chains.base import Chain\n",
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
+    "from langchain.llms import BaseLLM, OpenAI\n",
     "from langchain.prompts import PromptTemplate\n",
     "from langchain.prompts.base import StringPromptTemplate\n",
     "from langchain.schema import AgentAction, AgentFinish\n",
     "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain_community.llms import BaseLLM\n",
-    "from langchain_community.vectorstores import Chroma\n",
-    "from langchain_openai import ChatOpenAI, OpenAI, OpenAIEmbeddings\n",
+    "from langchain.vectorstores import Chroma\n",
     "from pydantic import BaseModel, Field"
    ]
   },