Change NamedTuple to BaseModel

.github/CONTRIBUTING.md

@@ -115,37 +115,8 @@ To get a report of current coverage, run the following:
 make coverage
 ```
 
-### Working with Optional Dependencies
-
-Langchain relies heavily on optional dependencies to keep the Langchain package lightweight.
-
-If you're adding a new dependency to Langchain, assume that it will be an optional dependency, and
-that most users won't have it installed.
-
-Users that do not have the dependency installed should be able to **import** your code without
-any side effects (no warnings, no errors, no exceptions). 
-
-To introduce the dependency to the pyproject.toml file correctly, please do the following: 
-
-1. Add the dependency to the main group as an optional dependency
-  ```bash
-  poetry add --optional [package_name]
-  ```
-2. Open pyproject.toml and add the dependency to the `extended_testing` extra
-3. Relock the poetry file to update the extra.
-  ```bash
-  poetry lock --no-update
-  ```
-4. Add a unit test that the very least attempts to import the new code. Ideally the unit
-test makes use of lightweight fixtures to test the logic of the code.
-5. Please use the `@pytest.mark.requires(package_name)` decorator for any tests that require the dependency.
-
 ### Testing
 
-See section about optional dependencies.
-
-#### Unit Tests
-
 Unit tests cover modular logic that does not require calls to outside APIs.
 
 To run unit tests:
@@ -162,20 +133,8 @@ make docker_tests
 
 If you add new logic, please add a unit test.
 
-
-
-#### Integration Tests
-
 Integration tests cover logic that requires making calls to outside APIs (often integration with other services).
 
-**warning** Almost no tests should be integration tests. 
-
-  Tests that require making network connections make it difficult for other
-  developers to test the code.
-
-  Instead favor relying on `responses` library and/or mock.patch to mock
-  requests using small fixtures.
-
 To run integration tests:
 
 ```bash

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,7 +1,7 @@
 # Your PR Title (What it does)
 
 <!--
-Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.
+Thank you for contributing to LangChain! Your PR will appear in our next release under the title you set. Please make sure it highlights your valuable contribution.
 
 Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.
 
@@ -14,17 +14,7 @@ Fixes # (issue)
 
 ## Before submitting
 
-<!-- If you're adding a new integration, please include:
-
-1. a test for the integration - favor unit tests that does not rely on network access.
-2. an example notebook showing its use
-
-
-See contribution guidelines for more information on how to write tests, lint
-etc:
-
-https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
--->
+<!-- If you're adding a new integration, include an integration test and an example notebook showing its use! -->
 
 ## Who can review?
 
@@ -32,25 +22,25 @@ Community members can review the PR once tests pass. Tag maintainers/contributor
 
 <!-- For a quicker response, figure out the right person to tag with @
 
-  @hwchase17 - project lead
+        @hwchase17 - project lead
 
-  Tracing / Callbacks
-  - @agola11
+        Tracing / Callbacks
+        - @agola11
 
-  Async
-  - @agola11
+        Async
+        - @agola11
 
-  DataLoaders
-  - @eyurtsev
+        DataLoaders
+        - @eyurtsev
 
-  Models
-  - @hwchase17
-  - @agola11
+        Models
+        - @hwchase17
+        - @agola11
 
-  Agents / Tools / Toolkits
-  - @vowelparrot
-
-  VectorStores / Retrievers / Memory
-  - @dev2049
+        Agents / Tools / Toolkits
+        - @vowelparrot
+        
+        VectorStores / Retrievers / Memory
+        - @dev2049
         
  -->

docs/ecosystem/modelscope.md

@@ -1,20 +0,0 @@
-# ModelScope
-
-This page covers how to use the modelscope ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific modelscope wrappers.
-
-## Installation and Setup
-
-* Install the Python SDK with `pip install modelscope`
-
-## Wrappers
-
-### Embeddings
-
-There exists a modelscope Embeddings wrapper, which you can access with 
-
-```python
-from langchain.embeddings import ModelScopeEmbeddings
-```
-
-For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/modelscope_hub.ipynb)

docs/getting_started/concepts.md

@@ -23,7 +23,7 @@ The results of these actions can then be fed back into the language model to gen
 ## ReAct
 
 `ReAct` is a prompting technique that combines Chain-of-Thought prompting with action plan generation.
-This induces the model to think about what action to take, then take it.
+This induces the to model to think about what action to take, then take it.
 
 - [Paper](https://arxiv.org/pdf/2210.03629.pdf)
 - [LangChain Example](../modules/agents/agents/examples/react.ipynb)

docs/integrations/beam.md

@@ -1,92 +0,0 @@
-# Beam
-
-This page covers how to use Beam within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Beam wrappers.
-
-## Installation and Setup
-
-- [Create an account](https://www.beam.cloud/)
-- Install the Beam CLI with `curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh`
-- Register API keys with `beam configure`
-- Set environment variables (`BEAM_CLIENT_ID`) and (`BEAM_CLIENT_SECRET`)
-- Install the Beam SDK `pip install beam-sdk`
-
-## Wrappers
-
-### LLM
-
-There exists a Beam LLM wrapper, which you can access with
-
-```python
-from langchain.llms.beam import Beam
-```
-
-## Define your Beam app.
-
-This is the environment you’ll be developing against once you start the app.
-It's also used to define the maximum response length from the model.
-```python
-llm = Beam(model_name="gpt2",
-           name="langchain-gpt2-test",
-           cpu=8,
-           memory="32Gi",
-           gpu="A10G",
-           python_version="python3.8",
-           python_packages=[
-               "diffusers[torch]>=0.10",
-               "transformers",
-               "torch",
-               "pillow",
-               "accelerate",
-               "safetensors",
-               "xformers",],
-           max_length="50",
-           verbose=False)
-```
-
-## Deploy your Beam app
-
-Once defined, you can deploy your Beam app by calling your model's `_deploy()` method.
-
-```python
-llm._deploy()
-```
-
-## Call your Beam app
-
-Once a beam model is deployed, it can be called by callying your model's `_call()` method.
-This returns the GPT2 text response to your prompt.
-
-```python
-response = llm._call("Running machine learning on a remote GPU")
-```
-
-An example script which deploys the model and calls it would be:
-
-```python
-from langchain.llms.beam import Beam
-import time
-
-llm = Beam(model_name="gpt2",
-           name="langchain-gpt2-test",
-           cpu=8,
-           memory="32Gi",
-           gpu="A10G",
-           python_version="python3.8",
-           python_packages=[
-               "diffusers[torch]>=0.10",
-               "transformers",
-               "torch",
-               "pillow",
-               "accelerate",
-               "safetensors",
-               "xformers",],
-           max_length="50",
-           verbose=False)
-
-llm._deploy()
-
-response = llm._call("Running machine learning on a remote GPU")
-
-print(response)
-```

docs/integrations/ctransformers.md

@@ -1,57 +0,0 @@
-# C Transformers
-
-This page covers how to use the [C Transformers](https://github.com/marella/ctransformers) library within LangChain.
-It is broken into two parts: installation and setup, and then references to specific C Transformers wrappers.
-
-## Installation and Setup
-
-- Install the Python package with `pip install ctransformers`
-- Download a supported [GGML model](https://huggingface.co/TheBloke) (see [Supported Models](https://github.com/marella/ctransformers#supported-models))
-
-## Wrappers
-
-### LLM
-
-There exists a CTransformers LLM wrapper, which you can access with:
-
-```python
-from langchain.llms import CTransformers
-```
-
-It provides a unified interface for all models:
-
-```python
-llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2')
-
-print(llm('AI is going to'))
-```
-
-If you are getting `illegal instruction` error, try using `lib='avx'` or `lib='basic'`:
-
-```py
-llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2', lib='avx')
-```
-
-It can be used with models hosted on the Hugging Face Hub:
-
-```py
-llm = CTransformers(model='marella/gpt-2-ggml')
-```
-
-If a model repo has multiple model files (`.bin` files), specify a model file using:
-
-```py
-llm = CTransformers(model='marella/gpt-2-ggml', model_file='ggml-model.bin')
-```
-
-Additional parameters can be passed using the `config` parameter:
-
-```py
-config = {'max_new_tokens': 256, 'repetition_penalty': 1.1}
-
-llm = CTransformers(model='marella/gpt-2-ggml', config=config)
-```
-
-See [Documentation](https://github.com/marella/ctransformers#config) for a list of available parameters.
-
-For a more detailed walkthrough of this, see [this notebook](../modules/models/llms/integrations/ctransformers.ipynb).

docs/integrations/momento.md

@@ -1,53 +0,0 @@
-# Momento
-
-This page covers how to use the [Momento](https://gomomento.com) ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Momento wrappers.
-
-## Installation and Setup
-
-- Sign up for a free account [here](https://docs.momentohq.com/getting-started) and get an auth token
-- Install the Momento Python SDK with `pip install momento`
-
-## Wrappers
-
-### Cache
-
-The Cache wrapper allows for [Momento](https://gomomento.com) to be used as a serverless, distributed, low-latency cache for LLM prompts and responses.
-
-#### Standard Cache
-
-The standard cache is the go-to use case for [Momento](https://gomomento.com) users in any environment.
-
-Import the cache as follows:
-
-```python
-from langchain.cache import MomentoCache
-```
-
-And set up like so:
-
-```python
-from datetime import timedelta
-from momento import CacheClient, Configurations, CredentialProvider
-import langchain
-
-# Instantiate the Momento client
-cache_client = CacheClient(
-    Configurations.Laptop.v1(),
-    CredentialProvider.from_environment_variable("MOMENTO_AUTH_TOKEN"),
-    default_ttl=timedelta(days=1))
-
-# Choose a Momento cache name of your choice
-cache_name = "langchain"
-
-# Instantiate the LLM cache
-langchain.llm_cache = MomentoCache(cache_client, cache_name)
-```
-
-### Memory
-
-Momento can be used as a distributed memory store for LLMs.
-
-#### Chat Message History Memory
-
-See [this notebook](../modules/memory/examples/momento_chat_message_history.ipynb) for a walkthrough of how to use Momento as a memory store for chat message history.

docs/integrations/vectara.md

@@ -1,40 +0,0 @@
-# Vectara
-
-
-What is Vectara?
-
-**Vectara Overview:**
-- Vectara is developer-first API platform for building conversational search applications
-- To use Vectara - first [sign up](https://console.vectara.com/signup) and create an account. Then create a corpus and an API key for indexing and searching.
-- You can use Vectara's [indexing API](https://docs.vectara.com/docs/indexing-apis/indexing) to add documents into Vectara's index
-- You can use Vectara's [Search API](https://docs.vectara.com/docs/search-apis/search) to query Vectara's index (which also supports Hybrid search implicitly).
-- You can use Vectara's integration with LangChain as a Vector store or using the Retriever abstraction.
-
-## Installation and Setup
-To use Vectara with LangChain no special installation steps are required. You just have to provide your customer_id, corpus ID, and an API key created within the Vectara console to enable indexing and searching.
-
-### VectorStore
-
-There exists a wrapper around the Vectara platform, allowing you to use it as a vectorstore, whether for semantic search or example selection.
-
-To import this vectorstore:
-```python
-from langchain.vectorstores import Vectara
-```
-
-To create an instance of the Vectara vectorstore:
-```python
-vectara = Vectara(
-    vectara_customer_id=customer_id, 
-    vectara_corpus_id=corpus_id, 
-    vectara_api_key=api_key
-)
-```
-The customer_id, corpus_id and api_key are optional, and if they are not supplied will be read from the environment variables `VECTARA_CUSTOMER_ID`, `VECTARA_CORPUS_ID` and `VECTARA_API_KEY`, respectively.
-
-
-For a more detailed walkthrough of the Vectara wrapper, see one of the two example notebooks:
-* [Chat Over Documents with Vectara](./vectara/vectara_chat.html)
-* [Vectara Text Generation](./vectara/vectara_text_generation.html)
-
-

docs/integrations/vectara/vectara_chat.ipynb

@@ -1,726 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "134a0785",
-   "metadata": {},
-   "source": [
-    "# Chat Over Documents with Vectara\n",
-    "\n",
-    "This notebook is based on the [chat_vector_db](https://github.com/hwchase17/langchain/blob/master/docs/modules/chains/index_examples/chat_vector_db.ipynb) notebook, but using Vectara as the vector database."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "70c4e529",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "from langchain.vectorstores import Vectara\n",
-    "from langchain.vectorstores.vectara import VectaraRetriever\n",
-    "from langchain.llms import OpenAI\n",
-    "from langchain.chains import ConversationalRetrievalChain"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "cdff94be",
-   "metadata": {},
-   "source": [
-    "Load in documents. You can replace this with a loader for whatever type of data you want"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "01c46e92",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.document_loaders import TextLoader\n",
-    "loader = TextLoader(\"../../modules/state_of_the_union.txt\")\n",
-    "documents = loader.load()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "239475d2",
-   "metadata": {},
-   "source": [
-    "We now split the documents, create embeddings for them, and put them in a vectorstore. This allows us to do semantic search over them."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "a8930cf7",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "vectorstore = Vectara.from_documents(documents, embedding=None)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "898b574b",
-   "metadata": {},
-   "source": [
-    "We can now create a memory object, which is neccessary to track the inputs/outputs and hold a conversation."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "af803fee",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.memory import ConversationBufferMemory\n",
-    "memory = ConversationBufferMemory(memory_key=\"chat_history\", return_messages=True)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3c96b118",
-   "metadata": {},
-   "source": [
-    "We now initialize the `ConversationalRetrievalChain`"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "7b4110f3",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "<class 'langchain.vectorstores.vectara.Vectara'>\n"
-     ]
-    }
-   ],
-   "source": [
-    "openai_api_key = os.environ['OPENAI_API_KEY']\n",
-    "llm = OpenAI(openai_api_key=openai_api_key, temperature=0)\n",
-    "retriever = VectaraRetriever(vectorstore, alpha=0.025, k=5, filter=None)\n",
-    "\n",
-    "print(type(vectorstore))\n",
-    "d = retriever.get_relevant_documents('What did the president say about Ketanji Brown Jackson')\n",
-    "\n",
-    "qa = ConversationalRetrievalChain.from_llm(llm, retriever, memory=memory)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "e8ce4fe9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = qa({\"question\": query})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "4c79862b",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender.\""
-      ]
-     },
-     "execution_count": 7,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result[\"answer\"]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "c697d9d1",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "query = \"Did he mention who she suceeded\"\n",
-    "result = qa({\"question\": query})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "ba0678f3",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "' Justice Stephen Breyer.'"
-      ]
-     },
-     "execution_count": 9,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result['answer']"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b3308b01-5300-4999-8cd3-22f16dae757e",
-   "metadata": {},
-   "source": [
-    "## Pass in chat history\n",
-    "\n",
-    "In the above example, we used a Memory object to track chat history. We can also just pass it in explicitly. In order to do this, we need to initialize a chain without any memory object."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "1b41a10b-bf68-4689-8f00-9aed7675e2ab",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0), vectorstore.as_retriever())"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "83f38c18-ac82-45f4-a79e-8b37ce1ae115",
-   "metadata": {},
-   "source": [
-    "Here's an example of asking a question with no chat history"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "id": "bc672290-8a8b-4828-a90c-f1bbdd6b3920",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "chat_history = []\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = qa({\"question\": query, \"chat_history\": chat_history})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 12,
-   "id": "6b62d758-c069-4062-88f0-21e7ea4710bf",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender.\""
-      ]
-     },
-     "execution_count": 12,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result[\"answer\"]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8c26a83d-c945-4458-b54a-c6bd7f391303",
-   "metadata": {},
-   "source": [
-    "Here's an example of asking a question with some chat history"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 13,
-   "id": "9c95460b-7116-4155-a9d2-c0fb027ee592",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "chat_history = [(query, result[\"answer\"])]\n",
-    "query = \"Did he mention who she suceeded\"\n",
-    "result = qa({\"question\": query, \"chat_history\": chat_history})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 14,
-   "id": "698ac00c-cadc-407f-9423-226b2d9258d0",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "' Justice Stephen Breyer.'"
-      ]
-     },
-     "execution_count": 14,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result['answer']"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0eaadf0f",
-   "metadata": {},
-   "source": [
-    "## Return Source Documents\n",
-    "You can also easily return source documents from the ConversationalRetrievalChain. This is useful for when you want to inspect what documents were returned."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 15,
-   "id": "562769c6",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "qa = ConversationalRetrievalChain.from_llm(llm, vectorstore.as_retriever(), return_source_documents=True)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 16,
-   "id": "ea478300",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "chat_history = []\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = qa({\"question\": query, \"chat_history\": chat_history})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 17,
-   "id": "4cb75b4e",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "Document(page_content='Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice. A former federal public defender.', metadata={'source': '../../modules/state_of_the_union.txt'})"
-      ]
-     },
-     "execution_count": 17,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result['source_documents'][0]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "669ede2f-d69f-4960-8468-8a768ce1a55f",
-   "metadata": {},
-   "source": [
-    "## ConversationalRetrievalChain with `search_distance`\n",
-    "If you are using a vector store that supports filtering by search distance, you can add a threshold value parameter."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 18,
-   "id": "f4f32c6f-8e49-44af-9116-8830b1fcc5f2",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "vectordbkwargs = {\"search_distance\": 0.9}"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 19,
-   "id": "1e251775-31e7-4679-b744-d4a57937f93a",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0), vectorstore.as_retriever(), return_source_documents=True)\n",
-    "chat_history = []\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = qa({\"question\": query, \"chat_history\": chat_history, \"vectordbkwargs\": vectordbkwargs})"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "99b96dae",
-   "metadata": {},
-   "source": [
-    "## ConversationalRetrievalChain with `map_reduce`\n",
-    "We can also use different types of combine document chains with the ConversationalRetrievalChain chain."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 20,
-   "id": "e53a9d66",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.chains import LLMChain\n",
-    "from langchain.chains.question_answering import load_qa_chain\n",
-    "from langchain.chains.conversational_retrieval.prompts import CONDENSE_QUESTION_PROMPT"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 21,
-   "id": "bf205e35",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)\n",
-    "doc_chain = load_qa_chain(llm, chain_type=\"map_reduce\")\n",
-    "\n",
-    "chain = ConversationalRetrievalChain(\n",
-    "    retriever=vectorstore.as_retriever(),\n",
-    "    question_generator=question_generator,\n",
-    "    combine_docs_chain=doc_chain,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 22,
-   "id": "78155887",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "chat_history = []\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = chain({\"question\": query, \"chat_history\": chat_history})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 23,
-   "id": "e54b5fa2",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "' The president did not mention Ketanji Brown Jackson.'"
-      ]
-     },
-     "execution_count": 23,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result['answer']"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a2fe6b14",
-   "metadata": {},
-   "source": [
-    "## ConversationalRetrievalChain with Question Answering with sources\n",
-    "\n",
-    "You can also use this chain with the question answering with sources chain."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 24,
-   "id": "d1058fd2",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.chains.qa_with_sources import load_qa_with_sources_chain"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 25,
-   "id": "a6594482",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "\n",
-    "question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)\n",
-    "doc_chain = load_qa_with_sources_chain(llm, chain_type=\"map_reduce\")\n",
-    "\n",
-    "chain = ConversationalRetrievalChain(\n",
-    "    retriever=vectorstore.as_retriever(),\n",
-    "    question_generator=question_generator,\n",
-    "    combine_docs_chain=doc_chain,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 26,
-   "id": "e2badd21",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "chat_history = []\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = chain({\"question\": query, \"chat_history\": chat_history})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 27,
-   "id": "edb31fe5",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "' The president did not mention Ketanji Brown Jackson.\\nSOURCES: ../../modules/state_of_the_union.txt'"
-      ]
-     },
-     "execution_count": 27,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result['answer']"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2324cdc6-98bf-4708-b8cd-02a98b1e5b67",
-   "metadata": {},
-   "source": [
-    "## ConversationalRetrievalChain with streaming to `stdout`\n",
-    "\n",
-    "Output from the chain will be streamed to `stdout` token by token in this example."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 28,
-   "id": "2efacec3-2690-4b05-8de3-a32fd2ac3911",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.chains.llm import LLMChain\n",
-    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
-    "from langchain.chains.conversational_retrieval.prompts import CONDENSE_QUESTION_PROMPT, QA_PROMPT\n",
-    "from langchain.chains.question_answering import load_qa_chain\n",
-    "\n",
-    "# Construct a ConversationalRetrievalChain with a streaming llm for combine docs\n",
-    "# and a separate, non-streaming llm for question generation\n",
-    "llm = OpenAI(temperature=0, openai_api_key=openai_api_key)\n",
-    "streaming_llm = OpenAI(streaming=True, callbacks=[StreamingStdOutCallbackHandler()], temperature=0, openai_api_key=openai_api_key)\n",
-    "\n",
-    "question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)\n",
-    "doc_chain = load_qa_chain(streaming_llm, chain_type=\"stuff\", prompt=QA_PROMPT)\n",
-    "\n",
-    "qa = ConversationalRetrievalChain(\n",
-    "    retriever=vectorstore.as_retriever(), combine_docs_chain=doc_chain, question_generator=question_generator)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 29,
-   "id": "fd6d43f4-7428-44a4-81bc-26fe88a98762",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      " The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender."
-     ]
-    }
-   ],
-   "source": [
-    "chat_history = []\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = qa({\"question\": query, \"chat_history\": chat_history})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 30,
-   "id": "5ab38978-f3e8-4fa7-808c-c79dec48379a",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      " Justice Stephen Breyer."
-     ]
-    }
-   ],
-   "source": [
-    "chat_history = [(query, result[\"answer\"])]\n",
-    "query = \"Did he mention who she suceeded\"\n",
-    "result = qa({\"question\": query, \"chat_history\": chat_history})\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f793d56b",
-   "metadata": {},
-   "source": [
-    "## get_chat_history Function\n",
-    "You can also specify a `get_chat_history` function, which can be used to format the chat_history string."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 31,
-   "id": "a7ba9d8c",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "def get_chat_history(inputs) -> str:\n",
-    "    res = []\n",
-    "    for human, ai in inputs:\n",
-    "        res.append(f\"Human:{human}\\nAI:{ai}\")\n",
-    "    return \"\\n\".join(res)\n",
-    "qa = ConversationalRetrievalChain.from_llm(llm, vectorstore.as_retriever(), get_chat_history=get_chat_history)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 32,
-   "id": "a3e33c0d",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "chat_history = []\n",
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "result = qa({\"question\": query, \"chat_history\": chat_history})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 33,
-   "id": "936dc62f",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender.\""
-      ]
-     },
-     "execution_count": 33,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "result['answer']"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "b8c26901",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.9"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/integrations/vectara/vectara_text_generation.ipynb

@@ -1,199 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Vectara Text Generation\n",
-    "\n",
-    "This notebook is based on [chat_vector_db](https://github.com/hwchase17/langchain/blob/master/docs/modules/chains/index_examples/question_answering.ipynb) and adapted to Vectara."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Prepare Data\n",
-    "\n",
-    "First, we prepare the data. For this example, we fetch a documentation site that consists of markdown files hosted on Github and split them into small enough Documents."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.llms import OpenAI\n",
-    "from langchain.docstore.document import Document\n",
-    "import requests\n",
-    "from langchain.vectorstores import Vectara\n",
-    "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain.prompts import PromptTemplate\n",
-    "import pathlib\n",
-    "import subprocess\n",
-    "import tempfile"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Cloning into '.'...\n"
-     ]
-    }
-   ],
-   "source": [
-    "def get_github_docs(repo_owner, repo_name):\n",
-    "    with tempfile.TemporaryDirectory() as d:\n",
-    "        subprocess.check_call(\n",
-    "            f\"git clone --depth 1 https://github.com/{repo_owner}/{repo_name}.git .\",\n",
-    "            cwd=d,\n",
-    "            shell=True,\n",
-    "        )\n",
-    "        git_sha = (\n",
-    "            subprocess.check_output(\"git rev-parse HEAD\", shell=True, cwd=d)\n",
-    "            .decode(\"utf-8\")\n",
-    "            .strip()\n",
-    "        )\n",
-    "        repo_path = pathlib.Path(d)\n",
-    "        markdown_files = list(repo_path.glob(\"*/*.md\")) + list(\n",
-    "            repo_path.glob(\"*/*.mdx\")\n",
-    "        )\n",
-    "        for markdown_file in markdown_files:\n",
-    "            with open(markdown_file, \"r\") as f:\n",
-    "                relative_path = markdown_file.relative_to(repo_path)\n",
-    "                github_url = f\"https://github.com/{repo_owner}/{repo_name}/blob/{git_sha}/{relative_path}\"\n",
-    "                yield Document(page_content=f.read(), metadata={\"source\": github_url})\n",
-    "\n",
-    "sources = get_github_docs(\"yirenlu92\", \"deno-manual-forked\")\n",
-    "\n",
-    "source_chunks = []\n",
-    "splitter = CharacterTextSplitter(separator=\" \", chunk_size=1024, chunk_overlap=0)\n",
-    "for source in sources:\n",
-    "    for chunk in splitter.split_text(source.page_content):\n",
-    "        source_chunks.append(chunk)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Set Up Vector DB\n",
-    "\n",
-    "Now that we have the documentation content in chunks, let's put all this information in a vector index for easy retrieval."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "search_index = Vectara.from_texts(source_chunks, embedding=None)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Set Up LLM Chain with Custom Prompt\n",
-    "\n",
-    "Next, let's set up a simple LLM chain but give it a custom prompt for blog post generation. Note that the custom prompt is parameterized and takes two inputs: `context`, which will be the documents fetched from the vector search, and `topic`, which is given by the user."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.chains import LLMChain\n",
-    "prompt_template = \"\"\"Use the context below to write a 400 word blog post about the topic below:\n",
-    "    Context: {context}\n",
-    "    Topic: {topic}\n",
-    "    Blog post:\"\"\"\n",
-    "\n",
-    "PROMPT = PromptTemplate(\n",
-    "    template=prompt_template, input_variables=[\"context\", \"topic\"]\n",
-    ")\n",
-    "\n",
-    "llm = OpenAI(openai_api_key=os.environ['OPENAI_API_KEY'], temperature=0)\n",
-    "\n",
-    "chain = LLMChain(llm=llm, prompt=PROMPT)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Generate Text\n",
-    "\n",
-    "Finally, we write a function to apply our inputs to the chain. The function takes an input parameter `topic`. We find the documents in the vector index that correspond to that `topic`, and use them as additional context in our simple LLM chain."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def generate_blog_post(topic):\n",
-    "    docs = search_index.similarity_search(topic, k=4)\n",
-    "    inputs = [{\"context\": doc.page_content, \"topic\": topic} for doc in docs]\n",
-    "    print(chain.apply(inputs))"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "[{'text': '\\n\\nEnvironment variables are an essential part of any development workflow. They provide a way to store and access information that is specific to the environment in which the code is running. This can be especially useful when working with different versions of a language or framework, or when running code on different machines.\\n\\nThe Deno CLI tasks extension provides a way to easily manage environment variables when running Deno commands. This extension provides a task definition for allowing you to create tasks that execute the `deno` CLI from within the editor. The template for the Deno CLI tasks has the following interface, which can be configured in a `tasks.json` within your workspace:\\n\\nThe task definition includes the `type` field, which should be set to `deno`, and the `command` field, which is the `deno` command to run (e.g. `run`, `test`, `cache`, etc.). Additionally, you can specify additional arguments to pass on the command line, the current working directory to execute the command, and any environment variables.\\n\\nUsing environment variables with the Deno CLI tasks extension is a great way to ensure that your code is running in the correct environment. For example, if you are running a test suite,'}, {'text': '\\n\\nEnvironment variables are an important part of any programming language, and they can be used to store and access data in a variety of ways. In this blog post, we\\'ll be taking a look at environment variables specifically for the shell.\\n\\nShell variables are similar to environment variables, but they won\\'t be exported to spawned commands. They are defined with the following syntax:\\n\\n```sh\\nVAR_NAME=value\\n```\\n\\nShell variables can be used to store and access data in a variety of ways. For example, you can use them to store values that you want to re-use, but don\\'t want to be available in any spawned processes.\\n\\nFor example, if you wanted to store a value and then use it in a command, you could do something like this:\\n\\n```sh\\nVAR=hello && echo $VAR && deno eval \"console.log(\\'Deno: \\' + Deno.env.get(\\'VAR\\'))\"\\n```\\n\\nThis would output the following:\\n\\n```\\nhello\\nDeno: undefined\\n```\\n\\nAs you can see, the value stored in the shell variable is not available in the spawned process.\\n\\n'}, {'text': '\\n\\nWhen it comes to developing applications, environment variables are an essential part of the process. Environment variables are used to store information that can be used by applications and scripts to customize their behavior. This is especially important when it comes to developing applications with Deno, as there are several environment variables that can impact the behavior of Deno.\\n\\nThe most important environment variable for Deno is `DENO_AUTH_TOKENS`. This environment variable is used to store authentication tokens that are used to access remote resources. This is especially important when it comes to accessing remote APIs or databases. Without the proper authentication tokens, Deno will not be able to access the remote resources.\\n\\nAnother important environment variable for Deno is `DENO_DIR`. This environment variable is used to store the directory where Deno will store its files. This includes the Deno executable, the Deno cache, and the Deno configuration files. By setting this environment variable, you can ensure that Deno will always be able to find the files it needs.\\n\\nFinally, there is the `DENO_PLUGINS` environment variable. This environment variable is used to store the list of plugins that Deno will use. This is important for customizing the'}, {'text': '\\n\\nEnvironment variables are a great way to store and access sensitive information in your Deno applications. Deno offers built-in support for environment variables with `Deno.env`, and you can also use a `.env` file to store and access environment variables. In this blog post, we\\'ll explore both of these options and how to use them in your Deno applications.\\n\\n## Built-in `Deno.env`\\n\\nThe Deno runtime offers built-in support for environment variables with [`Deno.env`](https://deno.land/api@v1.25.3?s=Deno.env). `Deno.env` has getter and setter methods. Here is example usage:\\n\\n```ts\\nDeno.env.set(\"FIREBASE_API_KEY\", \"examplekey123\");\\nDeno.env.set(\"FIREBASE_AUTH_DOMAIN\", \"firebasedomain.com\");\\n\\nconsole.log(Deno.env.get(\"FIREBASE_API_KEY\")); // examplekey123\\nconsole.log(Deno.env.get(\"FIREBASE_AUTH_'}]\n"
-     ]
-    }
-   ],
-   "source": [
-    "generate_blog_post(\"environment variables\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.9"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}

docs/modules/agents/agents/custom_agent_with_tool_retrieval.ipynb

@@ -149,7 +149,7 @@
     {
      "data": {
       "text/plain": [
-       "[Tool(name='Search', description='useful for when you need to answer questions about current events', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<bound method SerpAPIWrapper.run of SerpAPIWrapper(search_engine=<class 'serpapi.google_search.GoogleSearch'>, params={'engine': 'google', 'google_domain': 'google.com', 'gl': 'us', 'hl': 'en'}, serpapi_api_key='', aiosession=None)>, coroutine=None),\n",
+       "[Tool(name='Search', description='useful for when you need to answer questions about current events', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<bound method SerpAPIWrapper.run of SerpAPIWrapper(search_engine=<class 'serpapi.google_search.GoogleSearch'>, params={'engine': 'google', 'google_domain': 'google.com', 'gl': 'us', 'hl': 'en'}, serpapi_api_key='c657176b327b17e79b55306ab968d164ee2369a7c7fa5b3f8a5f7889903de882', aiosession=None)>, coroutine=None),\n",
        " Tool(name='foo-95', description='a silly function that you can use to get more information about the number 95', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None),\n",
        " Tool(name='foo-12', description='a silly function that you can use to get more information about the number 12', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None),\n",
        " Tool(name='foo-15', description='a silly function that you can use to get more information about the number 15', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None)]"

docs/modules/agents/getting_started.ipynb

@@ -22,7 +22,7 @@
     "\n",
     "- Tool: A function that performs a specific duty. This can be things like: Google Search, Database lookup, Python REPL, other chains. The interface for a tool is currently a function that is expected to have a string as an input, with a string as an output.\n",
     "- LLM: The language model powering the agent.\n",
-    "- Agent: The agent to use. This should be a string that references a support agent class. Because this notebook focuses on the simplest, highest level API, this only covers using the standard supported agents. If you want to implement a custom agent, see the documentation for [custom agents](agents/custom_agent.ipynb).\n",
+    "- Agent: The agent to use. This should be a string that references a support agent class. Because this notebook focuses on the simplest, highest level API, this only covers using the standard supported agents. If you want to implement a custom agent, see the documentation for custom agents (coming soon).\n",
     "\n",
     "**Agents**: For a list of supported agents and their specifications, see [here](agents.md).\n",
     "\n",

docs/modules/agents/toolkits/examples/csv.ipynb

@@ -35,7 +35,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 3,
    "id": "16c4dc59",
    "metadata": {},
    "outputs": [],
@@ -45,7 +45,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 4,
    "id": "46b9489d",
    "metadata": {},
    "outputs": [
@@ -58,10 +58,10 @@
       "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
       "\u001b[32;1m\u001b[1;3mThought: I need to count the number of rows\n",
       "Action: python_repl_ast\n",
-      "Action Input: df.shape[0]\u001b[0m\n",
+      "Action Input: len(df)\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3m891\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: There are 891 rows.\u001b[0m\n",
+      "Final Answer: There are 891 rows in the dataframe.\u001b[0m\n",
       "\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
@@ -69,10 +69,10 @@
     {
      "data": {
       "text/plain": [
-       "'There are 891 rows.'"
+       "'There are 891 rows in the dataframe.'"
       ]
      },
-     "execution_count": 5,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -83,7 +83,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 5,
    "id": "a96309be",
    "metadata": {},
    "outputs": [
@@ -110,7 +110,7 @@
        "'30 people have more than 3 siblings.'"
       ]
      },
-     "execution_count": 6,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -121,7 +121,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 6,
    "id": "964a09f7",
    "metadata": {},
    "outputs": [
@@ -136,15 +136,15 @@
       "Action: python_repl_ast\n",
       "Action Input: df['Age'].mean()\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3m29.69911764705882\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I can now calculate the square root\n",
       "Action: python_repl_ast\n",
       "Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3mNameError(\"name 'math' is not defined\")\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mname 'math' is not defined\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I need to import the math library\n",
       "Action: python_repl_ast\n",
       "Action Input: import math\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I can now calculate the square root\n",
       "Action: python_repl_ast\n",
       "Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3m5.449689683556195\u001b[0m\n",
@@ -160,7 +160,7 @@
        "'5.449689683556195'"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 6,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -169,59 +169,10 @@
     "agent.run(\"whats the square root of the average age?\")"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "09539c18",
-   "metadata": {},
-   "source": [
-    "### Multi CSV Example\n",
-    "\n",
-    "This next part shows how the agent can interact with multiple csv files passed in as a list."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "15f11fbd",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
-      "\u001b[32;1m\u001b[1;3mThought: I need to compare the age columns in both dataframes\n",
-      "Action: python_repl_ast\n",
-      "Action Input: len(df1[df1['Age'] != df2['Age']])\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3m177\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: 177 rows in the age column are different.\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'177 rows in the age column are different.'"
-      ]
-     },
-     "execution_count": 3,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "agent = create_csv_agent(OpenAI(temperature=0), ['titanic.csv', 'titanic_age_fillna.csv'], verbose=True)\n",
-    "agent.run(\"how many rows in the age column are different?\")"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "f2909808",
+   "id": "551de2be",
    "metadata": {},
    "outputs": [],
    "source": []

docs/modules/agents/toolkits/examples/pandas.ipynb

@@ -14,7 +14,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 7,
    "id": "0cdd9bf5",
    "metadata": {},
    "outputs": [],
@@ -60,10 +60,10 @@
       "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
       "\u001b[32;1m\u001b[1;3mThought: I need to count the number of rows\n",
       "Action: python_repl_ast\n",
-      "Action Input: df.shape[0]\u001b[0m\n",
+      "Action Input: len(df)\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3m891\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: There are 891 rows.\u001b[0m\n",
+      "Final Answer: There are 891 rows in the dataframe.\u001b[0m\n",
       "\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
@@ -71,7 +71,7 @@
     {
      "data": {
       "text/plain": [
-       "'There are 891 rows.'"
+       "'There are 891 rows in the dataframe.'"
       ]
      },
      "execution_count": 4,
@@ -138,20 +138,20 @@
       "Action: python_repl_ast\n",
       "Action Input: df['Age'].mean()\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3m29.69911764705882\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I can now calculate the square root\n",
       "Action: python_repl_ast\n",
       "Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3mNameError(\"name 'math' is not defined\")\u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mname 'math' is not defined\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I need to import the math library\n",
       "Action: python_repl_ast\n",
       "Action Input: import math\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
+      "Observation: \u001b[36;1m\u001b[1;3mNone\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m I can now calculate the square root\n",
       "Action: python_repl_ast\n",
       "Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
       "Observation: \u001b[36;1m\u001b[1;3m5.449689683556195\u001b[0m\n",
       "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: The square root of the average age is 5.449689683556195.\u001b[0m\n",
+      "Final Answer: 5.449689683556195\u001b[0m\n",
       "\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
@@ -159,7 +159,7 @@
     {
      "data": {
       "text/plain": [
-       "'The square root of the average age is 5.449689683556195.'"
+       "'5.449689683556195'"
       ]
      },
      "execution_count": 6,
@@ -171,71 +171,10 @@
     "agent.run(\"whats the square root of the average age?\")"
    ]
   },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "id": "c4bc0584",
-   "metadata": {},
-   "source": [
-    "### Multi DataFrame Example\n",
-    "\n",
-    "This next part shows how the agent can interact with multiple dataframes passed in as a list."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "42a15bd9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "df1 = df.copy()\n",
-    "df1[\"Age\"] = df1[\"Age\"].fillna(df1[\"Age\"].mean())"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "eba13b4d",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
-      "\u001b[32;1m\u001b[1;3mThought: I need to compare the age columns in both dataframes\n",
-      "Action: python_repl_ast\n",
-      "Action Input: len(df1[df1['Age'] != df2['Age']])\u001b[0m\n",
-      "Observation: \u001b[36;1m\u001b[1;3m177\u001b[0m\n",
-      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
-      "Final Answer: 177 rows in the age column are different.\u001b[0m\n",
-      "\n",
-      "\u001b[1m> Finished chain.\u001b[0m\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'177 rows in the age column are different.'"
-      ]
-     },
-     "execution_count": 8,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "agent = create_pandas_dataframe_agent(OpenAI(temperature=0), [df, df1], verbose=True)\n",
-    "agent.run(\"how many rows in the age column are different?\")"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "60d08a56",
+   "id": "eba13b4d",
    "metadata": {},
    "outputs": [],
    "source": []
@@ -257,7 +196,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,

docs/modules/agents/toolkits/examples/titanic_age_fillna.csv

@@ -1,892 +0,0 @@
-PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked
-1,0,3,"Braund, Mr. Owen Harris",male,22.0,1,0,A/5 21171,7.25,,S
-2,1,1,"Cumings, Mrs. John Bradley (Florence Briggs Thayer)",female,38.0,1,0,PC 17599,71.2833,C85,C
-3,1,3,"Heikkinen, Miss. Laina",female,26.0,0,0,STON/O2. 3101282,7.925,,S
-4,1,1,"Futrelle, Mrs. Jacques Heath (Lily May Peel)",female,35.0,1,0,113803,53.1,C123,S
-5,0,3,"Allen, Mr. William Henry",male,35.0,0,0,373450,8.05,,S
-6,0,3,"Moran, Mr. James",male,29.69911764705882,0,0,330877,8.4583,,Q
-7,0,1,"McCarthy, Mr. Timothy J",male,54.0,0,0,17463,51.8625,E46,S
-8,0,3,"Palsson, Master. Gosta Leonard",male,2.0,3,1,349909,21.075,,S
-9,1,3,"Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)",female,27.0,0,2,347742,11.1333,,S
-10,1,2,"Nasser, Mrs. Nicholas (Adele Achem)",female,14.0,1,0,237736,30.0708,,C
-11,1,3,"Sandstrom, Miss. Marguerite Rut",female,4.0,1,1,PP 9549,16.7,G6,S
-12,1,1,"Bonnell, Miss. Elizabeth",female,58.0,0,0,113783,26.55,C103,S
-13,0,3,"Saundercock, Mr. William Henry",male,20.0,0,0,A/5. 2151,8.05,,S
-14,0,3,"Andersson, Mr. Anders Johan",male,39.0,1,5,347082,31.275,,S
-15,0,3,"Vestrom, Miss. Hulda Amanda Adolfina",female,14.0,0,0,350406,7.8542,,S
-16,1,2,"Hewlett, Mrs. (Mary D Kingcome) ",female,55.0,0,0,248706,16.0,,S
-17,0,3,"Rice, Master. Eugene",male,2.0,4,1,382652,29.125,,Q
-18,1,2,"Williams, Mr. Charles Eugene",male,29.69911764705882,0,0,244373,13.0,,S
-19,0,3,"Vander Planke, Mrs. Julius (Emelia Maria Vandemoortele)",female,31.0,1,0,345763,18.0,,S
-20,1,3,"Masselmani, Mrs. Fatima",female,29.69911764705882,0,0,2649,7.225,,C
-21,0,2,"Fynney, Mr. Joseph J",male,35.0,0,0,239865,26.0,,S
-22,1,2,"Beesley, Mr. Lawrence",male,34.0,0,0,248698,13.0,D56,S
-23,1,3,"McGowan, Miss. Anna ""Annie""",female,15.0,0,0,330923,8.0292,,Q
-24,1,1,"Sloper, Mr. William Thompson",male,28.0,0,0,113788,35.5,A6,S
-25,0,3,"Palsson, Miss. Torborg Danira",female,8.0,3,1,349909,21.075,,S
-26,1,3,"Asplund, Mrs. Carl Oscar (Selma Augusta Emilia Johansson)",female,38.0,1,5,347077,31.3875,,S
-27,0,3,"Emir, Mr. Farred Chehab",male,29.69911764705882,0,0,2631,7.225,,C
-28,0,1,"Fortune, Mr. Charles Alexander",male,19.0,3,2,19950,263.0,C23 C25 C27,S
-29,1,3,"O'Dwyer, Miss. Ellen ""Nellie""",female,29.69911764705882,0,0,330959,7.8792,,Q
-30,0,3,"Todoroff, Mr. Lalio",male,29.69911764705882,0,0,349216,7.8958,,S
-31,0,1,"Uruchurtu, Don. Manuel E",male,40.0,0,0,PC 17601,27.7208,,C
-32,1,1,"Spencer, Mrs. William Augustus (Marie Eugenie)",female,29.69911764705882,1,0,PC 17569,146.5208,B78,C
-33,1,3,"Glynn, Miss. Mary Agatha",female,29.69911764705882,0,0,335677,7.75,,Q
-34,0,2,"Wheadon, Mr. Edward H",male,66.0,0,0,C.A. 24579,10.5,,S
-35,0,1,"Meyer, Mr. Edgar Joseph",male,28.0,1,0,PC 17604,82.1708,,C
-36,0,1,"Holverson, Mr. Alexander Oskar",male,42.0,1,0,113789,52.0,,S
-37,1,3,"Mamee, Mr. Hanna",male,29.69911764705882,0,0,2677,7.2292,,C
-38,0,3,"Cann, Mr. Ernest Charles",male,21.0,0,0,A./5. 2152,8.05,,S
-39,0,3,"Vander Planke, Miss. Augusta Maria",female,18.0,2,0,345764,18.0,,S
-40,1,3,"Nicola-Yarred, Miss. Jamila",female,14.0,1,0,2651,11.2417,,C
-41,0,3,"Ahlin, Mrs. Johan (Johanna Persdotter Larsson)",female,40.0,1,0,7546,9.475,,S
-42,0,2,"Turpin, Mrs. William John Robert (Dorothy Ann Wonnacott)",female,27.0,1,0,11668,21.0,,S
-43,0,3,"Kraeff, Mr. Theodor",male,29.69911764705882,0,0,349253,7.8958,,C
-44,1,2,"Laroche, Miss. Simonne Marie Anne Andree",female,3.0,1,2,SC/Paris 2123,41.5792,,C
-45,1,3,"Devaney, Miss. Margaret Delia",female,19.0,0,0,330958,7.8792,,Q
-46,0,3,"Rogers, Mr. William John",male,29.69911764705882,0,0,S.C./A.4. 23567,8.05,,S
-47,0,3,"Lennon, Mr. Denis",male,29.69911764705882,1,0,370371,15.5,,Q
-48,1,3,"O'Driscoll, Miss. Bridget",female,29.69911764705882,0,0,14311,7.75,,Q
-49,0,3,"Samaan, Mr. Youssef",male,29.69911764705882,2,0,2662,21.6792,,C
-50,0,3,"Arnold-Franchi, Mrs. Josef (Josefine Franchi)",female,18.0,1,0,349237,17.8,,S
-51,0,3,"Panula, Master. Juha Niilo",male,7.0,4,1,3101295,39.6875,,S
-52,0,3,"Nosworthy, Mr. Richard Cater",male,21.0,0,0,A/4. 39886,7.8,,S
-53,1,1,"Harper, Mrs. Henry Sleeper (Myna Haxtun)",female,49.0,1,0,PC 17572,76.7292,D33,C
-54,1,2,"Faunthorpe, Mrs. Lizzie (Elizabeth Anne Wilkinson)",female,29.0,1,0,2926,26.0,,S
-55,0,1,"Ostby, Mr. Engelhart Cornelius",male,65.0,0,1,113509,61.9792,B30,C
-56,1,1,"Woolner, Mr. Hugh",male,29.69911764705882,0,0,19947,35.5,C52,S
-57,1,2,"Rugg, Miss. Emily",female,21.0,0,0,C.A. 31026,10.5,,S
-58,0,3,"Novel, Mr. Mansouer",male,28.5,0,0,2697,7.2292,,C
-59,1,2,"West, Miss. Constance Mirium",female,5.0,1,2,C.A. 34651,27.75,,S
-60,0,3,"Goodwin, Master. William Frederick",male,11.0,5,2,CA 2144,46.9,,S
-61,0,3,"Sirayanian, Mr. Orsen",male,22.0,0,0,2669,7.2292,,C
-62,1,1,"Icard, Miss. Amelie",female,38.0,0,0,113572,80.0,B28,
-63,0,1,"Harris, Mr. Henry Birkhardt",male,45.0,1,0,36973,83.475,C83,S
-64,0,3,"Skoog, Master. Harald",male,4.0,3,2,347088,27.9,,S
-65,0,1,"Stewart, Mr. Albert A",male,29.69911764705882,0,0,PC 17605,27.7208,,C
-66,1,3,"Moubarek, Master. Gerios",male,29.69911764705882,1,1,2661,15.2458,,C
-67,1,2,"Nye, Mrs. (Elizabeth Ramell)",female,29.0,0,0,C.A. 29395,10.5,F33,S
-68,0,3,"Crease, Mr. Ernest James",male,19.0,0,0,S.P. 3464,8.1583,,S
-69,1,3,"Andersson, Miss. Erna Alexandra",female,17.0,4,2,3101281,7.925,,S
-70,0,3,"Kink, Mr. Vincenz",male,26.0,2,0,315151,8.6625,,S
-71,0,2,"Jenkin, Mr. Stephen Curnow",male,32.0,0,0,C.A. 33111,10.5,,S
-72,0,3,"Goodwin, Miss. Lillian Amy",female,16.0,5,2,CA 2144,46.9,,S
-73,0,2,"Hood, Mr. Ambrose Jr",male,21.0,0,0,S.O.C. 14879,73.5,,S
-74,0,3,"Chronopoulos, Mr. Apostolos",male,26.0,1,0,2680,14.4542,,C
-75,1,3,"Bing, Mr. Lee",male,32.0,0,0,1601,56.4958,,S
-76,0,3,"Moen, Mr. Sigurd Hansen",male,25.0,0,0,348123,7.65,F G73,S
-77,0,3,"Staneff, Mr. Ivan",male,29.69911764705882,0,0,349208,7.8958,,S
-78,0,3,"Moutal, Mr. Rahamin Haim",male,29.69911764705882,0,0,374746,8.05,,S
-79,1,2,"Caldwell, Master. Alden Gates",male,0.83,0,2,248738,29.0,,S
-80,1,3,"Dowdell, Miss. Elizabeth",female,30.0,0,0,364516,12.475,,S
-81,0,3,"Waelens, Mr. Achille",male,22.0,0,0,345767,9.0,,S
-82,1,3,"Sheerlinck, Mr. Jan Baptist",male,29.0,0,0,345779,9.5,,S
-83,1,3,"McDermott, Miss. Brigdet Delia",female,29.69911764705882,0,0,330932,7.7875,,Q
-84,0,1,"Carrau, Mr. Francisco M",male,28.0,0,0,113059,47.1,,S
-85,1,2,"Ilett, Miss. Bertha",female,17.0,0,0,SO/C 14885,10.5,,S
-86,1,3,"Backstrom, Mrs. Karl Alfred (Maria Mathilda Gustafsson)",female,33.0,3,0,3101278,15.85,,S
-87,0,3,"Ford, Mr. William Neal",male,16.0,1,3,W./C. 6608,34.375,,S
-88,0,3,"Slocovski, Mr. Selman Francis",male,29.69911764705882,0,0,SOTON/OQ 392086,8.05,,S
-89,1,1,"Fortune, Miss. Mabel Helen",female,23.0,3,2,19950,263.0,C23 C25 C27,S
-90,0,3,"Celotti, Mr. Francesco",male,24.0,0,0,343275,8.05,,S
-91,0,3,"Christmann, Mr. Emil",male,29.0,0,0,343276,8.05,,S
-92,0,3,"Andreasson, Mr. Paul Edvin",male,20.0,0,0,347466,7.8542,,S
-93,0,1,"Chaffee, Mr. Herbert Fuller",male,46.0,1,0,W.E.P. 5734,61.175,E31,S
-94,0,3,"Dean, Mr. Bertram Frank",male,26.0,1,2,C.A. 2315,20.575,,S
-95,0,3,"Coxon, Mr. Daniel",male,59.0,0,0,364500,7.25,,S
-96,0,3,"Shorney, Mr. Charles Joseph",male,29.69911764705882,0,0,374910,8.05,,S
-97,0,1,"Goldschmidt, Mr. George B",male,71.0,0,0,PC 17754,34.6542,A5,C
-98,1,1,"Greenfield, Mr. William Bertram",male,23.0,0,1,PC 17759,63.3583,D10 D12,C
-99,1,2,"Doling, Mrs. John T (Ada Julia Bone)",female,34.0,0,1,231919,23.0,,S
-100,0,2,"Kantor, Mr. Sinai",male,34.0,1,0,244367,26.0,,S
-101,0,3,"Petranec, Miss. Matilda",female,28.0,0,0,349245,7.8958,,S
-102,0,3,"Petroff, Mr. Pastcho (""Pentcho"")",male,29.69911764705882,0,0,349215,7.8958,,S
-103,0,1,"White, Mr. Richard Frasar",male,21.0,0,1,35281,77.2875,D26,S
-104,0,3,"Johansson, Mr. Gustaf Joel",male,33.0,0,0,7540,8.6542,,S
-105,0,3,"Gustafsson, Mr. Anders Vilhelm",male,37.0,2,0,3101276,7.925,,S
-106,0,3,"Mionoff, Mr. Stoytcho",male,28.0,0,0,349207,7.8958,,S
-107,1,3,"Salkjelsvik, Miss. Anna Kristine",female,21.0,0,0,343120,7.65,,S
-108,1,3,"Moss, Mr. Albert Johan",male,29.69911764705882,0,0,312991,7.775,,S
-109,0,3,"Rekic, Mr. Tido",male,38.0,0,0,349249,7.8958,,S
-110,1,3,"Moran, Miss. Bertha",female,29.69911764705882,1,0,371110,24.15,,Q
-111,0,1,"Porter, Mr. Walter Chamberlain",male,47.0,0,0,110465,52.0,C110,S
-112,0,3,"Zabour, Miss. Hileni",female,14.5,1,0,2665,14.4542,,C
-113,0,3,"Barton, Mr. David John",male,22.0,0,0,324669,8.05,,S
-114,0,3,"Jussila, Miss. Katriina",female,20.0,1,0,4136,9.825,,S
-115,0,3,"Attalah, Miss. Malake",female,17.0,0,0,2627,14.4583,,C
-116,0,3,"Pekoniemi, Mr. Edvard",male,21.0,0,0,STON/O 2. 3101294,7.925,,S
-117,0,3,"Connors, Mr. Patrick",male,70.5,0,0,370369,7.75,,Q
-118,0,2,"Turpin, Mr. William John Robert",male,29.0,1,0,11668,21.0,,S
-119,0,1,"Baxter, Mr. Quigg Edmond",male,24.0,0,1,PC 17558,247.5208,B58 B60,C
-120,0,3,"Andersson, Miss. Ellis Anna Maria",female,2.0,4,2,347082,31.275,,S
-121,0,2,"Hickman, Mr. Stanley George",male,21.0,2,0,S.O.C. 14879,73.5,,S
-122,0,3,"Moore, Mr. Leonard Charles",male,29.69911764705882,0,0,A4. 54510,8.05,,S
-123,0,2,"Nasser, Mr. Nicholas",male,32.5,1,0,237736,30.0708,,C
-124,1,2,"Webber, Miss. Susan",female,32.5,0,0,27267,13.0,E101,S
-125,0,1,"White, Mr. Percival Wayland",male,54.0,0,1,35281,77.2875,D26,S
-126,1,3,"Nicola-Yarred, Master. Elias",male,12.0,1,0,2651,11.2417,,C
-127,0,3,"McMahon, Mr. Martin",male,29.69911764705882,0,0,370372,7.75,,Q
-128,1,3,"Madsen, Mr. Fridtjof Arne",male,24.0,0,0,C 17369,7.1417,,S
-129,1,3,"Peter, Miss. Anna",female,29.69911764705882,1,1,2668,22.3583,F E69,C
-130,0,3,"Ekstrom, Mr. Johan",male,45.0,0,0,347061,6.975,,S
-131,0,3,"Drazenoic, Mr. Jozef",male,33.0,0,0,349241,7.8958,,C
-132,0,3,"Coelho, Mr. Domingos Fernandeo",male,20.0,0,0,SOTON/O.Q. 3101307,7.05,,S
-133,0,3,"Robins, Mrs. Alexander A (Grace Charity Laury)",female,47.0,1,0,A/5. 3337,14.5,,S
-134,1,2,"Weisz, Mrs. Leopold (Mathilde Francoise Pede)",female,29.0,1,0,228414,26.0,,S
-135,0,2,"Sobey, Mr. Samuel James Hayden",male,25.0,0,0,C.A. 29178,13.0,,S
-136,0,2,"Richard, Mr. Emile",male,23.0,0,0,SC/PARIS 2133,15.0458,,C
-137,1,1,"Newsom, Miss. Helen Monypeny",female,19.0,0,2,11752,26.2833,D47,S
-138,0,1,"Futrelle, Mr. Jacques Heath",male,37.0,1,0,113803,53.1,C123,S
-139,0,3,"Osen, Mr. Olaf Elon",male,16.0,0,0,7534,9.2167,,S
-140,0,1,"Giglio, Mr. Victor",male,24.0,0,0,PC 17593,79.2,B86,C
-141,0,3,"Boulos, Mrs. Joseph (Sultana)",female,29.69911764705882,0,2,2678,15.2458,,C
-142,1,3,"Nysten, Miss. Anna Sofia",female,22.0,0,0,347081,7.75,,S
-143,1,3,"Hakkarainen, Mrs. Pekka Pietari (Elin Matilda Dolck)",female,24.0,1,0,STON/O2. 3101279,15.85,,S
-144,0,3,"Burke, Mr. Jeremiah",male,19.0,0,0,365222,6.75,,Q
-145,0,2,"Andrew, Mr. Edgardo Samuel",male,18.0,0,0,231945,11.5,,S
-146,0,2,"Nicholls, Mr. Joseph Charles",male,19.0,1,1,C.A. 33112,36.75,,S
-147,1,3,"Andersson, Mr. August Edvard (""Wennerstrom"")",male,27.0,0,0,350043,7.7958,,S
-148,0,3,"Ford, Miss. Robina Maggie ""Ruby""",female,9.0,2,2,W./C. 6608,34.375,,S
-149,0,2,"Navratil, Mr. Michel (""Louis M Hoffman"")",male,36.5,0,2,230080,26.0,F2,S
-150,0,2,"Byles, Rev. Thomas Roussel Davids",male,42.0,0,0,244310,13.0,,S
-151,0,2,"Bateman, Rev. Robert James",male,51.0,0,0,S.O.P. 1166,12.525,,S
-152,1,1,"Pears, Mrs. Thomas (Edith Wearne)",female,22.0,1,0,113776,66.6,C2,S
-153,0,3,"Meo, Mr. Alfonzo",male,55.5,0,0,A.5. 11206,8.05,,S
-154,0,3,"van Billiard, Mr. Austin Blyler",male,40.5,0,2,A/5. 851,14.5,,S
-155,0,3,"Olsen, Mr. Ole Martin",male,29.69911764705882,0,0,Fa 265302,7.3125,,S
-156,0,1,"Williams, Mr. Charles Duane",male,51.0,0,1,PC 17597,61.3792,,C
-157,1,3,"Gilnagh, Miss. Katherine ""Katie""",female,16.0,0,0,35851,7.7333,,Q
-158,0,3,"Corn, Mr. Harry",male,30.0,0,0,SOTON/OQ 392090,8.05,,S
-159,0,3,"Smiljanic, Mr. Mile",male,29.69911764705882,0,0,315037,8.6625,,S
-160,0,3,"Sage, Master. Thomas Henry",male,29.69911764705882,8,2,CA. 2343,69.55,,S
-161,0,3,"Cribb, Mr. John Hatfield",male,44.0,0,1,371362,16.1,,S
-162,1,2,"Watt, Mrs. James (Elizabeth ""Bessie"" Inglis Milne)",female,40.0,0,0,C.A. 33595,15.75,,S
-163,0,3,"Bengtsson, Mr. John Viktor",male,26.0,0,0,347068,7.775,,S
-164,0,3,"Calic, Mr. Jovo",male,17.0,0,0,315093,8.6625,,S
-165,0,3,"Panula, Master. Eino Viljami",male,1.0,4,1,3101295,39.6875,,S
-166,1,3,"Goldsmith, Master. Frank John William ""Frankie""",male,9.0,0,2,363291,20.525,,S
-167,1,1,"Chibnall, Mrs. (Edith Martha Bowerman)",female,29.69911764705882,0,1,113505,55.0,E33,S
-168,0,3,"Skoog, Mrs. William (Anna Bernhardina Karlsson)",female,45.0,1,4,347088,27.9,,S
-169,0,1,"Baumann, Mr. John D",male,29.69911764705882,0,0,PC 17318,25.925,,S
-170,0,3,"Ling, Mr. Lee",male,28.0,0,0,1601,56.4958,,S
-171,0,1,"Van der hoef, Mr. Wyckoff",male,61.0,0,0,111240,33.5,B19,S
-172,0,3,"Rice, Master. Arthur",male,4.0,4,1,382652,29.125,,Q
-173,1,3,"Johnson, Miss. Eleanor Ileen",female,1.0,1,1,347742,11.1333,,S
-174,0,3,"Sivola, Mr. Antti Wilhelm",male,21.0,0,0,STON/O 2. 3101280,7.925,,S
-175,0,1,"Smith, Mr. James Clinch",male,56.0,0,0,17764,30.6958,A7,C
-176,0,3,"Klasen, Mr. Klas Albin",male,18.0,1,1,350404,7.8542,,S
-177,0,3,"Lefebre, Master. Henry Forbes",male,29.69911764705882,3,1,4133,25.4667,,S
-178,0,1,"Isham, Miss. Ann Elizabeth",female,50.0,0,0,PC 17595,28.7125,C49,C
-179,0,2,"Hale, Mr. Reginald",male,30.0,0,0,250653,13.0,,S
-180,0,3,"Leonard, Mr. Lionel",male,36.0,0,0,LINE,0.0,,S
-181,0,3,"Sage, Miss. Constance Gladys",female,29.69911764705882,8,2,CA. 2343,69.55,,S
-182,0,2,"Pernot, Mr. Rene",male,29.69911764705882,0,0,SC/PARIS 2131,15.05,,C
-183,0,3,"Asplund, Master. Clarence Gustaf Hugo",male,9.0,4,2,347077,31.3875,,S
-184,1,2,"Becker, Master. Richard F",male,1.0,2,1,230136,39.0,F4,S
-185,1,3,"Kink-Heilmann, Miss. Luise Gretchen",female,4.0,0,2,315153,22.025,,S
-186,0,1,"Rood, Mr. Hugh Roscoe",male,29.69911764705882,0,0,113767,50.0,A32,S
-187,1,3,"O'Brien, Mrs. Thomas (Johanna ""Hannah"" Godfrey)",female,29.69911764705882,1,0,370365,15.5,,Q
-188,1,1,"Romaine, Mr. Charles Hallace (""Mr C Rolmane"")",male,45.0,0,0,111428,26.55,,S
-189,0,3,"Bourke, Mr. John",male,40.0,1,1,364849,15.5,,Q
-190,0,3,"Turcin, Mr. Stjepan",male,36.0,0,0,349247,7.8958,,S
-191,1,2,"Pinsky, Mrs. (Rosa)",female,32.0,0,0,234604,13.0,,S
-192,0,2,"Carbines, Mr. William",male,19.0,0,0,28424,13.0,,S
-193,1,3,"Andersen-Jensen, Miss. Carla Christine Nielsine",female,19.0,1,0,350046,7.8542,,S
-194,1,2,"Navratil, Master. Michel M",male,3.0,1,1,230080,26.0,F2,S
-195,1,1,"Brown, Mrs. James Joseph (Margaret Tobin)",female,44.0,0,0,PC 17610,27.7208,B4,C
-196,1,1,"Lurette, Miss. Elise",female,58.0,0,0,PC 17569,146.5208,B80,C
-197,0,3,"Mernagh, Mr. Robert",male,29.69911764705882,0,0,368703,7.75,,Q
-198,0,3,"Olsen, Mr. Karl Siegwart Andreas",male,42.0,0,1,4579,8.4042,,S
-199,1,3,"Madigan, Miss. Margaret ""Maggie""",female,29.69911764705882,0,0,370370,7.75,,Q
-200,0,2,"Yrois, Miss. Henriette (""Mrs Harbeck"")",female,24.0,0,0,248747,13.0,,S
-201,0,3,"Vande Walle, Mr. Nestor Cyriel",male,28.0,0,0,345770,9.5,,S
-202,0,3,"Sage, Mr. Frederick",male,29.69911764705882,8,2,CA. 2343,69.55,,S
-203,0,3,"Johanson, Mr. Jakob Alfred",male,34.0,0,0,3101264,6.4958,,S
-204,0,3,"Youseff, Mr. Gerious",male,45.5,0,0,2628,7.225,,C
-205,1,3,"Cohen, Mr. Gurshon ""Gus""",male,18.0,0,0,A/5 3540,8.05,,S
-206,0,3,"Strom, Miss. Telma Matilda",female,2.0,0,1,347054,10.4625,G6,S
-207,0,3,"Backstrom, Mr. Karl Alfred",male,32.0,1,0,3101278,15.85,,S
-208,1,3,"Albimona, Mr. Nassef Cassem",male,26.0,0,0,2699,18.7875,,C
-209,1,3,"Carr, Miss. Helen ""Ellen""",female,16.0,0,0,367231,7.75,,Q
-210,1,1,"Blank, Mr. Henry",male,40.0,0,0,112277,31.0,A31,C
-211,0,3,"Ali, Mr. Ahmed",male,24.0,0,0,SOTON/O.Q. 3101311,7.05,,S
-212,1,2,"Cameron, Miss. Clear Annie",female,35.0,0,0,F.C.C. 13528,21.0,,S
-213,0,3,"Perkin, Mr. John Henry",male,22.0,0,0,A/5 21174,7.25,,S
-214,0,2,"Givard, Mr. Hans Kristensen",male,30.0,0,0,250646,13.0,,S
-215,0,3,"Kiernan, Mr. Philip",male,29.69911764705882,1,0,367229,7.75,,Q
-216,1,1,"Newell, Miss. Madeleine",female,31.0,1,0,35273,113.275,D36,C
-217,1,3,"Honkanen, Miss. Eliina",female,27.0,0,0,STON/O2. 3101283,7.925,,S
-218,0,2,"Jacobsohn, Mr. Sidney Samuel",male,42.0,1,0,243847,27.0,,S
-219,1,1,"Bazzani, Miss. Albina",female,32.0,0,0,11813,76.2917,D15,C
-220,0,2,"Harris, Mr. Walter",male,30.0,0,0,W/C 14208,10.5,,S
-221,1,3,"Sunderland, Mr. Victor Francis",male,16.0,0,0,SOTON/OQ 392089,8.05,,S
-222,0,2,"Bracken, Mr. James H",male,27.0,0,0,220367,13.0,,S
-223,0,3,"Green, Mr. George Henry",male,51.0,0,0,21440,8.05,,S
-224,0,3,"Nenkoff, Mr. Christo",male,29.69911764705882,0,0,349234,7.8958,,S
-225,1,1,"Hoyt, Mr. Frederick Maxfield",male,38.0,1,0,19943,90.0,C93,S
-226,0,3,"Berglund, Mr. Karl Ivar Sven",male,22.0,0,0,PP 4348,9.35,,S
-227,1,2,"Mellors, Mr. William John",male,19.0,0,0,SW/PP 751,10.5,,S
-228,0,3,"Lovell, Mr. John Hall (""Henry"")",male,20.5,0,0,A/5 21173,7.25,,S
-229,0,2,"Fahlstrom, Mr. Arne Jonas",male,18.0,0,0,236171,13.0,,S
-230,0,3,"Lefebre, Miss. Mathilde",female,29.69911764705882,3,1,4133,25.4667,,S
-231,1,1,"Harris, Mrs. Henry Birkhardt (Irene Wallach)",female,35.0,1,0,36973,83.475,C83,S
-232,0,3,"Larsson, Mr. Bengt Edvin",male,29.0,0,0,347067,7.775,,S
-233,0,2,"Sjostedt, Mr. Ernst Adolf",male,59.0,0,0,237442,13.5,,S
-234,1,3,"Asplund, Miss. Lillian Gertrud",female,5.0,4,2,347077,31.3875,,S
-235,0,2,"Leyson, Mr. Robert William Norman",male,24.0,0,0,C.A. 29566,10.5,,S
-236,0,3,"Harknett, Miss. Alice Phoebe",female,29.69911764705882,0,0,W./C. 6609,7.55,,S
-237,0,2,"Hold, Mr. Stephen",male,44.0,1,0,26707,26.0,,S
-238,1,2,"Collyer, Miss. Marjorie ""Lottie""",female,8.0,0,2,C.A. 31921,26.25,,S
-239,0,2,"Pengelly, Mr. Frederick William",male,19.0,0,0,28665,10.5,,S
-240,0,2,"Hunt, Mr. George Henry",male,33.0,0,0,SCO/W 1585,12.275,,S
-241,0,3,"Zabour, Miss. Thamine",female,29.69911764705882,1,0,2665,14.4542,,C
-242,1,3,"Murphy, Miss. Katherine ""Kate""",female,29.69911764705882,1,0,367230,15.5,,Q
-243,0,2,"Coleridge, Mr. Reginald Charles",male,29.0,0,0,W./C. 14263,10.5,,S
-244,0,3,"Maenpaa, Mr. Matti Alexanteri",male,22.0,0,0,STON/O 2. 3101275,7.125,,S
-245,0,3,"Attalah, Mr. Sleiman",male,30.0,0,0,2694,7.225,,C
-246,0,1,"Minahan, Dr. William Edward",male,44.0,2,0,19928,90.0,C78,Q
-247,0,3,"Lindahl, Miss. Agda Thorilda Viktoria",female,25.0,0,0,347071,7.775,,S
-248,1,2,"Hamalainen, Mrs. William (Anna)",female,24.0,0,2,250649,14.5,,S
-249,1,1,"Beckwith, Mr. Richard Leonard",male,37.0,1,1,11751,52.5542,D35,S
-250,0,2,"Carter, Rev. Ernest Courtenay",male,54.0,1,0,244252,26.0,,S
-251,0,3,"Reed, Mr. James George",male,29.69911764705882,0,0,362316,7.25,,S
-252,0,3,"Strom, Mrs. Wilhelm (Elna Matilda Persson)",female,29.0,1,1,347054,10.4625,G6,S
-253,0,1,"Stead, Mr. William Thomas",male,62.0,0,0,113514,26.55,C87,S
-254,0,3,"Lobb, Mr. William Arthur",male,30.0,1,0,A/5. 3336,16.1,,S
-255,0,3,"Rosblom, Mrs. Viktor (Helena Wilhelmina)",female,41.0,0,2,370129,20.2125,,S
-256,1,3,"Touma, Mrs. Darwis (Hanne Youssef Razi)",female,29.0,0,2,2650,15.2458,,C
-257,1,1,"Thorne, Mrs. Gertrude Maybelle",female,29.69911764705882,0,0,PC 17585,79.2,,C
-258,1,1,"Cherry, Miss. Gladys",female,30.0,0,0,110152,86.5,B77,S
-259,1,1,"Ward, Miss. Anna",female,35.0,0,0,PC 17755,512.3292,,C
-260,1,2,"Parrish, Mrs. (Lutie Davis)",female,50.0,0,1,230433,26.0,,S
-261,0,3,"Smith, Mr. Thomas",male,29.69911764705882,0,0,384461,7.75,,Q
-262,1,3,"Asplund, Master. Edvin Rojj Felix",male,3.0,4,2,347077,31.3875,,S
-263,0,1,"Taussig, Mr. Emil",male,52.0,1,1,110413,79.65,E67,S
-264,0,1,"Harrison, Mr. William",male,40.0,0,0,112059,0.0,B94,S
-265,0,3,"Henry, Miss. Delia",female,29.69911764705882,0,0,382649,7.75,,Q
-266,0,2,"Reeves, Mr. David",male,36.0,0,0,C.A. 17248,10.5,,S
-267,0,3,"Panula, Mr. Ernesti Arvid",male,16.0,4,1,3101295,39.6875,,S
-268,1,3,"Persson, Mr. Ernst Ulrik",male,25.0,1,0,347083,7.775,,S
-269,1,1,"Graham, Mrs. William Thompson (Edith Junkins)",female,58.0,0,1,PC 17582,153.4625,C125,S
-270,1,1,"Bissette, Miss. Amelia",female,35.0,0,0,PC 17760,135.6333,C99,S
-271,0,1,"Cairns, Mr. Alexander",male,29.69911764705882,0,0,113798,31.0,,S
-272,1,3,"Tornquist, Mr. William Henry",male,25.0,0,0,LINE,0.0,,S
-273,1,2,"Mellinger, Mrs. (Elizabeth Anne Maidment)",female,41.0,0,1,250644,19.5,,S
-274,0,1,"Natsch, Mr. Charles H",male,37.0,0,1,PC 17596,29.7,C118,C
-275,1,3,"Healy, Miss. Hanora ""Nora""",female,29.69911764705882,0,0,370375,7.75,,Q
-276,1,1,"Andrews, Miss. Kornelia Theodosia",female,63.0,1,0,13502,77.9583,D7,S
-277,0,3,"Lindblom, Miss. Augusta Charlotta",female,45.0,0,0,347073,7.75,,S
-278,0,2,"Parkes, Mr. Francis ""Frank""",male,29.69911764705882,0,0,239853,0.0,,S
-279,0,3,"Rice, Master. Eric",male,7.0,4,1,382652,29.125,,Q
-280,1,3,"Abbott, Mrs. Stanton (Rosa Hunt)",female,35.0,1,1,C.A. 2673,20.25,,S
-281,0,3,"Duane, Mr. Frank",male,65.0,0,0,336439,7.75,,Q
-282,0,3,"Olsson, Mr. Nils Johan Goransson",male,28.0,0,0,347464,7.8542,,S
-283,0,3,"de Pelsmaeker, Mr. Alfons",male,16.0,0,0,345778,9.5,,S
-284,1,3,"Dorking, Mr. Edward Arthur",male,19.0,0,0,A/5. 10482,8.05,,S
-285,0,1,"Smith, Mr. Richard William",male,29.69911764705882,0,0,113056,26.0,A19,S
-286,0,3,"Stankovic, Mr. Ivan",male,33.0,0,0,349239,8.6625,,C
-287,1,3,"de Mulder, Mr. Theodore",male,30.0,0,0,345774,9.5,,S
-288,0,3,"Naidenoff, Mr. Penko",male,22.0,0,0,349206,7.8958,,S
-289,1,2,"Hosono, Mr. Masabumi",male,42.0,0,0,237798,13.0,,S
-290,1,3,"Connolly, Miss. Kate",female,22.0,0,0,370373,7.75,,Q
-291,1,1,"Barber, Miss. Ellen ""Nellie""",female,26.0,0,0,19877,78.85,,S
-292,1,1,"Bishop, Mrs. Dickinson H (Helen Walton)",female,19.0,1,0,11967,91.0792,B49,C
-293,0,2,"Levy, Mr. Rene Jacques",male,36.0,0,0,SC/Paris 2163,12.875,D,C
-294,0,3,"Haas, Miss. Aloisia",female,24.0,0,0,349236,8.85,,S
-295,0,3,"Mineff, Mr. Ivan",male,24.0,0,0,349233,7.8958,,S
-296,0,1,"Lewy, Mr. Ervin G",male,29.69911764705882,0,0,PC 17612,27.7208,,C
-297,0,3,"Hanna, Mr. Mansour",male,23.5,0,0,2693,7.2292,,C
-298,0,1,"Allison, Miss. Helen Loraine",female,2.0,1,2,113781,151.55,C22 C26,S
-299,1,1,"Saalfeld, Mr. Adolphe",male,29.69911764705882,0,0,19988,30.5,C106,S
-300,1,1,"Baxter, Mrs. James (Helene DeLaudeniere Chaput)",female,50.0,0,1,PC 17558,247.5208,B58 B60,C
-301,1,3,"Kelly, Miss. Anna Katherine ""Annie Kate""",female,29.69911764705882,0,0,9234,7.75,,Q
-302,1,3,"McCoy, Mr. Bernard",male,29.69911764705882,2,0,367226,23.25,,Q
-303,0,3,"Johnson, Mr. William Cahoone Jr",male,19.0,0,0,LINE,0.0,,S
-304,1,2,"Keane, Miss. Nora A",female,29.69911764705882,0,0,226593,12.35,E101,Q
-305,0,3,"Williams, Mr. Howard Hugh ""Harry""",male,29.69911764705882,0,0,A/5 2466,8.05,,S
-306,1,1,"Allison, Master. Hudson Trevor",male,0.92,1,2,113781,151.55,C22 C26,S
-307,1,1,"Fleming, Miss. Margaret",female,29.69911764705882,0,0,17421,110.8833,,C
-308,1,1,"Penasco y Castellana, Mrs. Victor de Satode (Maria Josefa Perez de Soto y Vallejo)",female,17.0,1,0,PC 17758,108.9,C65,C
-309,0,2,"Abelson, Mr. Samuel",male,30.0,1,0,P/PP 3381,24.0,,C
-310,1,1,"Francatelli, Miss. Laura Mabel",female,30.0,0,0,PC 17485,56.9292,E36,C
-311,1,1,"Hays, Miss. Margaret Bechstein",female,24.0,0,0,11767,83.1583,C54,C
-312,1,1,"Ryerson, Miss. Emily Borie",female,18.0,2,2,PC 17608,262.375,B57 B59 B63 B66,C
-313,0,2,"Lahtinen, Mrs. William (Anna Sylfven)",female,26.0,1,1,250651,26.0,,S
-314,0,3,"Hendekovic, Mr. Ignjac",male,28.0,0,0,349243,7.8958,,S
-315,0,2,"Hart, Mr. Benjamin",male,43.0,1,1,F.C.C. 13529,26.25,,S
-316,1,3,"Nilsson, Miss. Helmina Josefina",female,26.0,0,0,347470,7.8542,,S
-317,1,2,"Kantor, Mrs. Sinai (Miriam Sternin)",female,24.0,1,0,244367,26.0,,S
-318,0,2,"Moraweck, Dr. Ernest",male,54.0,0,0,29011,14.0,,S
-319,1,1,"Wick, Miss. Mary Natalie",female,31.0,0,2,36928,164.8667,C7,S
-320,1,1,"Spedden, Mrs. Frederic Oakley (Margaretta Corning Stone)",female,40.0,1,1,16966,134.5,E34,C
-321,0,3,"Dennis, Mr. Samuel",male,22.0,0,0,A/5 21172,7.25,,S
-322,0,3,"Danoff, Mr. Yoto",male,27.0,0,0,349219,7.8958,,S
-323,1,2,"Slayter, Miss. Hilda Mary",female,30.0,0,0,234818,12.35,,Q
-324,1,2,"Caldwell, Mrs. Albert Francis (Sylvia Mae Harbaugh)",female,22.0,1,1,248738,29.0,,S
-325,0,3,"Sage, Mr. George John Jr",male,29.69911764705882,8,2,CA. 2343,69.55,,S
-326,1,1,"Young, Miss. Marie Grice",female,36.0,0,0,PC 17760,135.6333,C32,C
-327,0,3,"Nysveen, Mr. Johan Hansen",male,61.0,0,0,345364,6.2375,,S
-328,1,2,"Ball, Mrs. (Ada E Hall)",female,36.0,0,0,28551,13.0,D,S
-329,1,3,"Goldsmith, Mrs. Frank John (Emily Alice Brown)",female,31.0,1,1,363291,20.525,,S
-330,1,1,"Hippach, Miss. Jean Gertrude",female,16.0,0,1,111361,57.9792,B18,C
-331,1,3,"McCoy, Miss. Agnes",female,29.69911764705882,2,0,367226,23.25,,Q
-332,0,1,"Partner, Mr. Austen",male,45.5,0,0,113043,28.5,C124,S
-333,0,1,"Graham, Mr. George Edward",male,38.0,0,1,PC 17582,153.4625,C91,S
-334,0,3,"Vander Planke, Mr. Leo Edmondus",male,16.0,2,0,345764,18.0,,S
-335,1,1,"Frauenthal, Mrs. Henry William (Clara Heinsheimer)",female,29.69911764705882,1,0,PC 17611,133.65,,S
-336,0,3,"Denkoff, Mr. Mitto",male,29.69911764705882,0,0,349225,7.8958,,S
-337,0,1,"Pears, Mr. Thomas Clinton",male,29.0,1,0,113776,66.6,C2,S
-338,1,1,"Burns, Miss. Elizabeth Margaret",female,41.0,0,0,16966,134.5,E40,C
-339,1,3,"Dahl, Mr. Karl Edwart",male,45.0,0,0,7598,8.05,,S
-340,0,1,"Blackwell, Mr. Stephen Weart",male,45.0,0,0,113784,35.5,T,S
-341,1,2,"Navratil, Master. Edmond Roger",male,2.0,1,1,230080,26.0,F2,S
-342,1,1,"Fortune, Miss. Alice Elizabeth",female,24.0,3,2,19950,263.0,C23 C25 C27,S
-343,0,2,"Collander, Mr. Erik Gustaf",male,28.0,0,0,248740,13.0,,S
-344,0,2,"Sedgwick, Mr. Charles Frederick Waddington",male,25.0,0,0,244361,13.0,,S
-345,0,2,"Fox, Mr. Stanley Hubert",male,36.0,0,0,229236,13.0,,S
-346,1,2,"Brown, Miss. Amelia ""Mildred""",female,24.0,0,0,248733,13.0,F33,S
-347,1,2,"Smith, Miss. Marion Elsie",female,40.0,0,0,31418,13.0,,S
-348,1,3,"Davison, Mrs. Thomas Henry (Mary E Finck)",female,29.69911764705882,1,0,386525,16.1,,S
-349,1,3,"Coutts, Master. William Loch ""William""",male,3.0,1,1,C.A. 37671,15.9,,S
-350,0,3,"Dimic, Mr. Jovan",male,42.0,0,0,315088,8.6625,,S
-351,0,3,"Odahl, Mr. Nils Martin",male,23.0,0,0,7267,9.225,,S
-352,0,1,"Williams-Lambert, Mr. Fletcher Fellows",male,29.69911764705882,0,0,113510,35.0,C128,S
-353,0,3,"Elias, Mr. Tannous",male,15.0,1,1,2695,7.2292,,C
-354,0,3,"Arnold-Franchi, Mr. Josef",male,25.0,1,0,349237,17.8,,S
-355,0,3,"Yousif, Mr. Wazli",male,29.69911764705882,0,0,2647,7.225,,C
-356,0,3,"Vanden Steen, Mr. Leo Peter",male,28.0,0,0,345783,9.5,,S
-357,1,1,"Bowerman, Miss. Elsie Edith",female,22.0,0,1,113505,55.0,E33,S
-358,0,2,"Funk, Miss. Annie Clemmer",female,38.0,0,0,237671,13.0,,S
-359,1,3,"McGovern, Miss. Mary",female,29.69911764705882,0,0,330931,7.8792,,Q
-360,1,3,"Mockler, Miss. Helen Mary ""Ellie""",female,29.69911764705882,0,0,330980,7.8792,,Q
-361,0,3,"Skoog, Mr. Wilhelm",male,40.0,1,4,347088,27.9,,S
-362,0,2,"del Carlo, Mr. Sebastiano",male,29.0,1,0,SC/PARIS 2167,27.7208,,C
-363,0,3,"Barbara, Mrs. (Catherine David)",female,45.0,0,1,2691,14.4542,,C
-364,0,3,"Asim, Mr. Adola",male,35.0,0,0,SOTON/O.Q. 3101310,7.05,,S
-365,0,3,"O'Brien, Mr. Thomas",male,29.69911764705882,1,0,370365,15.5,,Q
-366,0,3,"Adahl, Mr. Mauritz Nils Martin",male,30.0,0,0,C 7076,7.25,,S
-367,1,1,"Warren, Mrs. Frank Manley (Anna Sophia Atkinson)",female,60.0,1,0,110813,75.25,D37,C
-368,1,3,"Moussa, Mrs. (Mantoura Boulos)",female,29.69911764705882,0,0,2626,7.2292,,C
-369,1,3,"Jermyn, Miss. Annie",female,29.69911764705882,0,0,14313,7.75,,Q
-370,1,1,"Aubart, Mme. Leontine Pauline",female,24.0,0,0,PC 17477,69.3,B35,C
-371,1,1,"Harder, Mr. George Achilles",male,25.0,1,0,11765,55.4417,E50,C
-372,0,3,"Wiklund, Mr. Jakob Alfred",male,18.0,1,0,3101267,6.4958,,S
-373,0,3,"Beavan, Mr. William Thomas",male,19.0,0,0,323951,8.05,,S
-374,0,1,"Ringhini, Mr. Sante",male,22.0,0,0,PC 17760,135.6333,,C
-375,0,3,"Palsson, Miss. Stina Viola",female,3.0,3,1,349909,21.075,,S
-376,1,1,"Meyer, Mrs. Edgar Joseph (Leila Saks)",female,29.69911764705882,1,0,PC 17604,82.1708,,C
-377,1,3,"Landergren, Miss. Aurora Adelia",female,22.0,0,0,C 7077,7.25,,S
-378,0,1,"Widener, Mr. Harry Elkins",male,27.0,0,2,113503,211.5,C82,C
-379,0,3,"Betros, Mr. Tannous",male,20.0,0,0,2648,4.0125,,C
-380,0,3,"Gustafsson, Mr. Karl Gideon",male,19.0,0,0,347069,7.775,,S
-381,1,1,"Bidois, Miss. Rosalie",female,42.0,0,0,PC 17757,227.525,,C
-382,1,3,"Nakid, Miss. Maria (""Mary"")",female,1.0,0,2,2653,15.7417,,C
-383,0,3,"Tikkanen, Mr. Juho",male,32.0,0,0,STON/O 2. 3101293,7.925,,S
-384,1,1,"Holverson, Mrs. Alexander Oskar (Mary Aline Towner)",female,35.0,1,0,113789,52.0,,S
-385,0,3,"Plotcharsky, Mr. Vasil",male,29.69911764705882,0,0,349227,7.8958,,S
-386,0,2,"Davies, Mr. Charles Henry",male,18.0,0,0,S.O.C. 14879,73.5,,S
-387,0,3,"Goodwin, Master. Sidney Leonard",male,1.0,5,2,CA 2144,46.9,,S
-388,1,2,"Buss, Miss. Kate",female,36.0,0,0,27849,13.0,,S
-389,0,3,"Sadlier, Mr. Matthew",male,29.69911764705882,0,0,367655,7.7292,,Q
-390,1,2,"Lehmann, Miss. Bertha",female,17.0,0,0,SC 1748,12.0,,C
-391,1,1,"Carter, Mr. William Ernest",male,36.0,1,2,113760,120.0,B96 B98,S
-392,1,3,"Jansson, Mr. Carl Olof",male,21.0,0,0,350034,7.7958,,S
-393,0,3,"Gustafsson, Mr. Johan Birger",male,28.0,2,0,3101277,7.925,,S
-394,1,1,"Newell, Miss. Marjorie",female,23.0,1,0,35273,113.275,D36,C
-395,1,3,"Sandstrom, Mrs. Hjalmar (Agnes Charlotta Bengtsson)",female,24.0,0,2,PP 9549,16.7,G6,S
-396,0,3,"Johansson, Mr. Erik",male,22.0,0,0,350052,7.7958,,S
-397,0,3,"Olsson, Miss. Elina",female,31.0,0,0,350407,7.8542,,S
-398,0,2,"McKane, Mr. Peter David",male,46.0,0,0,28403,26.0,,S
-399,0,2,"Pain, Dr. Alfred",male,23.0,0,0,244278,10.5,,S
-400,1,2,"Trout, Mrs. William H (Jessie L)",female,28.0,0,0,240929,12.65,,S
-401,1,3,"Niskanen, Mr. Juha",male,39.0,0,0,STON/O 2. 3101289,7.925,,S
-402,0,3,"Adams, Mr. John",male,26.0,0,0,341826,8.05,,S
-403,0,3,"Jussila, Miss. Mari Aina",female,21.0,1,0,4137,9.825,,S
-404,0,3,"Hakkarainen, Mr. Pekka Pietari",male,28.0,1,0,STON/O2. 3101279,15.85,,S
-405,0,3,"Oreskovic, Miss. Marija",female,20.0,0,0,315096,8.6625,,S
-406,0,2,"Gale, Mr. Shadrach",male,34.0,1,0,28664,21.0,,S
-407,0,3,"Widegren, Mr. Carl/Charles Peter",male,51.0,0,0,347064,7.75,,S
-408,1,2,"Richards, Master. William Rowe",male,3.0,1,1,29106,18.75,,S
-409,0,3,"Birkeland, Mr. Hans Martin Monsen",male,21.0,0,0,312992,7.775,,S
-410,0,3,"Lefebre, Miss. Ida",female,29.69911764705882,3,1,4133,25.4667,,S
-411,0,3,"Sdycoff, Mr. Todor",male,29.69911764705882,0,0,349222,7.8958,,S
-412,0,3,"Hart, Mr. Henry",male,29.69911764705882,0,0,394140,6.8583,,Q
-413,1,1,"Minahan, Miss. Daisy E",female,33.0,1,0,19928,90.0,C78,Q
-414,0,2,"Cunningham, Mr. Alfred Fleming",male,29.69911764705882,0,0,239853,0.0,,S
-415,1,3,"Sundman, Mr. Johan Julian",male,44.0,0,0,STON/O 2. 3101269,7.925,,S
-416,0,3,"Meek, Mrs. Thomas (Annie Louise Rowley)",female,29.69911764705882,0,0,343095,8.05,,S
-417,1,2,"Drew, Mrs. James Vivian (Lulu Thorne Christian)",female,34.0,1,1,28220,32.5,,S
-418,1,2,"Silven, Miss. Lyyli Karoliina",female,18.0,0,2,250652,13.0,,S
-419,0,2,"Matthews, Mr. William John",male,30.0,0,0,28228,13.0,,S
-420,0,3,"Van Impe, Miss. Catharina",female,10.0,0,2,345773,24.15,,S
-421,0,3,"Gheorgheff, Mr. Stanio",male,29.69911764705882,0,0,349254,7.8958,,C
-422,0,3,"Charters, Mr. David",male,21.0,0,0,A/5. 13032,7.7333,,Q
-423,0,3,"Zimmerman, Mr. Leo",male,29.0,0,0,315082,7.875,,S
-424,0,3,"Danbom, Mrs. Ernst Gilbert (Anna Sigrid Maria Brogren)",female,28.0,1,1,347080,14.4,,S
-425,0,3,"Rosblom, Mr. Viktor Richard",male,18.0,1,1,370129,20.2125,,S
-426,0,3,"Wiseman, Mr. Phillippe",male,29.69911764705882,0,0,A/4. 34244,7.25,,S
-427,1,2,"Clarke, Mrs. Charles V (Ada Maria Winfield)",female,28.0,1,0,2003,26.0,,S
-428,1,2,"Phillips, Miss. Kate Florence (""Mrs Kate Louise Phillips Marshall"")",female,19.0,0,0,250655,26.0,,S
-429,0,3,"Flynn, Mr. James",male,29.69911764705882,0,0,364851,7.75,,Q
-430,1,3,"Pickard, Mr. Berk (Berk Trembisky)",male,32.0,0,0,SOTON/O.Q. 392078,8.05,E10,S
-431,1,1,"Bjornstrom-Steffansson, Mr. Mauritz Hakan",male,28.0,0,0,110564,26.55,C52,S
-432,1,3,"Thorneycroft, Mrs. Percival (Florence Kate White)",female,29.69911764705882,1,0,376564,16.1,,S
-433,1,2,"Louch, Mrs. Charles Alexander (Alice Adelaide Slow)",female,42.0,1,0,SC/AH 3085,26.0,,S
-434,0,3,"Kallio, Mr. Nikolai Erland",male,17.0,0,0,STON/O 2. 3101274,7.125,,S
-435,0,1,"Silvey, Mr. William Baird",male,50.0,1,0,13507,55.9,E44,S
-436,1,1,"Carter, Miss. Lucile Polk",female,14.0,1,2,113760,120.0,B96 B98,S
-437,0,3,"Ford, Miss. Doolina Margaret ""Daisy""",female,21.0,2,2,W./C. 6608,34.375,,S
-438,1,2,"Richards, Mrs. Sidney (Emily Hocking)",female,24.0,2,3,29106,18.75,,S
-439,0,1,"Fortune, Mr. Mark",male,64.0,1,4,19950,263.0,C23 C25 C27,S
-440,0,2,"Kvillner, Mr. Johan Henrik Johannesson",male,31.0,0,0,C.A. 18723,10.5,,S
-441,1,2,"Hart, Mrs. Benjamin (Esther Ada Bloomfield)",female,45.0,1,1,F.C.C. 13529,26.25,,S
-442,0,3,"Hampe, Mr. Leon",male,20.0,0,0,345769,9.5,,S
-443,0,3,"Petterson, Mr. Johan Emil",male,25.0,1,0,347076,7.775,,S
-444,1,2,"Reynaldo, Ms. Encarnacion",female,28.0,0,0,230434,13.0,,S
-445,1,3,"Johannesen-Bratthammer, Mr. Bernt",male,29.69911764705882,0,0,65306,8.1125,,S
-446,1,1,"Dodge, Master. Washington",male,4.0,0,2,33638,81.8583,A34,S
-447,1,2,"Mellinger, Miss. Madeleine Violet",female,13.0,0,1,250644,19.5,,S
-448,1,1,"Seward, Mr. Frederic Kimber",male,34.0,0,0,113794,26.55,,S
-449,1,3,"Baclini, Miss. Marie Catherine",female,5.0,2,1,2666,19.2583,,C
-450,1,1,"Peuchen, Major. Arthur Godfrey",male,52.0,0,0,113786,30.5,C104,S
-451,0,2,"West, Mr. Edwy Arthur",male,36.0,1,2,C.A. 34651,27.75,,S
-452,0,3,"Hagland, Mr. Ingvald Olai Olsen",male,29.69911764705882,1,0,65303,19.9667,,S
-453,0,1,"Foreman, Mr. Benjamin Laventall",male,30.0,0,0,113051,27.75,C111,C
-454,1,1,"Goldenberg, Mr. Samuel L",male,49.0,1,0,17453,89.1042,C92,C
-455,0,3,"Peduzzi, Mr. Joseph",male,29.69911764705882,0,0,A/5 2817,8.05,,S
-456,1,3,"Jalsevac, Mr. Ivan",male,29.0,0,0,349240,7.8958,,C
-457,0,1,"Millet, Mr. Francis Davis",male,65.0,0,0,13509,26.55,E38,S
-458,1,1,"Kenyon, Mrs. Frederick R (Marion)",female,29.69911764705882,1,0,17464,51.8625,D21,S
-459,1,2,"Toomey, Miss. Ellen",female,50.0,0,0,F.C.C. 13531,10.5,,S
-460,0,3,"O'Connor, Mr. Maurice",male,29.69911764705882,0,0,371060,7.75,,Q
-461,1,1,"Anderson, Mr. Harry",male,48.0,0,0,19952,26.55,E12,S
-462,0,3,"Morley, Mr. William",male,34.0,0,0,364506,8.05,,S
-463,0,1,"Gee, Mr. Arthur H",male,47.0,0,0,111320,38.5,E63,S
-464,0,2,"Milling, Mr. Jacob Christian",male,48.0,0,0,234360,13.0,,S
-465,0,3,"Maisner, Mr. Simon",male,29.69911764705882,0,0,A/S 2816,8.05,,S
-466,0,3,"Goncalves, Mr. Manuel Estanslas",male,38.0,0,0,SOTON/O.Q. 3101306,7.05,,S
-467,0,2,"Campbell, Mr. William",male,29.69911764705882,0,0,239853,0.0,,S
-468,0,1,"Smart, Mr. John Montgomery",male,56.0,0,0,113792,26.55,,S
-469,0,3,"Scanlan, Mr. James",male,29.69911764705882,0,0,36209,7.725,,Q
-470,1,3,"Baclini, Miss. Helene Barbara",female,0.75,2,1,2666,19.2583,,C
-471,0,3,"Keefe, Mr. Arthur",male,29.69911764705882,0,0,323592,7.25,,S
-472,0,3,"Cacic, Mr. Luka",male,38.0,0,0,315089,8.6625,,S
-473,1,2,"West, Mrs. Edwy Arthur (Ada Mary Worth)",female,33.0,1,2,C.A. 34651,27.75,,S
-474,1,2,"Jerwan, Mrs. Amin S (Marie Marthe Thuillard)",female,23.0,0,0,SC/AH Basle 541,13.7917,D,C
-475,0,3,"Strandberg, Miss. Ida Sofia",female,22.0,0,0,7553,9.8375,,S
-476,0,1,"Clifford, Mr. George Quincy",male,29.69911764705882,0,0,110465,52.0,A14,S
-477,0,2,"Renouf, Mr. Peter Henry",male,34.0,1,0,31027,21.0,,S
-478,0,3,"Braund, Mr. Lewis Richard",male,29.0,1,0,3460,7.0458,,S
-479,0,3,"Karlsson, Mr. Nils August",male,22.0,0,0,350060,7.5208,,S
-480,1,3,"Hirvonen, Miss. Hildur E",female,2.0,0,1,3101298,12.2875,,S
-481,0,3,"Goodwin, Master. Harold Victor",male,9.0,5,2,CA 2144,46.9,,S
-482,0,2,"Frost, Mr. Anthony Wood ""Archie""",male,29.69911764705882,0,0,239854,0.0,,S
-483,0,3,"Rouse, Mr. Richard Henry",male,50.0,0,0,A/5 3594,8.05,,S
-484,1,3,"Turkula, Mrs. (Hedwig)",female,63.0,0,0,4134,9.5875,,S
-485,1,1,"Bishop, Mr. Dickinson H",male,25.0,1,0,11967,91.0792,B49,C
-486,0,3,"Lefebre, Miss. Jeannie",female,29.69911764705882,3,1,4133,25.4667,,S
-487,1,1,"Hoyt, Mrs. Frederick Maxfield (Jane Anne Forby)",female,35.0,1,0,19943,90.0,C93,S
-488,0,1,"Kent, Mr. Edward Austin",male,58.0,0,0,11771,29.7,B37,C
-489,0,3,"Somerton, Mr. Francis William",male,30.0,0,0,A.5. 18509,8.05,,S
-490,1,3,"Coutts, Master. Eden Leslie ""Neville""",male,9.0,1,1,C.A. 37671,15.9,,S
-491,0,3,"Hagland, Mr. Konrad Mathias Reiersen",male,29.69911764705882,1,0,65304,19.9667,,S
-492,0,3,"Windelov, Mr. Einar",male,21.0,0,0,SOTON/OQ 3101317,7.25,,S
-493,0,1,"Molson, Mr. Harry Markland",male,55.0,0,0,113787,30.5,C30,S
-494,0,1,"Artagaveytia, Mr. Ramon",male,71.0,0,0,PC 17609,49.5042,,C
-495,0,3,"Stanley, Mr. Edward Roland",male,21.0,0,0,A/4 45380,8.05,,S
-496,0,3,"Yousseff, Mr. Gerious",male,29.69911764705882,0,0,2627,14.4583,,C
-497,1,1,"Eustis, Miss. Elizabeth Mussey",female,54.0,1,0,36947,78.2667,D20,C
-498,0,3,"Shellard, Mr. Frederick William",male,29.69911764705882,0,0,C.A. 6212,15.1,,S
-499,0,1,"Allison, Mrs. Hudson J C (Bessie Waldo Daniels)",female,25.0,1,2,113781,151.55,C22 C26,S
-500,0,3,"Svensson, Mr. Olof",male,24.0,0,0,350035,7.7958,,S
-501,0,3,"Calic, Mr. Petar",male,17.0,0,0,315086,8.6625,,S
-502,0,3,"Canavan, Miss. Mary",female,21.0,0,0,364846,7.75,,Q
-503,0,3,"O'Sullivan, Miss. Bridget Mary",female,29.69911764705882,0,0,330909,7.6292,,Q
-504,0,3,"Laitinen, Miss. Kristina Sofia",female,37.0,0,0,4135,9.5875,,S
-505,1,1,"Maioni, Miss. Roberta",female,16.0,0,0,110152,86.5,B79,S
-506,0,1,"Penasco y Castellana, Mr. Victor de Satode",male,18.0,1,0,PC 17758,108.9,C65,C
-507,1,2,"Quick, Mrs. Frederick Charles (Jane Richards)",female,33.0,0,2,26360,26.0,,S
-508,1,1,"Bradley, Mr. George (""George Arthur Brayton"")",male,29.69911764705882,0,0,111427,26.55,,S
-509,0,3,"Olsen, Mr. Henry Margido",male,28.0,0,0,C 4001,22.525,,S
-510,1,3,"Lang, Mr. Fang",male,26.0,0,0,1601,56.4958,,S
-511,1,3,"Daly, Mr. Eugene Patrick",male,29.0,0,0,382651,7.75,,Q
-512,0,3,"Webber, Mr. James",male,29.69911764705882,0,0,SOTON/OQ 3101316,8.05,,S
-513,1,1,"McGough, Mr. James Robert",male,36.0,0,0,PC 17473,26.2875,E25,S
-514,1,1,"Rothschild, Mrs. Martin (Elizabeth L. Barrett)",female,54.0,1,0,PC 17603,59.4,,C
-515,0,3,"Coleff, Mr. Satio",male,24.0,0,0,349209,7.4958,,S
-516,0,1,"Walker, Mr. William Anderson",male,47.0,0,0,36967,34.0208,D46,S
-517,1,2,"Lemore, Mrs. (Amelia Milley)",female,34.0,0,0,C.A. 34260,10.5,F33,S
-518,0,3,"Ryan, Mr. Patrick",male,29.69911764705882,0,0,371110,24.15,,Q
-519,1,2,"Angle, Mrs. William A (Florence ""Mary"" Agnes Hughes)",female,36.0,1,0,226875,26.0,,S
-520,0,3,"Pavlovic, Mr. Stefo",male,32.0,0,0,349242,7.8958,,S
-521,1,1,"Perreault, Miss. Anne",female,30.0,0,0,12749,93.5,B73,S
-522,0,3,"Vovk, Mr. Janko",male,22.0,0,0,349252,7.8958,,S
-523,0,3,"Lahoud, Mr. Sarkis",male,29.69911764705882,0,0,2624,7.225,,C
-524,1,1,"Hippach, Mrs. Louis Albert (Ida Sophia Fischer)",female,44.0,0,1,111361,57.9792,B18,C
-525,0,3,"Kassem, Mr. Fared",male,29.69911764705882,0,0,2700,7.2292,,C
-526,0,3,"Farrell, Mr. James",male,40.5,0,0,367232,7.75,,Q
-527,1,2,"Ridsdale, Miss. Lucy",female,50.0,0,0,W./C. 14258,10.5,,S
-528,0,1,"Farthing, Mr. John",male,29.69911764705882,0,0,PC 17483,221.7792,C95,S
-529,0,3,"Salonen, Mr. Johan Werner",male,39.0,0,0,3101296,7.925,,S
-530,0,2,"Hocking, Mr. Richard George",male,23.0,2,1,29104,11.5,,S
-531,1,2,"Quick, Miss. Phyllis May",female,2.0,1,1,26360,26.0,,S
-532,0,3,"Toufik, Mr. Nakli",male,29.69911764705882,0,0,2641,7.2292,,C
-533,0,3,"Elias, Mr. Joseph Jr",male,17.0,1,1,2690,7.2292,,C
-534,1,3,"Peter, Mrs. Catherine (Catherine Rizk)",female,29.69911764705882,0,2,2668,22.3583,,C
-535,0,3,"Cacic, Miss. Marija",female,30.0,0,0,315084,8.6625,,S
-536,1,2,"Hart, Miss. Eva Miriam",female,7.0,0,2,F.C.C. 13529,26.25,,S
-537,0,1,"Butt, Major. Archibald Willingham",male,45.0,0,0,113050,26.55,B38,S
-538,1,1,"LeRoy, Miss. Bertha",female,30.0,0,0,PC 17761,106.425,,C
-539,0,3,"Risien, Mr. Samuel Beard",male,29.69911764705882,0,0,364498,14.5,,S
-540,1,1,"Frolicher, Miss. Hedwig Margaritha",female,22.0,0,2,13568,49.5,B39,C
-541,1,1,"Crosby, Miss. Harriet R",female,36.0,0,2,WE/P 5735,71.0,B22,S
-542,0,3,"Andersson, Miss. Ingeborg Constanzia",female,9.0,4,2,347082,31.275,,S
-543,0,3,"Andersson, Miss. Sigrid Elisabeth",female,11.0,4,2,347082,31.275,,S
-544,1,2,"Beane, Mr. Edward",male,32.0,1,0,2908,26.0,,S
-545,0,1,"Douglas, Mr. Walter Donald",male,50.0,1,0,PC 17761,106.425,C86,C
-546,0,1,"Nicholson, Mr. Arthur Ernest",male,64.0,0,0,693,26.0,,S
-547,1,2,"Beane, Mrs. Edward (Ethel Clarke)",female,19.0,1,0,2908,26.0,,S
-548,1,2,"Padro y Manent, Mr. Julian",male,29.69911764705882,0,0,SC/PARIS 2146,13.8625,,C
-549,0,3,"Goldsmith, Mr. Frank John",male,33.0,1,1,363291,20.525,,S
-550,1,2,"Davies, Master. John Morgan Jr",male,8.0,1,1,C.A. 33112,36.75,,S
-551,1,1,"Thayer, Mr. John Borland Jr",male,17.0,0,2,17421,110.8833,C70,C
-552,0,2,"Sharp, Mr. Percival James R",male,27.0,0,0,244358,26.0,,S
-553,0,3,"O'Brien, Mr. Timothy",male,29.69911764705882,0,0,330979,7.8292,,Q
-554,1,3,"Leeni, Mr. Fahim (""Philip Zenni"")",male,22.0,0,0,2620,7.225,,C
-555,1,3,"Ohman, Miss. Velin",female,22.0,0,0,347085,7.775,,S
-556,0,1,"Wright, Mr. George",male,62.0,0,0,113807,26.55,,S
-557,1,1,"Duff Gordon, Lady. (Lucille Christiana Sutherland) (""Mrs Morgan"")",female,48.0,1,0,11755,39.6,A16,C
-558,0,1,"Robbins, Mr. Victor",male,29.69911764705882,0,0,PC 17757,227.525,,C
-559,1,1,"Taussig, Mrs. Emil (Tillie Mandelbaum)",female,39.0,1,1,110413,79.65,E67,S
-560,1,3,"de Messemaeker, Mrs. Guillaume Joseph (Emma)",female,36.0,1,0,345572,17.4,,S
-561,0,3,"Morrow, Mr. Thomas Rowan",male,29.69911764705882,0,0,372622,7.75,,Q
-562,0,3,"Sivic, Mr. Husein",male,40.0,0,0,349251,7.8958,,S
-563,0,2,"Norman, Mr. Robert Douglas",male,28.0,0,0,218629,13.5,,S
-564,0,3,"Simmons, Mr. John",male,29.69911764705882,0,0,SOTON/OQ 392082,8.05,,S
-565,0,3,"Meanwell, Miss. (Marion Ogden)",female,29.69911764705882,0,0,SOTON/O.Q. 392087,8.05,,S
-566,0,3,"Davies, Mr. Alfred J",male,24.0,2,0,A/4 48871,24.15,,S
-567,0,3,"Stoytcheff, Mr. Ilia",male,19.0,0,0,349205,7.8958,,S
-568,0,3,"Palsson, Mrs. Nils (Alma Cornelia Berglund)",female,29.0,0,4,349909,21.075,,S
-569,0,3,"Doharr, Mr. Tannous",male,29.69911764705882,0,0,2686,7.2292,,C
-570,1,3,"Jonsson, Mr. Carl",male,32.0,0,0,350417,7.8542,,S
-571,1,2,"Harris, Mr. George",male,62.0,0,0,S.W./PP 752,10.5,,S
-572,1,1,"Appleton, Mrs. Edward Dale (Charlotte Lamson)",female,53.0,2,0,11769,51.4792,C101,S
-573,1,1,"Flynn, Mr. John Irwin (""Irving"")",male,36.0,0,0,PC 17474,26.3875,E25,S
-574,1,3,"Kelly, Miss. Mary",female,29.69911764705882,0,0,14312,7.75,,Q
-575,0,3,"Rush, Mr. Alfred George John",male,16.0,0,0,A/4. 20589,8.05,,S
-576,0,3,"Patchett, Mr. George",male,19.0,0,0,358585,14.5,,S
-577,1,2,"Garside, Miss. Ethel",female,34.0,0,0,243880,13.0,,S
-578,1,1,"Silvey, Mrs. William Baird (Alice Munger)",female,39.0,1,0,13507,55.9,E44,S
-579,0,3,"Caram, Mrs. Joseph (Maria Elias)",female,29.69911764705882,1,0,2689,14.4583,,C
-580,1,3,"Jussila, Mr. Eiriik",male,32.0,0,0,STON/O 2. 3101286,7.925,,S
-581,1,2,"Christy, Miss. Julie Rachel",female,25.0,1,1,237789,30.0,,S
-582,1,1,"Thayer, Mrs. John Borland (Marian Longstreth Morris)",female,39.0,1,1,17421,110.8833,C68,C
-583,0,2,"Downton, Mr. William James",male,54.0,0,0,28403,26.0,,S
-584,0,1,"Ross, Mr. John Hugo",male,36.0,0,0,13049,40.125,A10,C
-585,0,3,"Paulner, Mr. Uscher",male,29.69911764705882,0,0,3411,8.7125,,C
-586,1,1,"Taussig, Miss. Ruth",female,18.0,0,2,110413,79.65,E68,S
-587,0,2,"Jarvis, Mr. John Denzil",male,47.0,0,0,237565,15.0,,S
-588,1,1,"Frolicher-Stehli, Mr. Maxmillian",male,60.0,1,1,13567,79.2,B41,C
-589,0,3,"Gilinski, Mr. Eliezer",male,22.0,0,0,14973,8.05,,S
-590,0,3,"Murdlin, Mr. Joseph",male,29.69911764705882,0,0,A./5. 3235,8.05,,S
-591,0,3,"Rintamaki, Mr. Matti",male,35.0,0,0,STON/O 2. 3101273,7.125,,S
-592,1,1,"Stephenson, Mrs. Walter Bertram (Martha Eustis)",female,52.0,1,0,36947,78.2667,D20,C
-593,0,3,"Elsbury, Mr. William James",male,47.0,0,0,A/5 3902,7.25,,S
-594,0,3,"Bourke, Miss. Mary",female,29.69911764705882,0,2,364848,7.75,,Q
-595,0,2,"Chapman, Mr. John Henry",male,37.0,1,0,SC/AH 29037,26.0,,S
-596,0,3,"Van Impe, Mr. Jean Baptiste",male,36.0,1,1,345773,24.15,,S
-597,1,2,"Leitch, Miss. Jessie Wills",female,29.69911764705882,0,0,248727,33.0,,S
-598,0,3,"Johnson, Mr. Alfred",male,49.0,0,0,LINE,0.0,,S
-599,0,3,"Boulos, Mr. Hanna",male,29.69911764705882,0,0,2664,7.225,,C
-600,1,1,"Duff Gordon, Sir. Cosmo Edmund (""Mr Morgan"")",male,49.0,1,0,PC 17485,56.9292,A20,C
-601,1,2,"Jacobsohn, Mrs. Sidney Samuel (Amy Frances Christy)",female,24.0,2,1,243847,27.0,,S
-602,0,3,"Slabenoff, Mr. Petco",male,29.69911764705882,0,0,349214,7.8958,,S
-603,0,1,"Harrington, Mr. Charles H",male,29.69911764705882,0,0,113796,42.4,,S
-604,0,3,"Torber, Mr. Ernst William",male,44.0,0,0,364511,8.05,,S
-605,1,1,"Homer, Mr. Harry (""Mr E Haven"")",male,35.0,0,0,111426,26.55,,C
-606,0,3,"Lindell, Mr. Edvard Bengtsson",male,36.0,1,0,349910,15.55,,S
-607,0,3,"Karaic, Mr. Milan",male,30.0,0,0,349246,7.8958,,S
-608,1,1,"Daniel, Mr. Robert Williams",male,27.0,0,0,113804,30.5,,S
-609,1,2,"Laroche, Mrs. Joseph (Juliette Marie Louise Lafargue)",female,22.0,1,2,SC/Paris 2123,41.5792,,C
-610,1,1,"Shutes, Miss. Elizabeth W",female,40.0,0,0,PC 17582,153.4625,C125,S
-611,0,3,"Andersson, Mrs. Anders Johan (Alfrida Konstantia Brogren)",female,39.0,1,5,347082,31.275,,S
-612,0,3,"Jardin, Mr. Jose Neto",male,29.69911764705882,0,0,SOTON/O.Q. 3101305,7.05,,S
-613,1,3,"Murphy, Miss. Margaret Jane",female,29.69911764705882,1,0,367230,15.5,,Q
-614,0,3,"Horgan, Mr. John",male,29.69911764705882,0,0,370377,7.75,,Q
-615,0,3,"Brocklebank, Mr. William Alfred",male,35.0,0,0,364512,8.05,,S
-616,1,2,"Herman, Miss. Alice",female,24.0,1,2,220845,65.0,,S
-617,0,3,"Danbom, Mr. Ernst Gilbert",male,34.0,1,1,347080,14.4,,S
-618,0,3,"Lobb, Mrs. William Arthur (Cordelia K Stanlick)",female,26.0,1,0,A/5. 3336,16.1,,S
-619,1,2,"Becker, Miss. Marion Louise",female,4.0,2,1,230136,39.0,F4,S
-620,0,2,"Gavey, Mr. Lawrence",male,26.0,0,0,31028,10.5,,S
-621,0,3,"Yasbeck, Mr. Antoni",male,27.0,1,0,2659,14.4542,,C
-622,1,1,"Kimball, Mr. Edwin Nelson Jr",male,42.0,1,0,11753,52.5542,D19,S
-623,1,3,"Nakid, Mr. Sahid",male,20.0,1,1,2653,15.7417,,C
-624,0,3,"Hansen, Mr. Henry Damsgaard",male,21.0,0,0,350029,7.8542,,S
-625,0,3,"Bowen, Mr. David John ""Dai""",male,21.0,0,0,54636,16.1,,S
-626,0,1,"Sutton, Mr. Frederick",male,61.0,0,0,36963,32.3208,D50,S
-627,0,2,"Kirkland, Rev. Charles Leonard",male,57.0,0,0,219533,12.35,,Q
-628,1,1,"Longley, Miss. Gretchen Fiske",female,21.0,0,0,13502,77.9583,D9,S
-629,0,3,"Bostandyeff, Mr. Guentcho",male,26.0,0,0,349224,7.8958,,S
-630,0,3,"O'Connell, Mr. Patrick D",male,29.69911764705882,0,0,334912,7.7333,,Q
-631,1,1,"Barkworth, Mr. Algernon Henry Wilson",male,80.0,0,0,27042,30.0,A23,S
-632,0,3,"Lundahl, Mr. Johan Svensson",male,51.0,0,0,347743,7.0542,,S
-633,1,1,"Stahelin-Maeglin, Dr. Max",male,32.0,0,0,13214,30.5,B50,C
-634,0,1,"Parr, Mr. William Henry Marsh",male,29.69911764705882,0,0,112052,0.0,,S
-635,0,3,"Skoog, Miss. Mabel",female,9.0,3,2,347088,27.9,,S
-636,1,2,"Davis, Miss. Mary",female,28.0,0,0,237668,13.0,,S
-637,0,3,"Leinonen, Mr. Antti Gustaf",male,32.0,0,0,STON/O 2. 3101292,7.925,,S
-638,0,2,"Collyer, Mr. Harvey",male,31.0,1,1,C.A. 31921,26.25,,S
-639,0,3,"Panula, Mrs. Juha (Maria Emilia Ojala)",female,41.0,0,5,3101295,39.6875,,S
-640,0,3,"Thorneycroft, Mr. Percival",male,29.69911764705882,1,0,376564,16.1,,S
-641,0,3,"Jensen, Mr. Hans Peder",male,20.0,0,0,350050,7.8542,,S
-642,1,1,"Sagesser, Mlle. Emma",female,24.0,0,0,PC 17477,69.3,B35,C
-643,0,3,"Skoog, Miss. Margit Elizabeth",female,2.0,3,2,347088,27.9,,S
-644,1,3,"Foo, Mr. Choong",male,29.69911764705882,0,0,1601,56.4958,,S
-645,1,3,"Baclini, Miss. Eugenie",female,0.75,2,1,2666,19.2583,,C
-646,1,1,"Harper, Mr. Henry Sleeper",male,48.0,1,0,PC 17572,76.7292,D33,C
-647,0,3,"Cor, Mr. Liudevit",male,19.0,0,0,349231,7.8958,,S
-648,1,1,"Simonius-Blumer, Col. Oberst Alfons",male,56.0,0,0,13213,35.5,A26,C
-649,0,3,"Willey, Mr. Edward",male,29.69911764705882,0,0,S.O./P.P. 751,7.55,,S
-650,1,3,"Stanley, Miss. Amy Zillah Elsie",female,23.0,0,0,CA. 2314,7.55,,S
-651,0,3,"Mitkoff, Mr. Mito",male,29.69911764705882,0,0,349221,7.8958,,S
-652,1,2,"Doling, Miss. Elsie",female,18.0,0,1,231919,23.0,,S
-653,0,3,"Kalvik, Mr. Johannes Halvorsen",male,21.0,0,0,8475,8.4333,,S
-654,1,3,"O'Leary, Miss. Hanora ""Norah""",female,29.69911764705882,0,0,330919,7.8292,,Q
-655,0,3,"Hegarty, Miss. Hanora ""Nora""",female,18.0,0,0,365226,6.75,,Q
-656,0,2,"Hickman, Mr. Leonard Mark",male,24.0,2,0,S.O.C. 14879,73.5,,S
-657,0,3,"Radeff, Mr. Alexander",male,29.69911764705882,0,0,349223,7.8958,,S
-658,0,3,"Bourke, Mrs. John (Catherine)",female,32.0,1,1,364849,15.5,,Q
-659,0,2,"Eitemiller, Mr. George Floyd",male,23.0,0,0,29751,13.0,,S
-660,0,1,"Newell, Mr. Arthur Webster",male,58.0,0,2,35273,113.275,D48,C
-661,1,1,"Frauenthal, Dr. Henry William",male,50.0,2,0,PC 17611,133.65,,S
-662,0,3,"Badt, Mr. Mohamed",male,40.0,0,0,2623,7.225,,C
-663,0,1,"Colley, Mr. Edward Pomeroy",male,47.0,0,0,5727,25.5875,E58,S
-664,0,3,"Coleff, Mr. Peju",male,36.0,0,0,349210,7.4958,,S
-665,1,3,"Lindqvist, Mr. Eino William",male,20.0,1,0,STON/O 2. 3101285,7.925,,S
-666,0,2,"Hickman, Mr. Lewis",male,32.0,2,0,S.O.C. 14879,73.5,,S
-667,0,2,"Butler, Mr. Reginald Fenton",male,25.0,0,0,234686,13.0,,S
-668,0,3,"Rommetvedt, Mr. Knud Paust",male,29.69911764705882,0,0,312993,7.775,,S
-669,0,3,"Cook, Mr. Jacob",male,43.0,0,0,A/5 3536,8.05,,S
-670,1,1,"Taylor, Mrs. Elmer Zebley (Juliet Cummins Wright)",female,29.69911764705882,1,0,19996,52.0,C126,S
-671,1,2,"Brown, Mrs. Thomas William Solomon (Elizabeth Catherine Ford)",female,40.0,1,1,29750,39.0,,S
-672,0,1,"Davidson, Mr. Thornton",male,31.0,1,0,F.C. 12750,52.0,B71,S
-673,0,2,"Mitchell, Mr. Henry Michael",male,70.0,0,0,C.A. 24580,10.5,,S
-674,1,2,"Wilhelms, Mr. Charles",male,31.0,0,0,244270,13.0,,S
-675,0,2,"Watson, Mr. Ennis Hastings",male,29.69911764705882,0,0,239856,0.0,,S
-676,0,3,"Edvardsson, Mr. Gustaf Hjalmar",male,18.0,0,0,349912,7.775,,S
-677,0,3,"Sawyer, Mr. Frederick Charles",male,24.5,0,0,342826,8.05,,S
-678,1,3,"Turja, Miss. Anna Sofia",female,18.0,0,0,4138,9.8417,,S
-679,0,3,"Goodwin, Mrs. Frederick (Augusta Tyler)",female,43.0,1,6,CA 2144,46.9,,S
-680,1,1,"Cardeza, Mr. Thomas Drake Martinez",male,36.0,0,1,PC 17755,512.3292,B51 B53 B55,C
-681,0,3,"Peters, Miss. Katie",female,29.69911764705882,0,0,330935,8.1375,,Q
-682,1,1,"Hassab, Mr. Hammad",male,27.0,0,0,PC 17572,76.7292,D49,C
-683,0,3,"Olsvigen, Mr. Thor Anderson",male,20.0,0,0,6563,9.225,,S
-684,0,3,"Goodwin, Mr. Charles Edward",male,14.0,5,2,CA 2144,46.9,,S
-685,0,2,"Brown, Mr. Thomas William Solomon",male,60.0,1,1,29750,39.0,,S
-686,0,2,"Laroche, Mr. Joseph Philippe Lemercier",male,25.0,1,2,SC/Paris 2123,41.5792,,C
-687,0,3,"Panula, Mr. Jaako Arnold",male,14.0,4,1,3101295,39.6875,,S
-688,0,3,"Dakic, Mr. Branko",male,19.0,0,0,349228,10.1708,,S
-689,0,3,"Fischer, Mr. Eberhard Thelander",male,18.0,0,0,350036,7.7958,,S
-690,1,1,"Madill, Miss. Georgette Alexandra",female,15.0,0,1,24160,211.3375,B5,S
-691,1,1,"Dick, Mr. Albert Adrian",male,31.0,1,0,17474,57.0,B20,S
-692,1,3,"Karun, Miss. Manca",female,4.0,0,1,349256,13.4167,,C
-693,1,3,"Lam, Mr. Ali",male,29.69911764705882,0,0,1601,56.4958,,S
-694,0,3,"Saad, Mr. Khalil",male,25.0,0,0,2672,7.225,,C
-695,0,1,"Weir, Col. John",male,60.0,0,0,113800,26.55,,S
-696,0,2,"Chapman, Mr. Charles Henry",male,52.0,0,0,248731,13.5,,S
-697,0,3,"Kelly, Mr. James",male,44.0,0,0,363592,8.05,,S
-698,1,3,"Mullens, Miss. Katherine ""Katie""",female,29.69911764705882,0,0,35852,7.7333,,Q
-699,0,1,"Thayer, Mr. John Borland",male,49.0,1,1,17421,110.8833,C68,C
-700,0,3,"Humblen, Mr. Adolf Mathias Nicolai Olsen",male,42.0,0,0,348121,7.65,F G63,S
-701,1,1,"Astor, Mrs. John Jacob (Madeleine Talmadge Force)",female,18.0,1,0,PC 17757,227.525,C62 C64,C
-702,1,1,"Silverthorne, Mr. Spencer Victor",male,35.0,0,0,PC 17475,26.2875,E24,S
-703,0,3,"Barbara, Miss. Saiide",female,18.0,0,1,2691,14.4542,,C
-704,0,3,"Gallagher, Mr. Martin",male,25.0,0,0,36864,7.7417,,Q
-705,0,3,"Hansen, Mr. Henrik Juul",male,26.0,1,0,350025,7.8542,,S
-706,0,2,"Morley, Mr. Henry Samuel (""Mr Henry Marshall"")",male,39.0,0,0,250655,26.0,,S
-707,1,2,"Kelly, Mrs. Florence ""Fannie""",female,45.0,0,0,223596,13.5,,S
-708,1,1,"Calderhead, Mr. Edward Pennington",male,42.0,0,0,PC 17476,26.2875,E24,S
-709,1,1,"Cleaver, Miss. Alice",female,22.0,0,0,113781,151.55,,S
-710,1,3,"Moubarek, Master. Halim Gonios (""William George"")",male,29.69911764705882,1,1,2661,15.2458,,C
-711,1,1,"Mayne, Mlle. Berthe Antonine (""Mrs de Villiers"")",female,24.0,0,0,PC 17482,49.5042,C90,C
-712,0,1,"Klaber, Mr. Herman",male,29.69911764705882,0,0,113028,26.55,C124,S
-713,1,1,"Taylor, Mr. Elmer Zebley",male,48.0,1,0,19996,52.0,C126,S
-714,0,3,"Larsson, Mr. August Viktor",male,29.0,0,0,7545,9.4833,,S
-715,0,2,"Greenberg, Mr. Samuel",male,52.0,0,0,250647,13.0,,S
-716,0,3,"Soholt, Mr. Peter Andreas Lauritz Andersen",male,19.0,0,0,348124,7.65,F G73,S
-717,1,1,"Endres, Miss. Caroline Louise",female,38.0,0,0,PC 17757,227.525,C45,C
-718,1,2,"Troutt, Miss. Edwina Celia ""Winnie""",female,27.0,0,0,34218,10.5,E101,S
-719,0,3,"McEvoy, Mr. Michael",male,29.69911764705882,0,0,36568,15.5,,Q
-720,0,3,"Johnson, Mr. Malkolm Joackim",male,33.0,0,0,347062,7.775,,S
-721,1,2,"Harper, Miss. Annie Jessie ""Nina""",female,6.0,0,1,248727,33.0,,S
-722,0,3,"Jensen, Mr. Svend Lauritz",male,17.0,1,0,350048,7.0542,,S
-723,0,2,"Gillespie, Mr. William Henry",male,34.0,0,0,12233,13.0,,S
-724,0,2,"Hodges, Mr. Henry Price",male,50.0,0,0,250643,13.0,,S
-725,1,1,"Chambers, Mr. Norman Campbell",male,27.0,1,0,113806,53.1,E8,S
-726,0,3,"Oreskovic, Mr. Luka",male,20.0,0,0,315094,8.6625,,S
-727,1,2,"Renouf, Mrs. Peter Henry (Lillian Jefferys)",female,30.0,3,0,31027,21.0,,S
-728,1,3,"Mannion, Miss. Margareth",female,29.69911764705882,0,0,36866,7.7375,,Q
-729,0,2,"Bryhl, Mr. Kurt Arnold Gottfrid",male,25.0,1,0,236853,26.0,,S
-730,0,3,"Ilmakangas, Miss. Pieta Sofia",female,25.0,1,0,STON/O2. 3101271,7.925,,S
-731,1,1,"Allen, Miss. Elisabeth Walton",female,29.0,0,0,24160,211.3375,B5,S
-732,0,3,"Hassan, Mr. Houssein G N",male,11.0,0,0,2699,18.7875,,C
-733,0,2,"Knight, Mr. Robert J",male,29.69911764705882,0,0,239855,0.0,,S
-734,0,2,"Berriman, Mr. William John",male,23.0,0,0,28425,13.0,,S
-735,0,2,"Troupiansky, Mr. Moses Aaron",male,23.0,0,0,233639,13.0,,S
-736,0,3,"Williams, Mr. Leslie",male,28.5,0,0,54636,16.1,,S
-737,0,3,"Ford, Mrs. Edward (Margaret Ann Watson)",female,48.0,1,3,W./C. 6608,34.375,,S
-738,1,1,"Lesurer, Mr. Gustave J",male,35.0,0,0,PC 17755,512.3292,B101,C
-739,0,3,"Ivanoff, Mr. Kanio",male,29.69911764705882,0,0,349201,7.8958,,S
-740,0,3,"Nankoff, Mr. Minko",male,29.69911764705882,0,0,349218,7.8958,,S
-741,1,1,"Hawksford, Mr. Walter James",male,29.69911764705882,0,0,16988,30.0,D45,S
-742,0,1,"Cavendish, Mr. Tyrell William",male,36.0,1,0,19877,78.85,C46,S
-743,1,1,"Ryerson, Miss. Susan Parker ""Suzette""",female,21.0,2,2,PC 17608,262.375,B57 B59 B63 B66,C
-744,0,3,"McNamee, Mr. Neal",male,24.0,1,0,376566,16.1,,S
-745,1,3,"Stranden, Mr. Juho",male,31.0,0,0,STON/O 2. 3101288,7.925,,S
-746,0,1,"Crosby, Capt. Edward Gifford",male,70.0,1,1,WE/P 5735,71.0,B22,S
-747,0,3,"Abbott, Mr. Rossmore Edward",male,16.0,1,1,C.A. 2673,20.25,,S
-748,1,2,"Sinkkonen, Miss. Anna",female,30.0,0,0,250648,13.0,,S
-749,0,1,"Marvin, Mr. Daniel Warner",male,19.0,1,0,113773,53.1,D30,S
-750,0,3,"Connaghton, Mr. Michael",male,31.0,0,0,335097,7.75,,Q
-751,1,2,"Wells, Miss. Joan",female,4.0,1,1,29103,23.0,,S
-752,1,3,"Moor, Master. Meier",male,6.0,0,1,392096,12.475,E121,S
-753,0,3,"Vande Velde, Mr. Johannes Joseph",male,33.0,0,0,345780,9.5,,S
-754,0,3,"Jonkoff, Mr. Lalio",male,23.0,0,0,349204,7.8958,,S
-755,1,2,"Herman, Mrs. Samuel (Jane Laver)",female,48.0,1,2,220845,65.0,,S
-756,1,2,"Hamalainen, Master. Viljo",male,0.67,1,1,250649,14.5,,S
-757,0,3,"Carlsson, Mr. August Sigfrid",male,28.0,0,0,350042,7.7958,,S
-758,0,2,"Bailey, Mr. Percy Andrew",male,18.0,0,0,29108,11.5,,S
-759,0,3,"Theobald, Mr. Thomas Leonard",male,34.0,0,0,363294,8.05,,S
-760,1,1,"Rothes, the Countess. of (Lucy Noel Martha Dyer-Edwards)",female,33.0,0,0,110152,86.5,B77,S
-761,0,3,"Garfirth, Mr. John",male,29.69911764705882,0,0,358585,14.5,,S
-762,0,3,"Nirva, Mr. Iisakki Antino Aijo",male,41.0,0,0,SOTON/O2 3101272,7.125,,S
-763,1,3,"Barah, Mr. Hanna Assi",male,20.0,0,0,2663,7.2292,,C
-764,1,1,"Carter, Mrs. William Ernest (Lucile Polk)",female,36.0,1,2,113760,120.0,B96 B98,S
-765,0,3,"Eklund, Mr. Hans Linus",male,16.0,0,0,347074,7.775,,S
-766,1,1,"Hogeboom, Mrs. John C (Anna Andrews)",female,51.0,1,0,13502,77.9583,D11,S
-767,0,1,"Brewe, Dr. Arthur Jackson",male,29.69911764705882,0,0,112379,39.6,,C
-768,0,3,"Mangan, Miss. Mary",female,30.5,0,0,364850,7.75,,Q
-769,0,3,"Moran, Mr. Daniel J",male,29.69911764705882,1,0,371110,24.15,,Q
-770,0,3,"Gronnestad, Mr. Daniel Danielsen",male,32.0,0,0,8471,8.3625,,S
-771,0,3,"Lievens, Mr. Rene Aime",male,24.0,0,0,345781,9.5,,S
-772,0,3,"Jensen, Mr. Niels Peder",male,48.0,0,0,350047,7.8542,,S
-773,0,2,"Mack, Mrs. (Mary)",female,57.0,0,0,S.O./P.P. 3,10.5,E77,S
-774,0,3,"Elias, Mr. Dibo",male,29.69911764705882,0,0,2674,7.225,,C
-775,1,2,"Hocking, Mrs. Elizabeth (Eliza Needs)",female,54.0,1,3,29105,23.0,,S
-776,0,3,"Myhrman, Mr. Pehr Fabian Oliver Malkolm",male,18.0,0,0,347078,7.75,,S
-777,0,3,"Tobin, Mr. Roger",male,29.69911764705882,0,0,383121,7.75,F38,Q
-778,1,3,"Emanuel, Miss. Virginia Ethel",female,5.0,0,0,364516,12.475,,S
-779,0,3,"Kilgannon, Mr. Thomas J",male,29.69911764705882,0,0,36865,7.7375,,Q
-780,1,1,"Robert, Mrs. Edward Scott (Elisabeth Walton McMillan)",female,43.0,0,1,24160,211.3375,B3,S
-781,1,3,"Ayoub, Miss. Banoura",female,13.0,0,0,2687,7.2292,,C
-782,1,1,"Dick, Mrs. Albert Adrian (Vera Gillespie)",female,17.0,1,0,17474,57.0,B20,S
-783,0,1,"Long, Mr. Milton Clyde",male,29.0,0,0,113501,30.0,D6,S
-784,0,3,"Johnston, Mr. Andrew G",male,29.69911764705882,1,2,W./C. 6607,23.45,,S
-785,0,3,"Ali, Mr. William",male,25.0,0,0,SOTON/O.Q. 3101312,7.05,,S
-786,0,3,"Harmer, Mr. Abraham (David Lishin)",male,25.0,0,0,374887,7.25,,S
-787,1,3,"Sjoblom, Miss. Anna Sofia",female,18.0,0,0,3101265,7.4958,,S
-788,0,3,"Rice, Master. George Hugh",male,8.0,4,1,382652,29.125,,Q
-789,1,3,"Dean, Master. Bertram Vere",male,1.0,1,2,C.A. 2315,20.575,,S
-790,0,1,"Guggenheim, Mr. Benjamin",male,46.0,0,0,PC 17593,79.2,B82 B84,C
-791,0,3,"Keane, Mr. Andrew ""Andy""",male,29.69911764705882,0,0,12460,7.75,,Q
-792,0,2,"Gaskell, Mr. Alfred",male,16.0,0,0,239865,26.0,,S
-793,0,3,"Sage, Miss. Stella Anna",female,29.69911764705882,8,2,CA. 2343,69.55,,S
-794,0,1,"Hoyt, Mr. William Fisher",male,29.69911764705882,0,0,PC 17600,30.6958,,C
-795,0,3,"Dantcheff, Mr. Ristiu",male,25.0,0,0,349203,7.8958,,S
-796,0,2,"Otter, Mr. Richard",male,39.0,0,0,28213,13.0,,S
-797,1,1,"Leader, Dr. Alice (Farnham)",female,49.0,0,0,17465,25.9292,D17,S
-798,1,3,"Osman, Mrs. Mara",female,31.0,0,0,349244,8.6833,,S
-799,0,3,"Ibrahim Shawah, Mr. Yousseff",male,30.0,0,0,2685,7.2292,,C
-800,0,3,"Van Impe, Mrs. Jean Baptiste (Rosalie Paula Govaert)",female,30.0,1,1,345773,24.15,,S
-801,0,2,"Ponesell, Mr. Martin",male,34.0,0,0,250647,13.0,,S
-802,1,2,"Collyer, Mrs. Harvey (Charlotte Annie Tate)",female,31.0,1,1,C.A. 31921,26.25,,S
-803,1,1,"Carter, Master. William Thornton II",male,11.0,1,2,113760,120.0,B96 B98,S
-804,1,3,"Thomas, Master. Assad Alexander",male,0.42,0,1,2625,8.5167,,C
-805,1,3,"Hedman, Mr. Oskar Arvid",male,27.0,0,0,347089,6.975,,S
-806,0,3,"Johansson, Mr. Karl Johan",male,31.0,0,0,347063,7.775,,S
-807,0,1,"Andrews, Mr. Thomas Jr",male,39.0,0,0,112050,0.0,A36,S
-808,0,3,"Pettersson, Miss. Ellen Natalia",female,18.0,0,0,347087,7.775,,S
-809,0,2,"Meyer, Mr. August",male,39.0,0,0,248723,13.0,,S
-810,1,1,"Chambers, Mrs. Norman Campbell (Bertha Griggs)",female,33.0,1,0,113806,53.1,E8,S
-811,0,3,"Alexander, Mr. William",male,26.0,0,0,3474,7.8875,,S
-812,0,3,"Lester, Mr. James",male,39.0,0,0,A/4 48871,24.15,,S
-813,0,2,"Slemen, Mr. Richard James",male,35.0,0,0,28206,10.5,,S
-814,0,3,"Andersson, Miss. Ebba Iris Alfrida",female,6.0,4,2,347082,31.275,,S
-815,0,3,"Tomlin, Mr. Ernest Portage",male,30.5,0,0,364499,8.05,,S
-816,0,1,"Fry, Mr. Richard",male,29.69911764705882,0,0,112058,0.0,B102,S
-817,0,3,"Heininen, Miss. Wendla Maria",female,23.0,0,0,STON/O2. 3101290,7.925,,S
-818,0,2,"Mallet, Mr. Albert",male,31.0,1,1,S.C./PARIS 2079,37.0042,,C
-819,0,3,"Holm, Mr. John Fredrik Alexander",male,43.0,0,0,C 7075,6.45,,S
-820,0,3,"Skoog, Master. Karl Thorsten",male,10.0,3,2,347088,27.9,,S
-821,1,1,"Hays, Mrs. Charles Melville (Clara Jennings Gregg)",female,52.0,1,1,12749,93.5,B69,S
-822,1,3,"Lulic, Mr. Nikola",male,27.0,0,0,315098,8.6625,,S
-823,0,1,"Reuchlin, Jonkheer. John George",male,38.0,0,0,19972,0.0,,S
-824,1,3,"Moor, Mrs. (Beila)",female,27.0,0,1,392096,12.475,E121,S
-825,0,3,"Panula, Master. Urho Abraham",male,2.0,4,1,3101295,39.6875,,S
-826,0,3,"Flynn, Mr. John",male,29.69911764705882,0,0,368323,6.95,,Q
-827,0,3,"Lam, Mr. Len",male,29.69911764705882,0,0,1601,56.4958,,S
-828,1,2,"Mallet, Master. Andre",male,1.0,0,2,S.C./PARIS 2079,37.0042,,C
-829,1,3,"McCormack, Mr. Thomas Joseph",male,29.69911764705882,0,0,367228,7.75,,Q
-830,1,1,"Stone, Mrs. George Nelson (Martha Evelyn)",female,62.0,0,0,113572,80.0,B28,
-831,1,3,"Yasbeck, Mrs. Antoni (Selini Alexander)",female,15.0,1,0,2659,14.4542,,C
-832,1,2,"Richards, Master. George Sibley",male,0.83,1,1,29106,18.75,,S
-833,0,3,"Saad, Mr. Amin",male,29.69911764705882,0,0,2671,7.2292,,C
-834,0,3,"Augustsson, Mr. Albert",male,23.0,0,0,347468,7.8542,,S
-835,0,3,"Allum, Mr. Owen George",male,18.0,0,0,2223,8.3,,S
-836,1,1,"Compton, Miss. Sara Rebecca",female,39.0,1,1,PC 17756,83.1583,E49,C
-837,0,3,"Pasic, Mr. Jakob",male,21.0,0,0,315097,8.6625,,S
-838,0,3,"Sirota, Mr. Maurice",male,29.69911764705882,0,0,392092,8.05,,S
-839,1,3,"Chip, Mr. Chang",male,32.0,0,0,1601,56.4958,,S
-840,1,1,"Marechal, Mr. Pierre",male,29.69911764705882,0,0,11774,29.7,C47,C
-841,0,3,"Alhomaki, Mr. Ilmari Rudolf",male,20.0,0,0,SOTON/O2 3101287,7.925,,S
-842,0,2,"Mudd, Mr. Thomas Charles",male,16.0,0,0,S.O./P.P. 3,10.5,,S
-843,1,1,"Serepeca, Miss. Augusta",female,30.0,0,0,113798,31.0,,C
-844,0,3,"Lemberopolous, Mr. Peter L",male,34.5,0,0,2683,6.4375,,C
-845,0,3,"Culumovic, Mr. Jeso",male,17.0,0,0,315090,8.6625,,S
-846,0,3,"Abbing, Mr. Anthony",male,42.0,0,0,C.A. 5547,7.55,,S
-847,0,3,"Sage, Mr. Douglas Bullen",male,29.69911764705882,8,2,CA. 2343,69.55,,S
-848,0,3,"Markoff, Mr. Marin",male,35.0,0,0,349213,7.8958,,C
-849,0,2,"Harper, Rev. John",male,28.0,0,1,248727,33.0,,S
-850,1,1,"Goldenberg, Mrs. Samuel L (Edwiga Grabowska)",female,29.69911764705882,1,0,17453,89.1042,C92,C
-851,0,3,"Andersson, Master. Sigvard Harald Elias",male,4.0,4,2,347082,31.275,,S
-852,0,3,"Svensson, Mr. Johan",male,74.0,0,0,347060,7.775,,S
-853,0,3,"Boulos, Miss. Nourelain",female,9.0,1,1,2678,15.2458,,C
-854,1,1,"Lines, Miss. Mary Conover",female,16.0,0,1,PC 17592,39.4,D28,S
-855,0,2,"Carter, Mrs. Ernest Courtenay (Lilian Hughes)",female,44.0,1,0,244252,26.0,,S
-856,1,3,"Aks, Mrs. Sam (Leah Rosen)",female,18.0,0,1,392091,9.35,,S
-857,1,1,"Wick, Mrs. George Dennick (Mary Hitchcock)",female,45.0,1,1,36928,164.8667,,S
-858,1,1,"Daly, Mr. Peter Denis ",male,51.0,0,0,113055,26.55,E17,S
-859,1,3,"Baclini, Mrs. Solomon (Latifa Qurban)",female,24.0,0,3,2666,19.2583,,C
-860,0,3,"Razi, Mr. Raihed",male,29.69911764705882,0,0,2629,7.2292,,C
-861,0,3,"Hansen, Mr. Claus Peter",male,41.0,2,0,350026,14.1083,,S
-862,0,2,"Giles, Mr. Frederick Edward",male,21.0,1,0,28134,11.5,,S
-863,1,1,"Swift, Mrs. Frederick Joel (Margaret Welles Barron)",female,48.0,0,0,17466,25.9292,D17,S
-864,0,3,"Sage, Miss. Dorothy Edith ""Dolly""",female,29.69911764705882,8,2,CA. 2343,69.55,,S
-865,0,2,"Gill, Mr. John William",male,24.0,0,0,233866,13.0,,S
-866,1,2,"Bystrom, Mrs. (Karolina)",female,42.0,0,0,236852,13.0,,S
-867,1,2,"Duran y More, Miss. Asuncion",female,27.0,1,0,SC/PARIS 2149,13.8583,,C
-868,0,1,"Roebling, Mr. Washington Augustus II",male,31.0,0,0,PC 17590,50.4958,A24,S
-869,0,3,"van Melkebeke, Mr. Philemon",male,29.69911764705882,0,0,345777,9.5,,S
-870,1,3,"Johnson, Master. Harold Theodor",male,4.0,1,1,347742,11.1333,,S
-871,0,3,"Balkic, Mr. Cerin",male,26.0,0,0,349248,7.8958,,S
-872,1,1,"Beckwith, Mrs. Richard Leonard (Sallie Monypeny)",female,47.0,1,1,11751,52.5542,D35,S
-873,0,1,"Carlsson, Mr. Frans Olof",male,33.0,0,0,695,5.0,B51 B53 B55,S
-874,0,3,"Vander Cruyssen, Mr. Victor",male,47.0,0,0,345765,9.0,,S
-875,1,2,"Abelson, Mrs. Samuel (Hannah Wizosky)",female,28.0,1,0,P/PP 3381,24.0,,C
-876,1,3,"Najib, Miss. Adele Kiamie ""Jane""",female,15.0,0,0,2667,7.225,,C
-877,0,3,"Gustafsson, Mr. Alfred Ossian",male,20.0,0,0,7534,9.8458,,S
-878,0,3,"Petroff, Mr. Nedelio",male,19.0,0,0,349212,7.8958,,S
-879,0,3,"Laleff, Mr. Kristo",male,29.69911764705882,0,0,349217,7.8958,,S
-880,1,1,"Potter, Mrs. Thomas Jr (Lily Alexenia Wilson)",female,56.0,0,1,11767,83.1583,C50,C
-881,1,2,"Shelley, Mrs. William (Imanita Parrish Hall)",female,25.0,0,1,230433,26.0,,S
-882,0,3,"Markun, Mr. Johann",male,33.0,0,0,349257,7.8958,,S
-883,0,3,"Dahlberg, Miss. Gerda Ulrika",female,22.0,0,0,7552,10.5167,,S
-884,0,2,"Banfield, Mr. Frederick James",male,28.0,0,0,C.A./SOTON 34068,10.5,,S
-885,0,3,"Sutehall, Mr. Henry Jr",male,25.0,0,0,SOTON/OQ 392076,7.05,,S
-886,0,3,"Rice, Mrs. William (Margaret Norton)",female,39.0,0,5,382652,29.125,,Q
-887,0,2,"Montvila, Rev. Juozas",male,27.0,0,0,211536,13.0,,S
-888,1,1,"Graham, Miss. Margaret Edith",female,19.0,0,0,112053,30.0,B42,S
-889,0,3,"Johnston, Miss. Catherine Helen ""Carrie""",female,29.69911764705882,1,2,W./C. 6607,23.45,,S
-890,1,1,"Behr, Mr. Karl Howell",male,26.0,0,0,111369,30.0,C148,C
-891,0,3,"Dooley, Mr. Patrick",male,32.0,0,0,370376,7.75,,Q

docs/modules/agents/tools/examples/twilio.ipynb

@@ -1,107 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "dc23c48e",
-   "metadata": {},
-   "source": [
-    "# Twilio\n",
-    "\n",
-    "This notebook goes over how to use the [Twilio](https://www.twilio.com) API wrapper to send a text message."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c1a33b13",
-   "metadata": {},
-   "source": [
-    "## Setup\n",
-    "\n",
-    "To use this tool you need to install the Python Twilio package `twilio`"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "98b544b9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# !pip install twilio"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f7e883ae",
-   "metadata": {},
-   "source": [
-    "You'll also need to set up a Twilio account and get your credentials. You'll need your Account String Identifier (SID) and your Auth Token. You'll also need a number to send messages from.\n",
-    "\n",
-    "You can either pass these in to the TwilioAPIWrapper as named parameters `account_sid`, `auth_token`, `from_number`, or you can set the environment variables `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN`, `TWILIO_FROM_NUMBER`."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "36c133be",
-   "metadata": {},
-   "source": [
-    "## Sending a message"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "54bf5afd",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.utilities.twilio import TwilioAPIWrapper"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "31f8f382",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "twilio = TwilioAPIWrapper(\n",
-    "#     account_sid=\"foo\",\n",
-    "#     auth_token=\"bar\",\n",
-    "#     from_number=\"baz,\"\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5009d763",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "twilio.run(\"hello world\", \"+16162904619\")"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/modules/indexes/document_loaders.rst

@@ -41,11 +41,9 @@ For detailed instructions on how to get set up with Unstructured, see installati
    ./document_loaders/examples/html.ipynb
    ./document_loaders/examples/image.ipynb
    ./document_loaders/examples/jupyter_notebook.ipynb
-   ./document_loaders/examples/json.ipynb
    ./document_loaders/examples/markdown.ipynb
    ./document_loaders/examples/microsoft_powerpoint.ipynb
    ./document_loaders/examples/microsoft_word.ipynb
-   ./document_loaders/examples/odt.ipynb
    ./document_loaders/examples/pandas_dataframe.ipynb
    ./document_loaders/examples/pdf.ipynb
    ./document_loaders/examples/sitemap.ipynb
@@ -55,7 +53,6 @@ For detailed instructions on how to get set up with Unstructured, see installati
    ./document_loaders/examples/unstructured_file.ipynb
    ./document_loaders/examples/url.ipynb
    ./document_loaders/examples/web_base.ipynb
-   ./document_loaders/examples/weather.ipynb
    ./document_loaders/examples/whatsapp_chat.ipynb
 
 
@@ -83,7 +80,6 @@ We don't need any access permissions to these datasets and services.
    ./document_loaders/examples/ifixit.ipynb
    ./document_loaders/examples/imsdb.ipynb
    ./document_loaders/examples/mediawikidump.ipynb
-   ./document_loaders/examples/wikipedia.ipynb
    ./document_loaders/examples/youtube_transcript.ipynb
 
 
@@ -122,8 +118,6 @@ We need access tokens and sometime other parameters to get access to these datas
    ./document_loaders/examples/google_cloud_storage_file.ipynb
    ./document_loaders/examples/google_drive.ipynb
    ./document_loaders/examples/image_captions.ipynb
-   ./document_loaders/examples/iugu.ipynb
-   ./document_loaders/examples/joplin.ipynb
    ./document_loaders/examples/microsoft_onedrive.ipynb
    ./document_loaders/examples/modern_treasury.ipynb
    ./document_loaders/examples/notiondb.ipynb
@@ -136,5 +130,4 @@ We need access tokens and sometime other parameters to get access to these datas
    ./document_loaders/examples/slack.ipynb
    ./document_loaders/examples/spreedly.ipynb
    ./document_loaders/examples/stripe.ipynb
-   ./document_loaders/examples/tomarkdown.ipynb
    ./document_loaders/examples/twitter.ipynb

docs/modules/indexes/document_loaders/examples/bibtex.ipynb

@@ -1,190 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "bda1f3f5",
-   "metadata": {},
-   "source": [
-    "# BibTeX\n",
-    "\n",
-    "> BibTeX is a file format and reference management system commonly used in conjunction with LaTeX typesetting. It serves as a way to organize and store bibliographic information for academic and research documents.\n",
-    "\n",
-    "BibTeX files have a .bib extension and consist of plain text entries representing references to various publications, such as books, articles, conference papers, theses, and more. Each BibTeX entry follows a specific structure and contains fields for different bibliographic details like author names, publication title, journal or book title, year of publication, page numbers, and more.\n",
-    "\n",
-    "Bibtex files can also store the path to documents, such as `.pdf` files that can be retrieved."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1b7a1eef-7bf7-4e7d-8bfc-c4e27c9488cb",
-   "metadata": {},
-   "source": [
-    "## Installation\n",
-    "First, you need to install `bibtexparser` and `PyMuPDF`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 26,
-   "id": "b674aaea-ed3a-4541-8414-260a8f67f623",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "#!pip install bibtexparser pymupdf"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "95f05e1c-195e-4e2b-ae8e-8d6637f15be6",
-   "metadata": {},
-   "source": [
-    "## Examples"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e29b954c-1407-4797-ae21-6ba8937156be",
-   "metadata": {},
-   "source": [
-    "`BibtexLoader` has these arguments:\n",
-    "- `file_path`: the path the the `.bib` bibtex file\n",
-    "- optional `max_docs`: default=None, i.e. not limit. Use it to limit number of retrieved documents.\n",
-    "- optional `max_content_chars`: default=4000. Use it to limit the number of characters in a single document.\n",
-    "- optional `load_extra_meta`: default=False. By default only the most important fields from the bibtex entries: `Published` (publication year), `Title`, `Authors`, `Summary`, `Journal`, `Keywords`, and `URL`. If True, it will also try to load return `entry_id`, `note`, `doi`, and `links` fields. \n",
-    "- optional `file_pattern`: default=`r'[^:]+\\.pdf'`. Regex pattern to find files in the `file` entry. Default pattern supports `Zotero` flavour bibtex style and bare file path."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 27,
-   "id": "9bfd5e46",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.document_loaders import BibtexLoader"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 28,
-   "id": "01971b53",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Create a dummy bibtex file and download a pdf.\n",
-    "import urllib.request\n",
-    "\n",
-    "urllib.request.urlretrieve(\"https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf\", \"einstein1905.pdf\")\n",
-    "\n",
-    "bibtex_text = \"\"\"\n",
-    "    @article{einstein1915,\n",
-    "        title={Die Feldgleichungen der Gravitation},\n",
-    "        abstract={Die Grundgleichungen der Gravitation, die ich hier entwickeln werde, wurden von mir in einer Abhandlung: ,,Die formale Grundlage der allgemeinen Relativit{\\\"a}tstheorie`` in den Sitzungsberichten der Preu{\\ss}ischen Akademie der Wissenschaften 1915 ver{\\\"o}ffentlicht.},\n",
-    "        author={Einstein, Albert},\n",
-    "        journal={Sitzungsberichte der K{\\\"o}niglich Preu{\\ss}ischen Akademie der Wissenschaften},\n",
-    "        volume={1915},\n",
-    "        number={1},\n",
-    "        pages={844--847},\n",
-    "        year={1915},\n",
-    "        doi={10.1002/andp.19163540702},\n",
-    "        link={https://onlinelibrary.wiley.com/doi/abs/10.1002/andp.19163540702},\n",
-    "        file={einstein1905.pdf}\n",
-    "    }\n",
-    "    \"\"\"\n",
-    "# save bibtex_text to biblio.bib file\n",
-    "with open(\"./biblio.bib\", \"w\") as file:\n",
-    "    file.write(bibtex_text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 29,
-   "id": "2631f46b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "docs = BibtexLoader(\"./biblio.bib\").load()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 30,
-   "id": "33ef1fb2",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "{'id': 'einstein1915',\n",
-       " 'published_year': '1915',\n",
-       " 'title': 'Die Feldgleichungen der Gravitation',\n",
-       " 'publication': 'Sitzungsberichte der K{\"o}niglich Preu{\\\\ss}ischen Akademie der Wissenschaften',\n",
-       " 'authors': 'Einstein, Albert',\n",
-       " 'abstract': 'Die Grundgleichungen der Gravitation, die ich hier entwickeln werde, wurden von mir in einer Abhandlung: ,,Die formale Grundlage der allgemeinen Relativit{\"a}tstheorie`` in den Sitzungsberichten der Preu{\\\\ss}ischen Akademie der Wissenschaften 1915 ver{\"o}ffentlicht.',\n",
-       " 'url': 'https://doi.org/10.1002/andp.19163540702'}"
-      ]
-     },
-     "execution_count": 30,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "docs[0].metadata"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 31,
-   "id": "46969806-45a9-4c4d-a61b-cfb9658fc9de",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "ON THE ELECTRODYNAMICS OF MOVING\n",
-      "BODIES\n",
-      "By A. EINSTEIN\n",
-      "June 30, 1905\n",
-      "It is known that Maxwell’s electrodynamics—as usually understood at the\n",
-      "present time—when applied to moving bodies, leads to asymmetries which do\n",
-      "not appear to be inherent in the phenomena. Take, for example, the recipro-\n",
-      "cal electrodynamic action of a magnet and a conductor. The observable phe-\n",
-      "nomenon here depends only on the r\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(docs[0].page_content[:400])  # all pages of the pdf content"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/modules/indexes/document_loaders/examples/iugu.ipynb

@@ -1,86 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Iugu\n",
-    "\n",
-    ">[Iugu](https://www.iugu.com/) is a Brazilian services and software as a service (SaaS) company. It offers payment-processing software and application programming interfaces for e-commerce websites and mobile applications.\n",
-    "\n",
-    "This notebook covers how to load data from the `Iugu REST API` into a format that can be ingested into LangChain, along with example usage for vectorization."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "\n",
-    "\n",
-    "from langchain.document_loaders import IuguLoader\n",
-    "from langchain.indexes import VectorstoreIndexCreator"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The Iugu API requires an access token, which can be found inside of the Iugu dashboard.\n",
-    "\n",
-    "This document loader also requires a `resource` option which defines what data you want to load.\n",
-    "\n",
-    "Following resources are available:\n",
-    "\n",
-    "`Documentation` [Documentation](https://dev.iugu.com/reference/metadados)\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "iugu_loader = IuguLoader(\"charges\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Create a vectorstore retriver from the loader\n",
-    "# see https://python.langchain.com/en/latest/modules/indexes/getting_started.html for more details\n",
-    "\n",
-    "index = VectorstoreIndexCreator().from_loaders([iugu_loader])\n",
-    "iugu_doc_retriever = index.vectorstore.as_retriever()"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.12"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}

docs/modules/indexes/document_loaders/examples/joplin.ipynb

@@ -1,89 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "id": "1dc7df1d",
-   "metadata": {},
-   "source": [
-    "# Joplin\n",
-    "\n",
-    ">[Joplin](https://joplinapp.org/) is an open source note-taking app. Capture your thoughts and securely access them from any device.\n",
-    "\n",
-    "This notebook covers how to load documents from a `Joplin` database.\n",
-    "\n",
-    "`Joplin` has a [REST API](https://joplinapp.org/api/references/rest_api/) for accessing its local database. This loader uses the API to retrieve all notes in the database and their metadata. This requires an access token that can be obtained from the app by following these steps:\n",
-    "\n",
-    "1. Open the `Joplin` app. The app must stay open while the documents are being loaded.\n",
-    "2. Go to settings / options and select \"Web Clipper\".\n",
-    "3. Make sure that the Web Clipper service is enabled.\n",
-    "4. Under \"Advanced Options\", copy the authorization token.\n",
-    "\n",
-    "You may either initialize the loader directly with the access token, or store it in the environment variable JOPLIN_ACCESS_TOKEN.\n",
-    "\n",
-    "An alternative to this approach is to export the `Joplin`'s note database to Markdown files (optionally, with Front Matter metadata) and use a Markdown loader, such as ObsidianLoader, to load them."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "007c5cbf",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.document_loaders import JoplinLoader"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "a1caec59",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "loader = JoplinLoader(access_token=\"<access-token>\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "b1c30ff7",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "docs = loader.load()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "fa93b965",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.11"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/modules/indexes/document_loaders/examples/json_loader.ipynb

@@ -4,30 +4,28 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# JSON\n",
+    "# JSON Files\n",
     "\n",
-    ">[JSON (JavaScript Object Notation)](https://en.wikipedia.org/wiki/JSON) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other serializable values).\n",
+    "The `JSONLoader` uses a specified [jq schema](https://en.wikipedia.org/wiki/Jq_(programming_language)) to parse the JSON files.\n",
+    "\n",
+    "This notebook shows how to use the `JSONLoader` to load [JSON](https://en.wikipedia.org/wiki/JSON) files into documents. A few examples of `jq` schema extracting different parts of a JSON file are also shown.\n",
     "\n",
-    ">The `JSONLoader` uses a specified [jq schema](https://en.wikipedia.org/wiki/Jq_(programming_language)) to parse the JSON files. It uses the `jq` python package.\n",
     "Check this [manual](https://stedolan.github.io/jq/manual/#Basicfilters) for a detailed documentation of the `jq` syntax."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {
-    "tags": []
-   },
+   "execution_count": null,
+   "metadata": {},
    "outputs": [],
    "source": [
-    "#!pip install jq"
+    "!pip install jq"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 1,
    "metadata": {
-    "collapsed": true,
     "jupyter": {
      "outputs_hidden": true
     }
@@ -361,7 +359,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.9.16"
   }
  },
  "nbformat": 4,

docs/modules/indexes/document_loaders/examples/odt.ipynb

@@ -5,13 +5,9 @@
    "id": "22a849cc",
    "metadata": {},
    "source": [
-    "# Open Document Format (ODT)\n",
+    "## Unstructured ODT Loader\n",
     "\n",
-    ">The [Open Document Format for Office Applications (ODF)](https://en.wikipedia.org/wiki/OpenDocument), also known as `OpenDocument`, is an open file format for word processing documents, spreadsheets, presentations and graphics and using ZIP-compressed XML files. It was developed with the aim of providing an open, XML-based file format specification for office applications.\n",
-    "\n",
-    ">The standard is developed and maintained by a technical committee in the Organization for the Advancement of Structured Information Standards (`OASIS`) consortium. It was based on the Sun Microsystems specification for OpenOffice.org XML, the default format for `OpenOffice.org` and `LibreOffice`. It was originally developed for `StarOffice` \"to provide an open standard for office documents.\"\n",
-    "\n",
-    "The `UnstructuredODTLoader` is used to load `Open Office ODT` files."
+    "The `UnstructuredODTLoader` can be used to load Open Office ODT files."
    ]
   },
   {
@@ -72,7 +68,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.8.13"
   }
  },
  "nbformat": 4,

docs/modules/indexes/document_loaders/examples/tomarkdown.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# 2Markdown\n",
     "\n",
-    ">[2markdown](https://2markdown.com/) service transforms website content into structured markdown files.\n"
+    "Uses [2markdown](https://2markdown.com/) to convert any webpage into a standard markdown file"
    ]
   },
   {
@@ -17,7 +17,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# You will need to get your own API key. See https://2markdown.com/login\n",
+    "# You will need to get your own API key\n",
     "\n",
     "api_key = \"\""
    ]
@@ -56,7 +56,9 @@
    "cell_type": "code",
    "execution_count": 8,
    "id": "706304e9",
-   "metadata": {},
+   "metadata": {
+    "scrolled": false
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -218,7 +220,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,

docs/modules/indexes/document_loaders/examples/weather.ipynb

@@ -1,101 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "66a7777e",
-   "metadata": {},
-   "source": [
-    "# Weather\n",
-    "\n",
-    ">[OpenWeatherMap](https://openweathermap.org/) is an open source weather service provider\n",
-    "\n",
-    "This loader fetches the weather data from the OpenWeatherMap's OneCall API, using the pyowm Python package. You must initialize the loader with your OpenWeatherMap API token and the names of the cities you want the weather data for."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "9ec8a3b3",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.document_loaders import WeatherDataLoader"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "43128d8d",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "#!pip install pyowm"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "51b0f0db",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Set API key either by passing it in to constructor directly\n",
-    "# or by setting the environment variable \"OPENWEATHERMAP_API_KEY\".\n",
-    "\n",
-    "from getpass import getpass\n",
-    "\n",
-    "OPENWEATHERMAP_API_KEY = getpass()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "35d6809a",
-   "metadata": {
-    "pycharm": {
-     "name": "#%%\n"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "loader = WeatherDataLoader.from_params(['chennai','vellore'], openweathermap_api_key=OPENWEATHERMAP_API_KEY) "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "05fe33b9",
-   "metadata": {
-    "pycharm": {
-     "name": "#%%\n"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "documents = loader.load()\n",
-    "documents"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/modules/indexes/retrievers/examples/tf_idf.ipynb

@@ -16,17 +16,17 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "a801b57c",
    "metadata": {},
    "outputs": [],
    "source": [
-    "# !pip install scikit-learn\n"
+    "# !pip install scikit-learn"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "id": "393ac030",
    "metadata": {
     "tags": []
@@ -46,7 +46,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "id": "98b1c017",
    "metadata": {
     "tags": []
@@ -56,27 +56,6 @@
     "retriever = TFIDFRetriever.from_texts([\"foo\", \"bar\", \"world\", \"hello\", \"foo bar\"])"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "c016b266",
-   "metadata": {},
-   "source": [
-    "## Create a New Retriever with Documents\n",
-    "\n",
-    "You can now create a new retriever with the documents you created."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "53af4f00",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.schema import Document\n",
-    "retriever = TFIDFRetriever.from_documents([Document(page_content=\"foo\"), Document(page_content=\"bar\"), Document(page_content=\"world\"), Document(page_content=\"hello\"), Document(page_content=\"foo bar\")])"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "08437fa2",
@@ -89,7 +68,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 5,
    "id": "c0455218",
    "metadata": {
     "tags": []
@@ -101,7 +80,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 6,
    "id": "7dfa5c29",
    "metadata": {
     "tags": []
@@ -116,7 +95,7 @@
        " Document(page_content='world', metadata={})]"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 6,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -124,6 +103,14 @@
    "source": [
     "result"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "74bd9256",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -142,7 +129,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.10.6"
   }
  },
  "nbformat": 4,

docs/modules/indexes/retrievers/examples/zep_memorystore.ipynb

@@ -44,13 +44,22 @@
     "from uuid import uuid4\n",
     "\n",
     "# Set this to your Zep server URL\n",
-    "ZEP_API_URL = \"http://localhost:8000\""
+    "ZEP_API_URL = \"http://localhost:8000\"\n",
+    "\n",
+    "# Zep is async-first. Our sync APIs use an asyncio wrapper to run outside an app's event loop.\n",
+    "# This interferes with Jupyter's event loop, so we need to install nest_asyncio to run the\n",
+    "# Zep client in a notebook.\n",
+    "\n",
+    "# !pip install nest_asyncio  # Uncomment to install nest_asyncio\n",
+    "import nest_asyncio\n",
+    "\n",
+    "nest_asyncio.apply()"
    ],
    "metadata": {
     "collapsed": false,
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:03:27.863217Z",
-     "start_time": "2023-05-25T15:03:25.690273Z"
+     "end_time": "2023-05-18T20:09:20.355017Z",
+     "start_time": "2023-05-18T20:09:19.526069Z"
     }
    }
   },
@@ -81,8 +90,8 @@
    "metadata": {
     "collapsed": false,
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:03:29.118416Z",
-     "start_time": "2023-05-25T15:03:29.022464Z"
+     "end_time": "2023-05-18T20:09:20.424764Z",
+     "start_time": "2023-05-18T20:09:20.355626Z"
     }
    }
   },
@@ -161,8 +170,8 @@
    "metadata": {
     "collapsed": false,
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:03:30.271181Z",
-     "start_time": "2023-05-25T15:03:30.180442Z"
+     "end_time": "2023-05-18T20:09:20.603865Z",
+     "start_time": "2023-05-18T20:09:20.427041Z"
     }
    }
   },
@@ -185,7 +194,7 @@
    "outputs": [
     {
      "data": {
-      "text/plain": "[Document(page_content='Who was Octavia Butler?', metadata={'score': 0.7759001673780126, 'uuid': '3a82a02f-056e-4c6a-b960-67ebdf3b2b93', 'created_at': '2023-05-25T15:03:30.2041Z', 'role': 'human', 'token_count': 8}),\n Document(page_content=\"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\", metadata={'score': 0.7602262941130749, 'uuid': 'a2fc9c21-0897-46c8-bef7-6f5c0f71b04a', 'created_at': '2023-05-25T15:03:30.248065Z', 'role': 'ai', 'token_count': 27}),\n Document(page_content='Who were her contemporaries?', metadata={'score': 0.757553366415519, 'uuid': '41f9c41a-a205-41e1-b48b-a0a4cd943fc8', 'created_at': '2023-05-25T15:03:30.243995Z', 'role': 'human', 'token_count': 8}),\n Document(page_content='Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', metadata={'score': 0.7546211059317948, 'uuid': '34678311-0098-4f1a-8fd4-5615ac692deb', 'created_at': '2023-05-25T15:03:30.231427Z', 'role': 'ai', 'token_count': 31}),\n Document(page_content='Which books of hers were made into movies?', metadata={'score': 0.7496714959247069, 'uuid': '18046c3a-9666-4d3e-b4f0-43d1394732b7', 'created_at': '2023-05-25T15:03:30.236837Z', 'role': 'human', 'token_count': 11})]"
+      "text/plain": "[Document(page_content='Who was Octavia Butler?', metadata={'score': 0.7759001673780126, 'uuid': '3bedb2bf-aeaf-4849-924b-40a6d91e54b9', 'created_at': '2023-05-18T20:09:20.47556Z', 'role': 'human', 'token_count': 8})]"
      },
      "execution_count": 4,
      "metadata": {},
@@ -206,8 +215,8 @@
    "metadata": {
     "collapsed": false,
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:03:32.979155Z",
-     "start_time": "2023-05-25T15:03:32.590310Z"
+     "end_time": "2023-05-18T20:09:20.979411Z",
+     "start_time": "2023-05-18T20:09:20.604147Z"
     }
    }
   },
@@ -226,7 +235,7 @@
    "outputs": [
     {
      "data": {
-      "text/plain": "[Document(page_content='Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', metadata={'score': 0.8897321402776546, 'uuid': '1c09603a-52c1-40d7-9d69-29f26256029c', 'created_at': '2023-05-25T15:03:30.268257Z', 'role': 'ai', 'token_count': 56}),\n Document(page_content=\"Write a short synopsis of Butler's book, Parable of the Sower. What is it about?\", metadata={'score': 0.8857628682610436, 'uuid': 'f6706e8c-6c91-452f-8c1b-9559fd924657', 'created_at': '2023-05-25T15:03:30.265302Z', 'role': 'human', 'token_count': 23}),\n Document(page_content='Who was Octavia Butler?', metadata={'score': 0.7759670375149477, 'uuid': '3a82a02f-056e-4c6a-b960-67ebdf3b2b93', 'created_at': '2023-05-25T15:03:30.2041Z', 'role': 'human', 'token_count': 8}),\n Document(page_content=\"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\", metadata={'score': 0.7602854653476563, 'uuid': 'a2fc9c21-0897-46c8-bef7-6f5c0f71b04a', 'created_at': '2023-05-25T15:03:30.248065Z', 'role': 'ai', 'token_count': 27}),\n Document(page_content='You might want to read Ursula K. Le Guin or Joanna Russ.', metadata={'score': 0.7595293992240313, 'uuid': 'f22f2498-6118-4c74-8718-aa89ccd7e3d6', 'created_at': '2023-05-25T15:03:30.261198Z', 'role': 'ai', 'token_count': 18})]"
+      "text/plain": "[Document(page_content='Who was Octavia Butler?', metadata={'score': 0.7759001673780126, 'uuid': '3bedb2bf-aeaf-4849-924b-40a6d91e54b9', 'created_at': '2023-05-18T20:09:20.47556Z', 'role': 'human', 'token_count': 8}),\n Document(page_content='Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', metadata={'score': 0.7545887969667749, 'uuid': 'b32c0644-2dcb-4c1d-a445-6622e7ba82e5', 'created_at': '2023-05-18T20:09:20.512044Z', 'role': 'ai', 'token_count': 31})]"
      },
      "execution_count": 5,
      "metadata": {},
@@ -239,8 +248,8 @@
    "metadata": {
     "collapsed": false,
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:03:34.713354Z",
-     "start_time": "2023-05-25T15:03:34.577974Z"
+     "end_time": "2023-05-18T20:09:21.296699Z",
+     "start_time": "2023-05-18T20:09:20.983624Z"
     }
    }
   },

docs/modules/indexes/vectorstores/examples/opensearch.ipynb

@@ -12,8 +12,7 @@
     "\n",
     "This notebook shows how to use functionality related to the `OpenSearch` database.\n",
     "\n",
-    "To run, you should have an OpenSearch instance up and running: [see here for an easy Docker installation](https://hub.docker.com/r/opensearchproject/opensearch).\n",
-    "\n",
+    "To run, you should have the opensearch instance up and running: [here](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/)\n",
     "`similarity_search` by default performs the Approximate k-NN Search which uses one of the several algorithms like lucene, nmslib, faiss recommended for\n",
     "large datasets. To perform brute force search we have other search methods known as Script Scoring and Painless Scripting.\n",
     "Check [this](https://opensearch.org/docs/latest/search-plugins/knn/index/) for more details."
@@ -24,8 +23,7 @@
    "id": "94963977-9dfc-48b7-872a-53f2947f46c6",
    "metadata": {},
    "source": [
-    "## Installation\n",
-    "Install the Python client."
+    "## Installation"
    ]
   },
   {
@@ -63,7 +61,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "id": "aac9563e",
    "metadata": {},
    "outputs": [],
@@ -76,7 +74,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 2,
    "id": "a3c3999a",
    "metadata": {},
    "outputs": [],
@@ -100,32 +98,6 @@
     "`similarity_search` using `Approximate k-NN` Search with Custom Parameters"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "803fe12b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "    docs, \n",
-    "    embeddings, \n",
-    "    opensearch_url=\"http://localhost:9200\"\n",
-    ")\n",
-    "\n",
-    "# If using the default Docker installation, use this instantiation instead:\n",
-    "# docsearch = OpenSearchVectorSearch.from_documents(\n",
-    "#     docs, \n",
-    "#     embeddings, \n",
-    "#     opensearch_url=\"https://localhost:9200\", \n",
-    "#     http_auth=(\"admin\", \"admin\"),     \n",
-    "#     use_ssl = False,\n",
-    "#     verify_certs = False,\n",
-    "#     ssl_assert_hostname = False,\n",
-    "#     ssl_show_warn = False,\n",
-    "# )"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -137,8 +109,10 @@
    },
    "outputs": [],
    "source": [
+    "docsearch = OpenSearchVectorSearch.from_documents(docs, embeddings, opensearch_url=\"http://localhost:9200\")\n",
+    "\n",
     "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "docs = docsearch.similarity_search(query, k=10)"
+    "docs = docsearch.similarity_search(query)"
    ]
   },
   {
@@ -309,7 +283,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.10.6"
   }
  },
  "nbformat": 4,

docs/modules/indexes/vectorstores/examples/typesense.ipynb

@@ -1,229 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "source": [
-    "# Typesense\n",
-    "\n",
-    "> [Typesense](https://typesense.org) is an open source, in-memory search engine, that you can either [self-host](https://typesense.org/docs/guide/install-typesense.html#option-2-local-machine-self-hosting) or run on [Typesense Cloud](https://cloud.typesense.org/).\n",
-    ">\n",
-    "> Typesense focuses on performance by storing the entire index in RAM (with a backup on disk) and also focuses on providing an out-of-the-box developer experience by simplifying available options and setting good defaults.\n",
-    ">\n",
-    "> It also lets you combine attribute-based filtering together with vector queries, to fetch the most relevant documents."
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "This notebook shows you how to use Typesense as your VectorStore."
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "Let's first install our dependencies:"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "!pip install typesense openapi-schema-pydantic openai tiktoken"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "We want to use `OpenAIEmbeddings` so we have to get the OpenAI API Key."
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "import getpass\n",
-    "\n",
-    "os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key:')"
-   ],
-   "metadata": {
-    "collapsed": false,
-    "ExecuteTime": {
-     "end_time": "2023-05-23T22:48:02.968822Z",
-     "start_time": "2023-05-23T22:47:48.574094Z"
-    }
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "outputs": [],
-   "source": [
-    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
-    "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain.vectorstores import Typesense\n",
-    "from langchain.document_loaders import TextLoader"
-   ],
-   "metadata": {
-    "collapsed": false,
-    "ExecuteTime": {
-     "end_time": "2023-05-23T22:50:34.775893Z",
-     "start_time": "2023-05-23T22:50:34.771889Z"
-    }
-   }
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "Let's import our test dataset:"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 19,
-   "outputs": [],
-   "source": [
-    "loader = TextLoader('../../../state_of_the_union.txt')\n",
-    "documents = loader.load()\n",
-    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
-    "docs = text_splitter.split_documents(documents)\n",
-    "\n",
-    "embeddings = OpenAIEmbeddings()"
-   ],
-   "metadata": {
-    "collapsed": false,
-    "ExecuteTime": {
-     "end_time": "2023-05-23T22:56:19.093489Z",
-     "start_time": "2023-05-23T22:56:19.089Z"
-    }
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "docsearch = Typesense.from_documents(docs,\n",
-    "                                     embeddings,\n",
-    "                                     typesense_client_params={\n",
-    "                                         'host': 'localhost',   # Use xxx.a1.typesense.net for Typesense Cloud\n",
-    "                                         'port': '8108',        # Use 443 for Typesense Cloud\n",
-    "                                         'protocol': 'http',    # Use https for Typesense Cloud\n",
-    "                                         'typesense_api_key': 'xyz',\n",
-    "                                         'typesense_collection_name': 'lang-chain'\n",
-    "                                     })"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "## Similarity Search"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "found_docs = docsearch.similarity_search(query)"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "print(found_docs[0].page_content)"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "## Typesense as a Retriever\n",
-    "\n",
-    "Typesense, as all the other vector stores, is a LangChain Retriever, by using cosine similarity."
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "retriever = docsearch.as_retriever()\n",
-    "retriever"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "outputs": [],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "retriever.get_relevant_documents(query)[0]"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 2
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython2",
-   "version": "2.7.6"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}

docs/modules/indexes/vectorstores/examples/vectara.ipynb

@@ -1,318 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "683953b3",
-   "metadata": {},
-   "source": [
-    "# Vectara\n",
-    "\n",
-    ">[Vectara](https://Vectara.com/docs/) is a API platform for building LLM-powered applications. It provides a simple to use API for document indexing and query that is managed by Vectara and is optimized for performance and accuracy. \n",
-    "\n",
-    "\n",
-    "This notebook shows how to use functionality related to the `Vectara` vector database. \n",
-    "\n",
-    "See the [Vectara API documentation ](https://Vectara.com/docs/) for more information on how to use the API."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "7b2f111b-357a-4f42-9730-ef0603bdc1b5",
-   "metadata": {},
-   "source": [
-    "We want to use `OpenAIEmbeddings` so we have to get the OpenAI API Key."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "082e7e8b-ac52-430c-98d6-8f0924457642",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "OpenAI API Key:········\n"
-     ]
-    }
-   ],
-   "source": [
-    "import os\n",
-    "import getpass\n",
-    "\n",
-    "os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key:')"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "aac9563e",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:22.282884Z",
-     "start_time": "2023-04-04T10:51:21.408077Z"
-    },
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
-    "from langchain.text_splitter import CharacterTextSplitter\n",
-    "from langchain.vectorstores import Vectara\n",
-    "from langchain.document_loaders import TextLoader"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "a3c3999a",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:22.520144Z",
-     "start_time": "2023-04-04T10:51:22.285826Z"
-    },
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "loader = TextLoader('../../../state_of_the_union.txt')\n",
-    "documents = loader.load()\n",
-    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
-    "docs = text_splitter.split_documents(documents)\n",
-    "\n",
-    "embeddings = OpenAIEmbeddings()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "eeead681",
-   "metadata": {},
-   "source": [
-    "## Connecting to Vectara from LangChain\n",
-    "\n",
-    "The Vectara API provides simple API endpoints for indexing and querying."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "8429667e",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:22.525091Z",
-     "start_time": "2023-04-04T10:51:22.522015Z"
-    },
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "vectara = Vectara.from_documents(docs, embedding=None)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1f9215c8",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T09:27:29.920258Z",
-     "start_time": "2023-04-04T09:27:29.913714Z"
-    }
-   },
-   "source": [
-    "## Similarity search\n",
-    "\n",
-    "The simplest scenario for using Vectara is to perform a similarity search. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "a8c513ab",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:25.204469Z",
-     "start_time": "2023-04-04T10:51:24.855618Z"
-    },
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "found_docs = vectara.similarity_search(query)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "fc516993",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:25.220984Z",
-     "start_time": "2023-04-04T10:51:25.213943Z"
-    },
-    "tags": []
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice. A former federal public defender.\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(found_docs[0].page_content)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1bda9bf5",
-   "metadata": {},
-   "source": [
-    "## Similarity search with score\n",
-    "\n",
-    "Sometimes we might want to perform the search, but also obtain a relevancy score to know how good is a particular result."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "8804a21d",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:25.631585Z",
-     "start_time": "2023-04-04T10:51:25.227384Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "found_docs = vectara.similarity_search_with_score(query)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "756a6887",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:25.642282Z",
-     "start_time": "2023-04-04T10:51:25.635947Z"
-    }
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice. A former federal public defender.\n",
-      "\n",
-      "Score: 1.0046461\n"
-     ]
-    }
-   ],
-   "source": [
-    "document, score = found_docs[0]\n",
-    "print(document.page_content)\n",
-    "print(f\"\\nScore: {score}\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "691a82d6",
-   "metadata": {},
-   "source": [
-    "## Vectara as a Retriever\n",
-    "\n",
-    "Vectara, as all the other vector stores, is a LangChain Retriever, by using cosine similarity. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "id": "9427195f",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:26.031451Z",
-     "start_time": "2023-04-04T10:51:26.018763Z"
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "VectorStoreRetriever(vectorstore=<langchain.vectorstores.vectara.Vectara object at 0x156d3e830>, search_type='similarity', search_kwargs={})"
-      ]
-     },
-     "execution_count": 11,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "retriever = vectara.as_retriever()\n",
-    "retriever"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 15,
-   "id": "f3c70c31",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-04-04T10:51:26.495652Z",
-     "start_time": "2023-04-04T10:51:26.046407Z"
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "Document(page_content='Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice. A former federal public defender.', metadata={'source': '../../modules/state_of_the_union.txt'})"
-      ]
-     },
-     "execution_count": 15,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
-    "retriever.get_relevant_documents(query)[0]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "2300e785",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/modules/memory/examples/momento_chat_message_history.ipynb

@@ -1,86 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "91c6a7ef",
-   "metadata": {},
-   "source": [
-    "# Momento\n",
-    "\n",
-    "This notebook goes over how to use [Momento Cache](https://gomomento.com) to store chat message history using the `MomentoChatMessageHistory` class. See the Momento [docs](https://docs.momentohq.com/getting-started) for more detail on how to get set up with Momento.\n",
-    "\n",
-    "Note that, by default we will create a cache if one with the given name doesn't already exist.\n",
-    "\n",
-    "You'll need to get a Momento auth token to use this class. This can either be passed in to a momento.CacheClient if you'd like to instantiate that directly, as a named parameter `auth_token` to `MomentoChatMessageHistory.from_client_params`, or can just be set as an environment variable `MOMENTO_AUTH_TOKEN`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "d15e3302",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from datetime import timedelta\n",
-    "\n",
-    "from langchain.memory import MomentoChatMessageHistory\n",
-    "\n",
-    "session_id = \"foo\"\n",
-    "cache_name = \"langchain\"\n",
-    "ttl = timedelta(days=1),\n",
-    "history = MomentoChatMessageHistory.from_client_params(\n",
-    "    session_id, \n",
-    "    cache_name,\n",
-    "    ttl,\n",
-    ")\n",
-    "\n",
-    "history.add_user_message(\"hi!\")\n",
-    "\n",
-    "history.add_ai_message(\"whats up?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "64fc465e",
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[HumanMessage(content='hi!', additional_kwargs={}, example=False),\n",
-       " AIMessage(content='whats up?', additional_kwargs={}, example=False)]"
-      ]
-     },
-     "execution_count": 6,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "history.messages"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/modules/memory/examples/zep_memory.ipynb

@@ -39,8 +39,8 @@
    "execution_count": 1,
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:09:41.754535Z",
-     "start_time": "2023-05-25T15:09:40.897232Z"
+     "end_time": "2023-05-17T05:12:58.500164Z",
+     "start_time": "2023-05-17T05:12:57.688886Z"
     }
    },
    "outputs": [],
@@ -65,8 +65,30 @@
    "execution_count": 2,
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:09:41.762056Z",
-     "start_time": "2023-05-25T15:09:41.755238Z"
+     "end_time": "2023-05-17T05:12:58.506469Z",
+     "start_time": "2023-05-17T05:12:58.501324Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# Zep is async-first. Our sync APIs use an asyncio wrapper to run outside of an app's event loop.\n",
+    "# This interferes with Jupyter's event loop, so we need to install nest_asyncio to run the\n",
+    "# Zep client in a notebook.\n",
+    "\n",
+    "# !pip install nest_asyncio\n",
+    "\n",
+    "import nest_asyncio\n",
+    "\n",
+    "nest_asyncio.apply()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-05-17T05:12:58.514638Z",
+     "start_time": "2023-05-17T05:12:58.503811Z"
     }
    },
    "outputs": [
@@ -74,7 +96,7 @@
      "data": {
       "text/plain": "True"
      },
-     "execution_count": 2,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -96,11 +118,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": 4,
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:09:41.840440Z",
-     "start_time": "2023-05-25T15:09:41.762277Z"
+     "end_time": "2023-05-17T05:12:58.618633Z",
+     "start_time": "2023-05-17T05:12:58.510414Z"
     }
    },
    "outputs": [],
@@ -140,11 +162,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 5,
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:09:41.960661Z",
-     "start_time": "2023-05-25T15:09:41.842656Z"
+     "end_time": "2023-05-17T05:12:58.764097Z",
+     "start_time": "2023-05-17T05:12:58.622528Z"
     }
    },
    "outputs": [],
@@ -229,11 +251,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 6,
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:09:50.485377Z",
-     "start_time": "2023-05-25T15:09:41.962287Z"
+     "end_time": "2023-05-17T05:13:08.226487Z",
+     "start_time": "2023-05-17T05:12:58.760795Z"
     }
    },
    "outputs": [
@@ -244,17 +266,16 @@
       "\n",
       "\n",
       "\u001B[1m> Entering new AgentExecutor chain...\u001B[0m\n",
-      "\u001B[32;1m\u001B[1;3mThought: Do I need to use a tool? No\n",
-      "AI: Parable of the Sower is a prescient novel that speaks to the challenges facing contemporary society, such as climate change, economic inequality, and the rise of authoritarianism. It is a cautionary tale that warns of the dangers of ignoring these issues and the importance of taking action to address them.\u001B[0m\n",
+      "\u001B[32;1m\u001B[1;3mAI: Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.\u001B[0m\n",
       "\n",
       "\u001B[1m> Finished chain.\u001B[0m\n"
      ]
     },
     {
      "data": {
-      "text/plain": "'Parable of the Sower is a prescient novel that speaks to the challenges facing contemporary society, such as climate change, economic inequality, and the rise of authoritarianism. It is a cautionary tale that warns of the dangers of ignoring these issues and the importance of taking action to address them.'"
+      "text/plain": "'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.'"
      },
-     "execution_count": 5,
+     "execution_count": 6,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -279,11 +300,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:09:50.493438Z",
-     "start_time": "2023-05-25T15:09:50.479230Z"
+     "end_time": "2023-05-17T05:13:08.240995Z",
+     "start_time": "2023-05-17T05:13:08.227744Z"
     }
    },
    "outputs": [
@@ -291,19 +312,16 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "The conversation is about Octavia Butler. The AI describes her as an American science fiction author and mentions the\n",
-      "FX series Kindred as a well-known adaptation of her work. The human then asks about her contemporaries, and the AI lists \n",
-      "Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\n",
+      "The AI provides a summary of Octavia Butler's science fiction novel, Parable of the Sower, which follows the story of Lauren Olamina in a dystopian future. When asked for recommendations of other women sci-fi writers, the AI suggests Ursula K. Le Guin and Joanna Russ. The AI also mentions Butler's awards, including the Hugo Award, the Nebula Award, and the MacArthur Fellowship.\n",
       "\n",
       "\n",
-      "{'role': 'human', 'content': 'What awards did she win?', 'uuid': '9fa75c3c-edae-41e3-b9bc-9fcf16b523c9', 'created_at': '2023-05-25T15:09:41.91662Z', 'token_count': 8}\n",
-      "{'role': 'ai', 'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'uuid': 'def4636c-32cb-49ed-b671-32035a034712', 'created_at': '2023-05-25T15:09:41.919874Z', 'token_count': 21}\n",
-      "{'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'uuid': '6e87bd4a-bc23-451e-ae36-05a140415270', 'created_at': '2023-05-25T15:09:41.923771Z', 'token_count': 14}\n",
-      "{'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'uuid': 'f65d8dde-9ee8-4983-9da6-ba789b7e8aa4', 'created_at': '2023-05-25T15:09:41.935254Z', 'token_count': 18}\n",
-      "{'role': 'human', 'content': \"Write a short synopsis of Butler's book, Parable of the Sower. What is it about?\", 'uuid': '5678d056-7f05-4e70-b8e5-f85efa56db01', 'created_at': '2023-05-25T15:09:41.938974Z', 'token_count': 23}\n",
-      "{'role': 'ai', 'content': 'Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', 'uuid': '50d64946-9239-4327-83e6-71dcbdd16198', 'created_at': '2023-05-25T15:09:41.957437Z', 'token_count': 56}\n",
-      "{'role': 'human', 'content': \"WWhat is the book's relevance to the challenges facing contemporary society?\", 'uuid': 'a39cfc07-8858-480a-9026-fc47a8ef7001', 'created_at': '2023-05-25T15:09:50.469533Z', 'token_count': 16}\n",
-      "{'role': 'ai', 'content': 'Parable of the Sower is a prescient novel that speaks to the challenges facing contemporary society, such as climate change, economic inequality, and the rise of authoritarianism. It is a cautionary tale that warns of the dangers of ignoring these issues and the importance of taking action to address them.', 'uuid': 'a4ecf0fe-fdd0-4aad-b72b-efde2e6830cc', 'created_at': '2023-05-25T15:09:50.473793Z', 'token_count': 62}\n"
+      "{'role': 'ai', 'content': 'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.', 'uuid': '50836d01-9dfe-4dd9-8245-3dd67c4c5c73', 'created_at': '2023-05-17T05:13:08.221879Z', 'token_count': 0}\n",
+      "{'role': 'human', 'content': \"WWhat is the book's relevance to the challenges facing contemporary society?\", 'uuid': '4249ee05-22df-41b5-a8e3-e2f9386f9ca8', 'created_at': '2023-05-17T05:13:08.211224Z', 'token_count': 0}\n",
+      "{'role': 'ai', 'content': 'Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', 'uuid': 'd4806cc0-c75e-4c36-80e9-2d74b23aece0', 'created_at': '2023-05-17T05:12:58.757654Z', 'token_count': 56}\n",
+      "{'role': 'human', 'content': \"Write a short synopsis of Butler's book, Parable of the Sower. What is it about?\", 'uuid': '378f8b63-8ef8-4fe7-b0b2-abc3e73ce1b2', 'created_at': '2023-05-17T05:12:58.754882Z', 'token_count': 23}\n",
+      "{'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'uuid': 'b9e52976-3417-4d12-868c-34a197e8111d', 'created_at': '2023-05-17T05:12:58.75184Z', 'token_count': 18}\n",
+      "{'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'uuid': '64c606c8-3c4b-424b-855e-df71825c15d7', 'created_at': '2023-05-17T05:12:58.746194Z', 'token_count': 14}\n",
+      "{'role': 'ai', 'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'uuid': '39dab3bd-351b-45df-abec-bba056516700', 'created_at': '2023-05-17T05:12:58.743281Z', 'token_count': 21}\n"
      ]
     }
    ],
@@ -330,11 +348,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 8,
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2023-05-25T15:09:50.751203Z",
-     "start_time": "2023-05-25T15:09:50.495050Z"
+     "end_time": "2023-05-17T05:13:08.382204Z",
+     "start_time": "2023-05-17T05:13:08.240341Z"
     }
    },
    "outputs": [
@@ -342,16 +360,16 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "{'uuid': '6e87bd4a-bc23-451e-ae36-05a140415270', 'created_at': '2023-05-25T15:09:41.923771Z', 'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'token_count': 14} 0.9118298949424545\n",
-      "{'uuid': 'f65d8dde-9ee8-4983-9da6-ba789b7e8aa4', 'created_at': '2023-05-25T15:09:41.935254Z', 'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'token_count': 18} 0.8533024416448016\n",
-      "{'uuid': '52cfe3e8-b800-4dd8-a7dd-8e9e4764dfc8', 'created_at': '2023-05-25T15:09:41.913856Z', 'role': 'ai', 'content': \"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\", 'token_count': 27} 0.852352466457884\n",
-      "{'uuid': 'd40da612-0867-4a43-92ec-778b86490a39', 'created_at': '2023-05-25T15:09:41.858543Z', 'role': 'human', 'content': 'Who was Octavia Butler?', 'token_count': 8} 0.8235468913583194\n",
-      "{'uuid': '4fcfbce4-7bfa-44bd-879a-8cbf265bdcf9', 'created_at': '2023-05-25T15:09:41.893848Z', 'role': 'ai', 'content': 'Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', 'token_count': 31} 0.8204317130595353\n",
-      "{'uuid': 'def4636c-32cb-49ed-b671-32035a034712', 'created_at': '2023-05-25T15:09:41.919874Z', 'role': 'ai', 'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'token_count': 21} 0.8196714827228725\n",
-      "{'uuid': '862107de-8f6f-43c0-91fa-4441f01b2b3a', 'created_at': '2023-05-25T15:09:41.898149Z', 'role': 'human', 'content': 'Which books of hers were made into movies?', 'token_count': 11} 0.7954322970428519\n",
-      "{'uuid': '97164506-90fe-4c71-9539-69ebcd1d90a2', 'created_at': '2023-05-25T15:09:41.90887Z', 'role': 'human', 'content': 'Who were her contemporaries?', 'token_count': 8} 0.7942531405021976\n",
-      "{'uuid': '50d64946-9239-4327-83e6-71dcbdd16198', 'created_at': '2023-05-25T15:09:41.957437Z', 'role': 'ai', 'content': 'Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', 'token_count': 56} 0.78144769172694\n",
-      "{'uuid': 'c460ffd4-0715-4c69-b793-1092054973e6', 'created_at': '2023-05-25T15:09:41.903082Z', 'role': 'ai', 'content': \"The most well-known adaptation of Octavia Butler's work is the FX series Kindred, based on her novel of the same name.\", 'token_count': 29} 0.7811962820699464\n"
+      "{'uuid': '64c606c8-3c4b-424b-855e-df71825c15d7', 'created_at': '2023-05-17T05:12:58.746194Z', 'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'token_count': 14} 0.9119339814710342\n",
+      "{'uuid': 'b9e52976-3417-4d12-868c-34a197e8111d', 'created_at': '2023-05-17T05:12:58.75184Z', 'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'token_count': 18} 0.8533797599308901\n",
+      "{'uuid': 'e33fd150-d7f1-4539-84cd-61422208c991', 'created_at': '2023-05-17T05:12:58.718873Z', 'role': 'ai', 'content': \"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\", 'token_count': 27} 0.8523616510364422\n",
+      "{'uuid': '727327f0-b3a9-4564-bd07-1f6bf9e344f0', 'created_at': '2023-05-17T05:12:58.671189Z', 'role': 'human', 'content': 'Who was Octavia Butler?', 'token_count': 8} 0.8235829604682462\n",
+      "{'uuid': 'db4ecb8c-b887-4de4-bf50-1e497b242fa7', 'created_at': '2023-05-17T05:12:58.699086Z', 'role': 'ai', 'content': 'Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', 'token_count': 31} 0.8205039511484328\n",
+      "{'uuid': '39dab3bd-351b-45df-abec-bba056516700', 'created_at': '2023-05-17T05:12:58.743281Z', 'role': 'ai', 'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'token_count': 21} 0.8197832181782329\n",
+      "{'uuid': 'b809efd0-0611-4f22-b0a6-acfd19f99078', 'created_at': '2023-05-17T05:12:58.706624Z', 'role': 'human', 'content': 'Which books of hers were made into movies?', 'token_count': 11} 0.7955720292889512\n",
+      "{'uuid': 'a77a9b03-2eaa-45d0-8883-3fbf6e069a84', 'created_at': '2023-05-17T05:12:58.714159Z', 'role': 'human', 'content': 'Who were her contemporaries?', 'token_count': 8} 0.7943586007524974\n",
+      "{'uuid': 'd4806cc0-c75e-4c36-80e9-2d74b23aece0', 'created_at': '2023-05-17T05:12:58.757654Z', 'role': 'ai', 'content': 'Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', 'token_count': 56} 0.7815872510788034\n",
+      "{'uuid': '7800b21b-00aa-4f60-a30a-e1c645507e66', 'created_at': '2023-05-17T05:12:58.711137Z', 'role': 'ai', 'content': \"The most well-known adaptation of Octavia Butler's work is the FX series Kindred, based on her novel of the same name.\", 'token_count': 29} 0.781267456930058\n"
      ]
     }
    ],
@@ -360,6 +378,40 @@
     "for r in search_results:\n",
     "    print(r.message, r.dist)"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-05-17T05:13:08.382385Z",
+     "start_time": "2023-05-17T05:13:08.380270Z"
+    }
+   },
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "outputs": [],
+   "source": [],
+   "metadata": {
+    "collapsed": false,
+    "ExecuteTime": {
+     "end_time": "2023-05-17T05:13:08.384260Z",
+     "start_time": "2023-05-17T05:13:08.381802Z"
+    }
+   }
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "outputs": [],
+   "source": [],
+   "metadata": {
+    "collapsed": false
+   }
   }
  ],
  "metadata": {

docs/modules/memory/types/buffer.ipynb

@@ -30,7 +30,7 @@
    "outputs": [],
    "source": [
     "memory = ConversationBufferMemory()\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})"
    ]
   },
   {
@@ -70,7 +70,7 @@
    "outputs": [],
    "source": [
     "memory = ConversationBufferMemory(return_messages=True)\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})"
    ]
   },
   {

docs/modules/memory/types/buffer_window.ipynb

@@ -30,8 +30,8 @@
    "outputs": [],
    "source": [
     "memory = ConversationBufferWindowMemory( k=1)\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})\n",
-    "memory.save_context({\"input\": \"not much you\"}, {\"output\": \"not much\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})\n",
+    "memory.save_context({\"input\": \"not much you\"}, {\"ouput\": \"not much\"})"
    ]
   },
   {
@@ -71,8 +71,8 @@
    "outputs": [],
    "source": [
     "memory = ConversationBufferWindowMemory( k=1, return_messages=True)\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})\n",
-    "memory.save_context({\"input\": \"not much you\"}, {\"output\": \"not much\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})\n",
+    "memory.save_context({\"input\": \"not much you\"}, {\"ouput\": \"not much\"})"
    ]
   },
   {

docs/modules/memory/types/entity_summary_memory.ipynb

@@ -35,7 +35,7 @@
     "memory.load_memory_variables(_input)\n",
     "memory.save_context(\n",
     "    _input,\n",
-    "    {\"output\": \" That sounds like a great project! What kind of project are they working on?\"}\n",
+    "    {\"ouput\": \" That sounds like a great project! What kind of project are they working on?\"}\n",
     ")"
    ]
   },
@@ -73,7 +73,7 @@
     "memory.load_memory_variables(_input)\n",
     "memory.save_context(\n",
     "    _input,\n",
-    "    {\"output\": \" That sounds like a great project! What kind of project are they working on?\"}\n",
+    "    {\"ouput\": \" That sounds like a great project! What kind of project are they working on?\"}\n",
     ")"
    ]
   },

docs/modules/memory/types/kg.ipynb

@@ -32,8 +32,8 @@
    "source": [
     "llm = OpenAI(temperature=0)\n",
     "memory = ConversationKGMemory(llm=llm)\n",
-    "memory.save_context({\"input\": \"say hi to sam\"}, {\"output\": \"who is sam\"})\n",
-    "memory.save_context({\"input\": \"sam is a friend\"}, {\"output\": \"okay\"})"
+    "memory.save_context({\"input\": \"say hi to sam\"}, {\"ouput\": \"who is sam\"})\n",
+    "memory.save_context({\"input\": \"sam is a friend\"}, {\"ouput\": \"okay\"})"
    ]
   },
   {
@@ -73,8 +73,8 @@
    "outputs": [],
    "source": [
     "memory = ConversationKGMemory(llm=llm, return_messages=True)\n",
-    "memory.save_context({\"input\": \"say hi to sam\"}, {\"output\": \"who is sam\"})\n",
-    "memory.save_context({\"input\": \"sam is a friend\"}, {\"output\": \"okay\"})"
+    "memory.save_context({\"input\": \"say hi to sam\"}, {\"ouput\": \"who is sam\"})\n",
+    "memory.save_context({\"input\": \"sam is a friend\"}, {\"ouput\": \"okay\"})"
    ]
   },
   {

docs/modules/memory/types/summary.ipynb

@@ -30,7 +30,7 @@
    "outputs": [],
    "source": [
     "memory = ConversationSummaryMemory(llm=OpenAI(temperature=0))\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})"
    ]
   },
   {
@@ -70,7 +70,7 @@
    "outputs": [],
    "source": [
     "memory = ConversationSummaryMemory(llm=OpenAI(temperature=0), return_messages=True)\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})"
    ]
   },
   {

docs/modules/memory/types/token_buffer.ipynb

@@ -32,8 +32,8 @@
    "outputs": [],
    "source": [
     "memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=10)\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})\n",
-    "memory.save_context({\"input\": \"not much you\"}, {\"output\": \"not much\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})\n",
+    "memory.save_context({\"input\": \"not much you\"}, {\"ouput\": \"not much\"})"
    ]
   },
   {
@@ -73,8 +73,8 @@
    "outputs": [],
    "source": [
     "memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=10, return_messages=True)\n",
-    "memory.save_context({\"input\": \"hi\"}, {\"output\": \"whats up\"})\n",
-    "memory.save_context({\"input\": \"not much you\"}, {\"output\": \"not much\"})"
+    "memory.save_context({\"input\": \"hi\"}, {\"ouput\": \"whats up\"})\n",
+    "memory.save_context({\"input\": \"not much you\"}, {\"ouput\": \"not much\"})"
    ]
   },
   {

docs/modules/models/chat/integrations/google_vertex_ai_palm.ipynb

@@ -1,170 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Google Cloud Platform Vertex AI PaLM \n",
-    "\n",
-    "Note: This is seperate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
-    "\n",
-    "PaLM API on Vertex AI is a Preview offering, subject to the Pre-GA Offerings Terms of the [GCP Service Specific Terms](https://cloud.google.com/terms/service-terms). \n",
-    "\n",
-    "Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages). Further, by using PaLM API on Vertex AI, you agree to the Generative AI Preview [terms and conditions](https://cloud.google.com/trustedtester/aitos) (Preview Terms).\n",
-    "\n",
-    "For PaLM API on Vertex AI, you can process personal data as outlined in the Cloud Data Processing Addendum, subject to applicable restrictions and obligations in the Agreement (as defined in the Preview Terms).\n",
-    "\n",
-    "To use Vertex AI PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
-    "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",
-    "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n",
-    "\n",
-    "This codebase uses the `google.auth` library which first looks for the application credentials variable mentioned above, and then looks for system-level auth.\n",
-    "\n",
-    "For more information, see: \n",
-    "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n",
-    "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n",
-    "\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "#!pip install google-cloud-aiplatform"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "\n",
-    "from langchain.chat_models import ChatVertexAI\n",
-    "from langchain.prompts.chat import (\n",
-    "    ChatPromptTemplate,\n",
-    "    SystemMessagePromptTemplate,\n",
-    "    HumanMessagePromptTemplate,\n",
-    ")\n",
-    "from langchain.schema import (\n",
-    "    HumanMessage,\n",
-    "    SystemMessage\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "chat = ChatVertexAI()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "AIMessage(content='Sure, here is the translation of the sentence \"I love programming\" from English to French:\\n\\nJ\\'aime programmer.', additional_kwargs={}, example=False)"
-      ]
-     },
-     "execution_count": 4,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "messages = [\n",
-    "    SystemMessage(content=\"You are a helpful assistant that translates English to French.\"),\n",
-    "    HumanMessage(content=\"Translate this sentence from English to French. I love programming.\")\n",
-    "]\n",
-    "chat(messages)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "You can make use of templating by using a `MessagePromptTemplate`. You can build a `ChatPromptTemplate` from one or more `MessagePromptTemplates`. You can use `ChatPromptTemplate`'s `format_prompt` -- this returns a `PromptValue`, which you can convert to a string or Message object, depending on whether you want to use the formatted value as input to an llm or chat model.\n",
-    "\n",
-    "For convenience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "template=\"You are a helpful assistant that translates {input_language} to {output_language}.\"\n",
-    "system_message_prompt = SystemMessagePromptTemplate.from_template(template)\n",
-    "human_template=\"{text}\"\n",
-    "human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "AIMessage(content='Sure, here is the translation of \"I love programming\" in French:\\n\\nJ\\'aime programmer.', additional_kwargs={}, example=False)"
-      ]
-     },
-     "execution_count": 7,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])\n",
-    "\n",
-    "# get a chat completion from the formatted messages\n",
-    "chat(chat_prompt.format_prompt(input_language=\"English\", output_language=\"French\", text=\"I love programming.\").to_messages())"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.1"
-  },
-  "vscode": {
-   "interpreter": {
-    "hash": "cc99336516f23363341912c6723b01ace86f02e26b4290be1efc0677e2e2ec24"
-   }
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}

docs/modules/models/llms/examples/human_input_llm.ipynb

@@ -5,7 +5,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# How (and why) to use the human input LLM\n",
+    "# How (and why) to use the the human input LLM\n",
     "\n",
     "Similar to the fake LLM, LangChain provides a pseudo LLM class that can be used for testing, debugging, or educational purposes. This allows you to mock out calls to the LLM and simulate how a human would respond if they received the prompts.\n",
     "\n",
@@ -34,23 +34,6 @@
     "from langchain.agents import AgentType"
    ]
   },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Since we will use the `WikipediaQueryRun` tool in this notebook, you might need to install the `wikipedia` package if you haven't done so already."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%pip install wikipedia"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 4,
@@ -234,7 +217,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.11.3"
   },
   "orig_nbformat": 4,
   "vscode": {

docs/modules/models/llms/examples/llm_caching.ipynb

@@ -41,7 +41,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 3,
    "id": "f69f6283",
    "metadata": {},
    "outputs": [],
@@ -612,115 +612,6 @@
     "llm(\"Tell me joke\")"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "726fe754",
-   "metadata": {},
-   "source": [
-    "## Momento Cache\n",
-    "Use [Momento](../../../../integrations/momento.md) to cache prompts and responses.\n",
-    "\n",
-    "Requires momento to use, uncomment below to install:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "e8949f29",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# !pip install momento"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "56ea6a08",
-   "metadata": {},
-   "source": [
-    "You'll need to get a Momemto auth token to use this class. This can either be passed in to a momento.CacheClient if you'd like to instantiate that directly, as a named parameter `auth_token` to `MomentoChatMessageHistory.from_client_params`, or can just be set as an environment variable `MOMENTO_AUTH_TOKEN`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "2005f03a",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from datetime import timedelta\n",
-    "\n",
-    "from langchain.cache import MomentoCache\n",
-    "\n",
-    "\n",
-    "cache_name = \"langchain\"\n",
-    "ttl = timedelta(days=1)\n",
-    "langchain.llm_cache = MomentoCache.from_client_params(cache_name, ttl)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "c6a6c238",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "CPU times: user 40.7 ms, sys: 16.5 ms, total: 57.2 ms\n",
-      "Wall time: 1.73 s\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'"
-      ]
-     },
-     "execution_count": 10,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "%%time\n",
-    "# The first time, it is not yet in cache, so it should take longer\n",
-    "llm(\"Tell me a joke\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "id": "b8f78f9d",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "CPU times: user 3.16 ms, sys: 2.98 ms, total: 6.14 ms\n",
-      "Wall time: 57.9 ms\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!'"
-      ]
-     },
-     "execution_count": 11,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "%%time\n",
-    "# The second time it is, so it goes faster\n",
-    "# When run in the same region as the cache, latencies are single digit ms\n",
-    "llm(\"Tell me a joke\")"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "934943dc",
@@ -1018,9 +909,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "venv",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "venv"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -1032,7 +923,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.8.8"
   }
  },
  "nbformat": 4,

docs/modules/models/llms/integrations/beam.ipynb

@@ -1,159 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "J-yvaDTmTTza"
-   },
-   "source": [
-    "# Beam integration for langchain\n",
-    "\n",
-    "Calls the Beam API wrapper to deploy and make subsequent calls to an instance of the gpt2 LLM in a cloud deployment. Requires installation of the Beam library and registration of Beam Client ID and Client Secret. By calling the wrapper an instance of the model is created and run, with returned text relating to the prompt. Additional calls can then be made by directly calling the Beam API.\n",
-    "\n",
-    "[Create an account](https://www.beam.cloud/), if you don't have one already. Grab your API keys from the [dashboard](https://www.beam.cloud/dashboard/settings/api-keys)."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "CfTmesWtTfTS"
-   },
-   "source": [
-    "Install the Beam CLI"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "G_tCCurqR7Ik"
-   },
-   "outputs": [],
-   "source": [
-    "!curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "jJkcNqOdThQ7"
-   },
-   "source": [
-    "Register API Keys and set your beam client id and secret environment variables:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "7gQd6fszSEaH"
-   },
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "import subprocess\n",
-    "\n",
-    "beam_client_id = \"<Your beam client id>\"\n",
-    "beam_client_secret = \"<Your beam client secret>\"\n",
-    "\n",
-    "# Set the environment variables\n",
-    "os.environ['BEAM_CLIENT_ID'] = beam_client_id\n",
-    "os.environ['BEAM_CLIENT_SECRET'] = beam_client_secret\n",
-    "\n",
-    "# Run the beam configure command\n",
-    "!beam configure --clientId={beam_client_id} --clientSecret={beam_client_secret}"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "c20rkK18TrK2"
-   },
-   "source": [
-    "Install the Beam SDK:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "CH2Vop6ISNIf"
-   },
-   "outputs": [],
-   "source": [
-    "!pip install beam-sdk"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "XflOsp3bTwl1"
-   },
-   "source": [
-    "**Deploy and call Beam directly from langchain!**\n",
-    "\n",
-    "Note that a cold start might take a couple of minutes to return the response, but subsequent calls will be faster!"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "id": "KmaHxUqbSVnh"
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.llms.beam import Beam\n",
-    "\n",
-    "llm = Beam(model_name=\"gpt2\",\n",
-    "           name=\"langchain-gpt2-test\",\n",
-    "           cpu=8,\n",
-    "           memory=\"32Gi\",\n",
-    "           gpu=\"A10G\",\n",
-    "           python_version=\"python3.8\",\n",
-    "           python_packages=[\n",
-    "               \"diffusers[torch]>=0.10\",\n",
-    "               \"transformers\",\n",
-    "               \"torch\",\n",
-    "               \"pillow\",\n",
-    "               \"accelerate\",\n",
-    "               \"safetensors\",\n",
-    "               \"xformers\",],\n",
-    "           max_length=\"50\",\n",
-    "           verbose=False)\n",
-    "\n",
-    "llm._deploy()\n",
-    "\n",
-    "response = llm._call(\"Running machine learning on a remote GPU\")\n",
-    "\n",
-    "print(response)"
-   ]
-  }
- ],
- "metadata": {
-  "colab": {
-   "private_outputs": true,
-   "provenance": []
-  },
-  "gpuClass": "standard",
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 1
-}

docs/modules/models/llms/integrations/ctransformers.ipynb

@@ -1,125 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# C Transformers\n",
-    "\n",
-    "The [C Transformers](https://github.com/marella/ctransformers) library provides Python bindings for GGML models.\n",
-    "\n",
-    "This example goes over how to use LangChain to interact with `C Transformers` [models](https://github.com/marella/ctransformers#supported-models)."
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**Install**"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%pip install ctransformers"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**Load Model**"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.llms import CTransformers\n",
-    "\n",
-    "llm = CTransformers(model='marella/gpt-2-ggml')"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**Generate Text**"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(llm('AI is going to'))"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**Streaming**"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
-    "\n",
-    "llm = CTransformers(model='marella/gpt-2-ggml', callbacks=[StreamingStdOutCallbackHandler()])\n",
-    "\n",
-    "response = llm('AI is going to')"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**LLMChain**"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain import PromptTemplate, LLMChain\n",
-    "\n",
-    "template = \"\"\"Question: {question}\n",
-    "\n",
-    "Answer:\"\"\"\n",
-    "\n",
-    "prompt = PromptTemplate(template=template, input_variables=['question'])\n",
-    "\n",
-    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
-    "\n",
-    "response = llm_chain.run('What is AI?')"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "name": "python"
-  },
-  "orig_nbformat": 4
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

docs/modules/models/llms/integrations/databricks.ipynb

@@ -1,523 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {},
-     "inputWidgets": {},
-     "nuid": "5147e458-3b83-449e-9c2f-e7e1972e43fc",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "source": [
-    "# Databricks\n",
-    "\n",
-    "The [Databricks](https://www.databricks.com/) Lakehouse Platform unifies data, analytics, and AI on one platform.\n",
-    "\n",
-    "This example notebook shows how to wrap Databricks endpoints as LLMs in LangChain.\n",
-    "It supports two endpoint types:\n",
-    "* Serving endpoint, recommended for production and development,\n",
-    "* Cluster driver proxy app, recommended for iteractive development."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "bf07455f-aac9-4873-a8e7-7952af0f8c82",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.llms import Databricks"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {},
-     "inputWidgets": {},
-     "nuid": "94f6540e-40cd-4d9b-95d3-33d36f061dcc",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "source": [
-    "## Wrapping a serving endpoint\n",
-    "\n",
-    "Prerequisites:\n",
-    "* An LLM was registered and deployed to [a Databricks serving endpoint](https://docs.databricks.com/machine-learning/model-serving/index.html).\n",
-    "* You have [\"Can Query\" permission](https://docs.databricks.com/security/auth-authz/access-control/serving-endpoint-acl.html) to the endpoint.\n",
-    "\n",
-    "The expected MLflow model signature is:\n",
-    "  * inputs: `[{\"name\": \"prompt\", \"type\": \"string\"}, {\"name\": \"stop\", \"type\": \"list[string]\"}]`\n",
-    "  * outputs: `[{\"type\": \"string\"}]`\n",
-    "\n",
-    "If the model signature is incompatible or you want to insert extra configs, you can set `transform_input_fn` and `transform_output_fn` accordingly."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "7496dc7a-8a1a-4ce6-9648-4f69ed25275b",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'I am happy to hear that you are in good health and as always, you are appreciated.'"
-      ]
-     },
-     "execution_count": 4,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# If running a Databricks notebook attached to an interactive cluster in \"single user\" \n",
-    "# or \"no isolation shared\" mode, you only need to specify the endpoint name to create \n",
-    "# a `Databricks` instance to query a serving endpoint in the same workspace.\n",
-    "llm = Databricks(endpoint_name=\"dolly\")\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "0c86d952-4236-4a5e-bdac-cf4e3ccf3a16",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'Good'"
-      ]
-     },
-     "execution_count": 34,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "llm(\"How are you?\", stop=[\".\"])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "5f2507a2-addd-431d-9da5-dc2ae33783f6",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'I am fine. Thank you!'"
-      ]
-     },
-     "execution_count": 5,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# Otherwise, you can manually specify the Databricks workspace hostname and personal access token \n",
-    "# or set `DATABRICKS_HOST` and `DATABRICKS_API_TOKEN` environment variables, respectively.\n",
-    "# See https://docs.databricks.com/dev-tools/auth.html#databricks-personal-access-tokens\n",
-    "# We strongly recommend not exposing the API token explicitly inside a notebook.\n",
-    "# You can use Databricks secret manager to store your API token securely.\n",
-    "# See https://docs.databricks.com/dev-tools/databricks-utils.html#secrets-utility-dbutilssecrets\n",
-    "\n",
-    "import os\n",
-    "os.environ[\"DATABRICKS_API_TOKEN\"] = dbutils.secrets.get(\"myworkspace\", \"api_token\")\n",
-    "\n",
-    "llm = Databricks(host=\"myworkspace.cloud.databricks.com\", endpoint_name=\"dolly\")\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "9b54f8ce-ffe5-4c47-a3f0-b4ebde524a6a",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'I am fine.'"
-      ]
-     },
-     "execution_count": 5,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# If the serving endpoint accepts extra parameters like `temperature`,\n",
-    "# you can set them in `model_kwargs`.\n",
-    "llm = Databricks(endpoint_name=\"dolly\", model_kwargs={\"temperature\": 0.1})\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "50f172f5-ea1f-4ceb-8cf1-20289848de7b",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'I’m Excellent. You?'"
-      ]
-     },
-     "execution_count": 24,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# Use `transform_input_fn` and `transform_output_fn` if the serving endpoint\n",
-    "# expects a different input schema and does not return a JSON string,\n",
-    "# respectively, or you want to apply a prompt template on top.\n",
-    "\n",
-    "def transform_input(**request):\n",
-    "    full_prompt = f\"\"\"{request[\"prompt\"]}\n",
-    "    Be Concise.\n",
-    "    \"\"\"\n",
-    "    request[\"prompt\"] = full_prompt\n",
-    "    return request\n",
-    "\n",
-    "llm = Databricks(endpoint_name=\"dolly\", transform_input_fn=transform_input)\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {},
-     "inputWidgets": {},
-     "nuid": "8ea49319-a041-494d-afcd-87bcf00d5efb",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "source": [
-    "## Wrapping a cluster driver proxy app\n",
-    "\n",
-    "Prerequisites:\n",
-    "* An LLM loaded on a Databricks interactive cluster in \"single user\" or \"no isolation shared\" mode.\n",
-    "* A local HTTP server running on the driver node to serve the model at `\"/\"` using HTTP POST with JSON input/output.\n",
-    "* It uses a port number between `[3000, 8000]` and litens to the driver IP address or simply `0.0.0.0` instead of localhost only.\n",
-    "* You have \"Can Attach To\" permission to the cluster.\n",
-    "\n",
-    "The expected server schema (using JSON schema) is:\n",
-    "* inputs:\n",
-    "  ```json\n",
-    "  {\"type\": \"object\",\n",
-    "   \"properties\": {\n",
-    "      \"prompt\": {\"type\": \"string\"},\n",
-    "       \"stop\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}}},\n",
-    "    \"required\": [\"prompt\"]}\n",
-    "  ```\n",
-    "* outputs: `{\"type\": \"string\"}`\n",
-    "\n",
-    "If the server schema is incompatible or you want to insert extra configs, you can use `transform_input_fn` and `transform_output_fn` accordingly.\n",
-    "\n",
-    "The following is a minimal example for running a driver proxy app to serve an LLM:\n",
-    "\n",
-    "```python\n",
-    "from flask import Flask, request, jsonify\n",
-    "import torch\n",
-    "from transformers import pipeline, AutoTokenizer, StoppingCriteria\n",
-    "\n",
-    "model = \"databricks/dolly-v2-3b\"\n",
-    "tokenizer = AutoTokenizer.from_pretrained(model, padding_side=\"left\")\n",
-    "dolly = pipeline(model=model, tokenizer=tokenizer, trust_remote_code=True, device_map=\"auto\")\n",
-    "device = dolly.device\n",
-    "\n",
-    "class CheckStop(StoppingCriteria):\n",
-    "    def __init__(self, stop=None):\n",
-    "        super().__init__()\n",
-    "        self.stop = stop or []\n",
-    "        self.matched = \"\"\n",
-    "        self.stop_ids = [tokenizer.encode(s, return_tensors='pt').to(device) for s in self.stop]\n",
-    "    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs):\n",
-    "        for i, s in enumerate(self.stop_ids):\n",
-    "            if torch.all((s == input_ids[0][-s.shape[1]:])).item():\n",
-    "                self.matched = self.stop[i]\n",
-    "                return True\n",
-    "        return False\n",
-    "\n",
-    "def llm(prompt, stop=None, **kwargs):\n",
-    "  check_stop = CheckStop(stop)\n",
-    "  result = dolly(prompt, stopping_criteria=[check_stop], **kwargs)\n",
-    "  return result[0][\"generated_text\"].rstrip(check_stop.matched)\n",
-    "\n",
-    "app = Flask(\"dolly\")\n",
-    "\n",
-    "@app.route('/', methods=['POST'])\n",
-    "def serve_llm():\n",
-    "  resp = llm(**request.json)\n",
-    "  return jsonify(resp)\n",
-    "\n",
-    "app.run(host=\"0.0.0.0\", port=\"7777\")\n",
-    "```\n",
-    "\n",
-    "Once the server is running, you can create a `Databricks` instance to wrap it as an LLM."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "e3330a01-e738-4170-a176-9954aff56442",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'Hello, thank you for asking. It is wonderful to hear that you are well.'"
-      ]
-     },
-     "execution_count": 32,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# If running a Databricks notebook attached to the same cluster that runs the app,\n",
-    "# you only need to specify the driver port to create a `Databricks` instance.\n",
-    "llm = Databricks(cluster_driver_port=\"7777\")\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "39c121cf-0e44-4e31-91db-37fcac459677",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'I am well. You?'"
-      ]
-     },
-     "execution_count": 40,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# Otherwise, you can manually specify the cluster ID to use,\n",
-    "# as well as Databricks workspace hostname and personal access token.\n",
-    "\n",
-    "llm = Databricks(cluster_id=\"0000-000000-xxxxxxxx\", cluster_driver_port=\"7777\")\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "3d3de599-82fd-45e4-8d8b-bacfc49dc9ce",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'I am very well. It is a pleasure to meet you.'"
-      ]
-     },
-     "execution_count": 31,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# If the app accepts extra parameters like `temperature`,\n",
-    "# you can set them in `model_kwargs`.\n",
-    "llm = Databricks(cluster_driver_port=\"7777\", model_kwargs={\"temperature\": 0.1})\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "application/vnd.databricks.v1+cell": {
-     "cellMetadata": {
-      "byteLimit": 2048000,
-      "rowLimit": 10000
-     },
-     "inputWidgets": {},
-     "nuid": "853fae8e-8df4-41e6-9d45-7769f883fe80",
-     "showTitle": false,
-     "title": ""
-    }
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'I AM DOING GREAT THANK YOU.'"
-      ]
-     },
-     "execution_count": 32,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "# Use `transform_input_fn` and `transform_output_fn` if the app\n",
-    "# expects a different input schema and does not return a JSON string,\n",
-    "# respectively, or you want to apply a prompt template on top.\n",
-    "\n",
-    "def transform_input(**request):\n",
-    "    full_prompt = f\"\"\"{request[\"prompt\"]}\n",
-    "    Be Concise.\n",
-    "    \"\"\"\n",
-    "    request[\"prompt\"] = full_prompt\n",
-    "    return request\n",
-    "\n",
-    "def transform_output(response):\n",
-    "    return response.upper()\n",
-    "\n",
-    "llm = Databricks(\n",
-    "  cluster_driver_port=\"7777\",\n",
-    "  transform_input_fn=transform_input,\n",
-    "  transform_output_fn=transform_output)\n",
-    "\n",
-    "llm(\"How are you?\")"
-   ]
-  }
- ],
- "metadata": {
-  "application/vnd.databricks.v1+notebook": {
-   "dashboards": [],
-   "language": "python",
-   "notebookMetadata": {
-    "pythonIndentUnit": 2
-   },
-   "notebookName": "databricks",
-   "widgets": {}
-  },
-  "kernelspec": {
-   "display_name": "llm",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.10.10"
-  },
-  "orig_nbformat": 4
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}

docs/modules/models/llms/integrations/google_vertex_ai_palm.ipynb

@@ -1,138 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Google Cloud Platform Vertex AI PaLM \n",
-    "\n",
-    "Note: This is seperate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
-    "\n",
-    "PaLM API on Vertex AI is a Preview offering, subject to the Pre-GA Offerings Terms of the [GCP Service Specific Terms](https://cloud.google.com/terms/service-terms). \n",
-    "\n",
-    "Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages). Further, by using PaLM API on Vertex AI, you agree to the Generative AI Preview [terms and conditions](https://cloud.google.com/trustedtester/aitos) (Preview Terms).\n",
-    "\n",
-    "For PaLM API on Vertex AI, you can process personal data as outlined in the Cloud Data Processing Addendum, subject to applicable restrictions and obligations in the Agreement (as defined in the Preview Terms).\n",
-    "\n",
-    "To use Vertex AI PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
-    "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",
-    "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n",
-    "\n",
-    "This codebase uses the `google.auth` library which first looks for the application credentials variable mentioned above, and then looks for system-level auth.\n",
-    "\n",
-    "For more information, see: \n",
-    "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n",
-    "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n",
-    "\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "#!pip install google-cloud-aiplatform"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "\n",
-    "from langchain.llms import VertexAI\n",
-    "from langchain import PromptTemplate, LLMChain"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "template = \"\"\"Question: {question}\n",
-    "\n",
-    "Answer: Let's think step by step.\"\"\"\n",
-    "\n",
-    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm = VertexAI()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm_chain = LLMChain(prompt=prompt, llm=llm)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "'Justin Bieber was born on March 1, 1994. The Super Bowl in 1994 was won by the San Francisco 49ers.\\nThe final answer: San Francisco 49ers.'"
-      ]
-     },
-     "execution_count": 8,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "question = \"What NFL team won the Super Bowl in the year Justin Beiber was born?\"\n",
-    "\n",
-    "llm_chain.run(question)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.1"
-  },
-  "vscode": {
-   "interpreter": {
-    "hash": "cc99336516f23363341912c6723b01ace86f02e26b4290be1efc0677e2e2ec24"
-   }
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}

docs/modules/models/llms/integrations/mosaicml.ipynb

@@ -1,105 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# MosaicML\n",
-    "\n",
-    "[MosaicML](https://docs.mosaicml.com/en/latest/inference.html) offers a managed inference service. You can either use a variety of open source models, or deploy your own.\n",
-    "\n",
-    "This example goes over how to use LangChain to interact with MosaicML Inference for text completion."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# sign up for an account: https://forms.mosaicml.com/demo?utm_source=langchain\n",
-    "\n",
-    "from getpass import getpass\n",
-    "\n",
-    "MOSAICML_API_TOKEN = getpass()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "\n",
-    "os.environ[\"MOSAICML_API_TOKEN\"] = MOSAICML_API_TOKEN"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.llms import MosaicML\n",
-    "from langchain import PromptTemplate, LLMChain"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "template = \"\"\"Question: {question}\"\"\"\n",
-    "\n",
-    "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm = MosaicML(inject_instruction_format=True, model_kwargs={'do_sample': False})"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "llm_chain = LLMChain(prompt=prompt, llm=llm)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "question = \"What is one good reason why you should train a large language model on domain specific data?\"\n",
-    "\n",
-    "llm_chain.run(question)"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

docs/modules/models/llms/integrations/openai.ipynb

@@ -21,7 +21,7 @@
    },
    "outputs": [
     {
-     "name": "stdout",
+     "name": "stdin",
      "output_type": "stream",
      "text": [
       " ········\n"
@@ -127,20 +127,11 @@
     "\n",
     "llm_chain.run(question)"
    ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "58a9ddb1",
-   "metadata": {},
-   "source": [
-    "# if you are behind an explicit proxy, you can use the OPENAI_PROXY environment variable to pass through\n",
-    "os.environ[\"OPENAI_PROXY\"] = \"http://proxy.yourcompany.com:8080\""
-   ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3.11.1 64-bit",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -154,11 +145,11 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.1"
+   "version": "3.10.6"
   },
   "vscode": {
    "interpreter": {
-    "hash": "e971737741ff4ec9aff7dc6155a1060a59a8a6d52c757dbbe66bf8ee389494b1"
+    "hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
    }
   }
  },

docs/modules/models/llms/integrations/rellm_experimental.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Structured Decoding with RELLM\n",
     "\n",
-    "[RELLM](https://github.com/r2d4/rellm) is a library that wraps local Hugging Face pipeline models for structured decoding.\n",
+    "[RELLM](https://github.com/r2d4/rellm) is a library that wraps local HuggingFace pipeline models for structured decoding.\n",
     "\n",
     "It works by generating tokens one at a time. At each step, it masks tokens that don't conform to the provided partial regular expression.\n",
     "\n",
@@ -32,7 +32,7 @@
    "id": "66bd89f1-8daa-433d-bb8f-5b0b3ae34b00",
    "metadata": {},
    "source": [
-    "### Hugging Face Baseline\n",
+    "### HuggingFace Baseline\n",
     "\n",
     "First, let's establish a qualitative baseline by checking the output of the model without structured decoding."
    ]

docs/modules/models/text_embedding/examples/elasticsearch.ipynb

@@ -1,124 +0,0 @@
-{
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": []
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
-  "cells": [
-    {
-      "cell_type": "code",
-      "source": [
-        "!pip -q install elasticsearch langchain"
-      ],
-      "metadata": {
-        "id": "6dJxqebov4eU"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [
-        "import elasticsearch\n",
-        "from langchain.embeddings.elasticsearch import ElasticsearchEmbeddings"
-      ],
-      "metadata": {
-        "id": "RV7C3DUmv4aq"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [
-        "# Define the model ID\n",
-        "model_id = 'your_model_id'"
-      ],
-      "metadata": {
-        "id": "MrT3jplJvp09"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [
-        "# Instantiate ElasticsearchEmbeddings using credentials\n",
-        "embeddings = ElasticsearchEmbeddings.from_credentials(\n",
-        "    model_id,\n",
-        "    es_cloud_id='your_cloud_id', \n",
-        "    es_user='your_user', \n",
-        "    es_password='your_password'\n",
-        ")\n"
-      ],
-      "metadata": {
-        "id": "svtdnC-dvpxR"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [
-        "# Create embeddings for multiple documents\n",
-        "documents = [\n",
-        "    'This is an example document.', \n",
-        "    'Another example document to generate embeddings for.'\n",
-        "]\n",
-        "document_embeddings = embeddings.embed_documents(documents)\n"
-      ],
-      "metadata": {
-        "id": "7DXZAK7Kvpth"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [
-        "# Print document embeddings\n",
-        "for i, embedding in enumerate(document_embeddings):\n",
-        "    print(f\"Embedding for document {i+1}: {embedding}\")\n"
-      ],
-      "metadata": {
-        "id": "K8ra75W_vpqy"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [
-        "# Create an embedding for a single query\n",
-        "query = 'This is a single query.'\n",
-        "query_embedding = embeddings.embed_query(query)\n"
-      ],
-      "metadata": {
-        "id": "V4Q5kQo9vpna"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [
-        "# Print query embedding\n",
-        "print(f\"Embedding for query: {query_embedding}\")\n"
-      ],
-      "metadata": {
-        "id": "O0oQDzGKvpkz"
-      },
-      "execution_count": null,
-      "outputs": []
-    }
-  ]
-}

docs/modules/models/text_embedding/examples/google_vertex_ai_palm.ipynb

@@ -1,113 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Google Cloud Platform Vertex AI PaLM \n",
-    "\n",
-    "Note: This is seperate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
-    "\n",
-    "PaLM API on Vertex AI is a Preview offering, subject to the Pre-GA Offerings Terms of the [GCP Service Specific Terms](https://cloud.google.com/terms/service-terms). \n",
-    "\n",
-    "Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the [launch stage descriptions](https://cloud.google.com/products#product-launch-stages). Further, by using PaLM API on Vertex AI, you agree to the Generative AI Preview [terms and conditions](https://cloud.google.com/trustedtester/aitos) (Preview Terms).\n",
-    "\n",
-    "For PaLM API on Vertex AI, you can process personal data as outlined in the Cloud Data Processing Addendum, subject to applicable restrictions and obligations in the Agreement (as defined in the Preview Terms).\n",
-    "\n",
-    "To use Vertex AI PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
-    "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",
-    "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n",
-    "\n",
-    "This codebase uses the `google.auth` library which first looks for the application credentials variable mentioned above, and then looks for system-level auth.\n",
-    "\n",
-    "For more information, see: \n",
-    "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n",
-    "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n",
-    "\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "#!pip install google-cloud-aiplatform"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "\n",
-    "from langchain.embeddings import VertexAIEmbeddings"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "embeddings = VertexAIEmbeddings()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "text = \"This is a test document.\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "query_result = embeddings.embed_query(text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "doc_result = embeddings.embed_documents([text])"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.1"
-  },
-  "vscode": {
-   "interpreter": {
-    "hash": "cc99336516f23363341912c6723b01ace86f02e26b4290be1efc0677e2e2ec24"
-   }
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 4
-}

docs/modules/models/text_embedding/examples/minimax.ipynb

@@ -1,145 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# MiniMax\n",
-    "\n",
-    "[MiniMax](https://api.minimax.chat/document/guides/embeddings?id=6464722084cdc277dfaa966a) offers an embeddings service.\n",
-    "\n",
-    "This example goes over how to use LangChain to interact with MiniMax Inference for text embedding."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-05-24T15:13:15.397075Z",
-     "start_time": "2023-05-24T15:13:15.387540Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "\n",
-    "os.environ[\"MINIMAX_GROUP_ID\"] = \"MINIMAX_GROUP_ID\"\n",
-    "os.environ[\"MINIMAX_API_KEY\"] = \"MINIMAX_API_KEY\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-05-24T15:13:17.176956Z",
-     "start_time": "2023-05-24T15:13:15.399076Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "from langchain.embeddings import MiniMaxEmbeddings"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-05-24T15:13:17.193751Z",
-     "start_time": "2023-05-24T15:13:17.182053Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "embeddings = MiniMaxEmbeddings()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-05-24T15:13:17.844903Z",
-     "start_time": "2023-05-24T15:13:17.198751Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "query_text = \"This is a test query.\"\n",
-    "query_result = embeddings.embed_query(query_text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-05-24T15:13:18.605339Z",
-     "start_time": "2023-05-24T15:13:17.845906Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "document_text = \"This is a test document.\"\n",
-    "document_result = embeddings.embed_documents([document_text])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2023-05-24T15:13:18.620432Z",
-     "start_time": "2023-05-24T15:13:18.608335Z"
-    }
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Cosine similarity between document and query: 0.1573236279277012\n"
-     ]
-    }
-   ],
-   "source": [
-    "import numpy as np\n",
-    "\n",
-    "query_numpy = np.array(query_result)\n",
-    "document_numpy = np.array(document_result[0])\n",
-    "similarity = np.dot(query_numpy, document_numpy) / (np.linalg.norm(query_numpy)*np.linalg.norm(document_numpy))\n",
-    "print(f\"Cosine similarity between document and query: {similarity}\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

docs/modules/models/text_embedding/examples/modelscope_hub.ipynb

@@ -1,82 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# ModelScope\n",
-    "\n",
-    "Let's load the ModelScope Embedding class."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.embeddings import ModelScopeEmbeddings"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "model_id = \"damo/nlp_corom_sentence-embedding_english-base\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "embeddings = ModelScopeEmbeddings(model_id=model_id)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "text = \"This is a test document.\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "query_result = embeddings.embed_query(text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "doc_results = embeddings.embed_documents([\"foo\"])"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "chatgpt",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "name": "python",
-   "version": "3.9.15"
-  },
-  "orig_nbformat": 4
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

docs/modules/models/text_embedding/examples/mosaicml.ipynb

@@ -1,109 +0,0 @@
-{
- "cells": [
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# MosaicML embeddings\n",
-    "\n",
-    "[MosaicML](https://docs.mosaicml.com/en/latest/inference.html) offers a managed inference service. You can either use a variety of open source models, or deploy your own.\n",
-    "\n",
-    "This example goes over how to use LangChain to interact with MosaicML Inference for text embedding."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# sign up for an account: https://forms.mosaicml.com/demo?utm_source=langchain\n",
-    "\n",
-    "from getpass import getpass\n",
-    "\n",
-    "MOSAICML_API_TOKEN = getpass()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import os\n",
-    "\n",
-    "os.environ[\"MOSAICML_API_TOKEN\"] = MOSAICML_API_TOKEN"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from langchain.embeddings import MosaicMLInstructorEmbeddings"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "embeddings = MosaicMLInstructorEmbeddings(\n",
-    "    query_instruction=\"Represent the query for retrieval: \"\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "query_text = \"This is a test query.\"\n",
-    "query_result = embeddings.embed_query(query_text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "document_text = \"This is a test document.\"\n",
-    "document_result = embeddings.embed_documents([document_text])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import numpy as np\n",
-    "\n",
-    "query_numpy = np.array(query_result)\n",
-    "document_numpy = np.array(document_result[0])\n",
-    "similarity = np.dot(query_numpy, document_numpy) / (np.linalg.norm(query_numpy)*np.linalg.norm(document_numpy))\n",
-    "print(f\"Cosine similarity between document and query: {similarity}\")"
-   ]
-  }
- ],
- "metadata": {
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

docs/modules/models/text_embedding/examples/openai.ipynb

@@ -124,15 +124,12 @@
    "id": "aaad49f8",
    "metadata": {},
    "outputs": [],
-   "source": [
-    "# if you are behind an explicit proxy, you can use the OPENAI_PROXY environment variable to pass through\n",
-    "os.environ[\"OPENAI_PROXY\"] = \"http://proxy.yourcompany.com:8080\""
-   ]
+   "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3.11.1 64-bit",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -146,11 +143,11 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.1"
+   "version": "3.9.1"
   },
   "vscode": {
    "interpreter": {
-    "hash": "e971737741ff4ec9aff7dc6155a1060a59a8a6d52c757dbbe66bf8ee389494b1"
+    "hash": "7377c2ccc78bc62c2683122d48c8cd1fb85a53850a1b1fc29736ed39852c9885"
    }
   }
  },

docs/modules/prompts/output_parsers/examples/structured.ipynb

@@ -1,7 +1,6 @@
 {
  "cells": [
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "91871002",
    "metadata": {},
@@ -13,7 +12,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 23,
+   "execution_count": 3,
    "id": "b492997a",
    "metadata": {},
    "outputs": [],
@@ -25,7 +24,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "09473dce",
    "metadata": {},
@@ -35,7 +33,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 24,
+   "execution_count": 4,
    "id": "432ac44a",
    "metadata": {},
    "outputs": [],
@@ -48,7 +46,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "7b92ce96",
    "metadata": {},
@@ -58,7 +55,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 25,
+   "execution_count": 5,
    "id": "593cfc25",
    "metadata": {},
    "outputs": [],
@@ -72,7 +69,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "0943e783",
    "metadata": {},
@@ -82,7 +78,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 26,
+   "execution_count": 6,
    "id": "106f1ba6",
    "metadata": {},
    "outputs": [],
@@ -92,29 +88,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 27,
+   "execution_count": 7,
    "id": "86d9d24f",
    "metadata": {},
    "outputs": [],
    "source": [
-    "_input = prompt.format_prompt(question=\"what's the capital of france?\")\n",
+    "_input = prompt.format_prompt(question=\"what's the capital of france\")\n",
     "output = model(_input.to_string())"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 28,
+   "execution_count": 8,
    "id": "956bdc99",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "{'answer': 'Paris',\n",
-       " 'source': 'https://www.worldatlas.com/articles/what-is-the-capital-of-france.html'}"
+       "{'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}"
       ]
      },
-     "execution_count": 28,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -124,7 +119,6 @@
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "id": "da639285",
    "metadata": {},
@@ -134,7 +128,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
+   "execution_count": 9,
    "id": "8f483d7d",
    "metadata": {},
    "outputs": [],
@@ -144,7 +138,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 30,
+   "execution_count": 10,
    "id": "f761cbf1",
    "metadata": {},
    "outputs": [],
@@ -160,18 +154,18 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 31,
+   "execution_count": 11,
    "id": "edd73ae3",
    "metadata": {},
    "outputs": [],
    "source": [
-    "_input = prompt.format_prompt(question=\"what's the capital of france?\")\n",
+    "_input = prompt.format_prompt(question=\"what's the capital of france\")\n",
     "output = chat_model(_input.to_messages())"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 32,
+   "execution_count": 12,
    "id": "a3c8b91e",
    "metadata": {},
    "outputs": [
@@ -181,7 +175,7 @@
        "{'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}"
       ]
      },
-     "execution_count": 32,
+     "execution_count": 12,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -207,7 +201,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.11"
+   "version": "3.9.1"
   }
  },
  "nbformat": 4,

docs/modules/prompts/prompt_templates/getting_started.md

@@ -150,6 +150,7 @@ In this example, we'll create a prompt to generate word antonyms.
 ```python
 from langchain import PromptTemplate, FewShotPromptTemplate
 
+
 # First, create the list of few shot examples.
 examples = [
     {"word": "happy", "antonym": "sad"},
@@ -158,10 +159,10 @@ examples = [
 
 # Next, we specify the template to format the examples we have provided.
 # We use the `PromptTemplate` class for this.
-example_formatter_template = """Word: {word}
-Antonym: {antonym}
+example_formatter_template = """
+Word: {word}
+Antonym: {antonym}\n
 """
-
 example_prompt = PromptTemplate(
     input_variables=["word", "antonym"],
     template=example_formatter_template,
@@ -175,14 +176,14 @@ few_shot_prompt = FewShotPromptTemplate(
     example_prompt=example_prompt,
     # The prefix is some text that goes before the examples in the prompt.
     # Usually, this consists of intructions.
-    prefix="Give the antonym of every input\n",
+    prefix="Give the antonym of every input",
     # The suffix is some text that goes after the examples in the prompt.
     # Usually, this is where the user input will go
-    suffix="Word: {input}\nAntonym: ",
+    suffix="Word: {input}\nAntonym:",
     # The input variables are the variables that the overall prompt expects.
     input_variables=["input"],
     # The example_separator is the string we will use to join the prefix, examples, and suffix together with.
-    example_separator="\n",
+    example_separator="\n\n",
 )
 
 # We can now generate a prompt using the `format` method.
@@ -196,7 +197,7 @@ print(few_shot_prompt.format(input="big"))
 # -> Antonym: short
 # ->
 # -> Word: big
-# -> Antonym: 
+# -> Antonym:
 ```
 
 ## Select examples for a prompt template
@@ -228,11 +229,7 @@ example_selector = LengthBasedExampleSelector(
     example_prompt=example_prompt, 
     # This is the maximum length that the formatted examples should be.
     # Length is measured by the get_text_length function below.
-    max_length=25
-    # This is the function used to get the length of a string, which is used
-    # to determine which examples to include. It is commented out because
-    # it is provided as a default value if none is specified.
-    # get_text_length: Callable[[str], int] = lambda x: len(re.split("\n| ", x))
+    max_length=25,
 )
 
 # We can now use the `example_selector` to create a `FewShotPromptTemplate`.

docs/use_cases/agents/multi_modal_output_agent.ipynb

@@ -125,7 +125,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "def show_output(output):\n",
+    "def show_ouput(output):\n",
     "    \"\"\"Display the multi-modal output from the agent.\"\"\"\n",
     "    UUID_PATTERN = re.compile(\n",
     "        r\"([0-9A-Za-z]{8}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{12})\"\n",
@@ -168,7 +168,7 @@
     }
    ],
    "source": [
-    "show_output(output)"
+    "show_ouput(output)"
    ]
   },
   {
@@ -266,7 +266,7 @@
     }
    ],
    "source": [
-    "show_output(output)"
+    "show_ouput(output)"
    ]
   },
   {

docs/use_cases/multi_modal/image_agent.ipynb

@@ -125,7 +125,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "def show_output(output):\n",
+    "def show_ouput(output):\n",
     "    \"\"\"Display the multi-modal output from the agent.\"\"\"\n",
     "    UUID_PATTERN = re.compile(\n",
     "        r\"([0-9A-Za-z]{8}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{12})\"\n",
@@ -168,7 +168,7 @@
     }
    ],
    "source": [
-    "show_output(output)"
+    "show_ouput(output)"
    ]
   },
   {
@@ -221,7 +221,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "show_output(output)"
+    "show_ouput(output)"
    ]
   },
   {

langchain/agents/agent_toolkits/csv/base.py

@@ -1,5 +1,5 @@
 """Agent for working with csvs."""
-from typing import Any, List, Optional, Union
+from typing import Any, Optional
 
 from langchain.agents.agent import AgentExecutor
 from langchain.agents.agent_toolkits.pandas.base import create_pandas_dataframe_agent
@@ -8,27 +8,13 @@ from langchain.base_language import BaseLanguageModel
 
 def create_csv_agent(
     llm: BaseLanguageModel,
-    path: Union[str, List[str]],
+    path: str,
     pandas_kwargs: Optional[dict] = None,
-    **kwargs: Any,
+    **kwargs: Any
 ) -> AgentExecutor:
     """Create csv agent by loading to a dataframe and using pandas agent."""
-    try:
-        import pandas as pd
-    except ImportError:
-        raise ValueError(
-            "pandas package not found, please install with `pip install pandas`"
-        )
+    import pandas as pd
 
     _kwargs = pandas_kwargs or {}
-    if isinstance(path, str):
-        df = pd.read_csv(path, **_kwargs)
-    elif isinstance(path, list):
-        df = []
-        for item in path:
-            if not isinstance(item, str):
-                raise ValueError(f"Expected str, got {type(path)}")
-            df.append(pd.read_csv(item, **_kwargs))
-    else:
-        raise ValueError(f"Expected str or list, got {type(path)}")
+    df = pd.read_csv(path, **_kwargs)
     return create_pandas_dataframe_agent(llm, df, **kwargs)

langchain/agents/agent_toolkits/pandas/base.py

@@ -1,147 +1,24 @@
 """Agent for working with pandas objects."""
-from typing import Any, Dict, List, Optional, Tuple
+from typing import Any, Dict, List, Optional
 
 from langchain.agents.agent import AgentExecutor
 from langchain.agents.agent_toolkits.pandas.prompt import (
-    MULTI_DF_PREFIX,
     PREFIX,
     SUFFIX_NO_DF,
     SUFFIX_WITH_DF,
-    SUFFIX_WITH_MULTI_DF,
 )
 from langchain.agents.mrkl.base import ZeroShotAgent
 from langchain.base_language import BaseLanguageModel
 from langchain.callbacks.base import BaseCallbackManager
 from langchain.chains.llm import LLMChain
-from langchain.prompts.base import BasePromptTemplate
 from langchain.tools.python.tool import PythonAstREPLTool
 
 
-def _get_multi_prompt(
-    dfs: List[Any],
-    prefix: Optional[str] = None,
-    suffix: Optional[str] = None,
-    input_variables: Optional[List[str]] = None,
-    include_df_in_prompt: Optional[bool] = True,
-) -> Tuple[BasePromptTemplate, List[PythonAstREPLTool]]:
-    num_dfs = len(dfs)
-    if suffix is not None:
-        suffix_to_use = suffix
-        include_dfs_head = True
-    elif include_df_in_prompt:
-        suffix_to_use = SUFFIX_WITH_MULTI_DF
-        include_dfs_head = True
-    else:
-        suffix_to_use = SUFFIX_NO_DF
-        include_dfs_head = False
-    if input_variables is None:
-        input_variables = ["input", "agent_scratchpad", "num_dfs"]
-        if include_dfs_head:
-            input_variables += ["dfs_head"]
-
-    if prefix is None:
-        prefix = MULTI_DF_PREFIX
-
-    df_locals = {}
-    for i, dataframe in enumerate(dfs):
-        df_locals[f"df{i + 1}"] = dataframe
-    tools = [PythonAstREPLTool(locals=df_locals)]
-
-    prompt = ZeroShotAgent.create_prompt(
-        tools, prefix=prefix, suffix=suffix_to_use, input_variables=input_variables
-    )
-
-    partial_prompt = prompt.partial()
-    if "dfs_head" in input_variables:
-        dfs_head = "\n\n".join([d.head().to_markdown() for d in dfs])
-        partial_prompt = partial_prompt.partial(num_dfs=str(num_dfs), dfs_head=dfs_head)
-    if "num_dfs" in input_variables:
-        partial_prompt = partial_prompt.partial(num_dfs=str(num_dfs))
-    return partial_prompt, tools
-
-
-def _get_single_prompt(
-    df: Any,
-    prefix: Optional[str] = None,
-    suffix: Optional[str] = None,
-    input_variables: Optional[List[str]] = None,
-    include_df_in_prompt: Optional[bool] = True,
-) -> Tuple[BasePromptTemplate, List[PythonAstREPLTool]]:
-    if suffix is not None:
-        suffix_to_use = suffix
-        include_df_head = True
-    elif include_df_in_prompt:
-        suffix_to_use = SUFFIX_WITH_DF
-        include_df_head = True
-    else:
-        suffix_to_use = SUFFIX_NO_DF
-        include_df_head = False
-
-    if input_variables is None:
-        input_variables = ["input", "agent_scratchpad"]
-        if include_df_head:
-            input_variables += ["df_head"]
-
-    if prefix is None:
-        prefix = PREFIX
-
-    tools = [PythonAstREPLTool(locals={"df": df})]
-
-    prompt = ZeroShotAgent.create_prompt(
-        tools, prefix=prefix, suffix=suffix_to_use, input_variables=input_variables
-    )
-
-    partial_prompt = prompt.partial()
-    if "df_head" in input_variables:
-        partial_prompt = partial_prompt.partial(df_head=str(df.head().to_markdown()))
-    return partial_prompt, tools
-
-
-def _get_prompt_and_tools(
-    df: Any,
-    prefix: Optional[str] = None,
-    suffix: Optional[str] = None,
-    input_variables: Optional[List[str]] = None,
-    include_df_in_prompt: Optional[bool] = True,
-) -> Tuple[BasePromptTemplate, List[PythonAstREPLTool]]:
-    try:
-        import pandas as pd
-    except ImportError:
-        raise ValueError(
-            "pandas package not found, please install with `pip install pandas`"
-        )
-
-    if include_df_in_prompt is not None and suffix is not None:
-        raise ValueError("If suffix is specified, include_df_in_prompt should not be.")
-
-    if isinstance(df, list):
-        for item in df:
-            if not isinstance(item, pd.DataFrame):
-                raise ValueError(f"Expected pandas object, got {type(df)}")
-        return _get_multi_prompt(
-            df,
-            prefix=prefix,
-            suffix=suffix,
-            input_variables=input_variables,
-            include_df_in_prompt=include_df_in_prompt,
-        )
-    else:
-        if not isinstance(df, pd.DataFrame):
-            raise ValueError(f"Expected pandas object, got {type(df)}")
-        return _get_single_prompt(
-            df,
-            prefix=prefix,
-            suffix=suffix,
-            input_variables=input_variables,
-            include_df_in_prompt=include_df_in_prompt,
-        )
-
-
 def create_pandas_dataframe_agent(
     llm: BaseLanguageModel,
     df: Any,
     callback_manager: Optional[BaseCallbackManager] = None,
-    prefix: Optional[str] = None,
+    prefix: str = PREFIX,
     suffix: Optional[str] = None,
     input_variables: Optional[List[str]] = None,
     verbose: bool = False,
@@ -154,17 +31,39 @@ def create_pandas_dataframe_agent(
     **kwargs: Dict[str, Any],
 ) -> AgentExecutor:
     """Construct a pandas agent from an LLM and dataframe."""
+    try:
+        import pandas as pd
+    except ImportError:
+        raise ImportError(
+            "pandas package not found, please install with `pip install pandas`"
+        )
 
-    prompt, tools = _get_prompt_and_tools(
-        df,
-        prefix=prefix,
-        suffix=suffix,
-        input_variables=input_variables,
-        include_df_in_prompt=include_df_in_prompt,
+    if not isinstance(df, pd.DataFrame):
+        raise ValueError(f"Expected pandas object, got {type(df)}")
+    if include_df_in_prompt is not None and suffix is not None:
+        raise ValueError("If suffix is specified, include_df_in_prompt should not be.")
+    if suffix is not None:
+        suffix_to_use = suffix
+        if input_variables is None:
+            input_variables = ["df", "input", "agent_scratchpad"]
+    else:
+        if include_df_in_prompt:
+            suffix_to_use = SUFFIX_WITH_DF
+            input_variables = ["df", "input", "agent_scratchpad"]
+        else:
+            suffix_to_use = SUFFIX_NO_DF
+            input_variables = ["input", "agent_scratchpad"]
+    tools = [PythonAstREPLTool(locals={"df": df})]
+    prompt = ZeroShotAgent.create_prompt(
+        tools, prefix=prefix, suffix=suffix_to_use, input_variables=input_variables
     )
+    if "df" in input_variables:
+        partial_prompt = prompt.partial(df=str(df.head().to_markdown()))
+    else:
+        partial_prompt = prompt
     llm_chain = LLMChain(
         llm=llm,
-        prompt=prompt,
+        prompt=partial_prompt,
         callback_manager=callback_manager,
     )
     tool_names = [tool.name for tool in tools]

langchain/agents/agent_toolkits/pandas/prompt.py

@@ -4,10 +4,6 @@ PREFIX = """
 You are working with a pandas dataframe in Python. The name of the dataframe is `df`.
 You should use the tools below to answer the question posed of you:"""
 
-MULTI_DF_PREFIX = """
-You are working with {num_dfs} pandas dataframes in Python named df1, df2, etc. You 
-should use the tools below to answer the question posed of you:"""
-
 SUFFIX_NO_DF = """
 Begin!
 Question: {input}
@@ -15,15 +11,7 @@ Question: {input}
 
 SUFFIX_WITH_DF = """
 This is the result of `print(df.head())`:
-{df_head}
-
-Begin!
-Question: {input}
-{agent_scratchpad}"""
-
-SUFFIX_WITH_MULTI_DF = """
-This is the result of `print(df.head())` for each dataframe:
-{dfs_head}
+{df}
 
 Begin!
 Question: {input}

langchain/agents/chat/output_parser.py

@@ -1,8 +1,8 @@
+import json
 from typing import Union
 
 from langchain.agents.agent import AgentOutputParser
 from langchain.agents.chat.prompt import FORMAT_INSTRUCTIONS
-from langchain.output_parsers.json import parse_json_markdown
 from langchain.schema import AgentAction, AgentFinish, OutputParserException
 
 FINAL_ANSWER_ACTION = "Final Answer:"
@@ -18,7 +18,8 @@ class ChatOutputParser(AgentOutputParser):
                 {"output": text.split(FINAL_ANSWER_ACTION)[-1].strip()}, text
             )
         try:
-            response = parse_json_markdown(text)
+            action = text.split("```")[1]
+            response = json.loads(action.strip())
             return AgentAction(response["action"], response["action_input"], text)
 
         except Exception:

langchain/agents/conversational_chat/output_parser.py

@@ -1,10 +1,10 @@
 from __future__ import annotations
 
+import json
 from typing import Union
 
 from langchain.agents import AgentOutputParser
 from langchain.agents.conversational_chat.prompt import FORMAT_INSTRUCTIONS
-from langchain.output_parsers.json import parse_json_markdown
 from langchain.schema import AgentAction, AgentFinish, OutputParserException
 
 
@@ -14,7 +14,19 @@ class ConvoOutputParser(AgentOutputParser):
 
     def parse(self, text: str) -> Union[AgentAction, AgentFinish]:
         try:
-            response = parse_json_markdown(text)
+            cleaned_output = text.strip()
+            if "```json" in cleaned_output:
+                _, cleaned_output = cleaned_output.split("```json")
+            if "```" in cleaned_output:
+                cleaned_output, _ = cleaned_output.split("```")
+            if cleaned_output.startswith("```json"):
+                cleaned_output = cleaned_output[len("```json") :]
+            if cleaned_output.startswith("```"):
+                cleaned_output = cleaned_output[len("```") :]
+            if cleaned_output.endswith("```"):
+                cleaned_output = cleaned_output[: -len("```")]
+            cleaned_output = cleaned_output.strip()
+            response = json.loads(cleaned_output)
             action, action_input = response["action"], response["action_input"]
             if action == "Final Answer":
                 return AgentFinish({"output": action_input}, text)

langchain/agents/load_tools.py

@@ -46,7 +46,6 @@ from langchain.utilities.awslambda import LambdaWrapper
 from langchain.utilities.graphql import GraphQLAPIWrapper
 from langchain.utilities.searx_search import SearxSearchWrapper
 from langchain.utilities.serpapi import SerpAPIWrapper
-from langchain.utilities.twilio import TwilioAPIWrapper
 from langchain.utilities.wikipedia import WikipediaAPIWrapper
 from langchain.utilities.wolfram_alpha import WolframAlphaAPIWrapper
 from langchain.utilities.openweathermap import OpenWeatherMapAPIWrapper
@@ -219,14 +218,6 @@ def _get_serpapi(**kwargs: Any) -> BaseTool:
     )
 
 
-def _get_twilio(**kwargs: Any) -> BaseTool:
-    return Tool(
-        name="Text Message",
-        description="Useful for when you need to send a text message to a provided phone number.",
-        func=TwilioAPIWrapper(**kwargs).run,
-    )
-
-
 def _get_searx_search(**kwargs: Any) -> BaseTool:
     return SearxSearchRun(wrapper=SearxSearchWrapper(**kwargs))
 
@@ -295,7 +286,6 @@ _EXTRA_OPTIONAL_TOOLS: Dict[str, Tuple[Callable[[KwArg(Any)], BaseTool], List[st
         ["serper_api_key", "aiosession"],
     ),
     "serpapi": (_get_serpapi, ["serpapi_api_key", "aiosession"]),
-    "twilio": (_get_twilio, ["account_sid", "auth_token", "from_number"]),
     "searx-search": (_get_searx_search, ["searx_host", "engines", "aiosession"]),
     "wikipedia": (_get_wikipedia, ["top_k_results", "lang"]),
     "arxiv": (

langchain/agents/self_ask_with_search/output_parser.py

@@ -1,21 +1,24 @@
-from typing import Sequence, Union
+from typing import Union
 
 from langchain.agents.agent import AgentOutputParser
 from langchain.schema import AgentAction, AgentFinish, OutputParserException
 
 
 class SelfAskOutputParser(AgentOutputParser):
-    followups: Sequence[str] = ("Follow up:", "Followup:")
-    finish_string: str = "So the final answer is: "
-
     def parse(self, text: str) -> Union[AgentAction, AgentFinish]:
+        followup = "Follow up:"
         last_line = text.split("\n")[-1]
-        if not any([follow in last_line for follow in self.followups]):
-            if self.finish_string not in last_line:
-                raise OutputParserException(f"Could not parse output: {text}")
-            return AgentFinish({"output": last_line[len(self.finish_string) :]}, text)
 
-        after_colon = text.split(":")[-1].strip()
+        if followup not in last_line:
+            finish_string = "So the final answer is: "
+            if finish_string not in last_line:
+                raise OutputParserException(f"Could not parse output: {text}")
+            return AgentFinish({"output": last_line[len(finish_string) :]}, text)
+
+        after_colon = text.split(":")[-1]
+
+        if " " == after_colon[0]:
+            after_colon = after_colon[1:]
         return AgentAction("Intermediate Answer", after_colon, text)
 
     @property

langchain/base_language.py

@@ -58,16 +58,6 @@ class BaseLanguageModel(BaseModel, ABC):
     ) -> BaseMessage:
         """Predict message from messages."""
 
-    @abstractmethod
-    async def apredict(self, text: str, *, stop: Optional[Sequence[str]] = None) -> str:
-        """Predict text from text."""
-
-    @abstractmethod
-    async def apredict_messages(
-        self, messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None
-    ) -> BaseMessage:
-        """Predict message from messages."""
-
     def get_token_ids(self, text: str) -> List[int]:
         """Get the token present in the text."""
         return _get_token_ids_default_method(text)

langchain/cache.py

@@ -1,30 +1,14 @@
 """Beta Feature: base interface for cache."""
-from __future__ import annotations
-
 import hashlib
 import inspect
 import json
 from abc import ABC, abstractmethod
-from datetime import timedelta
-from typing import (
-    TYPE_CHECKING,
-    Any,
-    Callable,
-    Dict,
-    List,
-    Optional,
-    Tuple,
-    Type,
-    Union,
-    cast,
-)
+from typing import Any, Callable, Dict, List, Optional, Tuple, Type, Union, cast
 
 from sqlalchemy import Column, Integer, String, create_engine, select
 from sqlalchemy.engine.base import Engine
 from sqlalchemy.orm import Session
 
-from langchain.utils import get_from_env
-
 try:
     from sqlalchemy.orm import declarative_base
 except ImportError:
@@ -34,9 +18,6 @@ from langchain.embeddings.base import Embeddings
 from langchain.schema import Generation
 from langchain.vectorstores.redis import Redis as RedisVectorstore
 
-if TYPE_CHECKING:
-    import momento
-
 RETURN_VAL_TYPE = List[Generation]
 
 
@@ -45,39 +26,6 @@ def _hash(_input: str) -> str:
     return hashlib.md5(_input.encode()).hexdigest()
 
 
-def _dump_generations_to_json(generations: RETURN_VAL_TYPE) -> str:
-    """Dump generations to json.
-
-    Args:
-        generations (RETURN_VAL_TYPE): A list of language model generations.
-
-    Returns:
-        str: Json representing a list of generations.
-    """
-    return json.dumps([generation.dict() for generation in generations])
-
-
-def _load_generations_from_json(generations_json: str) -> RETURN_VAL_TYPE:
-    """Load generations from json.
-
-    Args:
-        generations_json (str): A string of json representing a list of generations.
-
-    Raises:
-        ValueError: Could not decode json string to list of generations.
-
-    Returns:
-        RETURN_VAL_TYPE: A list of generations.
-    """
-    try:
-        results = json.loads(generations_json)
-        return [Generation(**generation_dict) for generation_dict in results]
-    except json.JSONDecodeError:
-        raise ValueError(
-            f"Could not decode json to list of generations: {generations_json}"
-        )
-
-
 class BaseCache(ABC):
     """Base interface for cache."""
 
@@ -442,179 +390,3 @@ class GPTCache(BaseCache):
             gptcache_instance.flush()
 
         self.gptcache_dict.clear()
-
-
-def _ensure_cache_exists(cache_client: momento.CacheClient, cache_name: str) -> None:
-    """Create cache if it doesn't exist.
-
-    Raises:
-        SdkException: Momento service or network error
-        Exception: Unexpected response
-    """
-    from momento.responses import CreateCache
-
-    create_cache_response = cache_client.create_cache(cache_name)
-    if isinstance(create_cache_response, CreateCache.Success) or isinstance(
-        create_cache_response, CreateCache.CacheAlreadyExists
-    ):
-        return None
-    elif isinstance(create_cache_response, CreateCache.Error):
-        raise create_cache_response.inner_exception
-    else:
-        raise Exception(f"Unexpected response cache creation: {create_cache_response}")
-
-
-def _validate_ttl(ttl: Optional[timedelta]) -> None:
-    if ttl is not None and ttl <= timedelta(seconds=0):
-        raise ValueError(f"ttl must be positive but was {ttl}.")
-
-
-class MomentoCache(BaseCache):
-    """Cache that uses Momento as a backend. See https://gomomento.com/"""
-
-    def __init__(
-        self,
-        cache_client: momento.CacheClient,
-        cache_name: str,
-        *,
-        ttl: Optional[timedelta] = None,
-        ensure_cache_exists: bool = True,
-    ):
-        """Instantiate a prompt cache using Momento as a backend.
-
-        Note: to instantiate the cache client passed to MomentoCache,
-        you must have a Momento account. See https://gomomento.com/.
-
-        Args:
-            cache_client (CacheClient): The Momento cache client.
-            cache_name (str): The name of the cache to use to store the data.
-            ttl (Optional[timedelta], optional): The time to live for the cache items.
-                Defaults to None, ie use the client default TTL.
-            ensure_cache_exists (bool, optional): Create the cache if it doesn't
-                exist. Defaults to True.
-
-        Raises:
-            ImportError: Momento python package is not installed.
-            TypeError: cache_client is not of type momento.CacheClientObject
-            ValueError: ttl is non-null and non-negative
-        """
-        try:
-            from momento import CacheClient
-        except ImportError:
-            raise ImportError(
-                "Could not import momento python package. "
-                "Please install it with `pip install momento`."
-            )
-        if not isinstance(cache_client, CacheClient):
-            raise TypeError("cache_client must be a momento.CacheClient object.")
-        _validate_ttl(ttl)
-        if ensure_cache_exists:
-            _ensure_cache_exists(cache_client, cache_name)
-
-        self.cache_client = cache_client
-        self.cache_name = cache_name
-        self.ttl = ttl
-
-    @classmethod
-    def from_client_params(
-        cls,
-        cache_name: str,
-        ttl: timedelta,
-        *,
-        configuration: Optional[momento.config.Configuration] = None,
-        auth_token: Optional[str] = None,
-        **kwargs: Any,
-    ) -> MomentoCache:
-        """Construct cache from CacheClient parameters."""
-        try:
-            from momento import CacheClient, Configurations, CredentialProvider
-        except ImportError:
-            raise ImportError(
-                "Could not import momento python package. "
-                "Please install it with `pip install momento`."
-            )
-        if configuration is None:
-            configuration = Configurations.Laptop.v1()
-        auth_token = auth_token or get_from_env("auth_token", "MOMENTO_AUTH_TOKEN")
-        credentials = CredentialProvider.from_string(auth_token)
-        cache_client = CacheClient(configuration, credentials, default_ttl=ttl)
-        return cls(cache_client, cache_name, ttl=ttl, **kwargs)
-
-    def __key(self, prompt: str, llm_string: str) -> str:
-        """Compute cache key from prompt and associated model and settings.
-
-        Args:
-            prompt (str): The prompt run through the language model.
-            llm_string (str): The language model version and settings.
-
-        Returns:
-            str: The cache key.
-        """
-        return _hash(prompt + llm_string)
-
-    def lookup(self, prompt: str, llm_string: str) -> Optional[RETURN_VAL_TYPE]:
-        """Lookup llm generations in cache by prompt and associated model and settings.
-
-        Args:
-            prompt (str): The prompt run through the language model.
-            llm_string (str): The language model version and settings.
-
-        Raises:
-            SdkException: Momento service or network error
-
-        Returns:
-            Optional[RETURN_VAL_TYPE]: A list of language model generations.
-        """
-        from momento.responses import CacheGet
-
-        generations = []
-
-        get_response = self.cache_client.get(
-            self.cache_name, self.__key(prompt, llm_string)
-        )
-        if isinstance(get_response, CacheGet.Hit):
-            value = get_response.value_string
-            generations = _load_generations_from_json(value)
-        elif isinstance(get_response, CacheGet.Miss):
-            pass
-        elif isinstance(get_response, CacheGet.Error):
-            raise get_response.inner_exception
-        return generations if generations else None
-
-    def update(self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE) -> None:
-        """Store llm generations in cache.
-
-        Args:
-            prompt (str): The prompt run through the language model.
-            llm_string (str): The language model string.
-            return_val (RETURN_VAL_TYPE): A list of language model generations.
-
-        Raises:
-            SdkException: Momento service or network error
-            Exception: Unexpected response
-        """
-        key = self.__key(prompt, llm_string)
-        value = _dump_generations_to_json(return_val)
-        set_response = self.cache_client.set(self.cache_name, key, value, self.ttl)
-        from momento.responses import CacheSet
-
-        if isinstance(set_response, CacheSet.Success):
-            pass
-        elif isinstance(set_response, CacheSet.Error):
-            raise set_response.inner_exception
-        else:
-            raise Exception(f"Unexpected response: {set_response}")
-
-    def clear(self, **kwargs: Any) -> None:
-        """Clear the cache.
-
-        Raises:
-            SdkException: Momento service or network error
-        """
-        from momento.responses import CacheFlush
-
-        flush_response = self.cache_client.flush_cache(self.cache_name)
-        if isinstance(flush_response, CacheFlush.Success):
-            pass
-        elif isinstance(flush_response, CacheFlush.Error):
-            raise flush_response.inner_exception

langchain/callbacks/manager.py

@@ -888,12 +888,7 @@ def _configure(
                     handler.ensure_session()
                     callback_manager.add_handler(handler, True)
                 except Exception as e:
-                    logger.warning(
-                        "Unable to load requested LangChainTracer."
-                        " To disable this warning,"
-                        " unset the  LANGCHAIN_TRACING_V2 environment variables.",
-                        e,
-                    )
+                    logger.debug("Unable to load requested LangChainTracer", e)
         if open_ai is not None and not any(
             isinstance(handler, OpenAICallbackHandler)
             for handler in callback_manager.handlers

langchain/callbacks/openai_info.py

@@ -24,36 +24,20 @@ MODEL_COST_PER_1K_TOKENS = {
     "text-davinci-003": 0.02,
     "text-davinci-002": 0.02,
     "code-davinci-002": 0.02,
-    "ada-finetuned": 0.0016,
-    "babbage-finetuned": 0.0024,
-    "curie-finetuned": 0.012,
-    "davinci-finetuned": 0.12,
 }
 
 
-def standardize_model_name(
-    model_name: str,
-    is_completion: bool = False,
-) -> str:
-    model_name = model_name.lower()
-    if "ft-" in model_name:
-        return model_name.split(":")[0] + "-finetuned"
-    elif is_completion and model_name.startswith("gpt-4"):
-        return model_name + "-completion"
-    else:
-        return model_name
-
-
 def get_openai_token_cost_for_model(
     model_name: str, num_tokens: int, is_completion: bool = False
 ) -> float:
-    model_name = standardize_model_name(model_name, is_completion=is_completion)
-    if model_name not in MODEL_COST_PER_1K_TOKENS:
+    suffix = "-completion" if is_completion and model_name.startswith("gpt-4") else ""
+    model = model_name.lower() + suffix
+    if model not in MODEL_COST_PER_1K_TOKENS:
         raise ValueError(
             f"Unknown model: {model_name}. Please provide a valid OpenAI model name."
             "Known models are: " + ", ".join(MODEL_COST_PER_1K_TOKENS.keys())
         )
-    return MODEL_COST_PER_1K_TOKENS[model_name] * num_tokens / 1000
+    return MODEL_COST_PER_1K_TOKENS[model] * num_tokens / 1000
 
 
 class OpenAICallbackHandler(BaseCallbackHandler):
@@ -99,8 +83,8 @@ class OpenAICallbackHandler(BaseCallbackHandler):
         token_usage = response.llm_output["token_usage"]
         completion_tokens = token_usage.get("completion_tokens", 0)
         prompt_tokens = token_usage.get("prompt_tokens", 0)
-        model_name = standardize_model_name(response.llm_output.get("model_name", ""))
-        if model_name in MODEL_COST_PER_1K_TOKENS:
+        model_name = response.llm_output.get("model_name")
+        if model_name and model_name in MODEL_COST_PER_1K_TOKENS:
             completion_cost = get_openai_token_cost_for_model(
                 model_name, completion_tokens, is_completion=True
             )

langchain/callbacks/streaming_aiter.py

@@ -58,8 +58,7 @@ class AsyncIteratorCallbackHandler(AsyncCallbackHandler):
             )
 
             # Cancel the other task
-            if other:
-                other.pop().cancel()
+            other.pop().cancel()
 
             # Extract the value of the first completed task
             token_or_done = cast(Union[str, Literal[True]], done.pop().result())

langchain/chains/combine_documents/map_reduce.py

@@ -195,7 +195,9 @@ class MapReduceDocumentsChain(BaseCombineDocumentsChain):
             for docs in new_result_doc_list:
                 new_doc = _collapse_docs(docs, _collapse_docs_func, **kwargs)
                 result_docs.append(new_doc)
-            num_tokens = length_func(result_docs, **kwargs)
+            num_tokens = self.combine_document_chain.prompt_length(
+                result_docs, **kwargs
+            )
         if self.return_intermediate_steps:
             _results = [r[self.llm_chain.output_key] for r in results]
             extra_return_dict = {"intermediate_steps": _results}

langchain/chains/graph_qa/cypher.py

@@ -8,7 +8,7 @@ from pydantic import Field
 from langchain.base_language import BaseLanguageModel
 from langchain.callbacks.manager import CallbackManagerForChainRun
 from langchain.chains.base import Chain
-from langchain.chains.graph_qa.prompts import CYPHER_GENERATION_PROMPT, CYPHER_QA_PROMPT
+from langchain.chains.graph_qa.prompts import CYPHER_GENERATION_PROMPT, PROMPT
 from langchain.chains.llm import LLMChain
 from langchain.graphs.neo4j_graph import Neo4jGraph
 from langchain.prompts.base import BasePromptTemplate
@@ -45,7 +45,7 @@ class GraphCypherQAChain(Chain):
         cls,
         llm: BaseLanguageModel,
         *,
-        qa_prompt: BasePromptTemplate = CYPHER_QA_PROMPT,
+        qa_prompt: BasePromptTemplate = PROMPT,
         cypher_prompt: BasePromptTemplate = CYPHER_GENERATION_PROMPT,
         **kwargs: Any,
     ) -> GraphCypherQAChain:

langchain/chains/graph_qa/prompts.py

@@ -48,16 +48,3 @@ The question is:
 CYPHER_GENERATION_PROMPT = PromptTemplate(
     input_variables=["schema", "question"], template=CYPHER_GENERATION_TEMPLATE
 )
-
-CYPHER_QA_TEMPLATE = """You are an assistant that helps to form nice and human understandable answers.
-The information part contains the provided information that you can use to construct an answer.
-The provided information is authorative, you must never doubt it or try to use your internal knowledge to correct it.
-Make it sound like the information are coming from an AI assistant, but don't add any information.
-Information:
-{context}
-
-Question: {question}
-Helpful Answer:"""
-CYPHER_QA_PROMPT = PromptTemplate(
-    input_variables=["context", "question"], template=CYPHER_QA_TEMPLATE
-)

langchain/chains/llm_math/prompt.py

@@ -16,6 +16,7 @@ Answer: ${{Answer}}
 Begin.
 
 Question: What is 37593 * 67?
+
 ```text
 37593 * 67
 ```
@@ -25,16 +26,6 @@ Question: What is 37593 * 67?
 ```
 Answer: 2518731
 
-Question: 37593^(1/5)
-```text
-37593**(1/5)
-```
-...numexpr.evaluate("37593**(1/5)")...
-```output
-8.222831614237718
-```
-Answer: 8.222831614237718
-
 Question: {question}
 """
 

langchain/chains/query_constructor/base.py

@@ -22,7 +22,7 @@ from langchain.chains.query_constructor.prompt import (
     SCHEMA_WITH_LIMIT,
 )
 from langchain.chains.query_constructor.schema import AttributeInfo
-from langchain.output_parsers.json import parse_and_check_json_markdown
+from langchain.output_parsers.structured import parse_json_markdown
 from langchain.schema import BaseOutputParser, OutputParserException
 
 
@@ -33,7 +33,7 @@ class StructuredQueryOutputParser(BaseOutputParser[StructuredQuery]):
     def parse(self, text: str) -> StructuredQuery:
         try:
             expected_keys = ["query", "filter"]
-            parsed = parse_and_check_json_markdown(text, expected_keys)
+            parsed = parse_json_markdown(text, expected_keys)
             if len(parsed["query"]) == 0:
                 parsed["query"] = " "
             if parsed["filter"] == "NO_FILTER" or not parsed["filter"]:

langchain/chains/router/llm_router.py

@@ -9,7 +9,7 @@ from langchain.base_language import BaseLanguageModel
 from langchain.callbacks.manager import CallbackManagerForChainRun
 from langchain.chains import LLMChain
 from langchain.chains.router.base import RouterChain
-from langchain.output_parsers.json import parse_and_check_json_markdown
+from langchain.output_parsers.structured import parse_json_markdown
 from langchain.prompts import BasePromptTemplate
 from langchain.schema import BaseOutputParser, OutputParserException
 
@@ -77,7 +77,7 @@ class RouterOutputParser(BaseOutputParser[Dict[str, str]]):
     def parse(self, text: str) -> Dict[str, Any]:
         try:
             expected_keys = ["destination", "next_inputs"]
-            parsed = parse_and_check_json_markdown(text, expected_keys)
+            parsed = parse_json_markdown(text, expected_keys)
             if not isinstance(parsed["destination"], str):
                 raise ValueError("Expected 'destination' to be a string.")
             if not isinstance(parsed["next_inputs"], self.next_inputs_type):

langchain/chat_models/__init__.py

@@ -3,7 +3,6 @@ from langchain.chat_models.azure_openai import AzureChatOpenAI
 from langchain.chat_models.google_palm import ChatGooglePalm
 from langchain.chat_models.openai import ChatOpenAI
 from langchain.chat_models.promptlayer_openai import PromptLayerChatOpenAI
-from langchain.chat_models.vertexai import ChatVertexAI
 
 __all__ = [
     "ChatOpenAI",
@@ -11,5 +10,4 @@ __all__ = [
     "PromptLayerChatOpenAI",
     "ChatAnthropic",
     "ChatGooglePalm",
-    "ChatVertexAI",
 ]

langchain/chat_models/azure_openai.py

@@ -24,7 +24,6 @@ class AzureChatOpenAI(ChatOpenAI):
     - ``OPENAI_API_KEY``
     - ``OPENAI_API_BASE``
     - ``OPENAI_API_VERSION``
-    - ``OPENAI_PROXY``
 
     For exmaple, if you have `gpt-35-turbo` deployed, with the deployment name
     `35-turbo-dev`, the constructor should look like:
@@ -47,7 +46,6 @@ class AzureChatOpenAI(ChatOpenAI):
     openai_api_version: str = ""
     openai_api_key: str = ""
     openai_organization: str = ""
-    openai_proxy: str = ""
 
     @root_validator()
     def validate_environment(cls, values: Dict) -> Dict:
@@ -78,12 +76,6 @@ class AzureChatOpenAI(ChatOpenAI):
             "OPENAI_ORGANIZATION",
             default="",
         )
-        openai_proxy = get_from_dict_or_env(
-            values,
-            "openai_proxy",
-            "OPENAI_PROXY",
-            default="",
-        )
         try:
             import openai
 
@@ -93,8 +85,6 @@ class AzureChatOpenAI(ChatOpenAI):
             openai.api_key = openai_api_key
             if openai_organization:
                 openai.organization = openai_organization
-            if openai_proxy:
-                openai.proxy = {"http": openai_proxy, "https": openai_proxy}  # type: ignore[assignment]  # noqa: E501
         except ImportError:
             raise ImportError(
                 "Could not import openai python package. "

langchain/chat_models/base.py

@@ -183,19 +183,6 @@ class BaseChatModel(BaseLanguageModel, ABC):
         else:
             raise ValueError("Unexpected generation type")
 
-    async def _call_async(
-        self,
-        messages: List[BaseMessage],
-        stop: Optional[List[str]] = None,
-        callbacks: Callbacks = None,
-    ) -> BaseMessage:
-        result = await self.agenerate([messages], stop=stop, callbacks=callbacks)
-        generation = result.generations[0][0]
-        if isinstance(generation, ChatGeneration):
-            return generation.message
-        else:
-            raise ValueError("Unexpected generation type")
-
     def call_as_llm(self, message: str, stop: Optional[List[str]] = None) -> str:
         return self.predict(message, stop=stop)
 
@@ -216,23 +203,6 @@ class BaseChatModel(BaseLanguageModel, ABC):
             _stop = list(stop)
         return self(messages, stop=_stop)
 
-    async def apredict(self, text: str, *, stop: Optional[Sequence[str]] = None) -> str:
-        if stop is None:
-            _stop = None
-        else:
-            _stop = list(stop)
-        result = await self._call_async([HumanMessage(content=text)], stop=_stop)
-        return result.content
-
-    async def apredict_messages(
-        self, messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None
-    ) -> BaseMessage:
-        if stop is None:
-            _stop = None
-        else:
-            _stop = list(stop)
-        return await self._call_async(messages, stop=_stop)
-
     @property
     def _identifying_params(self) -> Mapping[str, Any]:
         """Get the identifying parameters."""

langchain/chat_models/openai.py

@@ -148,8 +148,6 @@ class ChatOpenAI(BaseChatModel):
     leave blank if not using a proxy or service emulator."""
     openai_api_base: Optional[str] = None
     openai_organization: Optional[str] = None
-    # to support explicit proxy for OpenAI
-    openai_proxy: Optional[str] = None
     request_timeout: Optional[Union[float, Tuple[float, float]]] = None
     """Timeout for requests to OpenAI completion API. Default is 600 seconds."""
     max_retries: int = 6
@@ -211,12 +209,6 @@ class ChatOpenAI(BaseChatModel):
             "OPENAI_API_BASE",
             default="",
         )
-        openai_proxy = get_from_dict_or_env(
-            values,
-            "openai_proxy",
-            "OPENAI_PROXY",
-            default="",
-        )
         try:
             import openai
 
@@ -230,8 +222,6 @@ class ChatOpenAI(BaseChatModel):
             openai.organization = openai_organization
         if openai_api_base:
             openai.api_base = openai_api_base
-        if openai_proxy:
-            openai.proxy = {"http": openai_proxy, "https": openai_proxy}  # type: ignore[assignment]  # noqa: E501
         try:
             values["client"] = openai.ChatCompletion
         except AttributeError:

langchain/chat_models/vertexai.py

@@ -1,137 +0,0 @@
-"""Wrapper around Google VertexAI chat-based models."""
-from dataclasses import dataclass, field
-from typing import Dict, List, Optional
-
-from pydantic import root_validator
-
-from langchain.callbacks.manager import (
-    AsyncCallbackManagerForLLMRun,
-    CallbackManagerForLLMRun,
-)
-from langchain.chat_models.base import BaseChatModel
-from langchain.llms.vertexai import _VertexAICommon
-from langchain.schema import (
-    AIMessage,
-    BaseMessage,
-    ChatGeneration,
-    ChatResult,
-    HumanMessage,
-    SystemMessage,
-)
-from langchain.utilities.vertexai import raise_vertex_import_error
-
-
-@dataclass
-class _MessagePair:
-    """InputOutputTextPair represents a pair of input and output texts."""
-
-    question: HumanMessage
-    answer: AIMessage
-
-
-@dataclass
-class _ChatHistory:
-    """InputOutputTextPair represents a pair of input and output texts."""
-
-    history: List[_MessagePair] = field(default_factory=list)
-    system_message: Optional[SystemMessage] = None
-
-
-def _parse_chat_history(history: List[BaseMessage]) -> _ChatHistory:
-    """Parse a sequence of messages into history.
-
-    A sequence should be either (SystemMessage, HumanMessage, AIMessage,
-    HumanMessage, AIMessage, ...) or (HumanMessage, AIMessage, HumanMessage,
-    AIMessage, ...).
-
-    Args:
-        history: The list of messages to re-create the history of the chat.
-    Returns:
-        A parsed chat history.
-    Raises:
-        ValueError: If a sequence of message is odd, or a human message is not followed
-            by a message from AI (e.g., Human, Human, AI or AI, AI, Human).
-    """
-    if not history:
-        return _ChatHistory()
-    first_message = history[0]
-    system_message = first_message if isinstance(first_message, SystemMessage) else None
-    chat_history = _ChatHistory(system_message=system_message)
-    messages_left = history[1:] if system_message else history
-    if len(messages_left) % 2 != 0:
-        raise ValueError(
-            f"Amount of messages in history should be even, got {len(messages_left)}!"
-        )
-    for question, answer in zip(messages_left[::2], messages_left[1::2]):
-        if not isinstance(question, HumanMessage) or not isinstance(answer, AIMessage):
-            raise ValueError(
-                "A human message should follow a bot one, "
-                f"got {question.type}, {answer.type}."
-            )
-        chat_history.history.append(_MessagePair(question=question, answer=answer))
-    return chat_history
-
-
-class ChatVertexAI(_VertexAICommon, BaseChatModel):
-    """Wrapper around Vertex AI large language models."""
-
-    model_name: str = "chat-bison"
-
-    @root_validator()
-    def validate_environment(cls, values: Dict) -> Dict:
-        """Validate that the python package exists in environment."""
-        cls._try_init_vertexai(values)
-        try:
-            from vertexai.preview.language_models import ChatModel
-        except ImportError:
-            raise_vertex_import_error()
-        values["client"] = ChatModel.from_pretrained(values["model_name"])
-        return values
-
-    def _generate(
-        self,
-        messages: List[BaseMessage],
-        stop: Optional[List[str]] = None,
-        run_manager: Optional[CallbackManagerForLLMRun] = None,
-    ) -> ChatResult:
-        """Generate next turn in the conversation.
-
-        Args:
-            messages: The history of the conversation as a list of messages.
-            stop: The list of stop words (optional).
-            run_manager: The Callbackmanager for LLM run, it's not used at the moment.
-
-        Returns:
-            The ChatResult that contains outputs generated by the model.
-
-        Raises:
-            ValueError: if the last message in the list is not from human.
-        """
-        if not messages:
-            raise ValueError(
-                "You should provide at least one message to start the chat!"
-            )
-        question = messages[-1]
-        if not isinstance(question, HumanMessage):
-            raise ValueError(
-                f"Last message in the list should be from human, got {question.type}."
-            )
-
-        history = _parse_chat_history(messages[:-1])
-        context = history.system_message.content if history.system_message else None
-        chat = self.client.start_chat(context=context, **self._default_params)
-        for pair in history.history:
-            chat._history.append((pair.question.content, pair.answer.content))
-        response = chat.send_message(question.content)
-        text = self._enforce_stop_words(response.text, stop)
-        return ChatResult(generations=[ChatGeneration(message=AIMessage(content=text))])
-
-    async def _agenerate(
-        self,
-        messages: List[BaseMessage],
-        stop: Optional[List[str]] = None,
-        run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
-    ) -> ChatResult:
-        raise NotImplementedError(
-            """Vertex AI doesn't support async requests at the moment."""
-        )

langchain/cli/docker-compose.yaml

@@ -19,7 +19,6 @@ services:
       - PORT=1984
       - LANGCHAIN_ENV=local_docker
       - LOG_LEVEL=warning
-      - OPENAI_API_KEY=${OPENAI_API_KEY}
     ports:
       - 1984:1984
     depends_on:

langchain/cli/main.py

@@ -1,5 +1,4 @@
 import argparse
-import json
 import logging
 import os
 import shutil
@@ -7,7 +6,7 @@ import subprocess
 from contextlib import contextmanager
 from pathlib import Path
 from subprocess import CalledProcessError
-from typing import Dict, Generator, List, Mapping, Optional, Union, cast
+from typing import Generator, List, Optional
 
 import requests
 import yaml
@@ -20,50 +19,6 @@ logger = logging.getLogger(__name__)
 _DIR = Path(__file__).parent
 
 
-def pprint_services(services_status: List[Mapping[str, Union[str, List[str]]]]) -> None:
-    # Loop through and collect Service, State, and Publishers["PublishedPorts"]
-    # for each service
-    services = []
-    for service in services_status:
-        service_status: Dict[str, str] = {
-            "Service": str(service["Service"]),
-            "Status": str(service["Status"]),
-        }
-        publishers = cast(List[Dict], service.get("Publishers", []))
-        if publishers:
-            service_status["PublishedPorts"] = ", ".join(
-                [str(publisher["PublishedPort"]) for publisher in publishers]
-            )
-        services.append(service_status)
-
-    max_service_len = max(len(service["Service"]) for service in services)
-    max_state_len = max(len(service["Status"]) for service in services)
-    service_message = [
-        "\n"
-        + "Service".ljust(max_service_len + 2)
-        + "Status".ljust(max_state_len + 2)
-        + "Published Ports"
-    ]
-    for service in services:
-        service_str = service["Service"].ljust(max_service_len + 2)
-        state_str = service["Status"].ljust(max_state_len + 2)
-        ports_str = service.get("PublishedPorts", "")
-        service_message.append(service_str + state_str + ports_str)
-
-    langchain_endpoint: str = "http://localhost:1984"
-    used_ngrok = any(["ngrok" in service["Service"] for service in services])
-    if used_ngrok:
-        langchain_endpoint = get_ngrok_url(auth_token=None)
-
-    service_message.append(
-        "\nTo connect, set the following environment variables"
-        " in your LangChain application:"
-        "\nLANGCHAIN_TRACING_V2=true"
-        f"\nLANGCHAIN_ENDPOINT={langchain_endpoint}"
-    )
-    logger.info("\n".join(service_message))
-
-
 def get_docker_compose_command() -> List[str]:
     """Get the correct docker compose command for this system."""
     try:
@@ -218,7 +173,6 @@ class PlusCommand:
         expose: bool = False,
         auth_token: Optional[str] = None,
         dev: bool = False,
-        openai_api_key: Optional[str] = None,
     ) -> None:
         """Run the LangChainPlus server locally.
 
@@ -226,16 +180,9 @@ class PlusCommand:
             expose: If True, expose the server to the internet using ngrok.
             auth_token: The ngrok authtoken to use (visible in the ngrok dashboard).
                 If not provided, ngrok server session length will be restricted.
-            dev: If True, use the development (rc) image of LangChainPlus.
-            openai_api_key: The OpenAI API key to use for LangChainPlus
-                If not provided, the OpenAI API Key will be read from the
-                OPENAI_API_KEY environment variable. If neither are provided,
-                some features of LangChainPlus will not be available.
         """
         if dev:
             os.environ["_LANGCHAINPLUS_IMAGE_PREFIX"] = "rc-"
-        if openai_api_key is not None:
-            os.environ["OPENAI_API_KEY"] = openai_api_key
         if expose:
             self._start_and_expose(auth_token=auth_token)
         else:
@@ -267,36 +214,6 @@ class PlusCommand:
             ]
         )
 
-    def status(self) -> None:
-        """Provide information about the status LangChainPlus server."""
-
-        command = [
-            *self.docker_compose_command,
-            "-f",
-            str(self.docker_compose_file),
-            "ps",
-            "--format",
-            "json",
-        ]
-
-        result = subprocess.run(
-            command,
-            stdout=subprocess.PIPE,
-            stderr=subprocess.PIPE,
-        )
-        try:
-            command_stdout = result.stdout.decode("utf-8")
-            services_status = json.loads(command_stdout)
-        except json.JSONDecodeError:
-            logger.error("Error checking LangChainPlus server status.")
-            return
-        if services_status:
-            logger.info("The LangChainPlus server is currently running.")
-            pprint_services(services_status)
-        else:
-            logger.info("The LangChainPlus server is not running.")
-            return
-
 
 def env() -> None:
     """Print the runtime environment information."""
@@ -333,20 +250,9 @@ def main() -> None:
         action="store_true",
         help="Use the development version of the LangChainPlus image.",
     )
-    server_start_parser.add_argument(
-        "--openai-api-key",
-        default=os.getenv("OPENAI_API_KEY"),
-        help="The OpenAI API key to use for LangChainPlus."
-        " If not provided, the OpenAI API Key will be read from the"
-        " OPENAI_API_KEY environment variable. If neither are provided,"
-        " some features of LangChainPlus will not be available.",
-    )
     server_start_parser.set_defaults(
         func=lambda args: server_command.start(
-            expose=args.expose,
-            auth_token=args.ngrok_authtoken,
-            dev=args.dev,
-            openai_api_key=args.openai_api_key,
+            expose=args.expose, auth_token=args.ngrok_authtoken, dev=args.dev
         )
     )
 
@@ -359,10 +265,7 @@ def main() -> None:
         "logs", description="Show the LangChainPlus server logs."
     )
     server_logs_parser.set_defaults(func=lambda args: server_command.logs())
-    server_status_parser = server_subparsers.add_parser(
-        "status", description="Show the LangChainPlus server status."
-    )
-    server_status_parser.set_defaults(func=lambda args: server_command.status())
+
     env_parser = subparsers.add_parser("env")
     env_parser.set_defaults(func=lambda args: env())
 

langchain/client/langchain.py

@@ -200,7 +200,7 @@ class LangChainPlusClient(BaseSettings):
         return Dataset(**result)
 
     @retry(stop=stop_after_attempt(3), wait=wait_fixed(0.5))
-    def read_run(self, run_id: Union[str, UUID]) -> Run:
+    def read_run(self, run_id: str) -> Run:
         """Read a run from the LangChain+ API."""
         response = self._get(f"/runs/{run_id}")
         raise_for_status_with_text(response)
@@ -268,22 +268,6 @@ class LangChainPlusClient(BaseSettings):
         raise_for_status_with_text(response)
         yield from [TracerSession(**session) for session in response.json()]
 
-    @xor_args(("session_name", "session_id"))
-    def delete_session(
-        self, *, session_name: Optional[str] = None, session_id: Optional[str] = None
-    ) -> None:
-        """Delete a session from the LangChain+ API."""
-        if session_name is not None:
-            session_id = self.read_session(session_name=session_name).id
-        elif session_id is None:
-            raise ValueError("Must provide session_name or session_id")
-        response = requests.delete(
-            self.api_url + f"/sessions/{session_id}",
-            headers=self._headers,
-        )
-        raise_for_status_with_text(response)
-        return None
-
     def create_dataset(self, dataset_name: str, description: str) -> Dataset:
         """Create a dataset in the LangChain+ API."""
         dataset = DatasetCreate(
@@ -376,7 +360,7 @@ class LangChainPlusClient(BaseSettings):
         return Example(**result)
 
     @retry(stop=stop_after_attempt(3), wait=wait_fixed(0.5))
-    def read_example(self, example_id: Union[str, UUID]) -> Example:
+    def read_example(self, example_id: str) -> Example:
         """Read an example from the LangChain+ API."""
         response = self._get(f"/examples/{example_id}")
         raise_for_status_with_text(response)

langchain/document_loaders/__init__.py

@@ -10,7 +10,6 @@ from langchain.document_loaders.azure_blob_storage_container import (
 from langchain.document_loaders.azure_blob_storage_file import (
     AzureBlobStorageFileLoader,
 )
-from langchain.document_loaders.bibtex import BibtexLoader
 from langchain.document_loaders.bigquery import BigQueryLoader
 from langchain.document_loaders.bilibili import BiliBiliLoader
 from langchain.document_loaders.blackboard import BlackboardLoader
@@ -47,7 +46,6 @@ from langchain.document_loaders.ifixit import IFixitLoader
 from langchain.document_loaders.image import UnstructuredImageLoader
 from langchain.document_loaders.image_captions import ImageCaptionLoader
 from langchain.document_loaders.imsdb import IMSDbLoader
-from langchain.document_loaders.joplin import JoplinLoader
 from langchain.document_loaders.json_loader import JSONLoader
 from langchain.document_loaders.markdown import UnstructuredMarkdownLoader
 from langchain.document_loaders.mastodon import MastodonTootsLoader
@@ -102,7 +100,6 @@ from langchain.document_loaders.unstructured import (
 from langchain.document_loaders.url import UnstructuredURLLoader
 from langchain.document_loaders.url_playwright import PlaywrightURLLoader
 from langchain.document_loaders.url_selenium import SeleniumURLLoader
-from langchain.document_loaders.weather import WeatherDataLoader
 from langchain.document_loaders.web_base import WebBaseLoader
 from langchain.document_loaders.whatsapp_chat import WhatsAppChatLoader
 from langchain.document_loaders.wikipedia import WikipediaLoader
@@ -130,7 +127,6 @@ __all__ = [
     "AzureBlobStorageContainerLoader",
     "AzureBlobStorageFileLoader",
     "BSHTMLLoader",
-    "BibtexLoader",
     "BigQueryLoader",
     "BiliBiliLoader",
     "BlackboardLoader",
@@ -163,7 +159,6 @@ __all__ = [
     "IFixitLoader",
     "IMSDbLoader",
     "ImageCaptionLoader",
-    "JoplinLoader",
     "JSONLoader",
     "MWDumpLoader",
     "MastodonTootsLoader",
@@ -217,7 +212,6 @@ __all__ = [
     "UnstructuredRTFLoader",
     "UnstructuredURLLoader",
     "UnstructuredWordDocumentLoader",
-    "WeatherDataLoader",
     "WebBaseLoader",
     "WhatsAppChatLoader",
     "WikipediaLoader",

langchain/document_loaders/bibtex.py

@@ -1,108 +0,0 @@
-import logging
-import re
-from pathlib import Path
-from typing import Any, Iterator, List, Mapping, Optional
-
-from langchain.docstore.document import Document
-from langchain.document_loaders.base import BaseLoader
-from langchain.utilities.bibtex import BibtexparserWrapper
-
-logger = logging.getLogger(__name__)
-
-
-class BibtexLoader(BaseLoader):
-    """Loads a bibtex file into a list of Documents.
-
-    Each document represents one entry from the bibtex file.
-
-    If a PDF file is present in the `file` bibtex field, the original PDF
-    is loaded into the document text. If no such file entry is present,
-    the `abstract` field is used instead.
-    """
-
-    def __init__(
-        self,
-        file_path: str,
-        *,
-        parser: Optional[BibtexparserWrapper] = None,
-        max_docs: Optional[int] = None,
-        max_content_chars: Optional[int] = 4_000,
-        load_extra_metadata: bool = False,
-        file_pattern: str = r"[^:]+\.pdf",
-    ):
-        """Initialize the BibtexLoader.
-
-        Args:
-            file_path: Path to the bibtex file.
-            max_docs: Max number of associated documents to load. Use -1 means
-                           no limit.
-        """
-        self.file_path = file_path
-        self.parser = parser or BibtexparserWrapper()
-        self.max_docs = max_docs
-        self.max_content_chars = max_content_chars
-        self.load_extra_metadata = load_extra_metadata
-        self.file_regex = re.compile(file_pattern)
-
-    def _load_entry(self, entry: Mapping[str, Any]) -> Optional[Document]:
-        import fitz
-
-        parent_dir = Path(self.file_path).parent
-        # regex is useful for Zotero flavor bibtex files
-        file_names = self.file_regex.findall(entry.get("file", ""))
-        if not file_names:
-            return None
-        texts: List[str] = []
-        for file_name in file_names:
-            try:
-                with fitz.open(parent_dir / file_name) as f:
-                    texts.extend(page.get_text() for page in f)
-            except FileNotFoundError as e:
-                logger.debug(e)
-        content = "\n".join(texts) or entry.get("abstract", "")
-        if self.max_content_chars:
-            content = content[: self.max_content_chars]
-        metadata = self.parser.get_metadata(entry, load_extra=self.load_extra_metadata)
-        return Document(
-            page_content=content,
-            metadata=metadata,
-        )
-
-    def lazy_load(self) -> Iterator[Document]:
-        """Load bibtex file using bibtexparser and get the article texts plus the
-
-        article metadata.
-
-        See https://bibtexparser.readthedocs.io/en/master/
-
-        Returns:
-            a list of documents with the document.page_content in text format
-        """
-        try:
-            import fitz  # noqa: F401
-        except ImportError:
-            raise ImportError(
-                "PyMuPDF package not found, please install it with "
-                "`pip install pymupdf`"
-            )
-
-        entries = self.parser.load_bibtex_entries(self.file_path)
-        if self.max_docs:
-            entries = entries[: self.max_docs]
-        for entry in entries:
-            doc = self._load_entry(entry)
-            if doc:
-                yield doc
-
-    def load(self) -> List[Document]:
-        """Load bibtex file documents from the given bibtex file path.
-
-        See https://bibtexparser.readthedocs.io/en/master/
-
-        Args:
-            file_path: the path to the bibtex file
-
-        Returns:
-            a list of documents with the document.page_content in text format
-        """
-        return list(self.lazy_load())

langchain/document_loaders/googledrive.py

@@ -31,7 +31,6 @@ class GoogleDriveLoader(BaseLoader, BaseModel):
     file_ids: Optional[List[str]] = None
     recursive: bool = False
     file_types: Optional[Sequence[str]] = None
-    load_trashed_files: bool = False
 
     @root_validator
     def validate_inputs(cls, values: Dict[str, Any]) -> Dict[str, Any]:
@@ -216,10 +215,8 @@ class GoogleDriveLoader(BaseLoader, BaseModel):
             _files = files
 
         returns = []
-        for file in files:
-            if file["trashed"] and not self.load_trashed_files:
-                continue
-            elif file["mimeType"] == "application/vnd.google-apps.document":
+        for file in _files:
+            if file["mimeType"] == "application/vnd.google-apps.document":
                 returns.append(self._load_document_from_id(file["id"]))  # type: ignore
             elif file["mimeType"] == "application/vnd.google-apps.spreadsheet":
                 returns.extend(self._load_sheet_from_id(file["id"]))  # type: ignore
@@ -227,6 +224,7 @@ class GoogleDriveLoader(BaseLoader, BaseModel):
                 returns.extend(self._load_file_from_id(file["id"]))  # type: ignore
             else:
                 pass
+
         return returns
 
     def _fetch_files_recursive(
@@ -240,7 +238,7 @@ class GoogleDriveLoader(BaseLoader, BaseModel):
                 pageSize=1000,
                 includeItemsFromAllDrives=True,
                 supportsAllDrives=True,
-                fields="nextPageToken, files(id, name, mimeType, parents, trashed)",
+                fields="nextPageToken, files(id, name, mimeType, parents)",
             )
             .execute()
         )

langchain/document_loaders/iugu.py

@@ -1,41 +0,0 @@
-"""Loader that fetches data from IUGU"""
-import json
-import urllib.request
-from typing import List, Optional
-
-from langchain.docstore.document import Document
-from langchain.document_loaders.base import BaseLoader
-from langchain.utils import get_from_env, stringify_dict
-
-IUGU_ENDPOINTS = {
-    "invoices": "https://api.iugu.com/v1/invoices",
-    "customers": "https://api.iugu.com/v1/customers",
-    "charges": "https://api.iugu.com/v1/charges",
-    "subscriptions": "https://api.iugu.com/v1/subscriptions",
-    "plans": "https://api.iugu.com/v1/plans",
-}
-
-
-class IuguLoader(BaseLoader):
-    def __init__(self, resource: str, api_token: Optional[str] = None) -> None:
-        self.resource = resource
-        api_token = api_token or get_from_env("api_token", "IUGU_API_TOKEN")
-        self.headers = {"Authorization": f"Bearer {api_token}"}
-
-    def _make_request(self, url: str) -> List[Document]:
-        request = urllib.request.Request(url, headers=self.headers)
-
-        with urllib.request.urlopen(request) as response:
-            json_data = json.loads(response.read().decode())
-            text = stringify_dict(json_data)
-            metadata = {"source": url}
-            return [Document(page_content=text, metadata=metadata)]
-
-    def _get_resource(self) -> List[Document]:
-        endpoint = IUGU_ENDPOINTS.get(self.resource)
-        if endpoint is None:
-            return []
-        return self._make_request(endpoint)
-
-    def load(self) -> List[Document]:
-        return self._get_resource()

langchain/document_loaders/joplin.py

@@ -1,88 +0,0 @@
-import json
-import urllib
-from datetime import datetime
-from typing import Iterator, List, Optional
-
-from langchain.document_loaders.base import BaseLoader
-from langchain.schema import Document
-from langchain.utils import get_from_env
-
-LINK_NOTE_TEMPLATE = "joplin://x-callback-url/openNote?id={id}"
-
-
-class JoplinLoader(BaseLoader):
-    """
-    Loader that fetches notes from Joplin.
-
-    In order to use this loader, you need to have Joplin running with the
-    Web Clipper enabled (look for "Web Clipper" in the app settings).
-
-    To get the access token, you need to go to the Web Clipper options and
-    under "Advanced Options" you will find the access token.
-
-    You can find more information about the Web Clipper service here:
-    https://joplinapp.org/clipper/
-    """
-
-    def __init__(
-        self,
-        access_token: Optional[str] = None,
-        port: int = 41184,
-        host: str = "localhost",
-    ) -> None:
-        access_token = access_token or get_from_env(
-            "access_token", "JOPLIN_ACCESS_TOKEN"
-        )
-        base_url = f"http://{host}:{port}"
-        self._get_note_url = (
-            f"{base_url}/notes?token={access_token}"
-            f"&fields=id,parent_id,title,body,created_time,updated_time&page={{page}}"
-        )
-        self._get_folder_url = (
-            f"{base_url}/folders/{{id}}?token={access_token}&fields=title"
-        )
-        self._get_tag_url = (
-            f"{base_url}/notes/{{id}}/tags?token={access_token}&fields=title"
-        )
-
-    def _get_notes(self) -> Iterator[Document]:
-        has_more = True
-        page = 1
-        while has_more:
-            req_note = urllib.request.Request(self._get_note_url.format(page=page))
-            with urllib.request.urlopen(req_note) as response:
-                json_data = json.loads(response.read().decode())
-                for note in json_data["items"]:
-                    metadata = {
-                        "source": LINK_NOTE_TEMPLATE.format(id=note["id"]),
-                        "folder": self._get_folder(note["parent_id"]),
-                        "tags": self._get_tags(note["id"]),
-                        "title": note["title"],
-                        "created_time": self._convert_date(note["created_time"]),
-                        "updated_time": self._convert_date(note["updated_time"]),
-                    }
-                    yield Document(page_content=note["body"], metadata=metadata)
-
-                has_more = json_data["has_more"]
-                page += 1
-
-    def _get_folder(self, folder_id: str) -> str:
-        req_folder = urllib.request.Request(self._get_folder_url.format(id=folder_id))
-        with urllib.request.urlopen(req_folder) as response:
-            json_data = json.loads(response.read().decode())
-            return json_data["title"]
-
-    def _get_tags(self, note_id: str) -> List[str]:
-        req_tag = urllib.request.Request(self._get_tag_url.format(id=note_id))
-        with urllib.request.urlopen(req_tag) as response:
-            json_data = json.loads(response.read().decode())
-            return [tag["title"] for tag in json_data["items"]]
-
-    def _convert_date(self, date: int) -> str:
-        return datetime.fromtimestamp(date / 1000).strftime("%Y-%m-%d %H:%M:%S")
-
-    def lazy_load(self) -> Iterator[Document]:
-        yield from self._get_notes()
-
-    def load(self) -> List[Document]:
-        return list(self.lazy_load())

langchain/document_loaders/readthedocs.py

@@ -1,6 +1,6 @@
 """Loader that loads ReadTheDocs documentation directory dump."""
 from pathlib import Path
-from typing import Any, List, Optional, Tuple, Union
+from typing import Any, List, Optional
 
 from langchain.docstore.document import Document
 from langchain.document_loaders.base import BaseLoader
@@ -11,31 +11,12 @@ class ReadTheDocsLoader(BaseLoader):
 
     def __init__(
         self,
-        path: Union[str, Path],
+        path: str,
         encoding: Optional[str] = None,
         errors: Optional[str] = None,
-        custom_html_tag: Optional[Tuple[str, dict]] = None,
         **kwargs: Optional[Any]
     ):
-        """
-        Initialize ReadTheDocsLoader
-
-        The loader loops over all files under `path` and extract the actual content of
-        the files by retrieving main html tags. Default main html tags include
-        `<main id="main-content>`, <`div role="main>`, and `<article role="main">`. You
-        can also define your own html tags by passing custom_html_tag, e.g.
-        `("div", "class=main")`. The loader iterates html tags with the order of
-        custom html tags (if exists) and default html tags. If any of the tags is not
-        empty, the loop will break and retrieve the content out of that tag.
-
-        Args:
-            path: The location of pulled readthedocs folder.
-            encoding: The encoding with which to open the documents.
-            errors: Specifies how encoding and decoding errors are to be handled—this
-                cannot be used in binary mode.
-            custom_html_tag: Optional custom html tag to retrieve the content from
-                files.
-        """
+        """Initialize path."""
         try:
             from bs4 import BeautifulSoup
         except ImportError:
@@ -51,50 +32,34 @@ class ReadTheDocsLoader(BaseLoader):
         except Exception as e:
             raise ValueError("Parsing kwargs do not appear valid") from e
 
-        self.file_path = Path(path)
+        self.file_path = path
         self.encoding = encoding
         self.errors = errors
-        self.custom_html_tag = custom_html_tag
         self.bs_kwargs = kwargs
 
     def load(self) -> List[Document]:
         """Load documents."""
+        from bs4 import BeautifulSoup
+
+        def _clean_data(data: str) -> str:
+            soup = BeautifulSoup(data, **self.bs_kwargs)
+            text = soup.find_all("main", {"id": "main-content"})
+
+            if len(text) == 0:
+                text = soup.find_all("div", {"role": "main"})
+
+            if len(text) != 0:
+                text = text[0].get_text()
+            else:
+                text = ""
+            return "\n".join([t for t in text.split("\n") if t])
+
         docs = []
-        for p in self.file_path.rglob("*"):
+        for p in Path(self.file_path).rglob("*"):
             if p.is_dir():
                 continue
             with open(p, encoding=self.encoding, errors=self.errors) as f:
-                text = self._clean_data(f.read())
+                text = _clean_data(f.read())
             metadata = {"source": str(p)}
             docs.append(Document(page_content=text, metadata=metadata))
         return docs
-
-    def _clean_data(self, data: str) -> str:
-        from bs4 import BeautifulSoup
-
-        soup = BeautifulSoup(data, **self.bs_kwargs)
-
-        # default tags
-        html_tags = [
-            ("div", {"role": "main"}),
-            ("main", {"id": "main-content"}),
-        ]
-
-        if self.custom_html_tag is not None:
-            html_tags.append(self.custom_html_tag)
-
-        text = None
-
-        # reversed order. check the custom one first
-        for tag, attrs in html_tags[::-1]:
-            text = soup.find(tag, attrs)
-            # if found, break
-            if text is not None:
-                break
-
-        if text is not None:
-            text = text.get_text()
-        else:
-            text = ""
-        # trim empty lines
-        return "\n".join([t for t in text.split("\n") if t])

langchain/document_loaders/weather.py

@@ -1,50 +0,0 @@
-"""Simple reader that reads weather data from OpenWeatherMap API"""
-from __future__ import annotations
-
-from datetime import datetime
-from typing import Iterator, List, Optional, Sequence
-
-from langchain.docstore.document import Document
-from langchain.document_loaders.base import BaseLoader
-from langchain.utilities.openweathermap import OpenWeatherMapAPIWrapper
-
-
-class WeatherDataLoader(BaseLoader):
-    """Weather Reader.
-
-    Reads the forecast & current weather of any location using OpenWeatherMap's free
-    API. Checkout 'https://openweathermap.org/appid' for more on how to generate a free
-    OpenWeatherMap API.
-    """
-
-    def __init__(
-        self,
-        client: OpenWeatherMapAPIWrapper,
-        places: Sequence[str],
-    ) -> None:
-        """Initialize with parameters."""
-        super().__init__()
-        self.client = client
-        self.places = places
-
-    @classmethod
-    def from_params(
-        cls, places: Sequence[str], *, openweathermap_api_key: Optional[str] = None
-    ) -> WeatherDataLoader:
-        client = OpenWeatherMapAPIWrapper(openweathermap_api_key=openweathermap_api_key)
-        return cls(client, places)
-
-    def lazy_load(
-        self,
-    ) -> Iterator[Document]:
-        """Lazily load weather data for the given locations."""
-        for place in self.places:
-            metadata = {"queried_at": datetime.now()}
-            content = self.client.run(place)
-            yield Document(page_content=content, metadata=metadata)
-
-    def load(
-        self,
-    ) -> List[Document]:
-        """Load weather data for the given locations."""
-        return list(self.lazy_load())

langchain/embeddings/__init__.py

@@ -7,7 +7,6 @@ from langchain.embeddings.aleph_alpha import (
     AlephAlphaSymmetricSemanticEmbedding,
 )
 from langchain.embeddings.cohere import CohereEmbeddings
-from langchain.embeddings.elasticsearch import ElasticsearchEmbeddings
 from langchain.embeddings.fake import FakeEmbeddings
 from langchain.embeddings.google_palm import GooglePalmEmbeddings
 from langchain.embeddings.huggingface import (
@@ -17,9 +16,6 @@ from langchain.embeddings.huggingface import (
 from langchain.embeddings.huggingface_hub import HuggingFaceHubEmbeddings
 from langchain.embeddings.jina import JinaEmbeddings
 from langchain.embeddings.llamacpp import LlamaCppEmbeddings
-from langchain.embeddings.minimax import MiniMaxEmbeddings
-from langchain.embeddings.modelscope_hub import ModelScopeEmbeddings
-from langchain.embeddings.mosaicml import MosaicMLInstructorEmbeddings
 from langchain.embeddings.openai import OpenAIEmbeddings
 from langchain.embeddings.sagemaker_endpoint import SagemakerEndpointEmbeddings
 from langchain.embeddings.self_hosted import SelfHostedEmbeddings
@@ -29,7 +25,6 @@ from langchain.embeddings.self_hosted_hugging_face import (
 )
 from langchain.embeddings.sentence_transformer import SentenceTransformerEmbeddings
 from langchain.embeddings.tensorflow_hub import TensorflowHubEmbeddings
-from langchain.embeddings.vertexai import VertexAIEmbeddings
 
 logger = logging.getLogger(__name__)
 
@@ -37,15 +32,12 @@ __all__ = [
     "OpenAIEmbeddings",
     "HuggingFaceEmbeddings",
     "CohereEmbeddings",
-    "ElasticsearchEmbeddings",
     "JinaEmbeddings",
     "LlamaCppEmbeddings",
     "HuggingFaceHubEmbeddings",
-    "ModelScopeEmbeddings",
     "TensorflowHubEmbeddings",
     "SagemakerEndpointEmbeddings",
     "HuggingFaceInstructEmbeddings",
-    "MosaicMLInstructorEmbeddings",
     "SelfHostedEmbeddings",
     "SelfHostedHuggingFaceEmbeddings",
     "SelfHostedHuggingFaceInstructEmbeddings",
@@ -54,8 +46,6 @@ __all__ = [
     "AlephAlphaSymmetricSemanticEmbedding",
     "SentenceTransformerEmbeddings",
     "GooglePalmEmbeddings",
-    "MiniMaxEmbeddings",
-    "VertexAIEmbeddings",
 ]
 
 

langchain/embeddings/elasticsearch.py

@@ -1,155 +0,0 @@
-from __future__ import annotations
-
-from typing import TYPE_CHECKING, List, Optional
-
-from langchain.utils import get_from_env
-
-if TYPE_CHECKING:
-    from elasticsearch.client import MlClient
-
-from langchain.embeddings.base import Embeddings
-
-
-class ElasticsearchEmbeddings(Embeddings):
-    """
-    Wrapper around Elasticsearch embedding models.
-
-    This class provides an interface to generate embeddings using a model deployed
-    in an Elasticsearch cluster. It requires an Elasticsearch connection object
-    and the model_id of the model deployed in the cluster.
-
-    In Elasticsearch you need to have an embedding model loaded and deployed.
-    - https://www.elastic.co/guide/en/elasticsearch/reference/current/infer-trained-model.html
-    - https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-deploy-models.html
-    """  # noqa: E501
-
-    def __init__(
-        self,
-        client: MlClient,
-        model_id: str,
-        *,
-        input_field: str = "text_field",
-    ):
-        """
-        Initialize the ElasticsearchEmbeddings instance.
-
-        Args:
-            client (MlClient): An Elasticsearch ML client object.
-            model_id (str): The model_id of the model deployed in the Elasticsearch
-                cluster.
-            input_field (str): The name of the key for the input text field in the
-                document. Defaults to 'text_field'.
-        """
-        self.client = client
-        self.model_id = model_id
-        self.input_field = input_field
-
-    @classmethod
-    def from_credentials(
-        cls,
-        model_id: str,
-        *,
-        es_cloud_id: Optional[str] = None,
-        es_user: Optional[str] = None,
-        es_password: Optional[str] = None,
-        input_field: str = "text_field",
-    ) -> ElasticsearchEmbeddings:
-        """Instantiate embeddings from Elasticsearch credentials.
-
-        Args:
-            model_id (str): The model_id of the model deployed in the Elasticsearch
-                cluster.
-            input_field (str): The name of the key for the input text field in the
-                document. Defaults to 'text_field'.
-            es_cloud_id: (str, optional): The Elasticsearch cloud ID to connect to.
-            es_user: (str, optional): Elasticsearch username.
-            es_password: (str, optional): Elasticsearch password.
-
-        Example Usage:
-            from langchain.embeddings import ElasticsearchEmbeddings
-
-            # Define the model ID and input field name (if different from default)
-            model_id = "your_model_id"
-            # Optional, only if different from 'text_field'
-            input_field = "your_input_field"
-
-            # Credentials can be passed in two ways. Either set the env vars
-            # ES_CLOUD_ID, ES_USER, ES_PASSWORD and they will be automatically pulled
-            # in, or pass them in directly as kwargs.
-            embeddings = ElasticsearchEmbeddings.from_credentials(
-                model_id,
-                input_field=input_field,
-                # es_cloud_id="foo",
-                # es_user="bar",
-                # es_password="baz",
-            )
-
-            documents = [
-                "This is an example document.",
-                "Another example document to generate embeddings for.",
-            ]
-            embeddings_generator.embed_documents(documents)
-        """
-        try:
-            from elasticsearch import Elasticsearch
-            from elasticsearch.client import MlClient
-        except ImportError:
-            raise ImportError(
-                "elasticsearch package not found, please install with 'pip install "
-                "elasticsearch'"
-            )
-
-        es_cloud_id = es_cloud_id or get_from_env("es_cloud_id", "ES_CLOUD_ID")
-        es_user = es_user or get_from_env("es_user", "ES_USER")
-        es_password = es_password or get_from_env("es_password", "ES_PASSWORD")
-
-        # Connect to Elasticsearch
-        es_connection = Elasticsearch(
-            cloud_id=es_cloud_id, basic_auth=(es_user, es_password)
-        )
-        client = MlClient(es_connection)
-        return cls(client, model_id, input_field=input_field)
-
-    def _embedding_func(self, texts: List[str]) -> List[List[float]]:
-        """
-        Generate embeddings for the given texts using the Elasticsearch model.
-
-        Args:
-            texts (List[str]): A list of text strings to generate embeddings for.
-
-        Returns:
-            List[List[float]]: A list of embeddings, one for each text in the input
-                list.
-        """
-        response = self.client.infer_trained_model(
-            model_id=self.model_id, docs=[{self.input_field: text} for text in texts]
-        )
-
-        embeddings = [doc["predicted_value"] for doc in response["inference_results"]]
-        return embeddings
-
-    def embed_documents(self, texts: List[str]) -> List[List[float]]:
-        """
-        Generate embeddings for a list of documents.
-
-        Args:
-            texts (List[str]): A list of document text strings to generate embeddings
-                for.
-
-        Returns:
-            List[List[float]]: A list of embeddings, one for each document in the input
-                list.
-        """
-        return self._embedding_func(texts)
-
-    def embed_query(self, text: str) -> List[float]:
-        """
-        Generate an embedding for a single query text.
-
-        Args:
-            text (str): The query text to generate an embedding for.
-
-        Returns:
-            List[float]: The embedding for the input query text.
-        """
-        return self._embedding_func([text])[0]

langchain/embeddings/minimax.py

@@ -1,163 +0,0 @@
-"""Wrapper around MiniMax APIs."""
-from __future__ import annotations
-
-import logging
-from typing import Any, Callable, Dict, List, Optional
-
-import requests
-from pydantic import BaseModel, Extra, root_validator
-from tenacity import (
-    before_sleep_log,
-    retry,
-    stop_after_attempt,
-    wait_exponential,
-)
-
-from langchain.embeddings.base import Embeddings
-from langchain.utils import get_from_dict_or_env
-
-logger = logging.getLogger(__name__)
-
-
-def _create_retry_decorator() -> Callable[[Any], Any]:
-    """Returns a tenacity retry decorator."""
-
-    multiplier = 1
-    min_seconds = 1
-    max_seconds = 4
-    max_retries = 6
-
-    return retry(
-        reraise=True,
-        stop=stop_after_attempt(max_retries),
-        wait=wait_exponential(multiplier=multiplier, min=min_seconds, max=max_seconds),
-        before_sleep=before_sleep_log(logger, logging.WARNING),
-    )
-
-
-def embed_with_retry(embeddings: MiniMaxEmbeddings, *args: Any, **kwargs: Any) -> Any:
-    """Use tenacity to retry the completion call."""
-    retry_decorator = _create_retry_decorator()
-
-    @retry_decorator
-    def _embed_with_retry(*args: Any, **kwargs: Any) -> Any:
-        return embeddings.embed(*args, **kwargs)
-
-    return _embed_with_retry(*args, **kwargs)
-
-
-class MiniMaxEmbeddings(BaseModel, Embeddings):
-    """Wrapper around MiniMax's embedding inference service.
-
-    To use, you should have the environment variable ``MINIMAX_GROUP_ID`` and
-    ``MINIMAX_API_KEY`` set with your API token, or pass it as a named parameter to
-    the constructor.
-
-    Example:
-        .. code-block:: python
-
-            from langchain.embeddings import MiniMaxEmbeddings
-            embeddings = MiniMaxEmbeddings()
-
-            query_text = "This is a test query."
-            query_result = embeddings.embed_query(query_text)
-
-            document_text = "This is a test document."
-            document_result = embeddings.embed_documents([document_text])
-
-    """
-
-    endpoint_url: str = "https://api.minimax.chat/v1/embeddings"
-    """Endpoint URL to use."""
-    model: str = "embo-01"
-    """Embeddings model name to use."""
-    embed_type_db: str = "db"
-    """For embed_documents"""
-    embed_type_query: str = "query"
-    """For embed_query"""
-
-    minimax_group_id: Optional[str] = None
-    """Group ID for MiniMax API."""
-    minimax_api_key: Optional[str] = None
-    """API Key for MiniMax API."""
-
-    class Config:
-        """Configuration for this pydantic object."""
-
-        extra = Extra.forbid
-
-    @root_validator()
-    def validate_environment(cls, values: Dict) -> Dict:
-        """Validate that group id and api key exists in environment."""
-        minimax_group_id = get_from_dict_or_env(
-            values, "minimax_group_id", "MINIMAX_GROUP_ID"
-        )
-        minimax_api_key = get_from_dict_or_env(
-            values, "minimax_api_key", "MINIMAX_API_KEY"
-        )
-        values["minimax_group_id"] = minimax_group_id
-        values["minimax_api_key"] = minimax_api_key
-        return values
-
-    def embed(
-        self,
-        texts: List[str],
-        embed_type: str,
-    ) -> List[List[float]]:
-        payload = {
-            "model": self.model,
-            "type": embed_type,
-            "texts": texts,
-        }
-
-        # HTTP headers for authorization
-        headers = {
-            "Authorization": f"Bearer {self.minimax_api_key}",
-            "Content-Type": "application/json",
-        }
-
-        params = {
-            "GroupId": self.minimax_group_id,
-        }
-
-        # send request
-        response = requests.post(
-            self.endpoint_url, params=params, headers=headers, json=payload
-        )
-        parsed_response = response.json()
-
-        # check for errors
-        if parsed_response["base_resp"]["status_code"] != 0:
-            raise ValueError(
-                f"MiniMax API returned an error: {parsed_response['base_resp']}"
-            )
-
-        embeddings = parsed_response["vectors"]
-
-        return embeddings
-
-    def embed_documents(self, texts: List[str]) -> List[List[float]]:
-        """Embed documents using a MiniMax embedding endpoint.
-
-        Args:
-            texts: The list of texts to embed.
-
-        Returns:
-            List of embeddings, one for each text.
-        """
-        embeddings = embed_with_retry(self, texts=texts, embed_type=self.embed_type_db)
-        return embeddings
-
-    def embed_query(self, text: str) -> List[float]:
-        """Embed a query using a MiniMax embedding endpoint.
-
-        Args:
-            text: The text to embed.
-
-        Returns:
-            Embeddings for the text.
-        """
-        embeddings = embed_with_retry(
-            self, texts=[text], embed_type=self.embed_type_query
-        )
-        return embeddings[0]

langchain/embeddings/modelscope_hub.py

@@ -1,72 +0,0 @@
-"""Wrapper around ModelScopeHub embedding models."""
-from typing import Any, List
-
-from pydantic import BaseModel, Extra
-
-from langchain.embeddings.base import Embeddings
-
-
-class ModelScopeEmbeddings(BaseModel, Embeddings):
-    """Wrapper around modelscope_hub embedding models.
-
-    To use, you should have the ``modelscope`` python package installed.
-
-    Example:
-        .. code-block:: python
-
-            from langchain.embeddings import ModelScopeEmbeddings
-            model_id = "damo/nlp_corom_sentence-embedding_english-base"
-            embed = ModelScopeEmbeddings(model_id=model_id)
-    """
-
-    embed: Any
-    model_id: str = "damo/nlp_corom_sentence-embedding_english-base"
-    """Model name to use."""
-
-    def __init__(self, **kwargs: Any):
-        """Initialize the modelscope"""
-        super().__init__(**kwargs)
-        try:
-            from modelscope.pipelines import pipeline
-            from modelscope.utils.constant import Tasks
-
-            self.embed = pipeline(Tasks.sentence_embedding, model=self.model_id)
-
-        except ImportError as e:
-            raise ImportError(
-                "Could not import some python packages."
-                "Please install it with `pip install modelscope`."
-            ) from e
-
-    class Config:
-        """Configuration for this pydantic object."""
-
-        extra = Extra.forbid
-
-    def embed_documents(self, texts: List[str]) -> List[List[float]]:
-        """Compute doc embeddings using a modelscope embedding model.
-
-        Args:
-            texts: The list of texts to embed.
-
-        Returns:
-            List of embeddings, one for each text.
-        """
-        texts = list(map(lambda x: x.replace("\n", " "), texts))
-        inputs = {"source_sentence": texts}
-        embeddings = self.embed(input=inputs)["text_embedding"]
-        return embeddings.tolist()
-
-    def embed_query(self, text: str) -> List[float]:
-        """Compute query embeddings using a modelscope embedding model.
-
-        Args:
-            text: The text to embed.
-
-        Returns:
-            Embeddings for the text.
-        """
-        text = text.replace("\n", " ")
-        inputs = {"source_sentence": [text]}
-        embedding = self.embed(input=inputs)["text_embedding"][0]
-        return embedding.tolist()

langchain/embeddings/mosaicml.py

@@ -1,137 +0,0 @@
-"""Wrapper around MosaicML APIs."""
-from typing import Any, Dict, List, Mapping, Optional, Tuple
-
-import requests
-from pydantic import BaseModel, Extra, root_validator
-
-from langchain.embeddings.base import Embeddings
-from langchain.utils import get_from_dict_or_env
-
-
-class MosaicMLInstructorEmbeddings(BaseModel, Embeddings):
-    """Wrapper around MosaicML's embedding inference service.
-
-    To use, you should have the
-    environment variable ``MOSAICML_API_TOKEN`` set with your API token, or pass
-    it as a named parameter to the constructor.
-
-    Example:
-        .. code-block:: python
-
-            from langchain.llms import MosaicMLInstructorEmbeddings
-            endpoint_url = (
-                "https://models.hosted-on.mosaicml.hosting/instructor-large/v1/predict"
-            )
-            mosaic_llm = MosaicMLInstructorEmbeddings(
-                endpoint_url=endpoint_url,
-                mosaicml_api_token="my-api-key"
-            )
-    """
-
-    endpoint_url: str = (
-        "https://models.hosted-on.mosaicml.hosting/instructor-large/v1/predict"
-    )
-    """Endpoint URL to use."""
-    embed_instruction: str = "Represent the document for retrieval: "
-    """Instruction used to embed documents."""
-    query_instruction: str = (
-        "Represent the question for retrieving supporting documents: "
-    )
-    """Instruction used to embed the query."""
-    retry_sleep: float = 1.0
-    """How long to try sleeping for if a rate limit is encountered"""
-
-    mosaicml_api_token: Optional[str] = None
-
-    class Config:
-        """Configuration for this pydantic object."""
-
-        extra = Extra.forbid
-
-    @root_validator()
-    def validate_environment(cls, values: Dict) -> Dict:
-        """Validate that api key and python package exists in environment."""
-        mosaicml_api_token = get_from_dict_or_env(
-            values, "mosaicml_api_token", "MOSAICML_API_TOKEN"
-        )
-        values["mosaicml_api_token"] = mosaicml_api_token
-        return values
-
-    @property
-    def _identifying_params(self) -> Mapping[str, Any]:
-        """Get the identifying parameters."""
-        return {"endpoint_url": self.endpoint_url}
-
-    def _embed(
-        self, input: List[Tuple[str, str]], is_retry: bool = False
-    ) -> List[List[float]]:
-        payload = {"input_strings": input}
-
-        # HTTP headers for authorization
-        headers = {
-            "Authorization": f"{self.mosaicml_api_token}",
-            "Content-Type": "application/json",
-        }
-
-        # send request
-        try:
-            response = requests.post(self.endpoint_url, headers=headers, json=payload)
-        except requests.exceptions.RequestException as e:
-            raise ValueError(f"Error raised by inference endpoint: {e}")
-
-        try:
-            parsed_response = response.json()
-
-            if "error" in parsed_response:
-                # if we get rate limited, try sleeping for 1 second
-                if (
-                    not is_retry
-                    and "rate limit exceeded" in parsed_response["error"].lower()
-                ):
-                    import time
-
-                    time.sleep(self.retry_sleep)
-
-                    return self._embed(input, is_retry=True)
-
-                raise ValueError(
-                    f"Error raised by inference API: {parsed_response['error']}"
-                )
-
-            if "data" not in parsed_response:
-                raise ValueError(
-                    f"Error raised by inference API, no key data: {parsed_response}"
-                )
-            embeddings = parsed_response["data"]
-        except requests.exceptions.JSONDecodeError as e:
-            raise ValueError(
-                f"Error raised by inference API: {e}.\nResponse: {response.text}"
-            )
-
-        return embeddings
-
-    def embed_documents(self, texts: List[str]) -> List[List[float]]:
-        """Embed documents using a MosaicML deployed instructor embedding model.
-
-        Args:
-            texts: The list of texts to embed.
-
-        Returns:
-            List of embeddings, one for each text.
-        """
-        instruction_pairs = [(self.embed_instruction, text) for text in texts]
-        embeddings = self._embed(instruction_pairs)
-        return embeddings
-
-    def embed_query(self, text: str) -> List[float]:
-        """Embed a query using a MosaicML deployed instructor embedding model.
-
-        Args:
-            text: The text to embed.
-
-        Returns:
-            Embeddings for the text.
-        """
-        instruction_pair = (self.query_instruction, text)
-        embedding = self._embed([instruction_pair])[0]
-        return embedding

langchain/embeddings/openai.py

@@ -91,7 +91,6 @@ class OpenAIEmbeddings(BaseModel, Embeddings):
             os.environ["OPENAI_API_BASE"] = "https://<your-endpoint.openai.azure.com/"
             os.environ["OPENAI_API_KEY"] = "your AzureOpenAI key"
             os.environ["OPENAI_API_VERSION"] = "2023-03-15-preview"
-            os.environ["OPENAI_PROXY"] = "http://your-corporate-proxy:8080"
 
             from langchain.embeddings.openai import OpenAIEmbeddings
             embeddings = OpenAIEmbeddings(
@@ -113,8 +112,6 @@ class OpenAIEmbeddings(BaseModel, Embeddings):
     openai_api_base: Optional[str] = None
     # to support Azure OpenAI Service custom endpoints
     openai_api_type: Optional[str] = None
-    # to support explicit proxy for OpenAI
-    openai_proxy: Optional[str] = None
     embedding_ctx_length: int = 8191
     openai_api_key: Optional[str] = None
     openai_organization: Optional[str] = None
@@ -151,12 +148,6 @@ class OpenAIEmbeddings(BaseModel, Embeddings):
             "OPENAI_API_TYPE",
             default="",
         )
-        openai_proxy = get_from_dict_or_env(
-            values,
-            "openai_proxy",
-            "OPENAI_PROXY",
-            default="",
-        )
         if openai_api_type in ("azure", "azure_ad", "azuread"):
             default_api_version = "2022-12-01"
         else:
@@ -185,8 +176,6 @@ class OpenAIEmbeddings(BaseModel, Embeddings):
                 openai.api_version = openai_api_version
             if openai_api_type:
                 openai.api_type = openai_api_type
-            if openai_proxy:
-                openai.proxy = {"http": openai_proxy, "https": openai_proxy}  # type: ignore[assignment]  # noqa: E501
             values["client"] = openai.Embedding
         except ImportError:
             raise ImportError(

langchain/embeddings/vertexai.py

@@ -1,47 +0,0 @@
-"""Wrapper around Google VertexAI embedding models."""
-from typing import Dict, List
-
-from pydantic import root_validator
-
-from langchain.embeddings.base import Embeddings
-from langchain.llms.vertexai import _VertexAICommon
-from langchain.utilities.vertexai import raise_vertex_import_error
-
-
-class VertexAIEmbeddings(_VertexAICommon, Embeddings):
-    model_name: str = "textembedding-gecko"
-
-    @root_validator()
-    def validate_environment(cls, values: Dict) -> Dict:
-        """Validates that the python package exists in environment."""
-        cls._try_init_vertexai(values)
-        try:
-            from vertexai.preview.language_models import TextEmbeddingModel
-        except ImportError:
-            raise_vertex_import_error()
-        values["client"] = TextEmbeddingModel.from_pretrained(values["model_name"])
-        return values
-
-    def embed_documents(self, texts: List[str]) -> List[List[float]]:
-        """Embed a list of strings.
-
-        Args:
-            texts: List[str] The list of strings to embed.
-
-        Returns:
-            List of embeddings, one for each text.
-        """
-        embeddings = self.client.get_embeddings(texts)
-        return [el.values for el in embeddings]
-
-    def embed_query(self, text: str) -> List[float]:
-        """Embed a text.
-
-        Args:
-            text: The text to embed.
-
-        Returns:
-            Embedding for the text.
-        """
-        embeddings = self.client.get_embeddings([text])
-        return embeddings[0].values

langchain/experimental/plan_and_execute/planners/chat_planner.py

@@ -25,7 +25,7 @@ SYSTEM_PROMPT = (
 
 class PlanningOutputParser(PlanOutputParser):
     def parse(self, text: str) -> Plan:
-        steps = [Step(value=v) for v in re.split("\n\s*\d+\. ", text)[1:]]
+        steps = [Step(value=v) for v in re.split("\n\d+\. ", text)[1:]]
         return Plan(steps=steps)
 
 

langchain/indexes/graph.py

@@ -28,15 +28,3 @@ class GraphIndexCreator(BaseModel):
         for triple in knowledge:
             graph.add_triple(triple)
         return graph
-
-    async def afrom_text(self, text: str) -> NetworkxEntityGraph:
-        """Create graph index from text asynchronously."""
-        if self.llm is None:
-            raise ValueError("llm should not be None")
-        graph = self.graph_type()
-        chain = LLMChain(llm=self.llm, prompt=KNOWLEDGE_TRIPLE_EXTRACTION_PROMPT)
-        output = await chain.apredict(text=text)
-        knowledge = parse_triples(output)
-        for triple in knowledge:
-            graph.add_triple(triple)
-        return graph