docs: Make deprecation docs bar bigger (#28674)

Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>

.github/workflows/_release.yml

@@ -78,6 +78,7 @@ jobs:
       - build
     uses:
       ./.github/workflows/_test_release.yml
+    permissions: write-all
     with:
       working-directory: ${{ inputs.working-directory }}
       dangerous-nonmaster-release: ${{ inputs.dangerous-nonmaster-release }}

.github/workflows/_test_release.yml

@@ -98,3 +98,4 @@ jobs:
           # This is *only for CI use* and is *extremely dangerous* otherwise!
           # https://github.com/pypa/gh-action-pypi-publish#tolerating-release-package-file-duplicates
           skip-existing: true
+          attestations: false

.github/workflows/codespell.yml

@@ -3,9 +3,9 @@ name: CI / cd . / make spell_check
 
 on:
   push:
-    branches: [master]
+    branches: [master, v0.1]
   pull_request:
-    branches: [master]
+    branches: [master, v0.1]
 
 permissions:
   contents: read

Makefile

@@ -17,7 +17,7 @@ clean: docs_clean api_docs_clean
 
 ## docs_build: Build the documentation.
 docs_build:
-	cd docs && make build-local
+	cd docs && make build
 
 ## docs_clean: Clean the documentation build artifacts.
 docs_clean:

cookbook/README.md

@@ -57,3 +57,4 @@ Notebook | Description
 [two_agent_debate_tools.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_agent_debate_tools.ipynb) | Simulate multi-agent dialogues where the agents can utilize various tools.
 [two_player_dnd.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/two_player_dnd.ipynb) | Simulate a two-player dungeons & dragons game, where a dialogue simulator class is used to coordinate the dialogue between the protagonist and the dungeon master.
 [wikibase_agent.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/wikibase_agent.ipynb) | Create a simple wikibase agent that utilizes sparql generation, with testing done on http://wikidata.org.
+[oracleai_demo.ipynb](https://github.com/langchain-ai/langchain/tree/master/cookbook/oracleai_demo.ipynb) | This guide outlines how to utilize Oracle AI Vector Search alongside Langchain for an end-to-end RAG pipeline, providing step-by-step examples. The process includes loading documents from various sources using OracleDocLoader, summarizing them either within or outside the database with OracleSummary, and generating embeddings similarly through OracleEmbeddings. It also covers chunking documents according to specific requirements using Advanced Oracle Capabilities from OracleTextSplitter, and finally, storing and indexing these documents in a Vector Store for querying with OracleVS.

cookbook/oracleai_demo.ipynb

@@ -0,0 +1,872 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Oracle AI Vector Search with Document Processing\n",
+    "Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords.\n",
+    "One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system. This is not only powerful but also significantly more effective because you don't need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems.\n",
+    "\n",
+    "In addition, because Oracle has been building database technologies for so long, your vectors can benefit from all of Oracle Database's most powerful features, like the following:\n",
+    "\n",
+    " * Partitioning Support\n",
+    " * Real Application Clusters scalability\n",
+    " * Exadata smart scans\n",
+    " * Shard processing across geographically distributed databases\n",
+    " * Transactions\n",
+    " * Parallel SQL\n",
+    " * Disaster recovery\n",
+    " * Security\n",
+    " * Oracle Machine Learning\n",
+    " * Oracle Graph Database\n",
+    " * Oracle Spatial and Graph\n",
+    " * Oracle Blockchain\n",
+    " * JSON\n",
+    "\n",
+    "This guide demonstrates how Oracle AI Vector Search can be used with Langchain to serve an end-to-end RAG pipeline. This guide goes through examples of:\n",
+    "\n",
+    " * Loading the documents from various sources using OracleDocLoader\n",
+    " * Summarizing them within/outside the database using OracleSummary\n",
+    " * Generating embeddings for them within/outside the database using OracleEmbeddings\n",
+    " * Chunking them according to different requirements using Advanced Oracle Capabilities from OracleTextSplitter\n",
+    " * Storing and Indexing them in a Vector Store and querying them for queries in OracleVS"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Prerequisites\n",
+    "\n",
+    "Please install Oracle Python Client driver to use Langchain with Oracle AI Vector Search. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# pip install oracledb"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Create Demo User\n",
+    "First, create a demo user with all the required privileges. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Connection successful!\n",
+      "User setup done!\n"
+     ]
+    }
+   ],
+   "source": [
+    "import sys\n",
+    "\n",
+    "import oracledb\n",
+    "\n",
+    "# please update with your username, password, hostname and service_name\n",
+    "# please make sure this user has sufficient privileges to perform all below\n",
+    "username = \"\"\n",
+    "password = \"\"\n",
+    "dsn = \"\"\n",
+    "\n",
+    "try:\n",
+    "    conn = oracledb.connect(user=username, password=password, dsn=dsn)\n",
+    "    print(\"Connection successful!\")\n",
+    "\n",
+    "    cursor = conn.cursor()\n",
+    "    cursor.execute(\n",
+    "        \"\"\"\n",
+    "    begin\n",
+    "        -- drop user\n",
+    "        begin\n",
+    "            execute immediate 'drop user testuser cascade';\n",
+    "        exception\n",
+    "            when others then\n",
+    "                dbms_output.put_line('Error setting up user.');\n",
+    "        end;\n",
+    "        execute immediate 'create user testuser identified by testuser';\n",
+    "        execute immediate 'grant connect, unlimited tablespace, create credential, create procedure, create any index to testuser';\n",
+    "        execute immediate 'create or replace directory DEMO_PY_DIR as ''/scratch/hroy/view_storage/hroy_devstorage/demo/orachain''';\n",
+    "        execute immediate 'grant read, write on directory DEMO_PY_DIR to public';\n",
+    "        execute immediate 'grant create mining model to testuser';\n",
+    "\n",
+    "        -- network access\n",
+    "        begin\n",
+    "            DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE(\n",
+    "                host => '*',\n",
+    "                ace => xs$ace_type(privilege_list => xs$name_list('connect'),\n",
+    "                                principal_name => 'testuser',\n",
+    "                                principal_type => xs_acl.ptype_db));\n",
+    "        end;\n",
+    "    end;\n",
+    "    \"\"\"\n",
+    "    )\n",
+    "    print(\"User setup done!\")\n",
+    "    cursor.close()\n",
+    "    conn.close()\n",
+    "except Exception as e:\n",
+    "    print(\"User setup failed!\")\n",
+    "    cursor.close()\n",
+    "    conn.close()\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Process Documents using Oracle AI\n",
+    "Let's think about a scenario that the users have some documents in Oracle Database or in a file system. They want to use the data for Oracle AI Vector Search using Langchain.\n",
+    "\n",
+    "For that, the users need to do some document preprocessing. The first step would be to read the documents, generate their summary(if needed) and then chunk/split them if needed. After that, they need to generate the embeddings for those chunks and store into Oracle AI Vector Store. Finally, the users will perform some semantic queries on those data. \n",
+    "\n",
+    "Oracle AI Vector Search Langchain library provides a range of document processing functionalities including document loading, splitting, generating summary and embeddings.\n",
+    "\n",
+    "In the following sections, we will go through how to use Oracle AI Langchain APIs to achieve each of these functionalities individually. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Connect to Demo User\n",
+    "The following sample code will show how to connect to Oracle Database. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 45,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Connection successful!\n"
+     ]
+    }
+   ],
+   "source": [
+    "import sys\n",
+    "\n",
+    "import oracledb\n",
+    "\n",
+    "# please update with your username, password, hostname and service_name\n",
+    "username = \"\"\n",
+    "password = \"\"\n",
+    "dsn = \"\"\n",
+    "\n",
+    "try:\n",
+    "    conn = oracledb.connect(user=username, password=password, dsn=dsn)\n",
+    "    print(\"Connection successful!\")\n",
+    "except Exception as e:\n",
+    "    print(\"Connection failed!\")\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Populate a Demo Table\n",
+    "Create a demo table and insert some sample documents."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 46,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Table created and populated.\n"
+     ]
+    }
+   ],
+   "source": [
+    "try:\n",
+    "    cursor = conn.cursor()\n",
+    "\n",
+    "    drop_table_sql = \"\"\"drop table demo_tab\"\"\"\n",
+    "    cursor.execute(drop_table_sql)\n",
+    "\n",
+    "    create_table_sql = \"\"\"create table demo_tab (id number, data clob)\"\"\"\n",
+    "    cursor.execute(create_table_sql)\n",
+    "\n",
+    "    insert_row_sql = \"\"\"insert into demo_tab values (:1, :2)\"\"\"\n",
+    "    rows_to_insert = [\n",
+    "        (\n",
+    "            1,\n",
+    "            \"If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.\",\n",
+    "        ),\n",
+    "        (\n",
+    "            2,\n",
+    "            \"A tablespace can be online (accessible) or offline (not accessible) whenever the database is open.\\nA tablespace is usually online so that its data is available to users. The SYSTEM tablespace and temporary tablespaces cannot be taken offline.\",\n",
+    "        ),\n",
+    "        (\n",
+    "            3,\n",
+    "            \"The database stores LOBs differently from other data types. Creating a LOB column implicitly creates a LOB segment and a LOB index. The tablespace containing the LOB segment and LOB index, which are always stored together, may be different from the tablespace containing the table.\\nSometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.\",\n",
+    "        ),\n",
+    "    ]\n",
+    "    cursor.executemany(insert_row_sql, rows_to_insert)\n",
+    "\n",
+    "    conn.commit()\n",
+    "\n",
+    "    print(\"Table created and populated.\")\n",
+    "    cursor.close()\n",
+    "except Exception as e:\n",
+    "    print(\"Table creation failed.\")\n",
+    "    cursor.close()\n",
+    "    conn.close()\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "Now that we have a demo user and a demo table with some data, we just need to do one more setup. For embedding and summary, we have a few provider options that the users can choose from such as database, 3rd party providers like ocigenai, huggingface, openai, etc. If the users choose to use 3rd party provider, they need to create a credential with corresponding authentication information. On the other hand, if the users choose to use 'database' as provider, they need to load an onnx model to Oracle Database for embeddings; however, for summary, they don't need to do anything."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Load ONNX Model\n",
+    "\n",
+    "To generate embeddings, Oracle provides a few provider options for users to choose from. The users can choose 'database' provider or some 3rd party providers like OCIGENAI, HuggingFace, etc.\n",
+    "\n",
+    "***Note*** If the users choose database option, they need to load an ONNX model to Oracle Database. The users do not need to load an ONNX model to Oracle Database if they choose to use 3rd party provider to generate embeddings.\n",
+    "\n",
+    "One of the core benefits of using an ONNX model is that the users do not need to transfer their data to 3rd party to generate embeddings. And also, since it does not involve any network or REST API calls, it may provide better performance.\n",
+    "\n",
+    "Here is the sample code to load an ONNX model to Oracle Database:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 47,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "ONNX model loaded.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain_community.embeddings.oracleai import OracleEmbeddings\n",
+    "\n",
+    "# please update with your related information\n",
+    "# make sure that you have onnx file in the system\n",
+    "onnx_dir = \"DEMO_PY_DIR\"\n",
+    "onnx_file = \"tinybert.onnx\"\n",
+    "model_name = \"demo_model\"\n",
+    "\n",
+    "try:\n",
+    "    OracleEmbeddings.load_onnx_model(conn, onnx_dir, onnx_file, model_name)\n",
+    "    print(\"ONNX model loaded.\")\n",
+    "except Exception as e:\n",
+    "    print(\"ONNX model loading failed!\")\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Create Credential\n",
+    "\n",
+    "On the other hand, if the users choose to use 3rd party provider to generate embeddings and summary, they need to create credential to access 3rd party provider's end points.\n",
+    "\n",
+    "***Note:*** The users do not need to create any credential if they choose to use 'database' provider to generate embeddings and summary. Should the users choose to 3rd party provider, they need to create credential for the 3rd party provider they want to use. \n",
+    "\n",
+    "Here is a sample example:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    cursor = conn.cursor()\n",
+    "    cursor.execute(\n",
+    "        \"\"\"\n",
+    "       declare\n",
+    "           jo json_object_t;\n",
+    "       begin\n",
+    "           -- HuggingFace\n",
+    "           dbms_vector_chain.drop_credential(credential_name  => 'HF_CRED');\n",
+    "           jo := json_object_t();\n",
+    "           jo.put('access_token', '<access_token>');\n",
+    "           dbms_vector_chain.create_credential(\n",
+    "               credential_name   =>  'HF_CRED',\n",
+    "               params            => json(jo.to_string));\n",
+    "\n",
+    "           -- OCIGENAI\n",
+    "           dbms_vector_chain.drop_credential(credential_name  => 'OCI_CRED');\n",
+    "           jo := json_object_t();\n",
+    "           jo.put('user_ocid','<user_ocid>');\n",
+    "           jo.put('tenancy_ocid','<tenancy_ocid>');\n",
+    "           jo.put('compartment_ocid','<compartment_ocid>');\n",
+    "           jo.put('private_key','<private_key>');\n",
+    "           jo.put('fingerprint','<fingerprint>');\n",
+    "           dbms_vector_chain.create_credential(\n",
+    "               credential_name   => 'OCI_CRED',\n",
+    "               params            => json(jo.to_string));\n",
+    "       end;\n",
+    "       \"\"\"\n",
+    "    )\n",
+    "    cursor.close()\n",
+    "    print(\"Credentials created.\")\n",
+    "except Exception as ex:\n",
+    "    cursor.close()\n",
+    "    raise"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Load Documents\n",
+    "The users can load the documents from Oracle Database or a file system or both. They just need to set the loader parameters accordingly. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters.\n",
+    "\n",
+    "The main benefit of using OracleDocLoader is that it can handle 150+ different file formats. You don't need to use different types of loader for different file formats. Here is the list formats that we support: [Oracle Text Supported Document Formats](https://docs.oracle.com/en/database/oracle/oracle-database/23/ccref/oracle-text-supported-document-formats.html)\n",
+    "\n",
+    "The following sample code will show how to do that:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 48,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of docs loaded: 3\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain_community.document_loaders.oracleai import OracleDocLoader\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "# loading from Oracle Database table\n",
+    "# make sure you have the table with this specification\n",
+    "loader_params = {}\n",
+    "loader_params = {\n",
+    "    \"owner\": \"testuser\",\n",
+    "    \"tablename\": \"demo_tab\",\n",
+    "    \"colname\": \"data\",\n",
+    "}\n",
+    "\n",
+    "\"\"\" load the docs \"\"\"\n",
+    "loader = OracleDocLoader(conn=conn, params=loader_params)\n",
+    "docs = loader.load()\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Number of docs loaded: {len(docs)}\")\n",
+    "# print(f\"Document-0: {docs[0].page_content}\") # content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Generate Summary\n",
+    "Now that the user loaded the documents, they may want to generate a summary for each document. The Oracle AI Vector Search Langchain library provides an API to do that. There are a few summary generation provider options including Database, OCIGENAI, HuggingFace and so on. The users can choose their preferred provider to generate a summary. Like before, they just need to set the summary parameters accordingly. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***Note:*** The users may need to set proxy if they want to use some 3rd party summary generation providers other than Oracle's in-house and default provider: 'database'. If you don't have proxy, please remove the proxy parameter when you instantiate the OracleSummary."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# proxy to be used when we instantiate summary and embedder object\n",
+    "proxy = \"\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The following sample code will show how to generate summary:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 49,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of Summaries: 3\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain_community.utilities.oracleai import OracleSummary\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "# using 'database' provider\n",
+    "summary_params = {\n",
+    "    \"provider\": \"database\",\n",
+    "    \"glevel\": \"S\",\n",
+    "    \"numParagraphs\": 1,\n",
+    "    \"language\": \"english\",\n",
+    "}\n",
+    "\n",
+    "# get the summary instance\n",
+    "# Remove proxy if not required\n",
+    "summ = OracleSummary(conn=conn, params=summary_params, proxy=proxy)\n",
+    "\n",
+    "list_summary = []\n",
+    "for doc in docs:\n",
+    "    summary = summ.get_summary(doc.page_content)\n",
+    "    list_summary.append(summary)\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Number of Summaries: {len(list_summary)}\")\n",
+    "# print(f\"Summary-0: {list_summary[0]}\") #content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Split Documents\n",
+    "The documents can be in different sizes: small, medium, large, or very large. The users like to split/chunk their documents into smaller pieces to generate embeddings. There are lots of different splitting customizations the users can do. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters.\n",
+    "\n",
+    "The following sample code will show how to do that:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 50,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of Chunks: 3\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain_community.document_loaders.oracleai import OracleTextSplitter\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "# split by default parameters\n",
+    "splitter_params = {\"normalize\": \"all\"}\n",
+    "\n",
+    "\"\"\" get the splitter instance \"\"\"\n",
+    "splitter = OracleTextSplitter(conn=conn, params=splitter_params)\n",
+    "\n",
+    "list_chunks = []\n",
+    "for doc in docs:\n",
+    "    chunks = splitter.split_text(doc.page_content)\n",
+    "    list_chunks.extend(chunks)\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Number of Chunks: {len(list_chunks)}\")\n",
+    "# print(f\"Chunk-0: {list_chunks[0]}\") # content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Generate Embeddings\n",
+    "Now that the documents are chunked as per requirements, the users may want to generate embeddings for these chunks. Oracle AI Vector Search provides a number of ways to generate embeddings. The users can load an ONNX embedding model to Oracle Database and use it to generate embeddings or use some 3rd party API's end points to generate embeddings. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***Note:*** The users may need to set proxy if they want to use some 3rd party embedding generation providers other than 'database' provider (aka using ONNX model)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# proxy to be used when we instantiate summary and embedder object\n",
+    "proxy = \"\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The following sample code will show how to generate embeddings:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 51,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Number of embeddings: 3\n"
+     ]
+    }
+   ],
+   "source": [
+    "from langchain_community.embeddings.oracleai import OracleEmbeddings\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "# using ONNX model loaded to Oracle Database\n",
+    "embedder_params = {\"provider\": \"database\", \"model\": \"demo_model\"}\n",
+    "\n",
+    "# get the embedding instance\n",
+    "# Remove proxy if not required\n",
+    "embedder = OracleEmbeddings(conn=conn, params=embedder_params, proxy=proxy)\n",
+    "\n",
+    "embeddings = []\n",
+    "for doc in docs:\n",
+    "    chunks = splitter.split_text(doc.page_content)\n",
+    "    for chunk in chunks:\n",
+    "        embed = embedder.embed_query(chunk)\n",
+    "        embeddings.append(embed)\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Number of embeddings: {len(embeddings)}\")\n",
+    "# print(f\"Embedding-0: {embeddings[0]}\") # content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create Oracle AI Vector Store\n",
+    "Now that you know how to use Oracle AI Langchain library APIs individually to process the documents, let us show how to integrate with Oracle AI Vector Store to facilitate the semantic searches."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First, let's import all the dependencies."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 52,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "\n",
+    "import oracledb\n",
+    "from langchain_community.document_loaders.oracleai import (\n",
+    "    OracleDocLoader,\n",
+    "    OracleTextSplitter,\n",
+    ")\n",
+    "from langchain_community.embeddings.oracleai import OracleEmbeddings\n",
+    "from langchain_community.utilities.oracleai import OracleSummary\n",
+    "from langchain_community.vectorstores import oraclevs\n",
+    "from langchain_community.vectorstores.oraclevs import OracleVS\n",
+    "from langchain_community.vectorstores.utils import DistanceStrategy\n",
+    "from langchain_core.documents import Document"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, let's combine all document processing stages together. Here is the sample code below:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 53,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Connection successful!\n",
+      "ONNX model loaded.\n",
+      "Number of total chunks with metadata: 3\n"
+     ]
+    }
+   ],
+   "source": [
+    "\"\"\"\n",
+    "In this sample example, we will use 'database' provider for both summary and embeddings.\n",
+    "So, we don't need to do the followings:\n",
+    "    - set proxy for 3rd party providers\n",
+    "    - create credential for 3rd party providers\n",
+    "\n",
+    "If you choose to use 3rd party provider, \n",
+    "please follow the necessary steps for proxy and credential.\n",
+    "\"\"\"\n",
+    "\n",
+    "# oracle connection\n",
+    "# please update with your username, password, hostname, and service_name\n",
+    "username = \"\"\n",
+    "password = \"\"\n",
+    "dsn = \"\"\n",
+    "\n",
+    "try:\n",
+    "    conn = oracledb.connect(user=username, password=password, dsn=dsn)\n",
+    "    print(\"Connection successful!\")\n",
+    "except Exception as e:\n",
+    "    print(\"Connection failed!\")\n",
+    "    sys.exit(1)\n",
+    "\n",
+    "\n",
+    "# load onnx model\n",
+    "# please update with your related information\n",
+    "onnx_dir = \"DEMO_PY_DIR\"\n",
+    "onnx_file = \"tinybert.onnx\"\n",
+    "model_name = \"demo_model\"\n",
+    "try:\n",
+    "    OracleEmbeddings.load_onnx_model(conn, onnx_dir, onnx_file, model_name)\n",
+    "    print(\"ONNX model loaded.\")\n",
+    "except Exception as e:\n",
+    "    print(\"ONNX model loading failed!\")\n",
+    "    sys.exit(1)\n",
+    "\n",
+    "\n",
+    "# params\n",
+    "# please update necessary fields with related information\n",
+    "loader_params = {\n",
+    "    \"owner\": \"testuser\",\n",
+    "    \"tablename\": \"demo_tab\",\n",
+    "    \"colname\": \"data\",\n",
+    "}\n",
+    "summary_params = {\n",
+    "    \"provider\": \"database\",\n",
+    "    \"glevel\": \"S\",\n",
+    "    \"numParagraphs\": 1,\n",
+    "    \"language\": \"english\",\n",
+    "}\n",
+    "splitter_params = {\"normalize\": \"all\"}\n",
+    "embedder_params = {\"provider\": \"database\", \"model\": \"demo_model\"}\n",
+    "\n",
+    "# instantiate loader, summary, splitter, and embedder\n",
+    "loader = OracleDocLoader(conn=conn, params=loader_params)\n",
+    "summary = OracleSummary(conn=conn, params=summary_params)\n",
+    "splitter = OracleTextSplitter(conn=conn, params=splitter_params)\n",
+    "embedder = OracleEmbeddings(conn=conn, params=embedder_params)\n",
+    "\n",
+    "# process the documents\n",
+    "chunks_with_mdata = []\n",
+    "for id, doc in enumerate(docs, start=1):\n",
+    "    summ = summary.get_summary(doc.page_content)\n",
+    "    chunks = splitter.split_text(doc.page_content)\n",
+    "    for ic, chunk in enumerate(chunks, start=1):\n",
+    "        chunk_metadata = doc.metadata.copy()\n",
+    "        chunk_metadata[\"id\"] = chunk_metadata[\"_oid\"] + \"$\" + str(id) + \"$\" + str(ic)\n",
+    "        chunk_metadata[\"document_id\"] = str(id)\n",
+    "        chunk_metadata[\"document_summary\"] = str(summ[0])\n",
+    "        chunks_with_mdata.append(\n",
+    "            Document(page_content=str(chunk), metadata=chunk_metadata)\n",
+    "        )\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Number of total chunks with metadata: {len(chunks_with_mdata)}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "At this point, we have processed the documents and generated chunks with metadata. Next, we will create Oracle AI Vector Store with those chunks.\n",
+    "\n",
+    "Here is the sample code how to do that:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 55,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Vector Store Table: oravs\n"
+     ]
+    }
+   ],
+   "source": [
+    "# create Oracle AI Vector Store\n",
+    "vectorstore = OracleVS.from_documents(\n",
+    "    chunks_with_mdata,\n",
+    "    embedder,\n",
+    "    client=conn,\n",
+    "    table_name=\"oravs\",\n",
+    "    distance_strategy=DistanceStrategy.DOT_PRODUCT,\n",
+    ")\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Vector Store Table: {vectorstore.table_name}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The above example creates a vector store with DOT_PRODUCT distance strategy. \n",
+    "\n",
+    "However, the users can create Oracle AI Vector Store provides different distance strategies. Please see the [comprehensive guide](/docs/integrations/vectorstores/oracle) for more information."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now that we have embeddings stored in vector stores, let's create an index on them to get better semantic search performance during query time.\n",
+    "\n",
+    "***Note*** If you are getting some insufficient memory error, please increase ***vector_memory_size*** in your database.\n",
+    "\n",
+    "Here is the sample code to create an index:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 56,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "oraclevs.create_index(\n",
+    "    conn, vectorstore, params={\"idx_name\": \"hnsw_oravs\", \"idx_type\": \"HNSW\"}\n",
+    ")\n",
+    "\n",
+    "print(\"Index created.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The above example creates a default HNSW index on the embeddings stored in 'oravs' table. The users can set different parameters as per their requirements. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters.\n",
+    "\n",
+    "Also, there are different types of vector indices that the users can create. Please see the [comprehensive guide](/docs/integrations/vectorstores/oracle) for more information.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Perform Semantic Search\n",
+    "All set!\n",
+    "\n",
+    "We have processed the documents, stored them to vector store, and then created index to get better query performance. Now let's do some semantic searches.\n",
+    "\n",
+    "Here is the sample code for this:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 58,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[Document(page_content='The database stores LOBs differently from other data types. Creating a LOB column implicitly creates a LOB segment and a LOB index. The tablespace containing the LOB segment and LOB index, which are always stored together, may be different from the tablespace containing the table. Sometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.', metadata={'_oid': '662f2f257677f3c2311a8ff999fd34e5', '_rowid': 'AAAR/xAAEAAAAAnAAC', 'id': '662f2f257677f3c2311a8ff999fd34e5$3$1', 'document_id': '3', 'document_summary': 'Sometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.\\n\\n'})]\n",
+      "[]\n",
+      "[(Document(page_content='The database stores LOBs differently from other data types. Creating a LOB column implicitly creates a LOB segment and a LOB index. The tablespace containing the LOB segment and LOB index, which are always stored together, may be different from the tablespace containing the table. Sometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.', metadata={'_oid': '662f2f257677f3c2311a8ff999fd34e5', '_rowid': 'AAAR/xAAEAAAAAnAAC', 'id': '662f2f257677f3c2311a8ff999fd34e5$3$1', 'document_id': '3', 'document_summary': 'Sometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.\\n\\n'}), 0.055675752460956573)]\n",
+      "[]\n",
+      "[Document(page_content='If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.', metadata={'_oid': '662f2f253acf96b33b430b88699490a2', '_rowid': 'AAAR/xAAEAAAAAnAAA', 'id': '662f2f253acf96b33b430b88699490a2$1$1', 'document_id': '1', 'document_summary': 'If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.\\n\\n'})]\n",
+      "[Document(page_content='If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.', metadata={'_oid': '662f2f253acf96b33b430b88699490a2', '_rowid': 'AAAR/xAAEAAAAAnAAA', 'id': '662f2f253acf96b33b430b88699490a2$1$1', 'document_id': '1', 'document_summary': 'If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.\\n\\n'})]\n"
+     ]
+    }
+   ],
+   "source": [
+    "query = \"What is Oracle AI Vector Store?\"\n",
+    "filter = {\"document_id\": [\"1\"]}\n",
+    "\n",
+    "# Similarity search without a filter\n",
+    "print(vectorstore.similarity_search(query, 1))\n",
+    "\n",
+    "# Similarity search with a filter\n",
+    "print(vectorstore.similarity_search(query, 1, filter=filter))\n",
+    "\n",
+    "# Similarity search with relevance score\n",
+    "print(vectorstore.similarity_search_with_score(query, 1))\n",
+    "\n",
+    "# Similarity search with relevance score with filter\n",
+    "print(vectorstore.similarity_search_with_score(query, 1, filter=filter))\n",
+    "\n",
+    "# Max marginal relevance search\n",
+    "print(vectorstore.max_marginal_relevance_search(query, 1, fetch_k=20, lambda_mult=0.5))\n",
+    "\n",
+    "# Max marginal relevance search with filter\n",
+    "print(\n",
+    "    vectorstore.max_marginal_relevance_search(\n",
+    "        query, 1, fetch_k=20, lambda_mult=0.5, filter=filter\n",
+    "    )\n",
+    ")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/Makefile

@@ -1,5 +1,5 @@
 # we build the docs in these stages:
-# 1. install quarto and python dependencies
+# 1. install vercel and python dependencies
 # 2. copy files from "source dir" to "intermediate dir"
 # 2. generate files like model feat table, etc in "intermediate dir"
 # 3. copy files to their right spots (e.g. langserve readme) in "intermediate dir"
@@ -7,14 +7,13 @@
 
 SOURCE_DIR = docs/
 INTERMEDIATE_DIR = build/intermediate/docs
-OUTPUT_DIR = build/output
-OUTPUT_DOCS_DIR = $(OUTPUT_DIR)/docs
+
+OUTPUT_NEW_DIR = build/output-new
+OUTPUT_NEW_DOCS_DIR = $(OUTPUT_NEW_DIR)/docs
 
 PYTHON = .venv/bin/python
 
-QUARTO_CMD ?= quarto
-
-PARTNER_DEPS_LIST := $(shell ls -1 ../libs/partners | grep -vE "airbyte|ibm" | xargs -I {} echo "../libs/partners/{}" | tr '\n' ' ')
+PARTNER_DEPS_LIST := $(shell find ../libs/partners -mindepth 1 -maxdepth 1 -type d -exec test -e "{}/pyproject.toml" \; -print | grep -vE "airbyte|ibm|ai21" | tr '\n' ' ')
 
 PORT ?= 3001
 
@@ -25,9 +24,6 @@ install-vercel-deps:
 	yum -y update
 	yum install gcc bzip2-devel libffi-devel zlib-devel wget tar gzip rsync -y
 
-	wget -q https://github.com/quarto-dev/quarto-cli/releases/download/v1.3.450/quarto-1.3.450-linux-amd64.tar.gz
-	tar -xzf quarto-1.3.450-linux-amd64.tar.gz
-
 install-py-deps:
 	python3 -m venv .venv
 	$(PYTHON) -m pip install --upgrade pip
@@ -49,32 +45,38 @@ generate-files:
 	wget -q https://raw.githubusercontent.com/langchain-ai/langserve/main/README.md -O $(INTERMEDIATE_DIR)/langserve.md
 	$(PYTHON) scripts/resolve_local_links.py $(INTERMEDIATE_DIR)/langserve.md https://github.com/langchain-ai/langserve/tree/main/
 
-	wget -q https://raw.githubusercontent.com/langchain-ai/langgraph/main/README.md -O $(INTERMEDIATE_DIR)/langgraph.md
-	$(PYTHON) scripts/resolve_local_links.py $(INTERMEDIATE_DIR)/langgraph.md https://github.com/langchain-ai/langgraph/tree/main/
-
-	$(PYTHON) scripts/generate_api_reference_links.py --docs_dir $(INTERMEDIATE_DIR)
-
 copy-infra:
-	mkdir -p $(OUTPUT_DIR)
-	cp -r src $(OUTPUT_DIR)
-	cp vercel.json $(OUTPUT_DIR)
-	cp babel.config.js $(OUTPUT_DIR)
-	cp -r data $(OUTPUT_DIR)
-	cp docusaurus.config.js $(OUTPUT_DIR)
-	cp package.json $(OUTPUT_DIR)
-	cp sidebars.js $(OUTPUT_DIR)
-	cp -r static $(OUTPUT_DIR)
-	cp yarn.lock $(OUTPUT_DIR)
+	mkdir -p $(OUTPUT_NEW_DIR)
+	cp -r src $(OUTPUT_NEW_DIR)
+	cp vercel.json $(OUTPUT_NEW_DIR)
+	cp babel.config.js $(OUTPUT_NEW_DIR)
+	cp -r data $(OUTPUT_NEW_DIR)
+	cp docusaurus.config.js $(OUTPUT_NEW_DIR)
+	cp package.json $(OUTPUT_NEW_DIR)
+	cp sidebars.js $(OUTPUT_NEW_DIR)
+	cp -r static $(OUTPUT_NEW_DIR)
+	cp yarn.lock $(OUTPUT_NEW_DIR)
 
-quarto-render:
-	$(QUARTO_CMD) render $(INTERMEDIATE_DIR) --output-dir $(OUTPUT_DOCS_DIR) --no-execute
-	mv $(OUTPUT_DOCS_DIR)/$(INTERMEDIATE_DIR)/* $(OUTPUT_DOCS_DIR)
-	rm -rf $(OUTPUT_DOCS_DIR)/build
+render:
+	$(PYTHON) scripts/notebook_convert.py $(INTERMEDIATE_DIR) $(OUTPUT_NEW_DOCS_DIR)
 
 md-sync:
-	rsync -avm --include="*/" --include="*.mdx" --include="*.md" --exclude="*" $(INTERMEDIATE_DIR)/ $(OUTPUT_DOCS_DIR)
+	rsync -avm --include="*/" --include="*.mdx" --include="*.md" --include="*.png" --exclude="*" $(INTERMEDIATE_DIR)/ $(OUTPUT_NEW_DOCS_DIR)
 
-build: install-py-deps generate-files copy-infra quarto-render md-sync
+generate-references:
+	$(PYTHON) scripts/generate_api_reference_links.py --docs_dir $(OUTPUT_NEW_DOCS_DIR)
+
+build: install-py-deps generate-files copy-infra render md-sync generate-references
+
+vercel-build: install-vercel-deps build
+	rm -rf docs
+	mv $(OUTPUT_NEW_DOCS_DIR) docs
+	rm -rf build
+	yarn run docusaurus build
+	mv build v0.1
+	mkdir build
+	mv v0.1 build
+	mv build/v0.1/404.html build
 
 start:
-	cd $(OUTPUT_DIR) && yarn && yarn start --port=$(PORT)
+	cd $(OUTPUT_NEW_DIR) && yarn && yarn start --port=$(PORT)

docs/docs/expression_language/how_to/decorator.ipynb

@@ -7,9 +7,9 @@
    "source": [
     "# Create a runnable with the @chain decorator\n",
     "\n",
-    "You can also turn an arbitrary function into a chain by adding a `@chain` decorator. This is functionaly equivalent to wrapping in a [`RunnableLambda`](/docs/expression_language/primitives/functions).\n",
+    "You can also turn an arbitrary function into a chain by adding a `@chain` decorator. This is functionally equivalent to wrapping in a [`RunnableLambda`](/docs/expression_language/primitives/functions).\n",
     "\n",
-    "This will have the benefit of improved observability by tracing your chain correctly. Any calls to runnables inside this function will be traced as nested childen.\n",
+    "This will have the benefit of improved observability by tracing your chain correctly. Any calls to runnables inside this function will be traced as nested children.\n",
     "\n",
     "It will also allow you to use this as any other runnable, compose it in chain, etc.\n",
     "\n",

docs/docs/expression_language/index.mdx

@@ -4,6 +4,7 @@ sidebar_class_name: hidden
 
 # LangChain Expression Language (LCEL)
 
+
 LangChain Expression Language, or LCEL, is a declarative way to easily compose chains together.
 LCEL was designed from day 1 to **support putting prototypes in production, with no code changes**, from the simplest “prompt + LLM” chain to the most complex chains (we’ve seen folks successfully run LCEL chains with 100s of steps in production). To highlight a few of the reasons you might want to use LCEL:
 

docs/docs/expression_language/interface.ipynb

@@ -16,6 +16,7 @@
    "id": "9a9acd2e",
    "metadata": {},
    "source": [
+
     "To make it as easy as possible to create custom chains, we've implemented a [\"Runnable\"](https://api.python.langchain.com/en/stable/runnables/langchain_core.runnables.base.Runnable.html#langchain_core.runnables.base.Runnable) protocol. Many LangChain components implement the `Runnable` protocol, including chat models, LLMs, output parsers, retrievers, prompt templates, and more. There are also several useful primitives for working with runnables, which you can read about [in this section](/docs/expression_language/primitives).\n",
     "\n",
     "This is a standard interface, which makes it easy to define custom chains as well as invoke them in a standard way. \n",

docs/docs/expression_language/why.ipynb

@@ -3,16 +3,18 @@
   {
    "cell_type": "raw",
    "id": "bc346658-6820-413a-bd8f-11bd3082fe43",
-   "metadata": {},
+   "metadata": {
+    "vscode": {
+     "languageId": "raw"
+    }
+   },
    "source": [
     "---\n",
     "sidebar_position: 0.5\n",
     "title: Advantages of LCEL\n",
     "---\n",
     "\n",
-    "```{=mdx}\n",
-    "import { ColumnContainer, Column } from \"@theme/Columns\";\n",
-    "```"
+    "import { ColumnContainer, Column } from \"@theme/Columns\";"
    ]
   },
   {
@@ -20,6 +22,7 @@
    "id": "919a5ae2-ed21-4923-b98f-723c111bac67",
    "metadata": {},
    "source": [
+    "\n",
     ":::{.callout-tip} \n",
     "We recommend reading the LCEL [Get started](/docs/expression_language/get_started) section first.\n",
     ":::"
@@ -56,13 +59,10 @@
     "## Invoke\n",
     "In the simplest case, we just want to pass in a topic string and get back a joke string:\n",
     "\n",
-    "```{=mdx}\n",
     "<ColumnContainer>\n",
     "\n",
     "<Column>\n",
     "\n",
-    "```\n",
-    "\n",
     "#### Without LCEL\n"
    ]
   },
@@ -102,11 +102,9 @@
    "metadata": {},
    "source": [
     "\n",
-    "```{=mdx}\n",
     "</Column>\n",
     "\n",
     "<Column>\n",
-    "```\n",
     "\n",
     "#### LCEL\n",
     "\n"
@@ -146,18 +144,15 @@
    "metadata": {},
    "source": [
     "\n",
-    "```{=mdx}\n",
     "</Column>\n",
     "</ColumnContainer>\n",
-    "```\n",
+    "\n",
     "## Stream\n",
     "If we want to stream results instead, we'll need to change our function:\n",
     "\n",
-    "```{=mdx}\n",
     "\n",
     "<ColumnContainer>\n",
     "<Column>\n",
-    "```\n",
     "\n",
     "#### Without LCEL\n",
     "\n"
@@ -198,11 +193,10 @@
    "id": "f8e36b0e-c7dc-4130-a51b-189d4b756c7f",
    "metadata": {},
    "source": [
-    "```{=mdx}\n",
     "</Column>\n",
     "\n",
     "<Column>\n",
-    "```\n",
+    "\n",
     "#### LCEL\n",
     "\n"
    ]
@@ -223,19 +217,18 @@
    "id": "b9b41e78-ddeb-44d0-a58b-a0ea0c99a761",
    "metadata": {},
    "source": [
-    "```{=mdx}\n",
     "</Column>\n",
     "</ColumnContainer>\n",
-    "```\n",
+    "\n",
     "\n",
     "## Batch\n",
     "\n",
     "If we want to run on a batch of inputs in parallel, we'll again need a new function:\n",
     "\n",
-    "```{=mdx}\n",
+    "\n",
     "<ColumnContainer>\n",
     "<Column>\n",
-    "```\n",
+    "\n",
     "\n",
     "#### Without LCEL\n",
     "\n"
@@ -263,11 +256,11 @@
    "id": "9b3e9d34-6775-43c1-93d8-684b58e341ab",
    "metadata": {},
    "source": [
-    "```{=mdx}\n",
+    "\n",
     "</Column>\n",
     "\n",
     "<Column>\n",
-    "```\n",
+    "\n",
     "#### LCEL\n",
     "\n"
    ]
@@ -287,18 +280,14 @@
    "id": "cc5ba36f-eec1-4fc1-8cfe-fa242a7f7809",
    "metadata": {},
    "source": [
-    "```{=mdx}\n",
     "</Column>\n",
     "</ColumnContainer>\n",
-    "```\n",
     "## Async\n",
     "\n",
     "If we need an asynchronous version:\n",
     "\n",
-    "```{=mdx}\n",
     "<ColumnContainer>\n",
     "<Column>\n",
-    "```\n",
     "\n",
     "#### Without LCEL\n",
     "\n"
@@ -334,11 +323,9 @@
    "id": "2f209290-498c-4c17-839e-ee9002919846",
    "metadata": {},
    "source": [
-    "```{=mdx}\n",
     "</Column>\n",
     "\n",
     "<Column>\n",
-    "```\n",
     "\n",
     "#### LCEL\n",
     "\n"
@@ -359,10 +346,9 @@
    "id": "1f282129-99a3-40f4-b67f-2d0718b1bea9",
    "metadata": {},
    "source": [
-    "```{=mdx}\n",
     "</Column>\n",
     "</ColumnContainer>\n",
-    "```\n",
+    "\n",
     "## Async Batch\n",
     "\n",
     "```{=mdx}\n",

docs/docs/get_started/installation.mdx

@@ -4,6 +4,7 @@ sidebar_position: 2
 
 # Installation
 
+
 ## Official release
 
 To install LangChain run:

docs/docs/get_started/introduction.mdx

@@ -5,6 +5,7 @@ sidebar_class_name: hidden
 
 # Introduction
 
+
 **LangChain** is a framework for developing applications powered by large language models (LLMs).
 
 LangChain simplifies every stage of the LLM application lifecycle:
@@ -13,12 +14,13 @@ LangChain simplifies every stage of the LLM application lifecycle:
 - **Deployment**: Turn any chain into an API with [LangServe](/docs/langserve).
 
 import ThemedImage from '@theme/ThemedImage';
+import useBaseUrl from '@docusaurus/useBaseUrl';
 
 <ThemedImage
   alt="Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers."
   sources={{
-    light: '/svg/langchain_stack.svg',
-    dark: '/svg/langchain_stack_dark.svg',
+    light: useBaseUrl('/svg/langchain_stack.svg'),
+    dark: useBaseUrl('/svg/langchain_stack_dark.svg'),
   }}
   title="LangChain Framework Overview"
 />
@@ -29,7 +31,7 @@ Concretely, the framework consists of the following open-source libraries:
 - **`langchain-community`**: Third party integrations.
   - Partner packages (e.g. **`langchain-openai`**, **`langchain-anthropic`**, etc.): Some integrations have been further split into their own lightweight packages that only depend on **`langchain-core`**.
 - **`langchain`**: Chains, agents, and retrieval strategies that make up an application's cognitive architecture.
-- **[langgraph](/docs/langgraph)**: Build robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph.
+- **[langgraph](https://langchain-ai.github.io/langgraph/)**: Build robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph.
 - **[langserve](/docs/langserve)**: Deploy LangChain chains as REST APIs.
 
 The broader ecosystem includes:
@@ -73,7 +75,7 @@ LangChain Expression Language (LCEL) is the foundation of many of LangChain's co
 ### [🦜🛠️ LangSmith](/docs/langsmith)
 Trace and evaluate your language model applications and intelligent agents to help you move from prototype to production.
 
-### [🦜🕸️ LangGraph](/docs/langgraph)
+### [🦜🕸️ LangGraph](https://langchain-ai.github.io/langgraph/)
 Build stateful, multi-actor applications with LLMs, built on top of (and intended to be used with) LangChain primitives.
 
 ### [🦜🏓 LangServe](/docs/langserve)

docs/docs/get_started/quickstart.mdx

@@ -4,6 +4,7 @@ sidebar_position: 1
 
 # Quickstart
 
+
 In this quickstart we'll show you how to:
 - Get setup with LangChain, LangSmith and LangServe
 - Use the most basic and common components of LangChain: prompt templates, models, and output parsers

docs/docs/guides/productionization/evaluation/comparison/index.mdx

@@ -17,7 +17,7 @@ Here's a summary of the key methods and properties of a comparison evaluator:
 - `requires_reference`: This property specifies whether this evaluator requires a reference label.
 
 :::note LangSmith Support
-The [run_on_dataset](https://api.python.langchain.com/en/latest/langchain_api_reference.html#module-langchain.smith) evaluation method is designed to evaluate only a single model at a time, and thus, doesn't support these evaluators.
+Pairwise evaluations are supported in LangSmith via the [`evaluate_comparative`](https://docs.smith.langchain.com/how_to_guides/evaluation/evaluate_pairwise) function.
 :::
 
 Detailed information about creating custom evaluators and the available built-in comparison evaluators is provided in the following sections.

docs/docs/guides/productionization/safety/hugging_face_prompt_injection.ipynb

@@ -50,7 +50,7 @@
     "from transformers import AutoTokenizer, pipeline\n",
     "\n",
     "# Using https://huggingface.co/protectai/deberta-v3-base-prompt-injection-v2\n",
-    "model_path = \"laiyer/deberta-v3-base-prompt-injection-v2\"\n",
+    "model_path = \"protectai/deberta-v3-base-prompt-injection-v2\"\n",
     "revision = None  # We recommend specifiying the revision to avoid breaking changes or supply chain attacks\n",
     "tokenizer = AutoTokenizer.from_pretrained(\n",
     "    model_path, revision=revision, model_input_names=[\"input_ids\", \"attention_mask\"]\n",

docs/docs/integrations/chat/deepinfra.ipynb

@@ -8,13 +8,8 @@
    "source": [
     "# DeepInfra\n",
     "\n",
-    "[DeepInfra](https://deepinfra.com/?utm_source=langchain) is a serverless inference as a service that provides access to a [variety of LLMs](https://deepinfra.com/models?utm_source=langchain) and [embeddings models](https://deepinfra.com/models?type=embeddings&utm_source=langchain). This notebook goes over how to use LangChain with DeepInfra for chat models."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
+    "[DeepInfra](https://deepinfra.com/?utm_source=langchain) is a serverless inference as a service that provides access to a [variety of LLMs](https://deepinfra.com/models?utm_source=langchain) and [embeddings models](https://deepinfra.com/models?type=embeddings&utm_source=langchain). This notebook goes over how to use LangChain with DeepInfra for chat models.\n",
+    "\n",
     "## Set the Environment API Key\n",
     "Make sure to get your API key from DeepInfra. You have to [Login](https://deepinfra.com/login?from=%2Fdash) and get a new token.\n",
     "\n",
@@ -24,7 +19,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
+   "id": "d4a7c55d-b235-4ca4-a579-c90cc9570da9",
    "metadata": {
     "tags": []
    },
@@ -32,70 +28,19 @@
    "source": [
     "# get a new token: https://deepinfra.com/login?from=%2Fdash\n",
     "\n",
+    "import os\n",
     "from getpass import getpass\n",
     "\n",
-    "DEEPINFRA_API_TOKEN = getpass()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "import os\n",
+    "from langchain_community.chat_models import ChatDeepInfra\n",
+    "from langchain_core.messages import HumanMessage\n",
+    "\n",
+    "DEEPINFRA_API_TOKEN = getpass()\n",
     "\n",
     "# or pass deepinfra_api_token parameter to the ChatDeepInfra constructor\n",
-    "os.environ[\"DEEPINFRA_API_TOKEN\"] = DEEPINFRA_API_TOKEN"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "d4a7c55d-b235-4ca4-a579-c90cc9570da9",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from langchain_community.chat_models import ChatDeepInfra\n",
-    "from langchain_core.messages import HumanMessage"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "70cf04e8-423a-4ff6-8b09-f11fb711c817",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "chat = ChatDeepInfra(model=\"meta-llama/Llama-2-7b-chat-hf\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "8199ef8f-eb8b-4253-9ea0-6c24a013ca4c",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "AIMessage(content=\" J'aime la programmation.\", additional_kwargs={}, example=False)"
-      ]
-     },
-     "execution_count": 3,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
+    "os.environ[\"DEEPINFRA_API_TOKEN\"] = DEEPINFRA_API_TOKEN\n",
+    "\n",
+    "chat = ChatDeepInfra(model=\"meta-llama/Llama-2-7b-chat-hf\")\n",
+    "\n",
     "messages = [\n",
     "    HumanMessage(\n",
     "        content=\"Translate this sentence from English to French. I love programming.\"\n",
@@ -115,7 +60,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "id": "93a21c5c-6ef9-4688-be60-b2e1f94842fb",
    "metadata": {
     "tags": []
@@ -127,53 +72,24 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "id": "c5fac0e9-05a4-4fc1-a3b3-e5bbb24b971b",
    "metadata": {
     "tags": []
    },
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "LLMResult(generations=[[ChatGeneration(text=\" J'aime programmer.\", generation_info=None, message=AIMessage(content=\" J'aime programmer.\", additional_kwargs={}, example=False))]], llm_output={}, run=[RunInfo(run_id=UUID('8cc8fb68-1c35-439c-96a0-695036a93652'))])"
-      ]
-     },
-     "execution_count": 5,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "await chat.agenerate([messages])"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": null,
    "id": "025be980-e50d-4a68-93dc-c9c7b500ce34",
    "metadata": {
     "tags": []
    },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      " J'aime la programmation."
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "AIMessage(content=\" J'aime la programmation.\", additional_kwargs={}, example=False)"
-      ]
-     },
-     "execution_count": 6,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "chat = ChatDeepInfra(\n",
     "    streaming=True,\n",

docs/docs/integrations/chat/kinetica.ipynb

@@ -123,7 +123,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [
     {
@@ -172,7 +172,7 @@
        "      <td>F</td>\n",
        "      <td>59836 Carla Causeway Suite 939\\nPort Eugene, I...</td>\n",
        "      <td>meltondenise@yahoo.com</td>\n",
-       "      <td>1997-09-09</td>\n",
+       "      <td>1997-11-23</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
@@ -181,7 +181,7 @@
        "      <td>M</td>\n",
        "      <td>3108 Christina Forges\\nPort Timothychester, KY...</td>\n",
        "      <td>erica80@hotmail.com</td>\n",
-       "      <td>1924-05-05</td>\n",
+       "      <td>1924-07-19</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
@@ -190,7 +190,7 @@
        "      <td>F</td>\n",
        "      <td>Unit 7405 Box 3052\\nDPO AE 09858</td>\n",
        "      <td>timothypotts@gmail.com</td>\n",
-       "      <td>1933-09-06</td>\n",
+       "      <td>1933-11-20</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
@@ -199,7 +199,7 @@
        "      <td>F</td>\n",
        "      <td>6408 Christopher Hill Apt. 459\\nNew Benjamin, ...</td>\n",
        "      <td>dadams@gmail.com</td>\n",
-       "      <td>1988-07-28</td>\n",
+       "      <td>1988-10-11</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>4</th>\n",
@@ -208,7 +208,7 @@
        "      <td>M</td>\n",
        "      <td>2241 Bell Gardens Suite 723\\nScottside, CA 38463</td>\n",
        "      <td>williamayala@gmail.com</td>\n",
-       "      <td>1930-12-19</td>\n",
+       "      <td>1931-03-04</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
@@ -233,14 +233,14 @@
        "\n",
        "    birthdate  \n",
        "id             \n",
-       "0  1997-09-09  \n",
-       "1  1924-05-05  \n",
-       "2  1933-09-06  \n",
-       "3  1988-07-28  \n",
-       "4  1930-12-19  "
+       "0  1997-11-23  \n",
+       "1  1924-07-19  \n",
+       "2  1933-11-20  \n",
+       "3  1988-10-11  \n",
+       "4  1931-03-04  "
       ]
      },
-     "execution_count": 2,
+     "execution_count": 1,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -646,7 +646,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.18"
+   "version": "3.11.4"
   }
  },
  "nbformat": 4,

docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb

@@ -9,6 +9,12 @@
    "source": [
     "# NVIDIA AI Foundation Endpoints\n",
     "\n",
+    ":::{.callout-caution}\n",
+    "\n",
+    "These are the LangChain v0.1 docs. Please refer to the updated [LangChain v0.2 docs](https://python.langchain.com/v0.2/docs/integrations/chat/nvidia_ai_endpoints/) instead.\n",
+    "\n",
+    ":::\n",
+    "\n",
     "The `ChatNVIDIA` class is a LangChain chat model that connects to [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/).\n",
     "\n",
     "\n",
@@ -1149,7 +1155,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.18"
+   "version": "3.11.9"
   }
  },
  "nbformat": 4,

docs/docs/integrations/chat/ollama_functions.ipynb

@@ -15,6 +15,12 @@
    "source": [
     "# OllamaFunctions\n",
     "\n",
+    ":::{.callout-caution}\n",
+    "\n",
+    "This was an experimental wrapper that bolted-on tool calling support to models that do not natively support it. The primary Ollama integration now supports tool calling, and should be used instead. See example usage in LangChain v0.2 documentation [here](https://python.langchain.com/v0.2/docs/integrations/chat/ollama/).\n",
+    "\n",
+    ":::\n",
+    "\n",
     "This notebook shows how to use an experimental wrapper around Ollama that gives it the same API as OpenAI Functions.\n",
     "\n",
     "Note that more powerful and capable models will perform better with complex schema and/or multiple functions. The examples below use llama3 and phi3 models.\n",

docs/docs/integrations/chat/openai.ipynb

@@ -17,6 +17,7 @@
    "source": [
     "# ChatOpenAI\n",
     "\n",
+
     "This notebook covers how to get started with OpenAI chat models."
    ]
   },
@@ -147,7 +148,7 @@
     "\n",
     "### ChatOpenAI.bind_tools()\n",
     "\n",
-    "With `ChatAnthropic.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an Anthropic tool schemas, which looks like:\n",
+    "With `ChatOpenAI.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an OpenAI tool schemas, which looks like:\n",
     "```\n",
     "{\n",
     "    \"name\": \"...\",\n",

docs/docs/integrations/chat/together.ipynb

@@ -0,0 +1,119 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "2970dd75-8ebf-4b51-8282-9b454b8f356d",
+   "metadata": {},
+   "source": [
+    "# Together AI\n",
+    "\n",
+    "[Together AI](https://www.together.ai/) offers an API to query [50+ leading open-source models](https://docs.together.ai/docs/inference-models) in a couple lines of code.\n",
+    "\n",
+    "This example goes over how to use LangChain to interact with Together AI models."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1c47fc36",
+   "metadata": {},
+   "source": [
+    "## Installation"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1ecdb29d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install --upgrade langchain-together"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "89883202",
+   "metadata": {},
+   "source": [
+    "## Environment\n",
+    "\n",
+    "To use Together AI, you'll need an API key which you can find here:\n",
+    "https://api.together.ai/settings/api-keys. This can be passed in as an init param\n",
+    "``together_api_key`` or set as environment variable ``TOGETHER_API_KEY``.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8304b4d9",
+   "metadata": {},
+   "source": [
+    "## Example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "637bb53f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Querying chat models with Together AI\n",
+    "\n",
+    "from langchain_together import ChatTogether\n",
+    "\n",
+    "# choose from our 50+ models here: https://docs.together.ai/docs/inference-models\n",
+    "chat = ChatTogether(\n",
+    "    # together_api_key=\"YOUR_API_KEY\",\n",
+    "    model=\"meta-llama/Llama-3-70b-chat-hf\",\n",
+    ")\n",
+    "\n",
+    "# stream the response back from the model\n",
+    "for m in chat.stream(\"Tell me fun things to do in NYC\"):\n",
+    "    print(m.content, end=\"\", flush=True)\n",
+    "\n",
+    "# if you don't want to do streaming, you can use the invoke method\n",
+    "# chat.invoke(\"Tell me fun things to do in NYC\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e7b7170d-d7c5-4890-9714-a37238343805",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Querying code and language models with Together AI\n",
+    "\n",
+    "from langchain_together import Together\n",
+    "\n",
+    "llm = Together(\n",
+    "    model=\"codellama/CodeLlama-70b-Python-hf\",\n",
+    "    # together_api_key=\"...\"\n",
+    ")\n",
+    "\n",
+    "print(llm.invoke(\"def bubble_sort(): \"))"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/docs/integrations/document_loaders/larksuite.ipynb

@@ -30,13 +30,24 @@
    "source": [
     "from getpass import getpass\n",
     "\n",
-    "from langchain_community.document_loaders.larksuite import LarkSuiteDocLoader\n",
+    "from langchain_community.document_loaders.larksuite import (\n",
+    "    LarkSuiteDocLoader,\n",
+    "    LarkSuiteWikiLoader,\n",
+    ")\n",
     "\n",
     "DOMAIN = input(\"larksuite domain\")\n",
     "ACCESS_TOKEN = getpass(\"larksuite tenant_access_token or user_access_token\")\n",
     "DOCUMENT_ID = input(\"larksuite document id\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "4b6b9a66",
+   "metadata": {},
+   "source": [
+    "## Load From Document"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 3,
@@ -65,6 +76,38 @@
     "pprint(docs)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "86f4a714",
+   "metadata": {},
+   "source": [
+    "## Load From Wiki"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "7332dfb9",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[Document(page_content='Test doc\\nThis is a test wiki doc.\\n', metadata={'document_id': 'TxOKdtMWaoSTDLxYS4ZcdEI7nwc', 'revision_id': 15, 'title': 'Test doc'})]\n"
+     ]
+    }
+   ],
+   "source": [
+    "from pprint import pprint\n",
+    "\n",
+    "DOCUMENT_ID = input(\"larksuite wiki id\")\n",
+    "larksuite_loader = LarkSuiteWikiLoader(DOMAIN, ACCESS_TOKEN, DOCUMENT_ID)\n",
+    "docs = larksuite_loader.load()\n",
+    "\n",
+    "pprint(docs)"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

docs/docs/integrations/document_loaders/oracleai.ipynb

@@ -0,0 +1,236 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Oracle AI Vector Search: Document Processing\n",
+    "Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords. One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system. This is not only powerful but also significantly more effective because you don't need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems.\n",
+    "\n",
+    "The guide demonstrates how to use Document Processing Capabilities within Oracle AI Vector Search to load and chunk documents using OracleDocLoader and OracleTextSplitter respectively."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Prerequisites\n",
+    "\n",
+    "Please install Oracle Python Client driver to use Langchain with Oracle AI Vector Search. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# pip install oracledb"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Connect to Oracle Database\n",
+    "The following sample code will show how to connect to Oracle Database. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "\n",
+    "import oracledb\n",
+    "\n",
+    "# please update with your username, password, hostname and service_name\n",
+    "username = \"<username>\"\n",
+    "password = \"<password>\"\n",
+    "dsn = \"<hostname>/<service_name>\"\n",
+    "\n",
+    "try:\n",
+    "    conn = oracledb.connect(user=username, password=password, dsn=dsn)\n",
+    "    print(\"Connection successful!\")\n",
+    "except Exception as e:\n",
+    "    print(\"Connection failed!\")\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's create a table and insert some sample docs to test."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    cursor = conn.cursor()\n",
+    "\n",
+    "    drop_table_sql = \"\"\"drop table if exists demo_tab\"\"\"\n",
+    "    cursor.execute(drop_table_sql)\n",
+    "\n",
+    "    create_table_sql = \"\"\"create table demo_tab (id number, data clob)\"\"\"\n",
+    "    cursor.execute(create_table_sql)\n",
+    "\n",
+    "    insert_row_sql = \"\"\"insert into demo_tab values (:1, :2)\"\"\"\n",
+    "    rows_to_insert = [\n",
+    "        (\n",
+    "            1,\n",
+    "            \"If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.\",\n",
+    "        ),\n",
+    "        (\n",
+    "            2,\n",
+    "            \"A tablespace can be online (accessible) or offline (not accessible) whenever the database is open.\\nA tablespace is usually online so that its data is available to users. The SYSTEM tablespace and temporary tablespaces cannot be taken offline.\",\n",
+    "        ),\n",
+    "        (\n",
+    "            3,\n",
+    "            \"The database stores LOBs differently from other data types. Creating a LOB column implicitly creates a LOB segment and a LOB index. The tablespace containing the LOB segment and LOB index, which are always stored together, may be different from the tablespace containing the table.\\nSometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.\",\n",
+    "        ),\n",
+    "    ]\n",
+    "    cursor.executemany(insert_row_sql, rows_to_insert)\n",
+    "\n",
+    "    conn.commit()\n",
+    "\n",
+    "    print(\"Table created and populated.\")\n",
+    "    cursor.close()\n",
+    "except Exception as e:\n",
+    "    print(\"Table creation failed.\")\n",
+    "    cursor.close()\n",
+    "    conn.close()\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Load Documents\n",
+    "The users can load the documents from Oracle Database or a file system or both. They just need to set the loader parameters accordingly. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters.\n",
+    "\n",
+    "The main benefit of using OracleDocLoader is that it can handle 150+ different file formats. You don't need to use different types of loader for different file formats. Here is the list of the formats that we support: [Oracle Text Supported Document Formats](https://docs.oracle.com/en/database/oracle/oracle-database/23/ccref/oracle-text-supported-document-formats.html)\n",
+    "\n",
+    "The following sample code will show how to do that:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.document_loaders.oracleai import OracleDocLoader\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "\"\"\"\n",
+    "# loading a local file\n",
+    "loader_params = {}\n",
+    "loader_params[\"file\"] = \"<file>\"\n",
+    "\n",
+    "# loading from a local directory\n",
+    "loader_params = {}\n",
+    "loader_params[\"dir\"] = \"<directory>\"\n",
+    "\"\"\"\n",
+    "\n",
+    "# loading from Oracle Database table\n",
+    "loader_params = {\n",
+    "    \"owner\": \"<owner>\",\n",
+    "    \"tablename\": \"demo_tab\",\n",
+    "    \"colname\": \"data\",\n",
+    "}\n",
+    "\n",
+    "\"\"\" load the docs \"\"\"\n",
+    "loader = OracleDocLoader(conn=conn, params=loader_params)\n",
+    "docs = loader.load()\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Number of docs loaded: {len(docs)}\")\n",
+    "# print(f\"Document-0: {docs[0].page_content}\") # content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Split Documents\n",
+    "The documents can be in different sizes: small, medium, large, or very large. The users like to split/chunk their documents into smaller pieces to generate embeddings. There are lots of different splitting customizations the users can do. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters.\n",
+    "\n",
+    "The following sample code will show how to do that:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.document_loaders.oracleai import OracleTextSplitter\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "\"\"\"\n",
+    "# Some examples\n",
+    "# split by chars, max 500 chars\n",
+    "splitter_params = {\"split\": \"chars\", \"max\": 500, \"normalize\": \"all\"}\n",
+    "\n",
+    "# split by words, max 100 words\n",
+    "splitter_params = {\"split\": \"words\", \"max\": 100, \"normalize\": \"all\"}\n",
+    "\n",
+    "# split by sentence, max 20 sentences\n",
+    "splitter_params = {\"split\": \"sentence\", \"max\": 20, \"normalize\": \"all\"}\n",
+    "\"\"\"\n",
+    "\n",
+    "# split by default parameters\n",
+    "splitter_params = {\"normalize\": \"all\"}\n",
+    "\n",
+    "# get the splitter instance\n",
+    "splitter = OracleTextSplitter(conn=conn, params=splitter_params)\n",
+    "\n",
+    "list_chunks = []\n",
+    "for doc in docs:\n",
+    "    chunks = splitter.split_text(doc.page_content)\n",
+    "    list_chunks.extend(chunks)\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Number of Chunks: {len(list_chunks)}\")\n",
+    "# print(f\"Chunk-0: {list_chunks[0]}\") # content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### End to End Demo\n",
+    "Please refer to our complete demo guide [Oracle AI Vector Search End-to-End Demo Guide](https://github.com/langchain-ai/langchain/tree/master/cookbook/oracleai_demo.ipynb) to build an end to end RAG pipeline with the help of Oracle AI Vector Search.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/docs/integrations/document_loaders/upstage.ipynb

@@ -4,7 +4,10 @@
    "cell_type": "raw",
    "id": "910f5772b6af13c9",
    "metadata": {
-    "collapsed": false
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
    },
    "source": [
     "---\n",
@@ -16,7 +19,10 @@
    "cell_type": "markdown",
    "id": "433f5422ad8e1efa",
    "metadata": {
-    "collapsed": false
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
    },
    "source": [
     "# UpstageLayoutAnalysisLoader\n",
@@ -41,9 +47,9 @@
     "\n",
     "Make sure to set the following environment variables:\n",
     "\n",
-    "- `UPSTAGE_DOCUMENT_AI_API_KEY`: Your Upstage Document AI API key. Read [Upstage developers document](https://developers.upstage.ai/docs/getting-started/quick-start) to get your API key.\n",
+    "- `UPSTAGE_API_KEY`: Your Upstage API key. Read [Upstage developers document](https://developers.upstage.ai/docs/getting-started/quick-start) to get your API key.\n",
     "\n",
-    "> As of April 2024, you need separate access tokens for Solar and Layout Analysis. The access tokens will be consolidated soon (hopefully in May) and you'll need just one key for all features."
+    "> The previously used UPSTAGE_DOCUMENT_AI_API_KEY is deprecated. However, the key previously used in UPSTAGE_DOCUMENT_AI_API_KEY can now be used in UPSTAGE_API_KEY."
    ]
   },
   {
@@ -63,7 +69,7 @@
    "source": [
     "import os\n",
     "\n",
-    "os.environ[\"UPSTAGE_DOCUMENT_AI_API_KEY\"] = \"YOUR_API_KEY\""
+    "os.environ[\"UPSTAGE_API_KEY\"] = \"YOUR_API_KEY\""
    ]
   },
   {
@@ -98,7 +104,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -112,7 +118,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.14"
+   "version": "3.12.3"
   }
  },
  "nbformat": 4,

docs/docs/integrations/llms/ipex_llm.ipynb

@@ -54,7 +54,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -76,7 +76,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -93,31 +93,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "897501860fe4452b836f816c72d955dd",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "2024-04-24 21:20:12,461 - INFO - Converting the current model to sym_int4 format......\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "llm = IpexLLM.from_model_id(\n",
     "    model_id=\"lmsys/vicuna-7b-v1.5\",\n",
@@ -134,29 +112,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/opt/anaconda3/envs/shane-langchain-3.11/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 0.3.0. Use RunnableSequence, e.g., `prompt | llm` instead.\n",
-      "  warn_deprecated(\n",
-      "/opt/anaconda3/envs/shane-langchain-3.11/lib/python3.11/site-packages/transformers/generation/utils.py:1369: UserWarning: Using `max_length`'s default (4096) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.\n",
-      "  warnings.warn(\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "AI stands for \"Artificial Intelligence.\" It refers to the development of computer systems that can perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. AI can be achieved through a combination of techniques such as machine learning, natural language processing, computer vision, and robotics. The ultimate goal of AI research is to create machines that can think and learn like humans, and can even exceed human capabilities in certain areas.\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
-    "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
+    "llm_chain = prompt | llm\n",
     "\n",
     "question = \"What is AI?\"\n",
     "output = llm_chain.invoke(question)"
@@ -179,7 +139,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -198,17 +158,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "2024-04-24 21:20:35,874 - INFO - Converting the current model to sym_int4 format......\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "llm_lowbit = IpexLLM.from_model_id_low_bit(\n",
     "    model_id=saved_lowbit_model_path,\n",
@@ -227,27 +179,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/opt/anaconda3/envs/shane-langchain-3.11/lib/python3.11/site-packages/transformers/generation/utils.py:1369: UserWarning: Using `max_length`'s default (4096) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.\n",
-      "  warnings.warn(\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "AI stands for \"Artificial Intelligence.\" It refers to the development of computer systems that can perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. AI can be achieved through a combination of techniques such as machine learning, natural language processing, computer vision, and robotics. The ultimate goal of AI research is to create machines that can think and learn like humans, and can even exceed human capabilities in certain areas.\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
-    "llm_chain = LLMChain(prompt=prompt, llm=llm_lowbit)\n",
+    "llm_chain = prompt | llm_lowbit\n",
+    "\n",
     "\n",
     "question = \"What is AI?\"\n",
     "output = llm_chain.invoke(question)"
@@ -256,7 +193,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "shane-diffusion",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -270,7 +207,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.9"
+   "version": "3.11.5"
   }
  },
  "nbformat": 4,

docs/docs/integrations/llms/konko.ipynb

@@ -2,7 +2,12 @@
  "cells": [
   {
    "cell_type": "raw",
-   "metadata": {},
+   "id": "b5f24c75",
+   "metadata": {
+    "vscode": {
+     "languageId": "raw"
+    }
+   },
    "source": [
     "---\n",
     "sidebar_label: Konko\n",
@@ -21,23 +26,12 @@
     "1. **Select** the right open source or proprietary LLMs for their application\n",
     "2. **Build** applications faster with integrations to leading application frameworks and fully managed APIs\n",
     "3. **Fine tune** smaller open-source LLMs to achieve industry-leading performance at a fraction of the cost\n",
-    "4. **Deploy production-scale APIs** that meet security, privacy, throughput, and latency SLAs without infrastructure set-up or administration using Konko AI's SOC 2 compliant, multi-cloud infrastructure\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0d896d07-82b4-4f38-8c37-f0bc8b0e4fe1",
-   "metadata": {},
-   "source": [
+    "4. **Deploy production-scale APIs** that meet security, privacy, throughput, and latency SLAs without infrastructure set-up or administration using Konko AI's SOC 2 compliant, multi-cloud infrastructure\n",
+    "\n",
     "This example goes over how to use LangChain to interact with `Konko` completion [models](https://docs.konko.ai/docs/list-of-models#konko-hosted-models-for-completion)\n",
     "\n",
-    "To run this notebook, you'll need Konko API key. Sign in to our web app to [create an API key](https://platform.konko.ai/settings/api-keys) to access models"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
+    "To run this notebook, you'll need Konko API key. Sign in to our web app to [create an API key](https://platform.konko.ai/settings/api-keys) to access models\n",
+    "\n",
     "#### Set Environment Variables\n",
     "\n",
     "1. You can set environment variables for \n",
@@ -48,13 +42,8 @@
     "```shell\n",
     "export KONKO_API_KEY={your_KONKO_API_KEY_here}\n",
     "export OPENAI_API_KEY={your_OPENAI_API_KEY_here} #Optional\n",
-    "```"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
+    "```\n",
+    "\n",
     "## Calling a model\n",
     "\n",
     "Find a model on the [Konko overview page](https://docs.konko.ai/docs/list-of-models)\n",
@@ -92,14 +81,6 @@
     "input_ = \"\"\"You are a helpful assistant. Explain Big Bang Theory briefly.\"\"\"\n",
     "print(llm.invoke(input_))"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "78148bf7-2211-40b4-93a7-e90139ab1169",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
@@ -118,7 +99,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.11.4"
   }
  },
  "nbformat": 4,

docs/docs/integrations/llms/llamacpp.ipynb

@@ -6,6 +6,7 @@
    "source": [
     "# Llama.cpp\n",
     "\n",
+
     "[llama-cpp-python](https://github.com/abetlen/llama-cpp-python) is a Python binding for [llama.cpp](https://github.com/ggerganov/llama.cpp).\n",
     "\n",
     "It supports inference for [many LLMs](https://github.com/ggerganov/llama.cpp#description) models, which can be accessed on [Hugging Face](https://huggingface.co/TheBloke).\n",

docs/docs/integrations/llms/ollama.ipynb

@@ -6,6 +6,7 @@
    "source": [
     "# Ollama\n",
     "\n",
+
     "[Ollama](https://ollama.ai/) allows you to run open-source large language models, such as Llama 2, locally.\n",
     "\n",
     "Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. \n",

docs/docs/integrations/llms/openai.ipynb

@@ -7,6 +7,7 @@
    "source": [
     "# OpenAI\n",
     "\n",
+
     "[OpenAI](https://platform.openai.com/docs/introduction) offers a spectrum of models with different levels of power suitable for different tasks.\n",
     "\n",
     "This example goes over how to use LangChain to interact with `OpenAI` [models](https://platform.openai.com/docs/models)"

docs/docs/integrations/llms/sambanova.ipynb

@@ -22,7 +22,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "**Sambaverse** allows you to interact with multiple open-source models. You can see the list of available models and interact with them in the [playground](https://sambaverse.sambanova.ai/playground)"
+    "**Sambaverse** allows you to interact with multiple open-source models. You can view the list of available models and interact with them in the [playground](https://sambaverse.sambanova.ai/playground).\n **Please note that Sambaverse's free offering is performance-limited.** Companies that are ready to evaluate the production tokens-per-second performance, volume throughput, and 10x lower total cost of ownership (TCO) of SambaNova should [contact us](https://sambaverse.sambanova.ai/contact-us) for a non-limited evaluation instance."
    ]
   },
   {

docs/docs/integrations/llms/together.ipynb

@@ -7,13 +7,17 @@
    "source": [
     "# Together AI\n",
     "\n",
-    "> The Together API makes it easy to fine-tune or run leading open-source models with a couple lines of code. We have integrated the world’s leading open-source models, including Llama-2, RedPajama, Falcon, Alpaca, Stable Diffusion XL, and more. Read more: https://together.ai\n",
+    "[Together AI](https://www.together.ai/) offers an API to query [50+ leading open-source models](https://docs.together.ai/docs/inference-models) in a couple lines of code.\n",
     "\n",
-    "To use, you'll need an API key which you can find here:\n",
-    "https://api.together.xyz/settings/api-keys. This can be passed in as init param\n",
-    "``together_api_key`` or set as environment variable ``TOGETHER_API_KEY``.\n",
-    "\n",
-    "Together API reference: https://docs.together.ai/reference"
+    "This example goes over how to use LangChain to interact with Together AI models."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1c47fc36",
+   "metadata": {},
+   "source": [
+    "## Installation"
    ]
   },
   {
@@ -23,45 +27,71 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "%pip install --upgrade --quiet langchain-together"
+    "%pip install --upgrade langchain-together"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "89883202",
+   "metadata": {},
+   "source": [
+    "## Environment\n",
+    "\n",
+    "To use Together AI, you'll need an API key which you can find here:\n",
+    "https://api.together.ai/settings/api-keys. This can be passed in as an init param\n",
+    "``together_api_key`` or set as environment variable ``TOGETHER_API_KEY``.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8304b4d9",
+   "metadata": {},
+   "source": [
+    "## Example"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
+   "id": "637bb53f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Querying chat models with Together AI\n",
+    "\n",
+    "from langchain_together import ChatTogether\n",
+    "\n",
+    "# choose from our 50+ models here: https://docs.together.ai/docs/inference-models\n",
+    "chat = ChatTogether(\n",
+    "    # together_api_key=\"YOUR_API_KEY\",\n",
+    "    model=\"meta-llama/Llama-3-70b-chat-hf\",\n",
+    ")\n",
+    "\n",
+    "# stream the response back from the model\n",
+    "for m in chat.stream(\"Tell me fun things to do in NYC\"):\n",
+    "    print(m.content, end=\"\", flush=True)\n",
+    "\n",
+    "# if you don't want to do streaming, you can use the invoke method\n",
+    "# chat.invoke(\"Tell me fun things to do in NYC\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
    "id": "e7b7170d-d7c5-4890-9714-a37238343805",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "A: A large language model is a neural network that is trained on a large amount of text data. It is able to generate text that is similar to the training data, and can be used for tasks such as language translation, question answering, and text summarization.\n",
-      "\n",
-      "A: A large language model is a neural network that is trained on a large amount of text data. It is able to generate text that is similar to the training data, and can be used for tasks such as language translation, question answering, and text summarization.\n",
-      "\n",
-      "A: A large language model is a neural network that is trained on\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
+    "# Querying code and language models with Together AI\n",
+    "\n",
     "from langchain_together import Together\n",
     "\n",
     "llm = Together(\n",
-    "    model=\"togethercomputer/RedPajama-INCITE-7B-Base\",\n",
-    "    temperature=0.7,\n",
-    "    max_tokens=128,\n",
-    "    top_k=1,\n",
+    "    model=\"codellama/CodeLlama-70b-Python-hf\",\n",
     "    # together_api_key=\"...\"\n",
     ")\n",
     "\n",
-    "input_ = \"\"\"You are a teacher with a deep knowledge of machine learning and AI. \\\n",
-    "You provide succinct and accurate answers. Answer the following question: \n",
-    "\n",
-    "What is a large language model?\"\"\"\n",
-    "print(llm.invoke(input_))"
+    "print(llm.invoke(\"def bubble_sort(): \"))"
    ]
   }
  ],

docs/docs/integrations/platforms/index.mdx

@@ -5,6 +5,7 @@ sidebar_class_name: hidden
 
 # Providers
 
+
 :::info
 
 If you'd like to write your own integration, see [Extending LangChain](/docs/guides/development/extending_langchain/).

docs/docs/integrations/providers/dspy.ipynb

@@ -13,7 +13,7 @@
     "\n",
     "This short tutorial demonstrates how this proof-of-concept feature works. *This will not give you the full power of DSPy or LangChain yet, but we will expand it if there's high demand.*\n",
     "\n",
-    "Note: this was slightly modified from the original example Omar wrote for DSPy. If you are interested in LangChain <> DSPy but coming from the DSPy side, I'd recommend checking that out. You can find that [here](https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/compiling_langchain.ipynb).\n",
+    "Note: this was slightly modified from the original example Omar wrote for DSPy. If you are interested in LangChain \\<\\> DSPy but coming from the DSPy side, I'd recommend checking that out. You can find that [here](https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/compiling_langchain.ipynb).\n",
     "\n",
     "Let's take a look at an example. In this example we will make a simple RAG pipeline. We will use DSPy to \"compile\" our program and learn an optimized prompt.\n",
     "\n",
@@ -218,7 +218,7 @@
    "id": "13c293d6-0806-42f5-a4aa-5b50d4cf38d2",
    "metadata": {},
    "source": [
-    "## LCEL <> DSPy\n",
+    "## LCEL \\<\\> DSPy\n",
     "\n",
     "In order to use LangChain with DSPy, you need to make two minor modifications\n",
     "\n",

docs/docs/integrations/providers/langchain_decorators.mdx

@@ -205,7 +205,7 @@ For chat models is very useful to define prompt as a set of message templates...
 def simulate_conversation(human_input:str, agent_role:str="a pirate"):
     """
     ## System message
-     - note the `:system` sufix inside the <prompt:_role_> tag
+     - note the `:system` suffix inside the <prompt:_role_> tag
      
 
     ```<prompt:system>

docs/docs/integrations/providers/nvidia.mdx

@@ -6,7 +6,7 @@
 
 > [NVIDIA AI Foundation Endpoints](https://www.nvidia.com/en-us/ai-data-science/foundation-models/) give users easy access to NVIDIA hosted API endpoints for 
 > NVIDIA AI Foundation Models like `Mixtral 8x7B`, `Llama 2`, `Stable Diffusion`, etc. These models, 
-> hosted on the [NVIDIA NGC catalog](https://catalog.ngc.nvidia.com/ai-foundation-models), are optimized, tested, and hosted on 
+> hosted on the [NVIDIA API catalog](https://build.nvidia.com/), are optimized, tested, and hosted on 
 > the NVIDIA AI platform, making them fast and easy to evaluate, further customize, 
 > and seamlessly run at peak performance on any accelerated stack.
 > 

docs/docs/integrations/providers/oracleai.mdx

@@ -0,0 +1,65 @@
+# OracleAI Vector Search
+Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords. One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system.
+This is not only powerful but also significantly more effective because you dont need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems.
+
+In addition, because Oracle has been building database technologies for so long, your vectors can benefit from all of Oracle Database's most powerful features, like the following:
+
+ * Partitioning Support
+ * Real Application Clusters scalability
+ * Exadata smart scans
+ * Shard processing across geographically distributed databases
+ * Transactions
+ * Parallel SQL
+ * Disaster recovery
+ * Security
+ * Oracle Machine Learning
+ * Oracle Graph Database
+ * Oracle Spatial and Graph
+ * Oracle Blockchain
+ * JSON
+
+
+## Document Loaders
+
+Please check the [usage example](/docs/integrations/document_loaders/oracleai).
+
+```python
+from langchain_community.document_loaders.oracleai import OracleDocLoader
+```
+
+## Text Splitter
+
+Please check the [usage example](/docs/integrations/document_loaders/oracleai).
+
+```python
+from langchain_community.document_loaders.oracleai import OracleTextSplitter
+```
+
+## Embeddings
+
+Please check the [usage example](/docs/integrations/text_embedding/oracleai).
+
+```python
+from langchain_community.embeddings.oracleai import OracleEmbeddings
+```
+
+## Summary
+
+Please check the [usage example](/docs/integrations/tools/oracleai).
+
+```python
+from langchain_community.utilities.oracleai import OracleSummary
+```
+
+## Vector Store
+
+Please check the [usage example](/docs/integrations/vectorstores/oracle).
+
+```python
+from langchain_community.vectorstores.oraclevs import OracleVS
+```
+
+## End to End Demo
+
+Please check the [Oracle AI Vector Search End-to-End Demo Guide](https://github.com/langchain-ai/langchain/blob/master/cookbook/oracleai_demo).
+

docs/docs/integrations/providers/together.ipynb

@@ -2,89 +2,102 @@
  "cells": [
   {
    "cell_type": "markdown",
+   "id": "2970dd75-8ebf-4b51-8282-9b454b8f356d",
    "metadata": {},
    "source": [
     "# Together AI\n",
     "\n",
-    "> [Together AI](https://together.ai) is a cloud platform for building and running generative AI.\n",
-    "> \n",
-    "> It makes it easy to fine-tune or run leading open-source models with a couple lines of code.\n",
-    "> We have integrated the world’s leading open-source models, including `Llama-2`, `RedPajama`, `Falcon`, `Alpaca`, `Stable Diffusion XL`, and more. Read mo\n",
+    "[Together AI](https://www.together.ai/) offers an API to query [50+ leading open-source models](https://docs.together.ai/docs/inference-models) in a couple lines of code.\n",
     "\n",
-    "## Installation and Setup\n",
-    "\n",
-    "To use, you'll need an API key which you can find [here](https://api.together.xyz/settings/api-keys).\n",
-    "\n",
-    "API key can be passed in as init param\n",
-    "``together_api_key`` or set as environment variable ``TOGETHER_API_KEY``.\n",
-    "\n",
-    "See details in the [Together API reference](https://docs.together.ai/reference)\n",
-    "\n",
-    "You will also need to install the `langchain-together` integration package:"
+    "This example goes over how to use LangChain to interact with Together AI models."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1c47fc36",
+   "metadata": {},
+   "source": [
+    "## Installation"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "1ecdb29d",
    "metadata": {},
    "outputs": [],
    "source": [
-    "%pip install --upgrade --quiet langchain-together"
+    "%pip install --upgrade langchain-together"
    ]
   },
   {
    "cell_type": "markdown",
+   "id": "89883202",
    "metadata": {},
    "source": [
-    "## LLMs\n",
+    "## Environment\n",
     "\n",
-    "See a [usage example](/docs/integrations/llms/together)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {
-    "id": "y8ku6X96sebl"
-   },
-   "outputs": [],
-   "source": [
-    "from langchain_together import Together"
+    "To use Together AI, you'll need an API key which you can find here:\n",
+    "https://api.together.ai/settings/api-keys. This can be passed in as an init param\n",
+    "``together_api_key`` or set as environment variable ``TOGETHER_API_KEY``.\n"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2024-04-03T18:49:24.701100Z",
-     "iopub.status.busy": "2024-04-03T18:49:24.700943Z",
-     "iopub.status.idle": "2024-04-03T18:49:24.705570Z",
-     "shell.execute_reply": "2024-04-03T18:49:24.704943Z",
-     "shell.execute_reply.started": "2024-04-03T18:49:24.701088Z"
-    }
-   },
+   "id": "8304b4d9",
+   "metadata": {},
    "source": [
-    "## Embedding models\n",
-    "\n",
-    "See a [usage example](/docs/integrations/text_embedding/together)."
+    "## Example"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
+   "id": "637bb53f",
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain_together.embeddings import TogetherEmbeddings"
+    "# Querying chat models with Together AI\n",
+    "\n",
+    "from langchain_together import ChatTogether\n",
+    "\n",
+    "# choose from our 50+ models here: https://docs.together.ai/docs/inference-models\n",
+    "chat = ChatTogether(\n",
+    "    # together_api_key=\"YOUR_API_KEY\",\n",
+    "    model=\"meta-llama/Llama-3-70b-chat-hf\",\n",
+    ")\n",
+    "\n",
+    "# stream the response back from the model\n",
+    "for m in chat.stream(\"Tell me fun things to do in NYC\"):\n",
+    "    print(m.content, end=\"\", flush=True)\n",
+    "\n",
+    "# if you don't want to do streaming, you can use the invoke method\n",
+    "# chat.invoke(\"Tell me fun things to do in NYC\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e7b7170d-d7c5-4890-9714-a37238343805",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Querying code and language models with Together AI\n",
+    "\n",
+    "from langchain_together import Together\n",
+    "\n",
+    "llm = Together(\n",
+    "    model=\"codellama/CodeLlama-70b-Python-hf\",\n",
+    "    # together_api_key=\"...\"\n",
+    ")\n",
+    "\n",
+    "print(llm.invoke(\"def bubble_sort(): \"))"
    ]
   }
  ],
  "metadata": {
-  "colab": {
-   "provenance": []
-  },
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": ".venv",
    "language": "python",
    "name": "python3"
   },
@@ -98,9 +111,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.11.4"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 4
+ "nbformat_minor": 5
 }

docs/docs/integrations/providers/upstage.ipynb

@@ -37,9 +37,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Get [API Keys](https://console.upstage.ai) and set environment variables `UPSTAGE_API_KEY` and `UPSTAGE_DOCUMENT_AI_API_KEY`.\n",
-    "\n",
-    "> As of April 2024, you need separate API Keys for Solar and Document AI(Layout Analysis). The API Keys will be consolidated soon (hopefully in May) and you'll need just one key for all features."
+    "Get [API Keys](https://console.upstage.ai) and set environment variable `UPSTAGE_API_KEY`."
    ]
   },
   {
@@ -75,8 +73,7 @@
    "source": [
     "import os\n",
     "\n",
-    "os.environ[\"UPSTAGE_API_KEY\"] = \"YOUR_API_KEY\"\n",
-    "os.environ[\"UPSTAGE_DOCUMENT_AI_API_KEY\"] = \"YOUR_DOCUMENT_AI_API_KEY\""
+    "os.environ[\"UPSTAGE_API_KEY\"] = \"YOUR_API_KEY\""
    ]
   },
   {
@@ -131,7 +128,10 @@
   {
    "cell_type": "markdown",
    "metadata": {
-    "collapsed": false
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
    },
    "source": [
     "### Groundedness Check"
@@ -141,7 +141,10 @@
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
-    "collapsed": false
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
    },
    "outputs": [],
    "source": [
@@ -160,7 +163,10 @@
   {
    "cell_type": "markdown",
    "metadata": {
-    "collapsed": false
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    }
    },
    "source": [
     "### Layout Analysis"
@@ -204,9 +210,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.11"
+   "version": "3.10.13"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 1
+ "nbformat_minor": 4
 }

docs/docs/integrations/text_embedding/nvidia_ai_endpoints.ipynb

@@ -85,9 +85,6 @@
     "import getpass\n",
     "import os\n",
     "\n",
-    "## API Key can be found by going to NVIDIA NGC -> AI Foundation Models -> (some model) -> Get API Code or similar.\n",
-    "## 10K free queries to any endpoint (which is a lot actually).\n",
-    "\n",
     "# del os.environ['NVIDIA_API_KEY']  ## delete key and reset\n",
     "if os.environ.get(\"NVIDIA_API_KEY\", \"\").startswith(\"nvapi-\"):\n",
     "    print(\"Valid NVIDIA_API_KEY already in environment. Delete to reset\")\n",
@@ -112,11 +109,7 @@
    "source": [
     "## Initialization\n",
     "\n",
-    "The main requirement when initializing an embedding model is to provide the model name. An example is `nvolveqa_40k` below.\n",
-    "\n",
-    "For `nvovleqa_40k`, you can also specify the `model_type` as `passage` or `query`. When doing retrieval, you will get best results if you embed the source documents with the `passage` type and the user queries with the `query` type.\n",
-    "\n",
-    "If not provided, the `embed_query` method will default to the `query` type, and the `embed_documents` mehod will default to the `passage` type."
+    "When initializing an embedding model you can select a model by passing it, e.g. `ai-embed-qa-4` below, or use the default by not passing any arguments."
    ]
   },
   {
@@ -129,10 +122,7 @@
    "source": [
     "from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings\n",
     "\n",
-    "embedder = NVIDIAEmbeddings(model=\"nvolveqa_40k\")\n",
-    "\n",
-    "# Alternatively, if you want to specify whether it will use the query or passage type\n",
-    "# embedder = NVIDIAEmbeddings(model=\"nvolveqa_40k\", model_type=\"passage\")"
+    "embedder = NVIDIAEmbeddings(model=\"ai-embed-qa-4\")"
    ]
   },
   {
@@ -156,7 +146,7 @@
     "id": "pcDu3v4CbmWk"
    },
    "source": [
-    "### **Similarity/Speed Test**\n",
+    "### **Similarity**\n",
     "\n",
     "The following is a quick test of the methods in terms of usage, format, and speed for the use case of embedding the following data points:\n",
     "\n",
@@ -250,7 +240,7 @@
     "s = time.perf_counter()\n",
     "# To use the \"query\" mode, we have to add it as an instance arg\n",
     "q_embeddings = NVIDIAEmbeddings(\n",
-    "    model=\"nvolveqa_40k\", model_type=\"query\"\n",
+    "    model=\"ai-embed-qa-4\", model_type=\"query\"\n",
     ").embed_documents(\n",
     "    [\n",
     "        \"What's the weather like in Komchatka?\",\n",
@@ -501,7 +491,7 @@
    "source": [
     "vectorstore = FAISS.from_texts(\n",
     "    [\"harrison worked at kensho\"],\n",
-    "    embedding=NVIDIAEmbeddings(model=\"nvolveqa_40k\"),\n",
+    "    embedding=NVIDIAEmbeddings(model=\"ai-embed-qa-4\"),\n",
     ")\n",
     "retriever = vectorstore.as_retriever()\n",
     "\n",
@@ -515,7 +505,7 @@
     "    ]\n",
     ")\n",
     "\n",
-    "model = ChatNVIDIA(model=\"mixtral_8x7b\")\n",
+    "model = ChatNVIDIA(model=\"ai-mixtral-8x7b-instruct\")\n",
     "\n",
     "chain = (\n",
     "    {\"context\": retriever, \"question\": RunnablePassthrough()}\n",

docs/docs/integrations/text_embedding/oracleai.ipynb

@@ -0,0 +1,262 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Oracle AI Vector Search: Generate Embeddings\n",
+    "Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords. One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system. This is not only powerful but also significantly more effective because you don't need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems.\n",
+    "\n",
+    "The guide demonstrates how to use Embedding Capabilities within Oracle AI Vector Search to generate embeddings for your documents using OracleEmbeddings."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Prerequisites\n",
+    "\n",
+    "Please install Oracle Python Client driver to use Langchain with Oracle AI Vector Search. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# pip install oracledb"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Connect to Oracle Database\n",
+    "The following sample code will show how to connect to Oracle Database. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "\n",
+    "import oracledb\n",
+    "\n",
+    "# please update with your username, password, hostname and service_name\n",
+    "username = \"<username>\"\n",
+    "password = \"<password>\"\n",
+    "dsn = \"<hostname>/<service_name>\"\n",
+    "\n",
+    "try:\n",
+    "    conn = oracledb.connect(user=username, password=password, dsn=dsn)\n",
+    "    print(\"Connection successful!\")\n",
+    "except Exception as e:\n",
+    "    print(\"Connection failed!\")\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For embedding, we have a few provider options that the users can choose from such as database, 3rd party providers like ocigenai, huggingface, openai, etc. If the users choose to use 3rd party provider, they need to create a credential with corresponding authentication information. On the other hand, if the users choose to use 'database' as provider, they need to load an onnx model to Oracle Database for embeddings."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Load ONNX Model\n",
+    "\n",
+    "To generate embeddings, Oracle provides a few provider options for users to choose from. The users can choose 'database' provider or some 3rd party providers like OCIGENAI, HuggingFace, etc.\n",
+    "\n",
+    "***Note*** If the users choose database option, they need to load an ONNX model to Oracle Database. The users do not need to load an ONNX model to Oracle Database if they choose to use 3rd party provider to generate embeddings.\n",
+    "\n",
+    "One of the core benefits of using an ONNX model is that the users do not need to transfer their data to 3rd party to generate embeddings. And also, since it does not involve any network or REST API calls, it may provide better performance.\n",
+    "\n",
+    "Here is the sample code to load an ONNX model to Oracle Database:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.embeddings.oracleai import OracleEmbeddings\n",
+    "\n",
+    "# please update with your related information\n",
+    "# make sure that you have onnx file in the system\n",
+    "onnx_dir = \"DEMO_DIR\"\n",
+    "onnx_file = \"tinybert.onnx\"\n",
+    "model_name = \"demo_model\"\n",
+    "\n",
+    "try:\n",
+    "    OracleEmbeddings.load_onnx_model(conn, onnx_dir, onnx_file, model_name)\n",
+    "    print(\"ONNX model loaded.\")\n",
+    "except Exception as e:\n",
+    "    print(\"ONNX model loading failed!\")\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Create Credential\n",
+    "\n",
+    "On the other hand, if the users choose to use 3rd party provider to generate embeddings, they need to create credential to access 3rd party provider's end points.\n",
+    "\n",
+    "***Note:*** The users do not need to create any credential if they choose to use 'database' provider to generate embeddings. Should the users choose to 3rd party provider, they need to create credential for the 3rd party provider they want to use. \n",
+    "\n",
+    "Here is a sample example:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    cursor = conn.cursor()\n",
+    "    cursor.execute(\n",
+    "        \"\"\"\n",
+    "       declare\n",
+    "           jo json_object_t;\n",
+    "       begin\n",
+    "           -- HuggingFace\n",
+    "           dbms_vector_chain.drop_credential(credential_name  => 'HF_CRED');\n",
+    "           jo := json_object_t();\n",
+    "           jo.put('access_token', '<access_token>');\n",
+    "           dbms_vector_chain.create_credential(\n",
+    "               credential_name   =>  'HF_CRED',\n",
+    "               params            => json(jo.to_string));\n",
+    "\n",
+    "           -- OCIGENAI\n",
+    "           dbms_vector_chain.drop_credential(credential_name  => 'OCI_CRED');\n",
+    "           jo := json_object_t();\n",
+    "           jo.put('user_ocid','<user_ocid>');\n",
+    "           jo.put('tenancy_ocid','<tenancy_ocid>');\n",
+    "           jo.put('compartment_ocid','<compartment_ocid>');\n",
+    "           jo.put('private_key','<private_key>');\n",
+    "           jo.put('fingerprint','<fingerprint>');\n",
+    "           dbms_vector_chain.create_credential(\n",
+    "               credential_name   => 'OCI_CRED',\n",
+    "               params            => json(jo.to_string));\n",
+    "       end;\n",
+    "       \"\"\"\n",
+    "    )\n",
+    "    cursor.close()\n",
+    "    print(\"Credentials created.\")\n",
+    "except Exception as ex:\n",
+    "    cursor.close()\n",
+    "    raise"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Generate Embeddings\n",
+    "Oracle AI Vector Search provides a number of ways to generate embeddings. The users can load an ONNX embedding model to Oracle Database and use it to generate embeddings or use some 3rd party API's end points to generate embeddings. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***Note:*** The users may need to set proxy if they want to use some 3rd party embedding generation providers other than 'database' provider (aka using ONNX model)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# proxy to be used when we instantiate summary and embedder object\n",
+    "proxy = \"<proxy>\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The following sample code will show how to generate embeddings:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.embeddings.oracleai import OracleEmbeddings\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "\"\"\"\n",
+    "# using ocigenai\n",
+    "embedder_params = {\n",
+    "    \"provider\": \"ocigenai\",\n",
+    "    \"credential_name\": \"OCI_CRED\",\n",
+    "    \"url\": \"https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/embedText\",\n",
+    "    \"model\": \"cohere.embed-english-light-v3.0\",\n",
+    "}\n",
+    "\n",
+    "# using huggingface\n",
+    "embedder_params = {\n",
+    "    \"provider\": \"huggingface\", \n",
+    "    \"credential_name\": \"HF_CRED\", \n",
+    "    \"url\": \"https://api-inference.huggingface.co/pipeline/feature-extraction/\", \n",
+    "    \"model\": \"sentence-transformers/all-MiniLM-L6-v2\", \n",
+    "    \"wait_for_model\": \"true\"\n",
+    "}\n",
+    "\"\"\"\n",
+    "\n",
+    "# using ONNX model loaded to Oracle Database\n",
+    "embedder_params = {\"provider\": \"database\", \"model\": \"demo_model\"}\n",
+    "\n",
+    "# Remove proxy if not required\n",
+    "embedder = OracleEmbeddings(conn=conn, params=embedder_params, proxy=proxy)\n",
+    "embed = embedder.embed_query(\"Hello World!\")\n",
+    "\n",
+    "\"\"\" verify \"\"\"\n",
+    "print(f\"Embedding generated by OracleEmbeddings: {embed}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### End to End Demo\n",
+    "Please refer to our complete demo guide [Oracle AI Vector Search End-to-End Demo Guide](https://github.com/langchain-ai/langchain/tree/master/cookbook/oracleai_demo.ipynb) to build an end to end RAG pipeline with the help of Oracle AI Vector Search.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/docs/integrations/text_embedding/sambanova.ipynb

@@ -0,0 +1,91 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# SambaNova\n",
+    "\n",
+    "**[SambaNova](https://sambanova.ai/)'s** [Sambastudio](https://sambanova.ai/technology/full-stack-ai-platform) is a platform for running your own open-source models\n",
+    "\n",
+    "This example goes over how to use LangChain to interact with SambaNova embedding models"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## SambaStudio\n",
+    "\n",
+    "**SambaStudio** allows you to train, run batch inference jobs, and deploy online inference endpoints to run open source models that you fine tuned yourself."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "A SambaStudio environment is required to deploy a model. Get more information at [sambanova.ai/products/enterprise-ai-platform-sambanova-suite](https://sambanova.ai/products/enterprise-ai-platform-sambanova-suite)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Register your environment variables:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "sambastudio_base_url = \"<Your SambaStudio environment URL>\"\n",
+    "sambastudio_project_id = \"<Your SambaStudio project id>\"\n",
+    "sambastudio_endpoint_id = \"<Your SambaStudio endpoint id>\"\n",
+    "sambastudio_api_key = \"<Your SambaStudio endpoint API key>\"\n",
+    "\n",
+    "# Set the environment variables\n",
+    "os.environ[\"SAMBASTUDIO_EMBEDDINGS_BASE_URL\"] = sambastudio_base_url\n",
+    "os.environ[\"SAMBASTUDIO_EMBEDDINGS_PROJECT_ID\"] = sambastudio_project_id\n",
+    "os.environ[\"SAMBASTUDIO_EMBEDDINGS_ENDPOINT_ID\"] = sambastudio_endpoint_id\n",
+    "os.environ[\"SAMBASTUDIO_EMBEDDINGS_API_KEY\"] = sambastudio_api_key"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Call SambaStudio hosted embeddings directly from LangChain!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.embeddings.sambanova import SambaStudioEmbeddings\n",
+    "\n",
+    "embeddings = SambaStudioEmbeddings()\n",
+    "\n",
+    "text = \"Hello, this is a test\"\n",
+    "result = embeddings.embed_query(text)\n",
+    "print(result)\n",
+    "\n",
+    "texts = [\"Hello, this is a test\", \"Hello, this is another test\"]\n",
+    "results = embeddings.embed_documents(texts)\n",
+    "print(results)"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/docs/integrations/toolkits/cassandra_database.ipynb

@@ -74,7 +74,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Enviroment Setup\n",
+    "## Environment Setup\n",
     "\n",
     "Install the following Python modules:\n",
     "\n",

docs/docs/integrations/toolkits/pandas.ipynb

@@ -46,7 +46,7 @@
    "source": [
     "## Using `ZERO_SHOT_REACT_DESCRIPTION`\n",
     "\n",
-    "This shows how to initialize the agent using the `ZERO_SHOT_REACT_DESCRIPTION` agent type. Note that this is an alternative to the above."
+    "This shows how to initialize the agent using the `ZERO_SHOT_REACT_DESCRIPTION` agent type."
    ]
   },
   {

docs/docs/integrations/tools/oracleai.ipynb

@@ -0,0 +1,174 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Oracle AI Vector Search: Generate Summary\n",
+    "Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords. One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system. This is not only powerful but also significantly more effective because you don't need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems.\n",
+    "\n",
+    "The guide demonstrates how to use Summary Capabilities within Oracle AI Vector Search to generate summary for your documents using OracleSummary."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Prerequisites\n",
+    "\n",
+    "Please install Oracle Python Client driver to use Langchain with Oracle AI Vector Search. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# pip install oracledb"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Connect to Oracle Database\n",
+    "The following sample code will show how to connect to Oracle Database. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "\n",
+    "import oracledb\n",
+    "\n",
+    "# please update with your username, password, hostname and service_name\n",
+    "username = \"<username>\"\n",
+    "password = \"<password>\"\n",
+    "dsn = \"<hostname>/<service_name>\"\n",
+    "\n",
+    "try:\n",
+    "    conn = oracledb.connect(user=username, password=password, dsn=dsn)\n",
+    "    print(\"Connection successful!\")\n",
+    "except Exception as e:\n",
+    "    print(\"Connection failed!\")\n",
+    "    sys.exit(1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Generate Summary\n",
+    "The Oracle AI Vector Search Langchain library provides APIs to generate summaries of documents. There are a few summary generation provider options including Database, OCIGENAI, HuggingFace and so on. The users can choose their preferred provider to generate a summary. They just need to set the summary parameters accordingly. Please refer to the Oracle AI Vector Search Guide book for complete information about these parameters."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "***Note:*** The users may need to set proxy if they want to use some 3rd party summary generation providers other than Oracle's in-house and default provider: 'database'. If you don't have proxy, please remove the proxy parameter when you instantiate the OracleSummary."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# proxy to be used when we instantiate summary and embedder object\n",
+    "proxy = \"<proxy>\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The following sample code will show how to generate summary:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.utilities.oracleai import OracleSummary\n",
+    "from langchain_core.documents import Document\n",
+    "\n",
+    "\"\"\"\n",
+    "# using 'ocigenai' provider\n",
+    "summary_params = {\n",
+    "    \"provider\": \"ocigenai\",\n",
+    "    \"credential_name\": \"OCI_CRED\",\n",
+    "    \"url\": \"https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/summarizeText\",\n",
+    "    \"model\": \"cohere.command\",\n",
+    "}\n",
+    "\n",
+    "# using 'huggingface' provider\n",
+    "summary_params = {\n",
+    "    \"provider\": \"huggingface\",\n",
+    "    \"credential_name\": \"HF_CRED\",\n",
+    "    \"url\": \"https://api-inference.huggingface.co/models/\",\n",
+    "    \"model\": \"facebook/bart-large-cnn\",\n",
+    "    \"wait_for_model\": \"true\"\n",
+    "}\n",
+    "\"\"\"\n",
+    "\n",
+    "# using 'database' provider\n",
+    "summary_params = {\n",
+    "    \"provider\": \"database\",\n",
+    "    \"glevel\": \"S\",\n",
+    "    \"numParagraphs\": 1,\n",
+    "    \"language\": \"english\",\n",
+    "}\n",
+    "\n",
+    "# get the summary instance\n",
+    "# Remove proxy if not required\n",
+    "summ = OracleSummary(conn=conn, params=summary_params, proxy=proxy)\n",
+    "summary = summ.get_summary(\n",
+    "    \"In the heart of the forest, \"\n",
+    "    + \"a lone fox ventured out at dusk, seeking a lost treasure. \"\n",
+    "    + \"With each step, memories flooded back, guiding its path. \"\n",
+    "    + \"As the moon rose high, illuminating the night, the fox unearthed \"\n",
+    "    + \"not gold, but a forgotten friendship, worth more than any riches.\"\n",
+    ")\n",
+    "\n",
+    "print(f\"Summary generated by OracleSummary: {summary}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### End to End Demo\n",
+    "Please refer to our complete demo guide [Oracle AI Vector Search End-to-End Demo Guide](https://github.com/langchain-ai/langchain/tree/master/cookbook/oracleai_demo.ipynb) to build an end to end RAG pipeline with the help of Oracle AI Vector Search.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/docs/integrations/vectorstores/chroma.ipynb

@@ -7,6 +7,7 @@
    "source": [
     "# Chroma\n",
     "\n",
+
     ">[Chroma](https://docs.trychroma.com/getting-started) is a AI-native open-source vector database focused on developer productivity and happiness. Chroma is licensed under Apache 2.0.\n",
     "\n",
     "\n",

docs/docs/integrations/vectorstores/faiss.ipynb

@@ -7,6 +7,7 @@
    "source": [
     "# Faiss\n",
     "\n",
+
     ">[Facebook AI Similarity Search (Faiss)](https://engineering.fb.com/2017/03/29/data-infrastructure/faiss-a-library-for-efficient-similarity-search/) is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM. It also contains supporting code for evaluation and parameter tuning.\n",
     "\n",
     "[Faiss documentation](https://faiss.ai/).\n",
@@ -83,7 +84,7 @@
    "metadata": {
     "tags": []
    },
-      "outputs": [
+   "outputs": [
     {
      "data": {
       "text/plain": [

docs/docs/integrations/vectorstores/oracle.ipynb

@@ -0,0 +1,469 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "dd33e9d5-9dba-4aac-9f7f-4cf9e6686593",
+   "metadata": {},
+   "source": [
+    "# Oracle AI Vector Search: Vector Store\n",
+    "\n",
+    "Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords.\n",
+    "One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system.\n",
+    "This is not only powerful but also significantly more effective because you dont need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems.\n",
+    "\n",
+    "In addition, because Oracle has been building database technologies for so long, your vectors can benefit from all of Oracle Database's most powerful features, like the following:\n",
+    "\n",
+    " * Partitioning Support\n",
+    " * Real Application Clusters scalability\n",
+    " * Exadata smart scans\n",
+    " * Shard processing across geographically distributed databases\n",
+    " * Transactions\n",
+    " * Parallel SQL\n",
+    " * Disaster recovery\n",
+    " * Security\n",
+    " * Oracle Machine Learning\n",
+    " * Oracle Graph Database\n",
+    " * Oracle Spatial and Graph\n",
+    " * Oracle Blockchain\n",
+    " * JSON"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7bd80054-c803-47e1-a259-c40ed073c37d",
+   "metadata": {},
+   "source": [
+    "### Prerequisites for using Langchain with Oracle AI Vector Search\n",
+    "\n",
+    "Please install Oracle Python Client driver to use Langchain with Oracle AI Vector Search. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2bbb989d-c6fb-4ab9-bafd-a95fd48538d0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# pip install oracledb"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0fceaa5a-95da-4ebd-8b8d-5e73bb653172",
+   "metadata": {},
+   "source": [
+    "### Connect to Oracle AI Vector Search"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4421e4b7-2c7e-4bcd-82b3-9576595edd0f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import oracledb\n",
+    "\n",
+    "username = \"username\"\n",
+    "password = \"password\"\n",
+    "dsn = \"ipaddress:port/orclpdb1\"\n",
+    "\n",
+    "try:\n",
+    "    connection = oracledb.connect(user=username, password=password, dsn=dsn)\n",
+    "    print(\"Connection successful!\")\n",
+    "except Exception as e:\n",
+    "    print(\"Connection failed!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b11cf362-01b0-485d-8527-31b0fbb5028e",
+   "metadata": {},
+   "source": [
+    "### Import the required dependencies to play with Oracle AI Vector Search"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "43ea59e3-2910-45a6-b195-5f06094bb7c9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.embeddings import HuggingFaceEmbeddings\n",
+    "from langchain_community.vectorstores import oraclevs\n",
+    "from langchain_community.vectorstores.oraclevs import OracleVS\n",
+    "from langchain_community.vectorstores.utils import DistanceStrategy\n",
+    "from langchain_core.documents import Document"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0aac10dc-a9cc-4fdb-901c-1b7a4bbbe5a7",
+   "metadata": {},
+   "source": [
+    "### Load Documents"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "70ac6982-b13a-4e8c-9c47-57c6d136ac60",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Define a list of documents (These dummy examples are 5 random documents from Oracle Concepts Manual )\n",
+    "\n",
+    "documents_json_list = [\n",
+    "    {\n",
+    "        \"id\": \"cncpt_15.5.3.2.2_P4\",\n",
+    "        \"text\": \"If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.\",\n",
+    "        \"link\": \"https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/logical-storage-structures.html#GUID-5387D7B2-C0CA-4C1E-811B-C7EB9B636442\",\n",
+    "    },\n",
+    "    {\n",
+    "        \"id\": \"cncpt_15.5.5_P1\",\n",
+    "        \"text\": \"A tablespace can be online (accessible) or offline (not accessible) whenever the database is open.\\nA tablespace is usually online so that its data is available to users. The SYSTEM tablespace and temporary tablespaces cannot be taken offline.\",\n",
+    "        \"link\": \"https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/logical-storage-structures.html#GUID-D02B2220-E6F5-40D9-AFB5-BC69BCEF6CD4\",\n",
+    "    },\n",
+    "    {\n",
+    "        \"id\": \"cncpt_22.3.4.3.1_P2\",\n",
+    "        \"text\": \"The database stores LOBs differently from other data types. Creating a LOB column implicitly creates a LOB segment and a LOB index. The tablespace containing the LOB segment and LOB index, which are always stored together, may be different from the tablespace containing the table.\\nSometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.\",\n",
+    "        \"link\": \"https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/concepts-for-database-developers.html#GUID-3C50EAB8-FC39-4BB3-B680-4EACCE49E866\",\n",
+    "    },\n",
+    "    {\n",
+    "        \"id\": \"cncpt_22.3.4.3.1_P3\",\n",
+    "        \"text\": \"The LOB segment stores data in pieces called chunks. A chunk is a logically contiguous set of data blocks and is the smallest unit of allocation for a LOB. A row in the table stores a pointer called a LOB locator, which points to the LOB index. When the table is queried, the database uses the LOB index to quickly locate the LOB chunks.\",\n",
+    "        \"link\": \"https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/concepts-for-database-developers.html#GUID-3C50EAB8-FC39-4BB3-B680-4EACCE49E866\",\n",
+    "    },\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "eaa942d6-5954-4898-8c32-3627b923a3a5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create Langchain Documents\n",
+    "\n",
+    "documents_langchain = []\n",
+    "\n",
+    "for doc in documents_json_list:\n",
+    "    metadata = {\"id\": doc[\"id\"], \"link\": doc[\"link\"]}\n",
+    "    doc_langchain = Document(page_content=doc[\"text\"], metadata=metadata)\n",
+    "    documents_langchain.append(doc_langchain)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6823f5e6-997c-4f15-927b-bd44c61f105f",
+   "metadata": {},
+   "source": [
+    "### Using AI Vector Search Create a bunch of Vector Stores with different distance strategies\n",
+    "\n",
+    "First we will create three vector stores each with different distance functions. Since we have not created indices in them yet, they will just create tables for now. Later we will use these vector stores to create HNSW indicies.\n",
+    "\n",
+    "You can manually connect to the Oracle Database and will see three tables \n",
+    "Documents_DOT, Documents_COSINE and Documents_EUCLIDEAN. \n",
+    "\n",
+    "We will then create three additional tables Documents_DOT_IVF, Documents_COSINE_IVF and Documents_EUCLIDEAN_IVF which will be used\n",
+    "to create IVF indicies on the tables instead of HNSW indices. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ed1b253e-5f5c-4a81-983c-74645213a170",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Ingest documents into Oracle Vector Store using different distance strategies\n",
+    "\n",
+    "model = HuggingFaceEmbeddings(model_name=\"sentence-transformers/all-mpnet-base-v2\")\n",
+    "\n",
+    "vector_store_dot = OracleVS.from_documents(\n",
+    "    documents_langchain,\n",
+    "    model,\n",
+    "    client=connection,\n",
+    "    table_name=\"Documents_DOT\",\n",
+    "    distance_strategy=DistanceStrategy.DOT_PRODUCT,\n",
+    ")\n",
+    "vector_store_max = OracleVS.from_documents(\n",
+    "    documents_langchain,\n",
+    "    model,\n",
+    "    client=connection,\n",
+    "    table_name=\"Documents_COSINE\",\n",
+    "    distance_strategy=DistanceStrategy.COSINE,\n",
+    ")\n",
+    "vector_store_euclidean = OracleVS.from_documents(\n",
+    "    documents_langchain,\n",
+    "    model,\n",
+    "    client=connection,\n",
+    "    table_name=\"Documents_EUCLIDEAN\",\n",
+    "    distance_strategy=DistanceStrategy.EUCLIDEAN_DISTANCE,\n",
+    ")\n",
+    "\n",
+    "# Ingest documents into Oracle Vector Store using different distance strategies\n",
+    "vector_store_dot_ivf = OracleVS.from_documents(\n",
+    "    documents_langchain,\n",
+    "    model,\n",
+    "    client=connection,\n",
+    "    table_name=\"Documents_DOT_IVF\",\n",
+    "    distance_strategy=DistanceStrategy.DOT_PRODUCT,\n",
+    ")\n",
+    "vector_store_max_ivf = OracleVS.from_documents(\n",
+    "    documents_langchain,\n",
+    "    model,\n",
+    "    client=connection,\n",
+    "    table_name=\"Documents_COSINE_IVF\",\n",
+    "    distance_strategy=DistanceStrategy.COSINE,\n",
+    ")\n",
+    "vector_store_euclidean_ivf = OracleVS.from_documents(\n",
+    "    documents_langchain,\n",
+    "    model,\n",
+    "    client=connection,\n",
+    "    table_name=\"Documents_EUCLIDEAN_IVF\",\n",
+    "    distance_strategy=DistanceStrategy.EUCLIDEAN_DISTANCE,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "77c29505-8688-4b87-9a99-e648fbb2d425",
+   "metadata": {},
+   "source": [
+    "### Demonstrating add, delete operations for texts, and basic similarity search\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "306563ae-577b-4bc7-8a92-3dd6a59310f5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def manage_texts(vector_stores):\n",
+    "    \"\"\"\n",
+    "    Adds texts to each vector store, demonstrates error handling for duplicate additions,\n",
+    "    and performs deletion of texts. Showcases similarity searches and index creation for each vector store.\n",
+    "\n",
+    "    Args:\n",
+    "    - vector_stores (list): A list of OracleVS instances.\n",
+    "    \"\"\"\n",
+    "    texts = [\"Rohan\", \"Shailendra\"]\n",
+    "    metadata = [\n",
+    "        {\"id\": \"100\", \"link\": \"Document Example Test 1\"},\n",
+    "        {\"id\": \"101\", \"link\": \"Document Example Test 2\"},\n",
+    "    ]\n",
+    "\n",
+    "    for i, vs in enumerate(vector_stores, start=1):\n",
+    "        # Adding texts\n",
+    "        try:\n",
+    "            vs.add_texts(texts, metadata)\n",
+    "            print(f\"\\n\\n\\nAdd texts complete for vector store {i}\\n\\n\\n\")\n",
+    "        except Exception as ex:\n",
+    "            print(f\"\\n\\n\\nExpected error on duplicate add for vector store {i}\\n\\n\\n\")\n",
+    "\n",
+    "        # Deleting texts using the value of 'id'\n",
+    "        vs.delete([metadata[0][\"id\"]])\n",
+    "        print(f\"\\n\\n\\nDelete texts complete for vector store {i}\\n\\n\\n\")\n",
+    "\n",
+    "        # Similarity search\n",
+    "        results = vs.similarity_search(\"How are LOBS stored in Oracle Database\", 2)\n",
+    "        print(f\"\\n\\n\\nSimilarity search results for vector store {i}: {results}\\n\\n\\n\")\n",
+    "\n",
+    "\n",
+    "vector_store_list = [\n",
+    "    vector_store_dot,\n",
+    "    vector_store_max,\n",
+    "    vector_store_euclidean,\n",
+    "    vector_store_dot_ivf,\n",
+    "    vector_store_max_ivf,\n",
+    "    vector_store_euclidean_ivf,\n",
+    "]\n",
+    "manage_texts(vector_store_list)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0980cb33-69cf-4547-842a-afdc4d6fa7d3",
+   "metadata": {},
+   "source": [
+    "### Demonstrating index creation with specific parameters for each distance strategy\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "46298a27-e309-456e-b2b8-771d9cb3be29",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def create_search_indices(connection):\n",
+    "    \"\"\"\n",
+    "    Creates search indices for the vector stores, each with specific parameters tailored to their distance strategy.\n",
+    "    \"\"\"\n",
+    "    # Index for DOT_PRODUCT strategy\n",
+    "    # Notice we are creating a HNSW index with default parameters\n",
+    "    # This will default to creating a HNSW index with 8 Parallel Workers and use the Default Accuracy used by Oracle AI Vector Search\n",
+    "    oraclevs.create_index(\n",
+    "        connection,\n",
+    "        vector_store_dot,\n",
+    "        params={\"idx_name\": \"hnsw_idx1\", \"idx_type\": \"HNSW\"},\n",
+    "    )\n",
+    "\n",
+    "    # Index for COSINE strategy with specific parameters\n",
+    "    # Notice we are creating a HNSW index with parallel 16 and Target Accuracy Specification as 97 percent\n",
+    "    oraclevs.create_index(\n",
+    "        connection,\n",
+    "        vector_store_max,\n",
+    "        params={\n",
+    "            \"idx_name\": \"hnsw_idx2\",\n",
+    "            \"idx_type\": \"HNSW\",\n",
+    "            \"accuracy\": 97,\n",
+    "            \"parallel\": 16,\n",
+    "        },\n",
+    "    )\n",
+    "\n",
+    "    # Index for EUCLIDEAN_DISTANCE strategy with specific parameters\n",
+    "    # Notice we are creating a HNSW index by specifying Power User Parameters which are neighbors = 64 and efConstruction = 100\n",
+    "    oraclevs.create_index(\n",
+    "        connection,\n",
+    "        vector_store_euclidean,\n",
+    "        params={\n",
+    "            \"idx_name\": \"hnsw_idx3\",\n",
+    "            \"idx_type\": \"HNSW\",\n",
+    "            \"neighbors\": 64,\n",
+    "            \"efConstruction\": 100,\n",
+    "        },\n",
+    "    )\n",
+    "\n",
+    "    # Index for DOT_PRODUCT strategy with specific parameters\n",
+    "    # Notice we are creating an IVF index with default parameters\n",
+    "    # This will default to creating an IVF index with 8 Parallel Workers and use the Default Accuracy used by Oracle AI Vector Search\n",
+    "    oraclevs.create_index(\n",
+    "        connection,\n",
+    "        vector_store_dot_ivf,\n",
+    "        params={\n",
+    "            \"idx_name\": \"ivf_idx1\",\n",
+    "            \"idx_type\": \"IVF\",\n",
+    "        },\n",
+    "    )\n",
+    "\n",
+    "    # Index for COSINE strategy with specific parameters\n",
+    "    # Notice we are creating an IVF index with parallel 32 and Target Accuracy Specification as 90 percent\n",
+    "    oraclevs.create_index(\n",
+    "        connection,\n",
+    "        vector_store_max_ivf,\n",
+    "        params={\n",
+    "            \"idx_name\": \"ivf_idx2\",\n",
+    "            \"idx_type\": \"IVF\",\n",
+    "            \"accuracy\": 90,\n",
+    "            \"parallel\": 32,\n",
+    "        },\n",
+    "    )\n",
+    "\n",
+    "    # Index for EUCLIDEAN_DISTANCE strategy with specific parameters\n",
+    "    # Notice we are creating an IVF index by specifying Power User Parameters which is neighbor_part = 64\n",
+    "    oraclevs.create_index(\n",
+    "        connection,\n",
+    "        vector_store_euclidean_ivf,\n",
+    "        params={\"idx_name\": \"ivf_idx3\", \"idx_type\": \"IVF\", \"neighbor_part\": 64},\n",
+    "    )\n",
+    "\n",
+    "    print(\"Index creation complete.\")\n",
+    "\n",
+    "\n",
+    "create_search_indices(connection)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7223d048-5c0b-4e91-a91b-a7daa9f86758",
+   "metadata": {},
+   "source": [
+    "### Now we will conduct a bunch of advanced searches on all six vector stores. Each of these three searches have a with and without filter version. The filter only selects the document with id 101 out and filters out everything else"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "37ca2e7d-9803-4260-95e7-62776d4fb820",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Conduct advanced searches after creating the indices\n",
+    "def conduct_advanced_searches(vector_stores):\n",
+    "    query = \"How are LOBS stored in Oracle Database\"\n",
+    "    # Constructing a filter for direct comparison against document metadata\n",
+    "    # This filter aims to include documents whose metadata 'id' is exactly '2'\n",
+    "    filter_criteria = {\"id\": [\"101\"]}  # Direct comparison filter\n",
+    "\n",
+    "    for i, vs in enumerate(vector_stores, start=1):\n",
+    "        print(f\"\\n--- Vector Store {i} Advanced Searches ---\")\n",
+    "        # Similarity search without a filter\n",
+    "        print(\"\\nSimilarity search results without filter:\")\n",
+    "        print(vs.similarity_search(query, 2))\n",
+    "\n",
+    "        # Similarity search with a filter\n",
+    "        print(\"\\nSimilarity search results with filter:\")\n",
+    "        print(vs.similarity_search(query, 2, filter=filter_criteria))\n",
+    "\n",
+    "        # Similarity search with relevance score\n",
+    "        print(\"\\nSimilarity search with relevance score:\")\n",
+    "        print(vs.similarity_search_with_score(query, 2))\n",
+    "\n",
+    "        # Similarity search with relevance score with filter\n",
+    "        print(\"\\nSimilarity search with relevance score with filter:\")\n",
+    "        print(vs.similarity_search_with_score(query, 2, filter=filter_criteria))\n",
+    "\n",
+    "        # Max marginal relevance search\n",
+    "        print(\"\\nMax marginal relevance search results:\")\n",
+    "        print(vs.max_marginal_relevance_search(query, 2, fetch_k=20, lambda_mult=0.5))\n",
+    "\n",
+    "        # Max marginal relevance search with filter\n",
+    "        print(\"\\nMax marginal relevance search results with filter:\")\n",
+    "        print(\n",
+    "            vs.max_marginal_relevance_search(\n",
+    "                query, 2, fetch_k=20, lambda_mult=0.5, filter=filter_criteria\n",
+    "            )\n",
+    "        )\n",
+    "\n",
+    "\n",
+    "conduct_advanced_searches(vector_store_list)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0da8c7e2-0db0-4363-b31b-a7a5e3f83717",
+   "metadata": {},
+   "source": [
+    "### End to End Demo\n",
+    "Please refer to our complete demo guide [Oracle AI Vector Search End-to-End Demo Guide](https://github.com/langchain-ai/langchain/tree/master/cookbook/oracleai_demo.ipynb) to build an end to end RAG pipeline with the help of Oracle AI Vector Search.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/docs/integrations/vectorstores/yellowbrick.ipynb

@@ -98,7 +98,7 @@
     "import psycopg2\n",
     "from IPython.display import Markdown, display\n",
     "from langchain.chains import LLMChain, RetrievalQAWithSourcesChain\n",
-    "from langchain_community.docstore.document import Document\n",
+    "from langchain.schema import Document\n",
     "from langchain_community.vectorstores import Yellowbrick\n",
     "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n",
     "from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
@@ -209,14 +209,12 @@
     "\n",
     "# Define the SQL statement to create a table\n",
     "create_table_query = f\"\"\"\n",
-    "CREATE TABLE if not exists {embedding_table} (\n",
-    "    id uuid,\n",
-    "    embedding_id integer,\n",
-    "    text character varying(60000),\n",
-    "    metadata character varying(1024),\n",
-    "    embedding double precision\n",
+    "CREATE TABLE IF NOT EXISTS {embedding_table} (\n",
+    "    doc_id uuid NOT NULL,\n",
+    "    embedding_id smallint NOT NULL,\n",
+    "    embedding double precision NOT NULL\n",
     ")\n",
-    "DISTRIBUTE ON (id);\n",
+    "DISTRIBUTE ON (doc_id);\n",
     "truncate table {embedding_table};\n",
     "\"\"\"\n",
     "\n",
@@ -257,6 +255,8 @@
     "    f\"postgres://{urlparse.quote(YBUSER)}:{YBPASSWORD}@{YBHOST}:5432/{YB_DOC_DATABASE}\"\n",
     ")\n",
     "\n",
+    "print(yellowbrick_doc_connection_string)\n",
+    "\n",
     "# Establish a connection to the Yellowbrick database\n",
     "conn = psycopg2.connect(yellowbrick_doc_connection_string)\n",
     "\n",
@@ -324,7 +324,7 @@
     "vector_store = Yellowbrick.from_documents(\n",
     "    documents=split_docs,\n",
     "    embedding=embeddings,\n",
-    "    connection_string=yellowbrick_connection_string,\n",
+    "    connection_info=yellowbrick_connection_string,\n",
     "    table=embedding_table,\n",
     ")\n",
     "\n",
@@ -403,6 +403,88 @@
     "print_result_sources(\"Whats an easy way to add users in bulk to Yellowbrick?\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "1f39fd30",
+   "metadata": {},
+   "source": [
+    "## Part 6: Introducing an Index to Increase Performance\n",
+    "\n",
+    "Yellowbrick also supports indexing using the Locality-Sensitive Hashing approach. This is an approximate nearest-neighbor search technique, and allows one to trade off similarity search time at the expense of accuracy. The index introduces two new tunable parameters:\n",
+    "\n",
+    "- The number of hyperplanes, which is provided as an argument to `create_lsh_index(num_hyperplanes)`. The more documents, the more hyperplanes are needed. LSH is a form of dimensionality reduction. The original embeddings are transformed into lower dimensional vectors where the number of components is the same as the number of hyperplanes.\n",
+    "- The Hamming distance, an integer representing the breadth of the search. Smaller Hamming distances result in faster retreival but lower accuracy.\n",
+    "\n",
+    "Here's how you can create an index on the embeddings we loaded into Yellowbrick. We'll also re-run the previous chat session, but this time the retrieval will use the index. Note that for such a small number of documents, you won't see the benefit of indexing in terms of performance."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "02ba61c4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "system_template = \"\"\"Use the following pieces of context to answer the users question.\n",
+    "Take note of the sources and include them in the answer in the format: \"SOURCES: source1 source2\", use \"SOURCES\" in capital letters regardless of the number of sources.\n",
+    "If you don't know the answer, just say that \"I don't know\", don't try to make up an answer.\n",
+    "----------------\n",
+    "{summaries}\"\"\"\n",
+    "messages = [\n",
+    "    SystemMessagePromptTemplate.from_template(system_template),\n",
+    "    HumanMessagePromptTemplate.from_template(\"{question}\"),\n",
+    "]\n",
+    "prompt = ChatPromptTemplate.from_messages(messages)\n",
+    "\n",
+    "vector_store = Yellowbrick(\n",
+    "    OpenAIEmbeddings(),\n",
+    "    yellowbrick_connection_string,\n",
+    "    embedding_table,  # Change the table name to reflect your embeddings\n",
+    ")\n",
+    "\n",
+    "lsh_params = Yellowbrick.IndexParams(\n",
+    "    Yellowbrick.IndexType.LSH, {\"num_hyperplanes\": 8, \"hamming_distance\": 2}\n",
+    ")\n",
+    "vector_store.create_index(lsh_params)\n",
+    "\n",
+    "chain_type_kwargs = {\"prompt\": prompt}\n",
+    "llm = ChatOpenAI(\n",
+    "    model_name=\"gpt-3.5-turbo\",  # Modify model_name if you have access to GPT-4\n",
+    "    temperature=0,\n",
+    "    max_tokens=256,\n",
+    ")\n",
+    "chain = RetrievalQAWithSourcesChain.from_chain_type(\n",
+    "    llm=llm,\n",
+    "    chain_type=\"stuff\",\n",
+    "    retriever=vector_store.as_retriever(\n",
+    "        k=5, search_kwargs={\"index_params\": lsh_params}\n",
+    "    ),\n",
+    "    return_source_documents=True,\n",
+    "    chain_type_kwargs=chain_type_kwargs,\n",
+    ")\n",
+    "\n",
+    "\n",
+    "def print_result_sources(query):\n",
+    "    result = chain(query)\n",
+    "    output_text = f\"\"\"### Question: \n",
+    "  {query}\n",
+    "  ### Answer: \n",
+    "  {result['answer']}\n",
+    "  ### Sources: \n",
+    "  {result['sources']}\n",
+    "  ### All relevant sources:\n",
+    "  {', '.join(list(set([doc.metadata['source'] for doc in result['source_documents']])))}\n",
+    "    \"\"\"\n",
+    "    display(Markdown(output_text))\n",
+    "\n",
+    "\n",
+    "# Use the chain to query\n",
+    "\n",
+    "print_result_sources(\"How many databases can be in a Yellowbrick Instance?\")\n",
+    "\n",
+    "print_result_sources(\"Whats an easy way to add users in bulk to Yellowbrick?\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "697c8a38",
@@ -418,9 +500,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "langchain_venv",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "langchain_venv"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {

docs/docs/modules/agents/agent_types/index.mdx

@@ -5,6 +5,7 @@ title: Types
 
 # Agent Types
 
+
 This categorizes all the available agents along a few dimensions.
 
 **Intended Model Type**

docs/docs/modules/agents/agent_types/react.ipynb

@@ -17,6 +17,7 @@
    "source": [
     "# ReAct\n",
     "\n",
+
     "This walkthrough showcases using an agent to implement the [ReAct](https://react-lm.github.io/) logic."
    ]
   },

docs/docs/modules/agents/agent_types/tool_calling.ipynb

@@ -16,6 +16,7 @@
    "source": [
     "# Tool calling agent\n",
     "\n",
+
     "[Tool calling](/docs/modules/model_io/chat/function_calling) allows a model to detect when one or more tools should be called and respond with the inputs that should be passed to those tools. In an API call, you can describe tools and have the model intelligently choose to output a structured object like JSON containing arguments to call these tools. The goal of tools APIs is to more reliably return valid and useful tool calls than what can be done using a generic text completion or chat API.\n",
     "\n",
     "We can take advantage of this structured output, combined with the fact that you can bind multiple tools to a [tool calling chat model](/docs/integrations/chat/) and\n",

docs/docs/modules/agents/agent_types/xml_agent.ipynb

@@ -208,7 +208,7 @@
    "source": [
     "# Custom XML Agents\n",
     "\n",
-    "**Note:** For greater customizability, we recommend checking out [LangGraph](/docs/langgraph).\n",
+    "**Note:** For greater customizability, we recommend checking out [LangGraph](https://langchain-ai.github.io/langgraph/).\n",
     "\n",
     "Here we provide an example of a custom XML Agent implementation, to give a sense for what `create_xml_agent` is doing under the hood."
    ]
@@ -365,7 +365,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.4"
+   "version": "3.10.5"
   }
  },
  "nbformat": 4,

docs/docs/modules/agents/how_to/custom_agent.ipynb

@@ -17,6 +17,7 @@
    "source": [
     "# Custom agent\n",
     "\n",
+
     "This notebook goes through how to create your own custom agent.\n",
     "\n",
     "In this example, we will use OpenAI Tool Calling to create this agent.\n",

docs/docs/modules/agents/index.ipynb

@@ -17,6 +17,7 @@
    "id": "f4c03f40-1328-412d-8a48-1db0cd481b77",
    "metadata": {},
    "source": [
+
     "The core idea of agents is to use a language model to choose a sequence of actions to take.\n",
     "In chains, a sequence of actions is hardcoded (in code).\n",
     "In agents, a language model is used as a reasoning engine to determine which actions to take and in which order.\n",
@@ -44,7 +45,7 @@
     "Agents have a lot of related functionality! Check out comprehensive guides including:\n",
     "\n",
     "- [Building a custom agent](/docs/modules/agents/how_to/custom_agent)\n",
-    "- [Streaming (of both intermediate steps and tokens](/docs/modules/agents/how_to/streaming)\n",
+    "- [Streaming (of both intermediate steps and tokens)](/docs/modules/agents/how_to/streaming)\n",
     "- [Building an agent that returns structured output](/docs/modules/agents/how_to/agent_structured)\n",
     "- Lots functionality around using AgentExecutor, including: [using it as an iterator](/docs/modules/agents/how_to/agent_iter), [handle parsing errors](/docs/modules/agents/how_to/handle_parsing_errors), [returning intermediate steps](/docs/modules/agents/how_to/intermediate_steps), [capping the max number of iterations](/docs/modules/agents/how_to/max_iterations), and [timeouts for agents](/docs/modules/agents/how_to/max_time_limit)"
    ]

docs/docs/modules/agents/quick_start.ipynb

@@ -18,6 +18,7 @@
    "source": [
     "# Quickstart\n",
     "\n",
+
     "To best understand the agent framework, let's build an agent that has two tools: one to look things up online, and one to look up specific data that we've loaded into a index.\n",
     "\n",
     "This will assume knowledge of [LLMs](/docs/modules/model_io/) and [retrieval](/docs/modules/data_connection/) so if you haven't already explored those sections, it is recommended you do so.\n",
@@ -279,7 +280,7 @@
    "id": "f8014c9d",
    "metadata": {},
    "source": [
-    "Now, we can initalize the agent with the LLM, the prompt, and the tools. The agent is responsible for taking in input and deciding what actions to take. Crucially, the Agent does not execute those actions - that is done by the AgentExecutor (next step). For more information about how to think about these components, see our [conceptual guide](/docs/modules/agents/concepts/)."
+    "Now, we can initialize the agent with the LLM, the prompt, and the tools. The agent is responsible for taking in input and deciding what actions to take. Crucially, the Agent does not execute those actions - that is done by the AgentExecutor (next step). For more information about how to think about these components, see our [conceptual guide](/docs/modules/agents/concepts/)."
    ]
   },
   {
@@ -705,7 +706,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.5"
   }
  },
  "nbformat": 4,

docs/docs/modules/chains.ipynb

@@ -18,20 +18,18 @@
    "id": "b872d874-ad6e-49b5-9435-66063a64d1a8",
    "metadata": {},
    "source": [
+
     "Chains refer to sequences of calls - whether to an LLM, a tool, or a data preprocessing step. The primary supported way to do this is with [LCEL](/docs/expression_language). \n",
     "\n",
-    "LCEL is great for constructing your own chains, but it's also nice to have chains that you can use off-the-shelf. There are two types of off-the-shelf chains that LangChain supports:\n",
+    "LCEL is great for constructing your chains, but it's also nice to have chains used off the shelf. There are two types of off-the-shelf chains that LangChain supports:\n",
     "\n",
     "- Chains that are built with LCEL. In this case, LangChain offers a higher-level constructor method. However, all that is being done under the hood is constructing a chain with LCEL. \n",
+    "- [Legacy] Chains constructed by subclassing from a legacy `Chain` class. These chains do not use LCEL under the hood but are the standalone classes.\n",
     "\n",
-    "- [Legacy] Chains constructed by subclassing from a legacy `Chain` class. These chains do not use LCEL under the hood but are rather standalone classes.\n",
-    "\n",
-    "We are working creating methods that create LCEL versions of all chains. We are doing this for a few reasons.\n",
+    "We are working on creating methods that create LCEL versions of all chains. We are doing this for a few reasons.\n",
     "\n",
     "1. Chains constructed in this way are nice because if you want to modify the internals of a chain you can simply modify the LCEL.\n",
-    "\n",
     "2. These chains natively support streaming, async, and batch out of the box.\n",
-    "\n",
     "3. These chains automatically get observability at each step.\n",
     "\n",
     "This page contains two lists. First, a list of all LCEL chain constructors. Second, a list of all legacy Chains."
@@ -44,32 +42,22 @@
    "source": [
     "## LCEL Chains\n",
     "\n",
-    "Below is a table of all LCEL chain constructors. In addition, we report on:\n",
+    "Below is a table of all `LCEL chain constructors`. \n",
     "\n",
-    "**Chain Constructor**\n",
-    "\n",
-    "The constructor function for this chain. These are all methods that return LCEL runnables. We also link to the API documentation.\n",
-    "\n",
-    "**Function Calling**\n",
-    "\n",
-    "Whether this requires OpenAI function calling.\n",
-    "\n",
-    "**Other Tools**\n",
-    "\n",
-    "What other tools (if any) are used in this chain.\n",
-    "\n",
-    "**When to Use**\n",
-    "\n",
-    "Our commentary on when to use this chain.\n",
+    "Table columns:\n",
     "\n",
+    "- **Chain Constructor:** The constructor function for this chain. These are all methods that return LCEL Runnables. We also link to the API documentation.\n",
+    "- **Function Calling:** Whether this requires OpenAI function calling.\n",
+    "- **Other Tools:** Other tools (if any) used in this chain.\n",
+    "- **When to Use:** Our commentary on when to use this chain.\n",
     "\n",
     "\n",
     "| Chain Constructor                | Function Calling      | Other Tools  | When to Use                                                                    |\n",
     "|----------------------------------|-------------------------|--------------|--------------------------------------------------------------------------------|\n",
-    "| [create_stuff_documents_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.stuff.create_stuff_documents_chain.html#langchain.chains.combine_documents.stuff.create_stuff_documents_chain)     |                         |              | This chain takes a list of documents and formats them all into a prompt, then passes that prompt to an LLM. It passes ALL documents, so you should make sure it fits within the context window the LLM you are using. |\n",
-    "| [create_openai_fn_runnable](https://api.python.langchain.com/en/latest/chains/langchain.chains.structured_output.base.create_openai_fn_runnable.html#langchain.chains.structured_output.base.create_openai_fn_runnable)     | ✅ |              | If you want to use OpenAI function calling to OPTIONALLY structured an output response. You may pass in multiple functions for it call, but it does not have to call it.                                     |\n",
+    "| [create_stuff_documents_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.stuff.create_stuff_documents_chain.html#langchain.chains.combine_documents.stuff.create_stuff_documents_chain)     |                         |              | This chain takes a list of documents and formats them all into a prompt, then passes that prompt to an LLM. It passes ALL documents, so you should make sure it fits within the context window of the LLM you are using. |\n",
+    "| [create_openai_fn_runnable](https://api.python.langchain.com/en/latest/chains/langchain.chains.structured_output.base.create_openai_fn_runnable.html#langchain.chains.structured_output.base.create_openai_fn_runnable)     | ✅ |              | If you want to use OpenAI function calling to OPTIONALLY structured an output response. You may pass in multiple functions for its call, but it does not have to call it.                                     |\n",
     "| [create_structured_output_runnable](https://api.python.langchain.com/en/latest/chains/langchain.chains.structured_output.base.create_structured_output_runnable.html#langchain.chains.structured_output.base.create_structured_output_runnable)   | ✅ |              | If you want to use OpenAI function calling to FORCE the LLM to respond with a certain function. You may only pass in one function, and the chain will ALWAYS return this response.                                      |\n",
-    "| [load_query_constructor_runnable](https://api.python.langchain.com/en/latest/chains/langchain.chains.query_constructor.base.load_query_constructor_runnable.html#langchain.chains.query_constructor.base.load_query_constructor_runnable)  |                         |              | Can be used to generate queries. You must specify a list of allowed operations, and then will return a runnable that converts a natural language query into those allowed operations.                                                                               |\n",
+    "| [load_query_constructor_runnable](https://api.python.langchain.com/en/latest/chains/langchain.chains.query_constructor.base.load_query_constructor_runnable.html#langchain.chains.query_constructor.base.load_query_constructor_runnable)  |                         |              | Can be used to generate queries. You must specify a list of allowed operations and then return a runnable that converts a natural language query into those allowed operations.                                                                               |\n",
     "| [create_sql_query_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.sql_database.query.create_sql_query_chain.html#langchain.chains.sql_database.query.create_sql_query_chain)           |                         | SQL Database | If you want to construct a query for a SQL database from natural language.                            |\n",
     "| [create_history_aware_retriever](https://api.python.langchain.com/en/latest/chains/langchain.chains.history_aware_retriever.create_history_aware_retriever.html#langchain.chains.history_aware_retriever.create_history_aware_retriever)   |                         | Retriever    | This chain takes in conversation history and then uses that to generate a search query which is passed to the underlying retriever.                       |\n",
     "| [create_retrieval_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.retrieval.create_retrieval_chain.html#langchain.chains.retrieval.create_retrieval_chain)           |                         | Retriever    | This chain takes in a user inquiry, which is then passed to the retriever to fetch relevant documents. Those documents (and original inputs) are then passed to an LLM to generate a response      |"
@@ -82,48 +70,39 @@
    "source": [
     "## Legacy Chains\n",
     "\n",
-    "Below we report on the legacy chain types that exist. We will maintain support for these until we are able to create a LCEL alternative. We report on:\n",
+    "Below are the `legacy chains`. We will maintain support for these until we create an LCEL alternative. \n",
     "\n",
-    "**Chain**\n",
+    "Table columns:\n",
     "\n",
-    "Name of the chain, or name of the constructor method. If constructor method, this will return a `Chain` subclass.\n",
-    "\n",
-    "**Function Calling**\n",
-    "\n",
-    "Whether this requires OpenAI Function Calling.\n",
-    "\n",
-    "**Other Tools**\n",
-    "\n",
-    "Other tools used in the chain.\n",
-    "\n",
-    "**When to Use**\n",
-    "\n",
-    "Our commentary on when to use.\n",
+    "- **Chain:** Name of the chain or name of the constructor method. If constructor method, this will return a `Chain` subclass.\n",
+    "- **Function Calling:** Whether chain requires OpenAI Function Calling.\n",
+    "- **Other Tools:** Other tools used in the chain.\n",
+    "- **When to Use:** Our commentary on when to use.\n",
     "\n",
     "| Chain                        |  Function Calling | Other Tools            | When to Use |\n",
     "|------------------------------|--------------------|------------------------|-------------|\n",
     "| [APIChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.api.base.APIChain.html#langchain.chains.api.base.APIChain)                    |                                            | Requests Wrapper               | This chain uses an LLM to convert a query into an API request, then executes that request, gets back a response, and then passes that request to an LLM to respond            |\n",
     "| [OpenAPIEndpointChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.api.openapi.chain.OpenAPIEndpointChain.html#langchain.chains.api.openapi.chain.OpenAPIEndpointChain)         |                                            | OpenAPI Spec           | Similar to APIChain, this chain is designed to interact with APIs. The main difference is this is optimized for ease of use with OpenAPI endpoints            |\n",
-    "| [ConversationalRetrievalChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.conversational_retrieval.base.ConversationalRetrievalChain.html#langchain.chains.conversational_retrieval.base.ConversationalRetrievalChain) |                                            | Retriever              |This chain can be used to have **conversations** with a document. It takes in a question and (optional) previous conversation history. If there is previous conversation history, it uses an LLM to rewrite the conversation into a query to send to a retriever (otherwise it just uses the newest user input). It then fetches those documents and passes them (along with the conversation) to an LLM to respond.             |\n",
-    "| [StuffDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.stuff.StuffDocumentsChain.html#langchain.chains.combine_documents.stuff.StuffDocumentsChain)           |                    |                        |This chain takes a list of documents and formats them all into a prompt, then passes that prompt to an LLM. It passes ALL documents, so you should make sure it fits within the context window the LLM you are using.             |\n",
-    "| [ReduceDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.reduce.ReduceDocumentsChain.html#langchain.chains.combine_documents.reduce.ReduceDocumentsChain)         |                                            |                        |This chain combines documents by iterative reducing them. It groups documents into chunks (less than some context length) then passes them into an LLM. It then takes the responses and continues to do this until it can fit everything into one final LLM call. Useful when you have a lot of documents, you want to have the LLM run over all of them, and you can do in parallel.              |\n",
-    "| [MapReduceDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain.html#langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain)      |                        |                                            |This chain first passes each document through an LLM, then reduces them using the ReduceDocumentsChain. Useful in the same situations as ReduceDocumentsChain, but does an initial LLM call before trying to reduce the documents.          |\n",
-    "| [RefineDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.refine.RefineDocumentsChain.html#langchain.chains.combine_documents.refine.RefineDocumentsChain)         |    |                                        |This chain collapses documents by generating an initial answer based on the first document and then looping over the remaining documents to *refine* its answer. This operates sequentially, so it cannot be parallelized. It is useful in similar situatations as MapReduceDocuments Chain, but for cases where you want to build up an answer by refining the previous answer (rather than parallelizing calls).                        |             |\n",
+    "| [ConversationalRetrievalChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.conversational_retrieval.base.ConversationalRetrievalChain.html#langchain.chains.conversational_retrieval.base.ConversationalRetrievalChain) |                                            | Retriever              |This chain can be used to have **conversations** with a document. It takes in a question and (optional) previous conversation history. If there is a previous conversation history, it uses an LLM to rewrite the conversation into a query to send to a retriever (otherwise it just uses the newest user input). It then fetches those documents and passes them (along with the conversation) to an LLM to respond.             |\n",
+    "| [StuffDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.stuff.StuffDocumentsChain.html#langchain.chains.combine_documents.stuff.StuffDocumentsChain)           |                    |                        |This chain takes a list of documents and formats them all into a prompt, then passes that prompt to an LLM. It passes ALL documents, so you should make sure it fits within the context window of the LLM you are using.             |\n",
+    "| [ReduceDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.reduce.ReduceDocumentsChain.html#langchain.chains.combine_documents.reduce.ReduceDocumentsChain)         |                                            |                        |This chain combines documents by iterative reducing them. It groups documents into chunks (less than some context length) and then passes them into an LLM. It then takes the responses and continues to do this until it can fit everything into one final LLM call. It is useful when you have a lot of documents, you want to have the LLM run over all of them, and you can do it in parallel.              |\n",
+    "| [MapReduceDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain.html#langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain)      |                        |                                            |This chain first passes each document through an LLM, then reduces them using the `ReduceDocumentsChain`. It is useful in the same situations as `ReduceDocumentsChain`, but does an initial LLM call before trying to reduce the documents.          |\n",
+    "| [RefineDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.refine.RefineDocumentsChain.html#langchain.chains.combine_documents.refine.RefineDocumentsChain)         |    |                                        |This chain collapses documents by generating an initial answer based on the first document and then looping over the remaining documents to *refine* its answer. This operates sequentially, so it cannot be parallelized. It is useful in similar situations as MapReduceDocuments Chain, but for cases where you want to build up an answer by refining the previous answer (rather than parallelizing calls).                        |             |\n",
     "| [MapRerankDocumentsChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.combine_documents.map_rerank.MapRerankDocumentsChain.html#langchain.chains.combine_documents.map_rerank.MapRerankDocumentsChain)      |                        |                    |                                      This calls on LLM on each document, asking it to not only answer but also produce a score of how confident it is. The answer with the highest confidence is then returned. This is useful when you have a lot of documents, but only want to answer based on a single document, rather than trying to combine answers (like Refine and Reduce methods do).|\n",
-    "| [ConstitutionalChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.constitutional_ai.base.ConstitutionalChain.html#langchain.chains.constitutional_ai.base.ConstitutionalChain)          |                                            |                        |This chain answers, then attempts to refine its answer based on constitutional principles that are provided. Use this when you want to enforce that a chain's answer follows some principles.             |\n",
-    "| [LLMChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.llm.LLMChain.html#langchain.chains.llm.LLMChain)                                 |  |                  |                        |This chain simply combines a prompt with an LLM and an output parser. The recommended way to do this is just to use LCEL.             |\n",
-    "| [ElasticsearchDatabaseChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.elasticsearch_database.base.ElasticsearchDatabaseChain.html#langchain.chains.elasticsearch_database.base.ElasticsearchDatabaseChain)                           |                    | ElasticSearch Instance |This chain converts a natural language question to an ElasticSearch query, and then runs it, and then summarizes the response. This is useful for when you want to ask natural language questions of an Elastic Search database             |\n",
+    "| [ConstitutionalChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.constitutional_ai.base.ConstitutionalChain.html#langchain.chains.constitutional_ai.base.ConstitutionalChain)          |                                            |                        |This chain answers, then attempts to refine its answer based on constitutional principles that are provided. Use this to enforce that a chain's answer follows some principles.             |\n",
+    "| [LLMChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.llm.LLMChain.html#langchain.chains.llm.LLMChain)                                 |  |                  |                        |This chain simply combines a prompt with an LLM and an output parser. The recommended way to do this is to use LCEL.             |\n",
+    "| [ElasticsearchDatabaseChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.elasticsearch_database.base.ElasticsearchDatabaseChain.html#langchain.chains.elasticsearch_database.base.ElasticsearchDatabaseChain)                           |                    | Elasticsearch Instance |This chain converts a natural language question to an `Elasticsearch` query, and then runs it, and then summarizes the response. This is useful for when you want to ask natural language questions of an `Elasticsearch` database             |\n",
     "| [FlareChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.flare.base.FlareChain.html#langchain.chains.flare.base.FlareChain)                   |                                            |                        |This implements [FLARE](https://arxiv.org/abs/2305.06983), an advanced retrieval technique. It is primarily meant as an exploratory advanced retrieval method.             |\n",
     "| [ArangoGraphQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.arangodb.ArangoGraphQAChain.html#langchain.chains.graph_qa.arangodb.ArangoGraphQAChain)                                   |                    |Arango Graph                        |This chain constructs an Arango query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.             |\n",
-    "|[GraphCypherQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.cypher.GraphCypherQAChain.html#langchain.chains.graph_qa.cypher.GraphCypherQAChain)                                                      |                    |A graph that works with Cypher query language                        |This chain constructs an Cypher query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.             |\n",
+    "|[GraphCypherQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.cypher.GraphCypherQAChain.html#langchain.chains.graph_qa.cypher.GraphCypherQAChain)                                                      |                    |A graph that works with Cypher query language                        |This chain constructs a Cypher query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.             |\n",
     "|[FalkorDBGraphQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.falkordb.FalkorDBQAChain.html#langchain.chains.graph_qa.falkordb.FalkorDBQAChain)                                                      |                    |Falkor Database                        | This chain constructs a FalkorDB query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.             |\n",
     "|[HugeGraphQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.hugegraph.HugeGraphQAChain.html#langchain.chains.graph_qa.hugegraph.HugeGraphQAChain)                                                     |                    |HugeGraph                        |This chain constructs an HugeGraph query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.              |\n",
     "|[KuzuQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.kuzu.KuzuQAChain.html#langchain.chains.graph_qa.kuzu.KuzuQAChain)                                                      |                    |Kuzu Graph                        |This chain constructs a Kuzu Graph query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.              |\n",
     "|[NebulaGraphQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.nebulagraph.NebulaGraphQAChain.html#langchain.chains.graph_qa.nebulagraph.NebulaGraphQAChain)                                                      |                    |Nebula Graph                        |This chain constructs a Nebula Graph query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.              |\n",
-    "|[NeptuneOpenCypherQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.neptune_cypher.NeptuneOpenCypherQAChain.html#langchain.chains.graph_qa.neptune_cypher.NeptuneOpenCypherQAChain)                                                     |                    |Neptune Graph                        |This chain constructs an Neptune Graph query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.              |\n",
-    "|[GraphSparqlChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.sparql.GraphSparqlQAChain.html#langchain.chains.graph_qa.sparql.GraphSparqlQAChain)                                                      |                    |Graph that works with SparQL                        |This chain constructs an SparQL query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.              |\n",
+    "|[NeptuneOpenCypherQAChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.neptune_cypher.NeptuneOpenCypherQAChain.html#langchain.chains.graph_qa.neptune_cypher.NeptuneOpenCypherQAChain)                                                     |                    |Neptune Graph                        |This chain constructs a Neptune Graph query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.              |\n",
+    "|[GraphSparqlChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.graph_qa.sparql.GraphSparqlQAChain.html#langchain.chains.graph_qa.sparql.GraphSparqlQAChain)                                                      |                    |Graph that works with SparQL                        |This chain constructs a SparQL query from natural language, executes that query against the graph, and then passes the results back to an LLM to respond.              |\n",
     "|[LLMMath](https://api.python.langchain.com/en/latest/chains/langchain.chains.llm_math.base.LLMMathChain.html#langchain.chains.llm_math.base.LLMMathChain)                                                      |                    |                        |This chain converts a user question to a math problem and then executes it (using [numexpr](https://github.com/pydata/numexpr))             |\n",
-    "|[LLMCheckerChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.llm_checker.base.LLMCheckerChain.html#langchain.chains.llm_checker.base.LLMCheckerChain)                                                      |                    |                        |This chain uses a second LLM call to varify its initial answer. Use this when you to have an extra layer of validation on the initial LLM call.             |\n",
+    "|[LLMCheckerChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.llm_checker.base.LLMCheckerChain.html#langchain.chains.llm_checker.base.LLMCheckerChain)                                                      |                    |                        |This chain uses a second LLM call to verify its initial answer. Use this when you have an extra layer of validation on the initial LLM call.             |\n",
     "|[LLMSummarizationChecker](https://api.python.langchain.com/en/latest/chains/langchain.chains.llm_summarization_checker.base.LLMSummarizationCheckerChain.html#langchain.chains.llm_summarization_checker.base.LLMSummarizationCheckerChain)                              |                        |                                            |This chain creates a summary using a sequence of LLM calls to make sure it is extra correct. Use this over the normal summarization chain when you are okay with multiple LLM calls (eg you care more about accuracy than speed/cost).             |\n",
     "|[create_citation_fuzzy_match_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.openai_functions.citation_fuzzy_match.create_citation_fuzzy_match_chain.html#langchain.chains.openai_functions.citation_fuzzy_match.create_citation_fuzzy_match_chain)                                                      |✅                    |                        |Uses OpenAI function calling to answer questions and cite its sources.             |\n",
     "|[create_extraction_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.openai_functions.extraction.create_extraction_chain.html#langchain.chains.openai_functions.extraction.create_extraction_chain)                              |                        ✅                    |                        |Uses OpenAI Function calling to extract information from text.             |\n",
@@ -131,7 +110,7 @@
     "|[get_openapi_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.openai_functions.openapi.get_openapi_chain.html#langchain.chains.openai_functions.openapi.get_openapi_chain)                              |                        ✅                    |OpenAPI Spec                        |Uses OpenAI function calling to query an OpenAPI.             |\n",
     "|[create_qa_with_structure_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.openai_functions.qa_with_structure.create_qa_with_structure_chain.html#langchain.chains.openai_functions.qa_with_structure.create_qa_with_structure_chain)                              |                        ✅                    |                        |Uses OpenAI function calling to do question answering over text and respond in a specific format.             |\n",
     "|[create_qa_with_sources_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.openai_functions.qa_with_structure.create_qa_with_sources_chain.html#langchain.chains.openai_functions.qa_with_structure.create_qa_with_sources_chain)                              |                        ✅                    |                        |Uses OpenAI function calling to answer questions with citations.             |\n",
-    "|[QAGenerationChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.qa_generation.base.QAGenerationChain.html#langchain.chains.qa_generation.base.QAGenerationChain)                                          |            |                    |Creates both questions and answers from documents. Can be used to generate question/answer pairs for evaluation of retrieval projects.                        |             \n",
+    "|[QAGenerationChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.qa_generation.base.QAGenerationChain.html#langchain.chains.qa_generation.base.QAGenerationChain)                                          |            |                    |Creates both questions and answers from documents. Used to generate question/answer pairs for evaluation of retrieval projects.                        |             \n",
     "|[RetrievalQAWithSourcesChain](https://api.python.langchain.com/en/latest/chains/langchain.chains.qa_with_sources.retrieval.RetrievalQAWithSourcesChain.html#langchain.chains.qa_with_sources.retrieval.RetrievalQAWithSourcesChain)                              |                        |                          Retriever                    |Does question answering over retrieved documents, and cites it sources. Use this when you want the answer response to have sources in the text response. Use this over `load_qa_with_sources_chain` when you want to use a retriever to fetch the relevant document as part of the chain (rather than pass them in).|             \n",
     "|[load_qa_with_sources_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.qa_with_sources.loading.load_qa_with_sources_chain.html#langchain.chains.qa_with_sources.loading.load_qa_with_sources_chain)                                                     | |Retriever                    |Does question answering over documents you pass in, and cites it sources. Use this when you want the answer response to have sources in the text response. Use this over RetrievalQAWithSources when you want to pass in the documents directly (rather than rely on a retriever to get them).|              \n",
     "|[RetrievalQA](https://api.python.langchain.com/en/latest/chains/langchain.chains.retrieval_qa.base.RetrievalQA.html#langchain.chains.retrieval_qa.base.RetrievalQA)   |                                            |Retriever                        |This chain first does a retrieval step to fetch relevant documents, then passes those documents into an LLM to generate a response.|\n",
@@ -168,7 +147,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.1"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,

docs/docs/modules/data_connection/document_loaders/index.mdx

@@ -4,6 +4,7 @@ sidebar_class_name: hidden
 ---
 # Document loaders
 
+
 :::info
 Head to [Integrations](/docs/integrations/document_loaders/) for documentation on built-in document loader integrations with 3rd-party tools.
 :::

docs/docs/modules/data_connection/document_loaders/pdf.mdx

@@ -4,6 +4,7 @@ keywords: [PyPDFDirectoryLoader, PyMuPDFLoader]
 
 # PDF
 
+
 >[Portable Document Format (PDF)](https://en.wikipedia.org/wiki/PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems.
 
 This covers how to load `PDF` documents into the Document format that we use downstream.

docs/docs/modules/data_connection/document_transformers/index.mdx

@@ -4,6 +4,7 @@ sidebar_class_name: hidden
 ---
 # Text Splitters
 
+
 Once you've loaded documents, you'll often want to transform them to better suit your application. The simplest example
 is you may want to split a long document into smaller chunks that can fit into your model's context window. LangChain
 has a number of built-in document transformers that make it easy to split, combine, filter, and otherwise manipulate documents.
@@ -25,27 +26,28 @@ That means there are two different axes along which you can customize your text
 
 ## Types of Text Splitters
 
-LangChain offers many different types of text splitters. These all live in the `langchain-text-splitters` package. Below is a table listing all of them, along with a few characteristics:
+LangChain offers many different types of `text splitters`. 
+These all live in the `langchain-text-splitters` package. 
 
-**Name**: Name of the text splitter
+Table columns:
 
-**Splits On**: How this text splitter splits text
-
-**Adds Metadata**: Whether or not this text splitter adds metadata about where each chunk came from.
-
-**Description**: Description of the splitter, including recommendation on when to use it.
+- **Name**: Name of the text splitter
+- **Classes**: Classes that implement this text splitter
+- **Splits On**: How this text splitter splits text
+- **Adds Metadata**: Whether or not this text splitter adds metadata about where each chunk came from.
+- **Description**: Description of the splitter, including recommendation on when to use it.
 
 
-| Name      | Splits On                             | Adds Metadata | Description                                                                                                                                                                             |
-|-----------|---------------------------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| Recursive | A list of user defined characters     |               | Recursively splits text. Splitting text recursively serves the purpose of trying to keep related pieces of text next to each other. This is the recommended way to start splitting text. |
-| HTML      | HTML specific characters              | ✅             | Splits text based on HTML-specific characters. Notably, this adds in relevant information about where that chunk came from (based on the HTML)                                          |
-| Markdown  | Markdown specific characters          | ✅             | Splits text based on Markdown-specific characters. Notably, this adds in relevant information about where that chunk came from (based on the Markdown)                                  |
-| Code      | Code (Python, JS) specific characters |               | Splits text based on characters specific to coding languages. 15 different languages are available to choose from.                                                                      |
-| Token     | Tokens                                |               | Splits text on tokens. There exist a few different ways to measure tokens.                                                                                                              |
-| Character | A user defined character              |               | Splits text based on a user defined character. One of the simpler methods.                                                                                                              |
-| [Experimental] Semantic Chunker | Sentences             |               | First splits on sentences. Then combines ones next to each other if they are semantically similar enough. Taken from [Greg Kamradt](https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb)                                                                                                              |
-| [AI21 Semantic Text Splitter](/docs/integrations/document_transformers/ai21_semantic_text_splitter) | Semantics             |    ✅           | Identifies distinct topics that form coherent pieces of text and splits along those. |
+| Name     | Classes                                                                                                                                                                                                             | Splits On                                                   | Adds Metadata | Description                                                                                                                                                                                                                                                                  |
+|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Recursive | [RecursiveCharacterTextSplitter](/docs/modules/data_connection/document_transformers/recursive_text_splitter), [RecursiveJsonSplitter](/docs/modules/data_connection/document_transformers/recursive_json_splitter) | A list of user defined characters     |               | Recursively splits text. This splitting is trying to keep related pieces of text next to each other. This is the `recommended way` to start splitting text.                                                                                                                    |
+| HTML      | [HTMLHeaderTextSplitter](/docs/modules/data_connection/document_transformers/HTML_header_metadata), [HTMLSectionSplitter](/docs/modules/data_connection/document_transformers/HTML_section_aware_splitter)          | HTML specific characters                                                                                                 | ✅             | Splits text based on HTML-specific characters. Notably, this adds in relevant information about where that chunk came from (based on the HTML)                                                                                                                               |
+| Markdown  | [MarkdownHeaderTextSplitter](/docs/modules/data_connection/document_transformers/markdown_header_metadata)                                                                                                          | Markdown specific characters                                                                                    | ✅             | Splits text based on Markdown-specific characters. Notably, this adds in relevant information about where that chunk came from (based on the Markdown)                                                                                                                       |
+| Code      | [many languages](/docs/modules/data_connection/document_transformers/code_splitter)                                                                                                                                 | Code (Python, JS) specific characters                                                                           |               | Splits text based on characters specific to coding languages. 15 different languages are available to choose from.                                                                                                                                                           |
+| Token    | [many classes](/docs/modules/data_connection/document_transformers/split_by_token)                                                                                                                                  | Tokens                                                                                                          |               | Splits text on tokens. There exist a few different ways to measure tokens.                                                                                                                                                                                                   |
+| Character  | [CharacterTextSplitter](/docs/modules/data_connection/document_transformers/character_text_splitter)                                                                                                                | A user defined character                                                                                        |               | Splits text based on a user defined character. One of the simpler methods.                                                                                                                                                                                                   |
+| [Experimental] Semantic Chunker | [SemanticChunker](/docs/modules/data_connection/document_transformers/semantic-chunker)                                                                                                                             | Sentences                                                                                                       |               | First splits on sentences. Then combines ones next to each other if they are semantically similar enough. Taken from [Greg Kamradt](https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb) |
+| AI21 Semantic Text Splitter | [AI21SemanticTextSplitter](/docs/integrations/document_transformers/ai21_semantic_text_splitter)                                                                                                                    |    ✅           | Identifies distinct topics that form coherent pieces of text and splits along those.                                                                                                                                                                                         |
 
 
 ## Evaluate text splitters

docs/docs/modules/data_connection/document_transformers/recursive_text_splitter.ipynb

@@ -7,6 +7,7 @@
    "source": [
     "# Recursively split by character\n",
     "\n",
+
     "This text splitter is the recommended one for generic text. It is parameterized by a list of characters. It tries to split on them in order until the chunks are small enough. The default list is `[\"\\n\\n\", \"\\n\", \" \", \"\"]`. This has the effect of trying to keep all paragraphs (and then sentences, and then words) together as long as possible, as those would generically seem to be the strongest semantically related pieces of text.\n",
     "\n",
     "1. How the text is split: by list of characters.\n",

docs/docs/modules/data_connection/document_transformers/semantic-chunker.ipynb

@@ -7,6 +7,7 @@
    "source": [
     "# Semantic Chunking\n",
     "\n",
+
     "Splits the text based on semantic similarity.\n",
     "\n",
     "Taken from Greg Kamradt's wonderful notebook:\n",

docs/docs/modules/data_connection/index.mdx

@@ -5,6 +5,7 @@ sidebar_class_name: hidden
 
 # Retrieval
 
+
 Many LLM applications require user-specific data that is not part of the model's training set.
 The primary way of accomplishing this is through Retrieval Augmented Generation (RAG).
 In this process, external data is *retrieved* and then passed to the LLM when doing the *generation* step.

docs/docs/modules/data_connection/indexing.ipynb

@@ -60,7 +60,7 @@
     "   * document addition by id (`add_documents` method with `ids` argument)\n",
     "   * delete by id (`delete` method with `ids` argument)\n",
     "\n",
-    "Compatible Vectorstores: `AnalyticDB`, `AstraDB`, `AzureCosmosDBVectorSearch`, `AzureSearch`, `AwaDB`, `Bagel`, `Cassandra`, `Chroma`, `CouchbaseVectorStore`, `DashVector`, `DatabricksVectorSearch`, `DeepLake`, `Dingo`, `ElasticVectorSearch`, `ElasticsearchStore`, `FAISS`, `HanaDB`, `LanceDB`, `Milvus`, `MyScale`, `OpenSearchVectorSearch`, `PGVector`, `Pinecone`, `Qdrant`, `Redis`, `Rockset`, `ScaNN`, `SupabaseVectorStore`, `SurrealDBStore`, `TimescaleVector`, `UpstashVectorStore`, `Vald`, `VDMS`, `Vearch`, `VespaStore`, `Weaviate`, `ZepVectorStore`, `TencentVectorDB`, `OpenSearchVectorSearch`.\n",
+    "Compatible Vectorstores: `AnalyticDB`, `AstraDB`, `AzureCosmosDBVectorSearch`, `AzureSearch`, `AwaDB`, `Bagel`, `Cassandra`, `Chroma`, `CouchbaseVectorStore`, `DashVector`, `DatabricksVectorSearch`, `DeepLake`, `Dingo`, `ElasticVectorSearch`, `ElasticsearchStore`, `FAISS`, `HanaDB`, `LanceDB`, `Milvus`, `MyScale`, `OpenSearchVectorSearch`, `PGVector`, `Pinecone`, `Qdrant`, `Redis`, `Rockset`, `ScaNN`, `SupabaseVectorStore`, `SurrealDBStore`, `TimescaleVector`, `UpstashVectorStore`, `Vald`, `VDMS`, `Vearch`, `VespaStore`, `Weaviate`, `ZepVectorStore`, `TencentVectorDB`, `OpenSearchVectorSearch`, `Yellowbrick`.\n",
     "  \n",
     "## Caution\n",
     "\n",

docs/docs/modules/data_connection/retrievers/index.mdx

@@ -6,36 +6,33 @@ sidebar_class_name: hidden
 
 # Retrievers
 
+
 A retriever is an interface that returns documents given an unstructured query. It is more general than a vector store.
 A retriever does not need to be able to store documents, only to return (or retrieve) them. Vector stores can be used
 as the backbone of a retriever, but there are other types of retrievers as well.
 
-Retrievers accept a string query as input and return a list of `Document`'s as output.
+Retrievers accept a string `query` as input and return a list of `Document`'s as output.
 
 ## Advanced Retrieval Types
 
-LangChain provides several advanced retrieval types. A full list is below, along with the following information:
+Table columns:
 
-**Name**: Name of the retrieval algorithm.
+- **Name**: Name of the retrieval algorithm.
+- **Index Type**: Which index type (if any) this relies on.
+- **Uses an LLM**: Whether this retrieval method uses an LLM.
+- **When to Use**: Our commentary on when you should considering using this retrieval method.
+- **Description**: Description of what this retrieval algorithm is doing.
 
-**Index Type**: Which index type (if any) this relies on.
-
-**Uses an LLM**: Whether this retrieval method uses an LLM.
-
-**When to Use**: Our commentary on when you should considering using this retrieval method.
-
-**Description**: Description of what this retrieval algorithm is doing.
-
-| Name                      | Index Type                   | Uses an LLM               | When to Use                                                                                                                                   | Description                                                                                                                                                                                                                                                                                      |
-|---------------------------|------------------------------|---------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| [Vectorstore](./vectorstore)               | Vectorstore                  | No                        | If you are just getting started and looking for something quick and easy.                                                                     | This is the simplest method and the one that is easiest to get started with. It involves creating embeddings for each piece of text.                                                                                                                                                             |
-| [ParentDocument](./parent_document_retriever)            | Vectorstore + Document Store | No                        | If your pages have lots of smaller pieces of distinct information that are best indexed by themselves, but best retrieved all together.       | This involves indexing multiple chunks for each document. Then you find the chunks that are most similar in embedding space, but you retrieve the whole parent document and return that (rather than individual chunks).                                                                         |
-| [Multi Vector](multi_vector)              | Vectorstore + Document Store | Sometimes during indexing | If you are able to extract information from documents that you think is more relevant to index than the text itself.                          | This involves creating multiple vectors for each document. Each vector could be created in a myriad of ways - examples include summaries of the text and hypothetical questions.                                                                                                                 |
-| [Self Query](./self_query)               | Vectorstore                  | Yes                       | If users are asking questions that are better answered by fetching documents based on metadata rather than similarity with the text.          | This uses an LLM to transform user input into two things: (1) a string to look up semantically, (2) a metadata filer to go along with it. This is useful because oftentimes questions are about the METADATA of documents (not the content itself).                                              |
-| [Contextual Compression](./contextual_compression)    | Any                          | Sometimes                 | If you are finding that your retrieved documents contain too much irrelevant information and are distracting the LLM.                         | This puts a post-processing step on top of another retriever and extracts only the most relevant information from retrieved documents. This can be done with embeddings or an LLM.                                                                                                               |
-| [Time-Weighted Vectorstore](./time_weighted_vectorstore) | Vectorstore                  | No                        | If you have timestamps associated with your documents, and you want to retrieve the most recent ones                                          | This fetches documents based on a combination of semantic similarity (as in normal vector retrieval) and recency (looking at timestamps of indexed documents)                                                                                                                                    |
-| [Multi-Query Retriever](./MultiQueryRetriever)     | Any                          | Yes                       | If users are asking questions that are complex and require multiple pieces of distinct information to respond                                 | This uses an LLM to generate multiple queries from the original one. This is useful when the original query needs pieces of information about multiple topics to be properly answered. By generating multiple queries, we can then fetch documents for each of them.                             |
-| [Ensemble](./ensemble)                  | Any                          | No                        | If you have multiple retrieval methods and want to try combining them.                                                                        | This fetches documents from multiple retrievers and then combines them.                                                                                                                                                                                                                          |
+| Name                      | Index Type                   | Uses an LLM               | When to Use                                                                                                                                   | Description                                                                                                                                                                                                                                                                                       |
+|---------------------------|------------------------------|---------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [Vectorstore](./vectorstore)               | Vectorstore                  | No                        | If you are just getting started and looking for something quick and easy.                                                                     | This is the simplest method and the one that is easiest to get started with. It creates embeddings for each piece of text.                                                                                                                                                                        |
+| [ParentDocument](./parent_document_retriever)            | Vectorstore + Document Store | No                        | If your pages have lots of smaller pieces of distinct information that are best indexed by themselves, but best retrieved all together.       | This indexes multiple chunks for each document. Then you find the chunks that are most similar in embedding space, but you retrieve the whole parent document and return that (rather than individual chunks).                                                                                    |
+| [Multi Vector](multi_vector)              | Vectorstore + Document Store | Sometimes during indexing | If you are able to extract information from documents that you think is more relevant to index than the text itself.                          | This creates multiple vectors for each document. Each vector could be created in a myriad of ways - examples include summaries of the text and hypothetical questions.                                                                                                                            |
+| [Self Query](./self_query)               | Vectorstore                  | Yes                       | If users are asking questions that are better answered by fetching documents based on metadata rather than similarity with the text.          | This uses an LLM to transform user input into two things: (1) a string to look up semantically, (2) a metadata filer to go along with it. This is useful because oftentimes questions are about the METADATA of documents (not the content itself).                                               |
+| [Contextual Compression](./contextual_compression)    | Any                          | Sometimes                 | If you are finding that your retrieved documents contain too much irrelevant information and are distracting the LLM.                         | This puts a post-processing step on top of another retriever and extracts only the most relevant information from retrieved documents. This can be done with embeddings or an LLM.                                                                                                                |
+| [Time-Weighted Vectorstore](./time_weighted_vectorstore) | Vectorstore                  | No                        | If you have timestamps associated with your documents, and you want to retrieve the most recent ones                                          | This fetches documents based on a combination of semantic similarity (as in normal vector retrieval) and recency (looking at timestamps of indexed documents)                                                                                                                                     |
+| [Multi-Query Retriever](./MultiQueryRetriever)     | Any                          | Yes                       | If users are asking questions that are complex and require multiple pieces of distinct information to respond                                 | This uses an LLM to generate multiple queries from the original one. This is useful when the original query needs pieces of information about multiple topics to be properly answered. By generating multiple queries, we can then fetch documents for each of them.                              |
+| [Ensemble](./ensemble)                  | Any                          | No                        | If you have multiple retrieval methods and want to try combining them.                                                                        | This fetches documents from multiple retrievers and then combines them.                                                                                                                                                                                                                    |
 | [Long-Context Reorder](./long_context_reorder)      | Any                          | No                        | If you are working with a long-context model and noticing that it's not paying attention to information in the middle of retrieved documents. | This fetches documents from an underlying retriever, and then reorders them so that the most similar are near the beginning and end. This is useful because it's been shown that for longer context models they sometimes don't pay attention to information in the middle of the context window. |
 
 

docs/docs/modules/data_connection/retrievers/vectorstore.ipynb

@@ -17,6 +17,7 @@
    "source": [
     "# Vector store-backed retriever\n",
     "\n",
+
     "A vector store retriever is a retriever that uses a vector store to retrieve documents. It is a lightweight wrapper around the vector store class to make it conform to the retriever interface.\n",
     "It uses the search methods implemented by a vector store, like similarity search and MMR, to query the texts in the vector store.\n",
     "\n",

docs/docs/modules/data_connection/text_embedding/index.mdx

@@ -4,6 +4,7 @@ sidebar_class_name: hidden
 ---
 # Text embedding models
 
+
 :::info
 Head to [Integrations](/docs/integrations/text_embedding/) for documentation on built-in integrations with text embedding model providers.
 :::

docs/docs/modules/data_connection/vectorstores/index.mdx

@@ -4,6 +4,7 @@ sidebar_class_name: hidden
 ---
 # Vector stores
 
+
 :::info
 Head to [Integrations](/docs/integrations/vectorstores/) for documentation on built-in integrations with 3rd-party vector stores.
 :::
@@ -56,6 +57,50 @@ documents = text_splitter.split_documents(raw_documents)
 db = Chroma.from_documents(documents, OpenAIEmbeddings())
 ```
 
+  </TabItem>
+  <TabItem value="pinecone" label="Pinecone">
+
+This walkthrough uses the `Pinecone` vector database, which provides broad functionality to store and search over vectors.
+
+```bash
+pip install langchain-pinecone
+```
+
+We want to use OpenAIEmbeddings so we have to get the OpenAI API Key.
+
+```python
+import os
+import getpass
+
+os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key:')
+```
+
+```python
+from langchain_community.document_loaders import TextLoader
+from langchain_openai import OpenAIEmbeddings
+from langchain_text_splitters import CharacterTextSplitter
+
+# Load the document, split it into chunks, and embed each chunk.
+loader = TextLoader("../../modules/state_of_the_union.txt")
+documents = loader.load()
+text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
+docs = text_splitter.split_documents(documents)
+
+embeddings = OpenAIEmbeddings()
+```
+
+Next, go to the [Pinecone console](https://app.pinecone.io) and create a new index with `dimension=1536` called "langchain-test-index". Then, copy the API key and index name.
+
+```python
+from langchain_pinecone import PineconeVectorStore
+
+os.environ['PINECONE_API_KEY'] = '<YOUR_PINECONE_API_KEY>'
+
+index_name = "langchain-test-index"
+
+# Connect to Pinecone index and insert the chunked docs as contents
+docsearch = PineconeVectorStore.from_documents(docs, embeddings, index_name=index_name)
+```
   </TabItem>
   <TabItem value="faiss" label="FAISS">
 
@@ -280,4 +325,4 @@ I’ve worked on these issues a long time.
 I know what works: Investing in crime prevention and community police officers who’ll walk the beat, who’ll know the neighborhood, and who can restore trust and safety.
 ```
 
-</CodeOutputBlock>
+</CodeOutputBlock>

docs/docs/modules/index.mdx

@@ -4,6 +4,7 @@ sidebar_class_name: hidden
 
 # Components
 
+
 LangChain provides standard, extendable interfaces and external integrations for the following main components:
 
 ## [Model I/O](/docs/modules/model_io/)

docs/docs/modules/model_io/chat/function_calling.ipynb

@@ -18,6 +18,7 @@
    "source": [
     "# Tool calling\n",
     "\n",
+
     "```{=mdx}\n",
     ":::info\n",
     "We use the term \"tool calling\" interchangeably with \"function calling\". Although\n",

docs/docs/modules/model_io/chat/index.mdx

@@ -5,6 +5,7 @@ sidebar_class_name: hidden
 
 # Chat Models
 
+
 Chat Models are a core component of LangChain.
 
 A chat model is a language model that uses chat messages as inputs and returns chat messages as outputs (as opposed to using plain text).

docs/docs/modules/model_io/llms/index.mdx

@@ -5,6 +5,7 @@ sidebar_class_name: hidden
 
 # LLMs
 
+
 Large Language Models (LLMs) are a core component of LangChain.
 LangChain does not serve its own LLMs, but rather provides a standard interface for interacting with many different LLMs. To be specific, this interface is one that takes as input a string and returns a string.
 

docs/docs/modules/model_io/output_parsers/index.mdx

@@ -5,6 +5,7 @@ sidebar_class_name: hidden
 ---
 # Output Parsers
 
+
 Output parsers are responsible for taking the output of an LLM and transforming it to a more suitable format. This is very useful when you are using LLMs to generate any form of structured data.
 
 Besides having a large collection of different types of output parsers, one distinguishing benefit of LangChain OutputParsers is that many of them support streaming.
@@ -15,34 +16,30 @@ See [this quick-start guide](./quick_start) for an introduction to output parser
 
 ## Output Parser Types
 
-LangChain has lots of different types of output parsers. This is a list of output parsers LangChain supports. The table below has various pieces of information:
+LangChain has lots of different types of `output parsers`.
 
-**Name**: The name of the output parser
+Table columns:
 
-**Supports Streaming**: Whether the output parser supports streaming.
-
-**Has Format Instructions**: Whether the output parser has format instructions. This is generally available except when (a) the desired schema is not specified in the prompt but rather in other parameters (like OpenAI function calling), or (b) when the OutputParser wraps another OutputParser.
-
-**Calls LLM**: Whether this output parser itself calls an LLM. This is usually only done by output parsers that attempt to correct misformatted output.
-
-**Input Type**: Expected input type. Most output parsers work on both strings and messages, but some (like OpenAI Functions) need a message with specific kwargs.
-
-**Output Type**: The output type of the object returned by the parser.
-
-**Description**: Our commentary on this output parser and when to use it.
+- **Name**: The name of the output parser
+- **Supports Streaming**: Whether the output parser supports streaming.
+- **Has Format Instructions**: Whether the output parser has format instructions. This is generally available except when (a) the desired schema is not specified in the prompt but rather in other parameters (like OpenAI function calling), or (b) when the OutputParser wraps another OutputParser.
+- **Calls LLM**: Whether this output parser itself calls an LLM. This is usually only done by output parsers that attempt to correct misformatted output.
+- **Input Type**: Expected input type. Most output parsers work on both strings and messages, but some (like OpenAI Functions) need a message with specific kwargs.
+- **Output Type**: The output type of the object returned by the parser.
+- **Description**: Our commentary on this output parser and when to use it.
 
 | Name            | Supports Streaming | Has Format Instructions       | Calls LLM | Input Type                       | Output Type          | Description                                                                                                                                                                                                                                              |
 |-----------------|--------------------|-------------------------------|-----------|----------------------------------|----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | [OpenAITools](./types/openai_tools) |                   | (Passes `tools` to model) |           | `Message` (with `tool_choice`)   | JSON object          | Uses latest OpenAI function calling args `tools` and `tool_choice` to structure the return output. If you are using a model that supports function calling, this is generally the most reliable method.                                                                                        |
 | [OpenAIFunctions](./types/openai_functions) | ✅                  | (Passes `functions` to model) |           | `Message` (with `function_call`) | JSON object          | Uses legacy OpenAI function calling args `functions` and `function_call` to structure the return output.                                                                                        |
-| [JSON](./types/json)            | ✅                  | ✅                             |           | `str` \| `Message`               | JSON object          | Returns a JSON object as specified. You can specify a Pydantic model and it will return JSON for that model. Probably the most reliable output parser for getting structured data that does NOT use function calling.                                    |
+| [JSON](./types/json)            | ✅                  | ✅                             |           | `str` \| `Message`               | JSON object          | Returns a JSON object as specified. You specify a Pydantic model and it will return JSON for that model. Probably the most reliable output parser for getting structured data that does NOT use function calling.                                    |
 | [XML](./types/xml)            | ✅                  | ✅                             |           | `str` \| `Message`                 | `dict`               | Returns a dictionary of tags. Use when XML output is needed. Use with models that are good at writing XML (like Anthropic's).                                                                                                                            |
 | [CSV](./types/csv)           | ✅                  | ✅                             |           | `str` \| `Message`                 | `List[str]`          | Returns a list of comma separated values.                                                                                                                                                                                                                |
 | [OutputFixing](./types/output_fixing)    |                    |                               | ✅         | `str` \| `Message`                 |                      | Wraps another output parser. If that output parser errors, then this will pass the error message and the bad output to an LLM and ask it to fix the output.                                                                                              |
-| [RetryWithError](./types/retry)  |                    |                               | ✅         | `str` \| `Message`                 |                      | Wraps another output parser. If that output parser errors, then this will pass the original inputs, the bad output, and the error message to an LLM and ask it to fix it. Compared to OutputFixingParser, this one also sends the original instructions. |
+| [RetryWithError](./types/retry)  |                    |                               | ✅         | `str` \| `Message`                 |                      | Wraps another output parser. If that output parser errors, then this will pass the original inputs, the bad output, and the error message to an LLM and ask it to fix it. Compared to `OutputFixingParser`, this one also sends the original instructions. |
 | [Pydantic](./types/pydantic)        |                    | ✅                             |           | `str` \| `Message`                 | `pydantic.BaseModel` | Takes a user defined Pydantic model and returns data in that format.                                                                                                                                                                                     |
 | [YAML](./types/yaml)        |                    | ✅                             |           | `str` \| `Message`                 | `pydantic.BaseModel` | Takes a user defined Pydantic model and returns data in that format. Uses YAML to encode it.                                                                                                                                                                                    |
 | [PandasDataFrame](./types/pandas_dataframe) |                    | ✅                             |           | `str` \| `Message`                 | `dict`               | Useful for doing operations with pandas DataFrames.                                                                                                                                                                                                      |
 | [Enum](./types/enum)            |                    | ✅                             |           | `str` \| `Message`                 | `Enum`               | Parses response into one of the provided enum values.                                                                                                                                                                                                    |
 | [Datetime](./types/datetime)        |                    | ✅                             |           | `str` \| `Message`                 | `datetime.datetime`  | Parses response into a datetime string.                                                                                                                                                                                                                  |
-| [Structured](./types/structured)      |                    | ✅                             |           | `str` \| `Message`                 | `Dict[str, str]`     | An output parser that returns structured information. It is less powerful than other output parsers since it only allows for fields to be strings. This can be useful when you are working with smaller LLMs.                                            |
+| [Structured](./types/structured)      |                    | ✅                             |           | `str` \| `Message`                 | `Dict[str, str]`     | An output parser that returns structured information. It is less powerful than other output parsers since it only allows for fields to be strings. This useful when you are working with smaller LLMs.                                            |

docs/docs/modules/model_io/prompts/index.mdx

@@ -4,6 +4,7 @@ sidebar_class_name: hidden
 ---
 # Prompts
 
+
 A prompt for a language model is a set of instructions or input provided by a user to
 guide the model's response, helping it understand the context and generate relevant
 and coherent language-based output, such as answering questions, completing sentences,

docs/docs/modules/model_io/prompts/quick_start.ipynb

@@ -18,6 +18,7 @@
    "source": [
     "# Quick reference\n",
     "\n",
+
     "Prompt templates are predefined recipes for generating prompts for language models.\n",
     "\n",
     "A template may include instructions, few-shot examples, and specific context and\n",

docs/docs/modules/tools/custom_tools.ipynb

@@ -7,6 +7,7 @@
    "source": [
     "# Defining Custom Tools\n",
     "\n",
+
     "When constructing your own agent, you will need to provide it with a list of Tools that it can use. Besides the actual function that is called, the Tool consists of several components:\n",
     "\n",
     "- `name` (str), is required and must be unique within a set of tools provided to an agent\n",

docs/docs/modules/tools/index.ipynb

@@ -18,6 +18,7 @@
    "source": [
     "# Tools\n",
     "\n",
+
     "Tools are interfaces that an agent, chain, or LLM can use to interact with the world.\n",
     "They combine a few things:\n",
     "\n",

docs/docs/use_cases/apis.ipynb

@@ -16,7 +16,7 @@
    "id": "a15e6a18",
    "metadata": {},
    "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/apis.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/v0.1/docs/docs/use_cases/apis.ipynb)\n",
     "\n",
     "## Use case \n",
     "\n",

docs/docs/use_cases/chatbots/index.ipynb

@@ -15,6 +15,7 @@
    "source": [
     "# Chatbots\n",
     "\n",
+
     "## Overview\n",
     "\n",
     "Chatbots are one of the most popular use-cases for LLMs. The core features of chatbots are that they can have long-running, stateful conversations and can answer user questions using relevant information.\n",

docs/docs/use_cases/chatbots/quickstart.ipynb

@@ -15,6 +15,7 @@
    "source": [
     "# Quickstart\n",
     "\n",
+
     "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/chatbots.ipynb)"
    ]
   },

docs/docs/use_cases/chatbots/tool_usage.ipynb

@@ -15,6 +15,7 @@
    "source": [
     "# Tool usage\n",
     "\n",
+
     "This section will cover how to create conversational agents: chatbots that can interact with other systems and APIs using tools.\n",
     "\n",
     "Before reading this guide, we recommend you read both [the chatbot quickstart](/docs/use_cases/chatbots/quickstart) in this section and be familiar with [the documentation on agents](/docs/modules/agents/).\n",

docs/docs/use_cases/code_understanding.ipynb

@@ -14,7 +14,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/code_understanding.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/v0.1/docs/docs/use_cases/code_understanding.ipynb)\n",
     "\n",
     "## Use case\n",
     "\n",

docs/docs/use_cases/data_generation.ipynb

@@ -17,7 +17,7 @@
    "id": "aa3571cc",
    "metadata": {},
    "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/data_generation.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/v0.1/docs/docs/use_cases/data_generation.ipynb)\n",
     "\n",
     "## Use case\n",
     "\n",

docs/docs/use_cases/extraction/index.ipynb

@@ -18,6 +18,7 @@
    "source": [
     "## Overview\n",
     "\n",
+
     "Large Language Models (LLMs) are emerging as an extremely capable technology for powering information extraction applications.\n",
     "\n",
     "Classical solutions to information extraction rely on a combination of people, (many) hand-crafted rules (e.g., regular expressions), and custom fine-tuned ML models.\n",
@@ -56,9 +57,9 @@
     "\n",
     "Head to the [Guidelines](/docs/use_cases/extraction/guidelines) page to see a list of opinionated guidelines on how to get the best performance for extraction use cases.\n",
     "\n",
-    "## Use Case Accelerant\n",
+    "## Reference Application\n",
     "\n",
-    "[langchain-extract](https://github.com/langchain-ai/langchain-extract) is a starter repo that implements a simple web server for information extraction from text and files using LLMs. It is build using **FastAPI**, **LangChain** and **Postgresql**. Feel free to adapt it to your own use cases.\n",
+    "[langchain-extract](https://github.com/langchain-ai/langchain-extract) is a starter repo that implements a simple web server for information extraction from text and files using LLMs. It is built using **FastAPI**, **LangChain** and **Postgresql**. Feel free to adapt it to your own use cases.\n",
     "\n",
     "## Other Resources\n",
     "\n",

docs/docs/use_cases/graph/constructing.ipynb

@@ -65,7 +65,7 @@
    "metadata": {},
    "outputs": [
     {
-     "name": "stdout",
+     "name": "stdin",
      "output_type": "stream",
      "text": [
       " ········\n"
@@ -128,7 +128,7 @@
     "from langchain_experimental.graph_transformers import LLMGraphTransformer\n",
     "from langchain_openai import ChatOpenAI\n",
     "\n",
-    "llm = ChatOpenAI(temperature=0, model_name=\"gpt-4-0125-preview\")\n",
+    "llm = ChatOpenAI(temperature=0, model_name=\"gpt-4-turbo\")\n",
     "\n",
     "llm_transformer = LLMGraphTransformer(llm=llm)"
    ]
@@ -149,8 +149,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Nodes:[Node(id='Marie Curie', type='Person'), Node(id='Polish', type='Nationality'), Node(id='French', type='Nationality'), Node(id='Physicist', type='Occupation'), Node(id='Chemist', type='Occupation'), Node(id='Radioactivity', type='Field'), Node(id='Nobel Prize', type='Award'), Node(id='Pierre Curie', type='Person'), Node(id='University Of Paris', type='Organization')]\n",
-      "Relationships:[Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Polish', type='Nationality'), type='NATIONALITY'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='French', type='Nationality'), type='NATIONALITY'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Physicist', type='Occupation'), type='OCCUPATION'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Chemist', type='Occupation'), type='OCCUPATION'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Radioactivity', type='Field'), type='RESEARCH_FIELD'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Nobel Prize', type='Award'), type='AWARD_WINNER'), Relationship(source=Node(id='Pierre Curie', type='Person'), target=Node(id='Nobel Prize', type='Award'), type='AWARD_WINNER'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='University Of Paris', type='Organization'), type='PROFESSOR')]\n"
+      "Nodes:[Node(id='Marie Curie', type='Person'), Node(id='Pierre Curie', type='Person'), Node(id='University Of Paris', type='Organization')]\n",
+      "Relationships:[Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Pierre Curie', type='Person'), type='MARRIED'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='University Of Paris', type='Organization'), type='PROFESSOR')]\n"
      ]
     }
    ],
@@ -158,7 +158,7 @@
     "from langchain_core.documents import Document\n",
     "\n",
     "text = \"\"\"\n",
-    "Marie Curie, was a Polish and naturalised-French physicist and chemist who conducted pioneering research on radioactivity.\n",
+    "Marie Curie, born in 1867, was a Polish and naturalised-French physicist and chemist who conducted pioneering research on radioactivity.\n",
     "She was the first woman to win a Nobel Prize, the first person to win a Nobel Prize twice, and the only person to win a Nobel Prize in two scientific fields.\n",
     "Her husband, Pierre Curie, was a co-winner of her first Nobel Prize, making them the first-ever married couple to win the Nobel Prize and launching the Curie family legacy of five Nobel Prizes.\n",
     "She was, in 1906, the first woman to become a professor at the University of Paris.\n",
@@ -191,8 +191,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Nodes:[Node(id='Marie Curie', type='Person'), Node(id='Polish', type='Country'), Node(id='French', type='Country'), Node(id='Pierre Curie', type='Person'), Node(id='University Of Paris', type='Organization')]\n",
-      "Relationships:[Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Polish', type='Country'), type='NATIONALITY'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='French', type='Country'), type='NATIONALITY'), Relationship(source=Node(id='Pierre Curie', type='Person'), target=Node(id='Marie Curie', type='Person'), type='SPOUSE'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='University Of Paris', type='Organization'), type='WORKED_AT')]\n"
+      "Nodes:[Node(id='Marie Curie', type='Person'), Node(id='Pierre Curie', type='Person'), Node(id='University Of Paris', type='Organization')]\n",
+      "Relationships:[Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Pierre Curie', type='Person'), type='SPOUSE'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='University Of Paris', type='Organization'), type='WORKED_AT')]\n"
      ]
     }
    ],
@@ -218,6 +218,41 @@
     "![graph_construction2.png](../../../static/img/graph_construction2.png)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The `node_properties` parameter enables the extraction of node properties, allowing the creation of a more detailed graph.\n",
+    "When set to `True`, LLM autonomously identifies and extracts relevant node properties.\n",
+    "Conversely, if `node_properties` is defined as a list of strings, the LLM selectively retrieves only the specified properties from the text."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Nodes:[Node(id='Marie Curie', type='Person', properties={'born_year': '1867'}), Node(id='Pierre Curie', type='Person'), Node(id='University Of Paris', type='Organization')]\n",
+      "Relationships:[Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='Pierre Curie', type='Person'), type='SPOUSE'), Relationship(source=Node(id='Marie Curie', type='Person'), target=Node(id='University Of Paris', type='Organization'), type='WORKED_AT')]\n"
+     ]
+    }
+   ],
+   "source": [
+    "llm_transformer_props = LLMGraphTransformer(\n",
+    "    llm=llm,\n",
+    "    allowed_nodes=[\"Person\", \"Country\", \"Organization\"],\n",
+    "    allowed_relationships=[\"NATIONALITY\", \"LOCATED_IN\", \"WORKED_AT\", \"SPOUSE\"],\n",
+    "    node_properties=[\"born_year\"],\n",
+    ")\n",
+    "graph_documents_props = llm_transformer_props.convert_to_graph_documents(documents)\n",
+    "print(f\"Nodes:{graph_documents_props[0].nodes}\")\n",
+    "print(f\"Relationships:{graph_documents_props[0].relationships}\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -229,11 +264,11 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [],
    "source": [
-    "graph.add_graph_documents(graph_documents_filtered)"
+    "graph.add_graph_documents(graph_documents_props)"
    ]
   }
  ],

docs/docs/use_cases/graph/mapping.ipynb

@@ -28,18 +28,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "18294435-182d-48da-bcab-5b8945b6d9cf",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Note: you may need to restart the kernel to use updated packages.\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "%pip install --upgrade --quiet  langchain langchain-community langchain-openai neo4j"
    ]
@@ -166,20 +158,10 @@
    "execution_count": 5,
    "id": "e1a19424-6046-40c2-81d1-f3b88193a293",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/tomazbratanic/anaconda3/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The function `create_structured_output_chain` was deprecated in LangChain 0.1.1 and will be removed in 0.2.0. Use create_structured_output_runnable instead.\n",
-      "  warn_deprecated(\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from typing import List, Optional\n",
     "\n",
-    "from langchain.chains.openai_functions import create_structured_output_chain\n",
     "from langchain_core.prompts import ChatPromptTemplate\n",
     "from langchain_core.pydantic_v1 import BaseModel, Field\n",
     "from langchain_openai import ChatOpenAI\n",
@@ -211,7 +193,7 @@
     ")\n",
     "\n",
     "\n",
-    "entity_chain = create_structured_output_chain(Entities, llm, prompt)"
+    "entity_chain = prompt | llm.with_structured_output(Entities)"
    ]
   },
   {
@@ -231,8 +213,7 @@
     {
      "data": {
       "text/plain": [
-       "{'question': 'Who played in Casino movie?',\n",
-       " 'function': Entities(names=['Casino'])}"
+       "Entities(names=['Casino'])"
       ]
      },
      "execution_count": 6,
@@ -278,9 +259,9 @@
     "\"\"\"\n",
     "\n",
     "\n",
-    "def map_to_database(values):\n",
+    "def map_to_database(entities: Entities) -> Optional[str]:\n",
     "    result = \"\"\n",
-    "    for entity in values.names:\n",
+    "    for entity in entities.names:\n",
     "        response = graph.query(match_query, {\"value\": entity})\n",
     "        try:\n",
     "            result += f\"{entity} maps to {response[0]['result']} {response[0]['type']} in database\\n\"\n",
@@ -289,7 +270,7 @@
     "    return result\n",
     "\n",
     "\n",
-    "map_to_database(entities[\"function\"])"
+    "map_to_database(entities)"
    ]
   },
   {
@@ -334,7 +315,7 @@
     "cypher_response = (\n",
     "    RunnablePassthrough.assign(names=entity_chain)\n",
     "    | RunnablePassthrough.assign(\n",
-    "        entities_list=lambda x: map_to_database(x[\"names\"][\"function\"]),\n",
+    "        entities_list=lambda x: map_to_database(x[\"names\"]),\n",
     "        schema=lambda _: graph.get_schema,\n",
     "    )\n",
     "    | cypher_prompt\n",
@@ -429,7 +410,7 @@
     {
      "data": {
       "text/plain": [
-       "'Joe Pesci, Robert De Niro, Sharon Stone, and James Woods played in the movie \"Casino\".'"
+       "'Robert De Niro, James Woods, Joe Pesci, and Sharon Stone played in the movie \"Casino\".'"
       ]
      },
      "execution_count": 11,
@@ -466,7 +447,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.9.18"
   }
  },
  "nbformat": 4,

docs/docs/use_cases/question_answering/chat_history.ipynb

@@ -17,6 +17,7 @@
    "source": [
     "# Add chat history\n",
     "\n",
+
     "In many Q&A applications we want to allow the user to have a back-and-forth conversation, meaning the application needs some sort of \"memory\" of past questions and answers, and some logic for incorporating those into its current thinking.\n",
     "\n",
     "In this guide we focus on **adding logic for incorporating historical messages.** Further details on chat history management is [covered here](/docs/expression_language/how_to/message_history).\n",

docs/docs/use_cases/question_answering/hybrid.ipynb

@@ -0,0 +1,392 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "14d3fd06",
+   "metadata": {
+    "id": "14d3fd06"
+   },
+   "source": [
+    "# Hybrid Search\n",
+    "\n",
+    "The standard search in LangChain is done by vector similarity. However, a number of vectorstores implementations (Astra DB, ElasticSearch, Neo4J, AzureSearch, ...) also support more advanced search combining vector similarity search and other search techniques (full-text, BM25, and so on). This is generally referred to as \"Hybrid\" search.\n",
+    "\n",
+    "**Step 1: Make sure the vectorstore you are using supports hybrid search**\n",
+    "\n",
+    "At the moment, there is no unified way to perform hybrid search in LangChain. Each vectorstore may have their own way to do it. This is generally exposed as a keyword argument that is passed in during `similarity_search`. By reading the documentation or source code, figure out whether the vectorstore you are using supports hybrid search, and, if so, how to use it.\n",
+    "\n",
+    "**Step 2: Add that parameter as a configurable field for the chain**\n",
+    "\n",
+    "This will let you easily call the chain and configure any relevant flags at runtime. See [this documentation](/docs/expression_language/primitives/configure) for more information on configuration.\n",
+    "\n",
+    "**Step 3: Call the chain with that configurable field**\n",
+    "\n",
+    "Now, at runtime you can call this chain with configurable field.\n",
+    "\n",
+    "## Code Example\n",
+    "\n",
+    "Let's see a concrete example of what this looks like in code. We will use the Cassandra/CQL interface of Astra DB for this example.\n",
+    "\n",
+    "Install the following Python package:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c2efe35eea197769",
+   "metadata": {
+    "id": "c2efe35eea197769",
+    "outputId": "527275b4-076e-4b22-945c-e41a59188116"
+   },
+   "outputs": [],
+   "source": [
+    "!pip install \"cassio>=0.1.7\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b4ef96d44341cd84",
+   "metadata": {
+    "collapsed": false,
+    "id": "b4ef96d44341cd84"
+   },
+   "source": [
+    "Get the [connection secrets](https://docs.datastax.com/en/astra/astra-db-vector/get-started/quickstart.html).\n",
+    "\n",
+    "Initialize cassio:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cb2cef097277c32e",
+   "metadata": {
+    "id": "cb2cef097277c32e",
+    "outputId": "4c3d05a0-319a-44a0-8ec3-0a9c78453132"
+   },
+   "outputs": [],
+   "source": [
+    "import cassio\n",
+    "\n",
+    "cassio.init(\n",
+    "    database_id=\"Your database ID\",\n",
+    "    token=\"Your application token\",\n",
+    "    keyspace=\"Your key space\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e1e51444877f45eb",
+   "metadata": {
+    "collapsed": false,
+    "id": "e1e51444877f45eb"
+   },
+   "source": [
+    "Create the Cassandra VectorStore with a standard [index analyzer](https://docs.datastax.com/en/astra/astra-db-vector/cql/use-analyzers-with-cql.html). The index analyzer is needed to enable term matching."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7345de3c",
+   "metadata": {
+    "id": "7345de3c",
+    "outputId": "d38bcee0-0134-4ac6-8d35-afcce282481b"
+   },
+   "outputs": [],
+   "source": [
+    "from cassio.table.cql import STANDARD_ANALYZER\n",
+    "from langchain_community.vectorstores import Cassandra\n",
+    "from langchain_openai import OpenAIEmbeddings\n",
+    "\n",
+    "embeddings = OpenAIEmbeddings()\n",
+    "vectorstore = Cassandra(\n",
+    "    embedding=embeddings,\n",
+    "    table_name=\"test_hybrid\",\n",
+    "    body_index_options=[STANDARD_ANALYZER],\n",
+    "    session=None,\n",
+    "    keyspace=None,\n",
+    ")\n",
+    "\n",
+    "vectorstore.add_texts(\n",
+    "    [\n",
+    "        \"In 2023, I visited Paris\",\n",
+    "        \"In 2022, I visited New York\",\n",
+    "        \"In 2021, I visited New Orleans\",\n",
+    "    ]\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "73887f23bbab978c",
+   "metadata": {
+    "collapsed": false,
+    "id": "73887f23bbab978c"
+   },
+   "source": [
+    "If we do a standard similarity search, we get all the documents:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3c2a39fa",
+   "metadata": {
+    "id": "3c2a39fa",
+    "outputId": "5290085b-896c-4c81-9b40-c315331b7009"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='In 2022, I visited New York'),\n",
+       "Document(page_content='In 2023, I visited Paris'),\n",
+       "Document(page_content='In 2021, I visited New Orleans')]"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "vectorstore.as_retriever().invoke(\"What city did I visit last?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "78d4c3c79e67d8c3",
+   "metadata": {
+    "collapsed": false,
+    "id": "78d4c3c79e67d8c3"
+   },
+   "source": [
+    "The Astra DB vectorstore `body_search` argument can be used to filter the search on the term `new`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "56393baa",
+   "metadata": {
+    "id": "56393baa",
+    "outputId": "d1c939f3-342f-4df4-94a3-d25429b5a25e"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Document(page_content='In 2022, I visited New York'),\n",
+       "Document(page_content='In 2021, I visited New Orleans')]"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "vectorstore.as_retriever(search_kwargs={\"body_search\": \"new\"}).invoke(\n",
+    "    \"What city did I visit last?\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "88ae97ed",
+   "metadata": {
+    "id": "88ae97ed"
+   },
+   "source": [
+    "We can now create the chain that we will use to do question-answering over"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "62707b4f",
+   "metadata": {
+    "id": "62707b4f"
+   },
+   "outputs": [],
+   "source": [
+    "from langchain_core.output_parsers import StrOutputParser\n",
+    "from langchain_core.prompts import ChatPromptTemplate\n",
+    "from langchain_core.runnables import (\n",
+    "    ConfigurableField,\n",
+    "    RunnablePassthrough,\n",
+    ")\n",
+    "from langchain_openai import ChatOpenAI"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b6778ffa",
+   "metadata": {
+    "id": "b6778ffa"
+   },
+   "source": [
+    "This is basic question-answering chain set up."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "44a865f6",
+   "metadata": {
+    "id": "44a865f6"
+   },
+   "outputs": [],
+   "source": [
+    "template = \"\"\"Answer the question based only on the following context:\n",
+    "{context}\n",
+    "Question: {question}\n",
+    "\"\"\"\n",
+    "prompt = ChatPromptTemplate.from_template(template)\n",
+    "\n",
+    "model = ChatOpenAI()\n",
+    "\n",
+    "retriever = vectorstore.as_retriever()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "72125166",
+   "metadata": {
+    "id": "72125166"
+   },
+   "source": [
+    "Here we mark the retriever as having a configurable field. All vectorstore retrievers have `search_kwargs` as a field. This is just a dictionary, with vectorstore specific fields"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "babbadff",
+   "metadata": {
+    "id": "babbadff"
+   },
+   "outputs": [],
+   "source": [
+    "configurable_retriever = retriever.configurable_fields(\n",
+    "    search_kwargs=ConfigurableField(\n",
+    "        id=\"search_kwargs\",\n",
+    "        name=\"Search Kwargs\",\n",
+    "        description=\"The search kwargs to use\",\n",
+    "    )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2d481b70",
+   "metadata": {
+    "id": "2d481b70"
+   },
+   "source": [
+    "We can now create the chain using our configurable retriever"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "210b0446",
+   "metadata": {
+    "id": "210b0446"
+   },
+   "outputs": [],
+   "source": [
+    "chain = (\n",
+    "    {\"context\": configurable_retriever, \"question\": RunnablePassthrough()}\n",
+    "    | prompt\n",
+    "    | model\n",
+    "    | StrOutputParser()\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a38037b2",
+   "metadata": {
+    "id": "a38037b2",
+    "outputId": "1ea14996-5965-4a5e-9678-b9c35ce5c6de"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Paris"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke(\"What city did I visit last?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7f6458c3",
+   "metadata": {
+    "id": "7f6458c3"
+   },
+   "source": [
+    "We can now invoke the chain with configurable options. `search_kwargs` is the id of the configurable field. The value is the search kwargs to use for Astra DB."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9gYLqBTH8BFz",
+   "metadata": {
+    "id": "9gYLqBTH8BFz",
+    "outputId": "4358a2e6-f306-48f1-dd5c-781ac8a33e89"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "New York"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.invoke(\n",
+    "    \"What city did I visit last?\",\n",
+    "    config={\"configurable\": {\"search_kwargs\": {\"body_search\": \"new\"}}},\n",
+    ")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/docs/use_cases/question_answering/index.ipynb

@@ -80,6 +80,7 @@
     "- [Returning sources](/docs/use_cases/question_answering/sources): How to return the source documents used in a particular generation.\n",
     "- [Streaming](/docs/use_cases/question_answering/streaming): How to stream final answers as well as intermediate steps.\n",
     "- [Adding chat history](/docs/use_cases/question_answering/chat_history): How to add chat history to a Q&A app.\n",
+    "- [Hybrid search](/docs/use_cases/question_answering/hybrid): How to do hybrid search.\n",
     "- [Per-user retrieval](/docs/use_cases/question_answering/per_user): How to do retrieval when each user has their own private data.\n",
     "- [Using agents](/docs/use_cases/question_answering/conversational_retrieval_agents): How to use agents for Q&A.\n",
     "- [Using local models](/docs/use_cases/question_answering/local_retrieval_qa): How to use local models for Q&A."

docs/docs/use_cases/question_answering/quickstart.mdx

@@ -5,6 +5,7 @@ title: Quickstart
 
 # Quickstart
 
+
 LangChain has a number of components designed to help build
 question-answering applications, and RAG applications more generally. To
 familiarize ourselves with these, we’ll build a simple Q&A application

docs/docs/use_cases/sql/agents.ipynb

@@ -15,6 +15,7 @@
    "source": [
     "# Agents\n",
     "\n",
+
     "LangChain has a SQL Agent which provides a more flexible way of interacting with SQL Databases than a chain. The main advantages of using the SQL Agent are:\n",
     "\n",
     "- It can answer questions based on the databases' schema as well as on the databases' content (like describing a specific table).\n",
@@ -352,7 +353,7 @@
     "\n",
     "To optimize agent performance, we can provide a custom prompt with domain-specific knowledge. In this case we'll create a few shot prompt with an example selector, that will dynamically build the few shot prompt based on the user input. This will help the model make better queries by inserting relevant queries in the prompt that the model can use as reference.\n",
     "\n",
-    "First we need some user input <> SQL query examples:"
+    "First we need some user input \\<\\> SQL query examples:"
    ]
   },
   {
@@ -798,7 +799,7 @@
     "\n",
     "Under the hood, `create_sql_agent` is just passing in SQL tools to more generic agent constructors. To learn more about the built-in generic agent types as well as how to build custom agents, head to the [Agents Modules](/docs/modules/agents/).\n",
     "\n",
-    "The built-in `AgentExecutor` runs a simple Agent action -> Tool call -> Agent action... loop. To build more complex agent runtimes, head to the [LangGraph section](/docs/langgraph)."
+    "The built-in `AgentExecutor` runs a simple Agent action -> Tool call -> Agent action... loop. To build more complex agent runtimes, head to the [LangGraph section](https://langchain-ai.github.io/langgraph/)."
    ]
   }
  ],

docs/docs/use_cases/sql/index.ipynb

@@ -15,6 +15,7 @@
    "source": [
     "# SQL\n",
     "\n",
+
     "One of the most common types of databases that we can build Q&A systems for are SQL databases. LangChain comes with a number of built-in chains and agents that are compatible with any SQL dialect supported by SQLAlchemy (e.g., MySQL, PostgreSQL, Oracle SQL, Databricks, SQLite). They enable use cases such as:\n",
     "\n",
     "* Generating queries that will be run based on natural language questions,\n",

docs/docs/use_cases/sql/quickstart.ipynb

@@ -15,6 +15,7 @@
    "source": [
     "# Quickstart\n",
     "\n",
+
     "In this guide we'll go over the basic ways to create a Q&A chain and agent over a SQL database. These systems will allow us to ask a question about the data in a SQL database and get back a natural language answer. The main difference between the two is that our agent can query the database in a loop as many time as it needs to answer the question.\n",
     "\n",
     "## ⚠️ Security note ⚠️\n",

docs/docs/use_cases/summarization.ipynb

@@ -16,7 +16,8 @@
    "id": "cf13f702",
    "metadata": {},
    "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/summarization.ipynb)\n",
+
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/v0.1/docs/docs/use_cases/summarization.ipynb)\n",
     "\n",
     "## Use case\n",
     "\n",
@@ -589,9 +590,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "poetry-venv",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "poetry-venv"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -603,7 +604,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.10.5"
   }
  },
  "nbformat": 4,

docs/docs/use_cases/tagging.ipynb

@@ -16,7 +16,7 @@
    "id": "a0507a4b",
    "metadata": {},
    "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/tagging.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/v0.1/docs/docs/use_cases/tagging.ipynb)\n",
     "\n",
     "## Use case\n",
     "\n",

docs/docs/use_cases/web_scraping.ipynb

@@ -16,7 +16,7 @@
    "id": "6605e7f7",
    "metadata": {},
    "source": [
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/web_scraping.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain/blob/v0.1/docs/docs/use_cases/web_scraping.ipynb)\n",
     "\n",
     "## Use case\n",
     "\n",

docs/docusaurus.config.js

@@ -9,6 +9,8 @@ require("dotenv").config();
 const baseLightCodeBlockTheme = require("prism-react-renderer/themes/vsLight");
 const baseDarkCodeBlockTheme = require("prism-react-renderer/themes/vsDark");
 
+const baseUrl = "/v0.1/";
+
 /** @type {import('@docusaurus/types').Config} */
 const config = {
   title: "🦜️🔗 LangChain",
@@ -18,7 +20,7 @@ const config = {
   url: "https://python.langchain.com",
   // Set the /<baseUrl>/ pathname under which your site is served
   // For GitHub pages deployment, it is often '/<projectName>/'
-  baseUrl: "/",
+  baseUrl: baseUrl,
   trailingSlash: true,
   onBrokenLinks: "throw",
   onBrokenMarkdownLinks: "throw",
@@ -118,6 +120,13 @@ const config = {
   themeConfig:
     /** @type {import('@docusaurus/preset-classic').ThemeConfig} */
     ({
+      announcementBar: {
+        content:
+          'This is documentation for LangChain v0.1, which is no longer actively maintained. Check out the docs for the <a href="https://python.langchain.com/docs/introduction">latest version here</a>.',
+        backgroundColor: "#FF0000",
+        textColor: "#FFFFFF",
+        isCloseable: false,
+      },
       docs: {
         sidebar: {
           hideable: true,
@@ -165,7 +174,7 @@ const config = {
             position: "left",
           },
           {
-            href: "https://api.python.langchain.com",
+            href: "https://api.python.langchain.com/en/v0.1/",
             label: "API Reference",
             position: "left",
           },
@@ -205,6 +214,25 @@ const config = {
               },
             ]
           },
+          {
+            type: "dropdown",
+            label: "v0.1",
+            position: "right",
+            items: [
+              {
+                label: "Latest",
+                href: "https://python.langchain.com/docs/introduction"
+              },
+              {
+                label: "v0.2",
+                href: "https://python.langchain.com/v0.2/docs/introduction"
+              },
+              {
+                label: "v0.1",
+                href: "/docs/get_started/introduction"
+              }
+            ]
+          },
           {
             type: "dropdown",
             label: "🦜️🔗",
@@ -318,7 +346,7 @@ const config = {
     }),
 
   scripts: [
-    "/js/google_analytics.js",
+    baseUrl + "js/google_analytics.js",
     {
       src: "https://www.googletagmanager.com/gtag/js?id=G-9B66JQQH2F",
       async: true,

docs/package.json

@@ -5,7 +5,7 @@
   "scripts": {
     "docusaurus": "docusaurus",
     "start": "rm -rf ./docs/api && docusaurus start",
-    "build": "bash vercel_build.sh && rm -rf ./build && docusaurus build",
+    "build": "make vercel-build",
     "swizzle": "docusaurus swizzle",
     "deploy": "docusaurus deploy",
     "clear": "docusaurus clear",