diff --git a/docs/docs/integrations/chat/ai21.ipynb b/docs/docs/integrations/chat/ai21.ipynb index 1818fef0ab7..b519548521d 100644 --- a/docs/docs/integrations/chat/ai21.ipynb +++ b/docs/docs/integrations/chat/ai21.ipynb @@ -1,318 +1,316 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "4cebeec0", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: AI21 Labs\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatAI21\n", - "\n", - "## Overview\n", - "\n", - "This notebook covers how to get started with AI21 chat models.\n", - "Note that different chat models support different parameters. See the [AI21 documentation](https://docs.ai21.com/reference) to learn more about the parameters in your chosen model.\n", - "[See all AI21's LangChain components.](https://pypi.org/project/langchain-ai21/) \n", - "\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/__package_name_short_snake__) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatAI21](https://python.langchain.com/api_reference/ai21/chat_models/langchain_ai21.chat_models.ChatAI21.html#langchain_ai21.chat_models.ChatAI21) | [langchain-ai21](https://python.langchain.com/api_reference/ai21/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ai21?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ai21?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "\n", - "## Setup" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "### Credentials\n", - "\n", - "We'll need to get an [AI21 API key](https://docs.ai21.com/) and set the `AI21_API_KEY` environment variable:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "import os\n", - "from getpass import getpass\n", - "\n", - "if \"AI21_API_KEY\" not in os.environ:\n", - " os.environ[\"AI21_API_KEY\"] = getpass()" - ] - }, - { - "cell_type": "markdown", - "id": "f6844fff-3702-4489-ab74-732f69f3b9d7", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7c2e19d3-7c58-4470-9e1a-718b27a32056", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "98e22f31-8acc-42d6-916d-415d1263c56e", - "metadata": {}, - "source": [ - "### Installation" - ] - }, - { - "cell_type": "markdown", - "id": "f9699cd9-58f2-450e-aa64-799e66906c0f", - "metadata": {}, - "source": [ - "!pip install -qU langchain-ai21" - ] - }, - { - "cell_type": "markdown", - "id": "4828829d3da430ce", - "metadata": { - "collapsed": false, - "jupyter": { - "outputs_hidden": false + "cells": [ + { + "cell_type": "raw", + "id": "4cebeec0", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: AI21 Labs\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatAI21\n", + "\n", + "## Overview\n", + "\n", + "This notebook covers how to get started with AI21 chat models.\n", + "Note that different chat models support different parameters. See the [AI21 documentation](https://docs.ai21.com/reference) to learn more about the parameters in your chosen model.\n", + "[See all AI21's LangChain components.](https://pypi.org/project/langchain-ai21/)\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/__package_name_short_snake__) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatAI21](https://python.langchain.com/api_reference/ai21/chat_models/langchain_ai21.chat_models.ChatAI21.html#langchain_ai21.chat_models.ChatAI21) | [langchain-ai21](https://python.langchain.com/api_reference/ai21/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ai21?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ai21?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "\n", + "## Setup" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "### Credentials\n", + "\n", + "We'll need to get an [AI21 API key](https://docs.ai21.com/) and set the `AI21_API_KEY` environment variable:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import os\n", + "from getpass import getpass\n", + "\n", + "if \"AI21_API_KEY\" not in os.environ:\n", + " os.environ[\"AI21_API_KEY\"] = getpass()" + ] + }, + { + "cell_type": "markdown", + "id": "f6844fff-3702-4489-ab74-732f69f3b9d7", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7c2e19d3-7c58-4470-9e1a-718b27a32056", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "98e22f31-8acc-42d6-916d-415d1263c56e", + "metadata": {}, + "source": [ + "### Installation" + ] + }, + { + "cell_type": "markdown", + "id": "f9699cd9-58f2-450e-aa64-799e66906c0f", + "metadata": {}, + "source": [ + "!pip install -qU langchain-ai21" + ] + }, + { + "cell_type": "markdown", + "id": "4828829d3da430ce", + "metadata": { + "collapsed": false, + "jupyter": { + "outputs_hidden": false + } + }, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c40756fb-cbf8-4d44-a293-3989d707237e", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_ai21 import ChatAI21\n", + "\n", + "llm = ChatAI21(model=\"jamba-instruct\", temperature=0)" + ] + }, + { + "cell_type": "markdown", + "id": "2bdc5d68-2a19-495e-8c04-d11adc86d3ae", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "46b982dc-5d8a-46da-a711-81c03ccd6adc", + "metadata": {}, + "outputs": [], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "markdown", + "id": "10a30f84-b531-4fd5-8b5b-91512fbdc75b", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39353473fce5dd2e", + "metadata": { + "collapsed": false, + "jupyter": { + "outputs_hidden": false + } + }, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "39c0ccd229927eab", + "metadata": {}, + "source": "# Tool Calls / Function Calling" + }, + { + "cell_type": "markdown", + "id": "2bf6b40be07fe2d4", + "metadata": {}, + "source": "This example shows how to use tool calling with AI21 models:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a181a28df77120fb", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from getpass import getpass\n", + "\n", + "from langchain_ai21.chat_models import ChatAI21\n", + "from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage\n", + "from langchain_core.tools import tool\n", + "from langchain_core.utils.function_calling import convert_to_openai_tool\n", + "\n", + "if \"AI21_API_KEY\" not in os.environ:\n", + " os.environ[\"AI21_API_KEY\"] = getpass()\n", + "\n", + "\n", + "@tool\n", + "def get_weather(location: str, date: str) -> str:\n", + " \"\"\"“Provide the weather for the specified location on the given date.”\"\"\"\n", + " if location == \"New York\" and date == \"2024-12-05\":\n", + " return \"25 celsius\"\n", + " elif location == \"New York\" and date == \"2024-12-06\":\n", + " return \"27 celsius\"\n", + " elif location == \"London\" and date == \"2024-12-05\":\n", + " return \"22 celsius\"\n", + " return \"32 celsius\"\n", + "\n", + "\n", + "llm = ChatAI21(model=\"jamba-1.5-mini\")\n", + "\n", + "llm_with_tools = llm.bind_tools([convert_to_openai_tool(get_weather)])\n", + "\n", + "chat_messages = [\n", + " SystemMessage(\n", + " content=\"You are a helpful assistant. You can use the provided tools \"\n", + " \"to assist with various tasks and provide accurate information\"\n", + " )\n", + "]\n", + "\n", + "human_messages = [\n", + " HumanMessage(\n", + " content=\"What is the forecast for the weather in New York on December 5, 2024?\"\n", + " ),\n", + " HumanMessage(content=\"And what about the 2024-12-06?\"),\n", + " HumanMessage(content=\"OK, thank you.\"),\n", + " HumanMessage(content=\"What is the expected weather in London on December 5, 2024?\"),\n", + "]\n", + "\n", + "\n", + "for human_message in human_messages:\n", + " print(f\"User: {human_message.content}\")\n", + " chat_messages.append(human_message)\n", + " response = llm_with_tools.invoke(chat_messages)\n", + " chat_messages.append(response)\n", + " if response.tool_calls:\n", + " tool_call = response.tool_calls[0]\n", + " if tool_call[\"name\"] == \"get_weather\":\n", + " weather = get_weather.invoke(\n", + " {\n", + " \"location\": tool_call[\"args\"][\"location\"],\n", + " \"date\": tool_call[\"args\"][\"date\"],\n", + " }\n", + " )\n", + " chat_messages.append(\n", + " ToolMessage(content=weather, tool_call_id=tool_call[\"id\"])\n", + " )\n", + " llm_answer = llm_with_tools.invoke(chat_messages)\n", + " print(f\"Assistant: {llm_answer.content}\")\n", + " else:\n", + " print(f\"Assistant: {response.content}\")" + ] + }, + { + "cell_type": "markdown", + "id": "e79de691-9dd6-4697-b57e-59a4a3cc073a", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatAI21 features and configurations head to the API reference: https://python.langchain.com/api_reference/ai21/chat_models/langchain_ai21.chat_models.ChatAI21.html" + ] } - }, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c40756fb-cbf8-4d44-a293-3989d707237e", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_ai21 import ChatAI21\n", - "\n", - "llm = ChatAI21(model=\"jamba-instruct\", temperature=0)" - ] - }, - { - "cell_type": "markdown", - "id": "2bdc5d68-2a19-495e-8c04-d11adc86d3ae", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "46b982dc-5d8a-46da-a711-81c03ccd6adc", - "metadata": {}, - "outputs": [], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "markdown", - "id": "10a30f84-b531-4fd5-8b5b-91512fbdc75b", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "39353473fce5dd2e", - "metadata": { - "collapsed": false, - "jupyter": { - "outputs_hidden": false + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" } - }, - "outputs": [], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] }, - { - "cell_type": "markdown", - "id": "39c0ccd229927eab", - "metadata": {}, - "source": "# Tool Calls / Function Calling" - }, - { - "cell_type": "markdown", - "id": "2bf6b40be07fe2d4", - "metadata": {}, - "source": "This example shows how to use tool calling with AI21 models:" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a181a28df77120fb", - "metadata": {}, - "outputs": [], - "source": [ - "import os\n", - "from getpass import getpass\n", - "\n", - "from langchain_ai21.chat_models import ChatAI21\n", - "from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage\n", - "from langchain_core.tools import tool\n", - "from langchain_core.utils.function_calling import convert_to_openai_tool\n", - "\n", - "if \"AI21_API_KEY\" not in os.environ:\n", - " os.environ[\"AI21_API_KEY\"] = getpass()\n", - "\n", - "\n", - "@tool\n", - "def get_weather(location: str, date: str) -> str:\n", - " \"\"\"“Provide the weather for the specified location on the given date.”\"\"\"\n", - " if location == \"New York\" and date == \"2024-12-05\":\n", - " return \"25 celsius\"\n", - " elif location == \"New York\" and date == \"2024-12-06\":\n", - " return \"27 celsius\"\n", - " elif location == \"London\" and date == \"2024-12-05\":\n", - " return \"22 celsius\"\n", - " return \"32 celsius\"\n", - "\n", - "\n", - "llm = ChatAI21(model=\"jamba-1.5-mini\")\n", - "\n", - "llm_with_tools = llm.bind_tools([convert_to_openai_tool(get_weather)])\n", - "\n", - "chat_messages = [\n", - " SystemMessage(\n", - " content=\"You are a helpful assistant. You can use the provided tools \"\n", - " \"to assist with various tasks and provide accurate information\"\n", - " )\n", - "]\n", - "\n", - "human_messages = [\n", - " HumanMessage(\n", - " content=\"What is the forecast for the weather in New York on December 5, 2024?\"\n", - " ),\n", - " HumanMessage(content=\"And what about the 2024-12-06?\"),\n", - " HumanMessage(content=\"OK, thank you.\"),\n", - " HumanMessage(content=\"What is the expected weather in London on December 5, 2024?\"),\n", - "]\n", - "\n", - "\n", - "for human_message in human_messages:\n", - " print(f\"User: {human_message.content}\")\n", - " chat_messages.append(human_message)\n", - " response = llm_with_tools.invoke(chat_messages)\n", - " chat_messages.append(response)\n", - " if response.tool_calls:\n", - " tool_call = response.tool_calls[0]\n", - " if tool_call[\"name\"] == \"get_weather\":\n", - " weather = get_weather.invoke(\n", - " {\n", - " \"location\": tool_call[\"args\"][\"location\"],\n", - " \"date\": tool_call[\"args\"][\"date\"],\n", - " }\n", - " )\n", - " chat_messages.append(\n", - " ToolMessage(content=weather, tool_call_id=tool_call[\"id\"])\n", - " )\n", - " llm_answer = llm_with_tools.invoke(chat_messages)\n", - " print(f\"Assistant: {llm_answer.content}\")\n", - " else:\n", - " print(f\"Assistant: {response.content}\")" - ] - }, - { - "cell_type": "markdown", - "id": "e79de691-9dd6-4697-b57e-59a4a3cc073a", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatAI21 features and configurations head to the API reference: https://python.langchain.com/api_reference/ai21/chat_models/langchain_ai21.chat_models.ChatAI21.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/anthropic.ipynb b/docs/docs/integrations/chat/anthropic.ipynb index d5970040d4d..f055d408593 100644 --- a/docs/docs/integrations/chat/anthropic.ipynb +++ b/docs/docs/integrations/chat/anthropic.ipynb @@ -1,993 +1,991 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Anthropic\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatAnthropic\n", - "\n", - "This notebook provides a quick overview for getting started with Anthropic [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatAnthropic features and configurations head to the [API reference](https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html).\n", - "\n", - "Anthropic has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in the [Anthropic docs](https://docs.anthropic.com/en/docs/models-overview).\n", - "\n", - "\n", - ":::info AWS Bedrock and Google VertexAI\n", - "\n", - "Note that certain Anthropic models can also be accessed via AWS Bedrock and Google VertexAI. See the [ChatBedrock](/docs/integrations/chat/bedrock/) and [ChatVertexAI](/docs/integrations/chat/google_vertex_ai_palm/) integrations to use Anthropic models via these services.\n", - "\n", - ":::\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/anthropic) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatAnthropic](https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html) | [langchain-anthropic](https://python.langchain.com/api_reference/anthropic/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-anthropic?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-anthropic?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access Anthropic models you'll need to create an Anthropic account, get an API key, and install the `langchain-anthropic` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to https://console.anthropic.com/ to sign up for Anthropic and generate an API key. Once you've done this set the ANTHROPIC_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"ANTHROPIC_API_KEY\" not in os.environ:\n", - " os.environ[\"ANTHROPIC_API_KEY\"] = getpass.getpass(\"Enter your Anthropic API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Anthropic integration lives in the `langchain-anthropic` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-anthropic" - ] - }, - { - "cell_type": "markdown", - "id": "fe4993ad-4a9b-4021-8ebd-f0fbbc739f49", - "metadata": {}, - "source": [ - ":::info This guide requires ``langchain-anthropic>=0.3.10``\n", - "\n", - ":::" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_anthropic import ChatAnthropic\n", - "\n", - "llm = ChatAnthropic(\n", - " model=\"claude-3-5-sonnet-20240620\",\n", - " temperature=0,\n", - " max_tokens=1024,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation\n" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore la programmation.\", response_metadata={'id': 'msg_018Nnu76krRPq8HvgKLW4F8T', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 29, 'output_tokens': 11}}, id='run-57e9295f-db8a-48dc-9619-babd2bedd891-0', usage_metadata={'input_tokens': 29, 'output_tokens': 11, 'total_tokens': 40})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Anthropic\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore la programmation.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=\"Here's the German translation:\\n\\nIch liebe Programmieren.\", response_metadata={'id': 'msg_01GhkRtQZUkA5Ge9hqmD8HGY', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 23, 'output_tokens': 18}}, id='run-da5906b4-b200-4e08-b81a-64d4453643b6-0', usage_metadata={'input_tokens': 23, 'output_tokens': 18, 'total_tokens': 41})" - ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## Content blocks\n", - "\n", - "Content from a single Anthropic AI message can either be a single string or a **list of content blocks**. For example when an Anthropic model invokes a tool, the tool invocation is part of the message content (as well as being exposed in the standardized `AIMessage.tool_calls`):" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "4a374a24-2534-4e6f-825b-30fab7bbe0cb", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[{'text': \"To answer this question, we'll need to check the current weather in both Los Angeles (LA) and New York (NY). I'll use the GetWeather function to retrieve this information for both cities.\",\n", - " 'type': 'text'},\n", - " {'id': 'toolu_01Ddzj5PkuZkrjF4tafzu54A',\n", - " 'input': {'location': 'Los Angeles, CA'},\n", - " 'name': 'GetWeather',\n", - " 'type': 'tool_use'},\n", - " {'id': 'toolu_012kz4qHZQqD4qg8sFPeKqpP',\n", - " 'input': {'location': 'New York, NY'},\n", - " 'name': 'GetWeather',\n", - " 'type': 'tool_use'}]" - ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from pydantic import BaseModel, Field\n", - "\n", - "\n", - "class GetWeather(BaseModel):\n", - " \"\"\"Get the current weather in a given location\"\"\"\n", - "\n", - " location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n", - "\n", - "\n", - "llm_with_tools = llm.bind_tools([GetWeather])\n", - "ai_msg = llm_with_tools.invoke(\"Which city is hotter today: LA or NY?\")\n", - "ai_msg.content" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "6b4a1ead-952c-489f-a8d4-355d3fb55f3f", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[{'name': 'GetWeather',\n", - " 'args': {'location': 'Los Angeles, CA'},\n", - " 'id': 'toolu_01Ddzj5PkuZkrjF4tafzu54A'},\n", - " {'name': 'GetWeather',\n", - " 'args': {'location': 'New York, NY'},\n", - " 'id': 'toolu_012kz4qHZQqD4qg8sFPeKqpP'}]" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "ai_msg.tool_calls" - ] - }, - { - "cell_type": "markdown", - "id": "6e36d25c-f358-49e5-aefa-b99fbd3fec6b", - "metadata": {}, - "source": [ - "## Extended thinking\n", - "\n", - "Claude 3.7 Sonnet supports an [extended thinking](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking) feature, which will output the step-by-step reasoning process that led to its final answer.\n", - "\n", - "To use it, specify the `thinking` parameter when initializing `ChatAnthropic`. It can also be passed in as a kwarg during invocation.\n", - "\n", - "You will need to specify a token budget to use this feature. See usage example below:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "a34cf93b-8522-43a6-a3f3-8a189ddf54a7", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[\n", - " {\n", - " \"signature\": \"ErUBCkYIARgCIkCx7bIPj35jGPHpoVOB2y5hvPF8MN4lVK75CYGftmVNlI4axz2+bBbSexofWsN1O/prwNv8yPXnIXQmwT6zrJsKEgwJzvks0yVRZtaGBScaDOm9xcpOxbuhku1zViIw9WDgil/KZL8DsqWrhVpC6TzM0RQNCcsHcmgmyxbgG9g8PR0eJGLxCcGoEw8zMQu1Kh1hQ1/03hZ2JCOgigpByR9aNPTwwpl64fQUe6WwIw==\",\n", - " \"thinking\": \"To find the cube root of 50.653, I need to find the value of $x$ such that $x^3 = 50.653$.\\n\\nI can try to estimate this first. \\n$3^3 = 27$\\n$4^3 = 64$\\n\\nSo the cube root of 50.653 will be somewhere between 3 and 4, but closer to 4.\\n\\nLet me try to compute this more precisely. I can use the cube root function:\\n\\ncube root of 50.653 = 50.653^(1/3)\\n\\nLet me calculate this:\\n50.653^(1/3) \\u2248 3.6998\\n\\nLet me verify:\\n3.6998^3 \\u2248 50.6533\\n\\nThat's very close to 50.653, so I'm confident that the cube root of 50.653 is approximately 3.6998.\\n\\nActually, let me compute this more precisely:\\n50.653^(1/3) \\u2248 3.69981\\n\\nLet me verify once more:\\n3.69981^3 \\u2248 50.652998\\n\\nThat's extremely close to 50.653, so I'll say that the cube root of 50.653 is approximately 3.69981.\",\n", - " \"type\": \"thinking\"\n", - " },\n", - " {\n", - " \"text\": \"The cube root of 50.653 is approximately 3.6998.\\n\\nTo verify: 3.6998\\u00b3 = 50.6530, which is very close to our original number.\",\n", - " \"type\": \"text\"\n", - " }\n", - "]\n" - ] - } - ], - "source": [ - "import json\n", - "\n", - "from langchain_anthropic import ChatAnthropic\n", - "\n", - "llm = ChatAnthropic(\n", - " model=\"claude-3-7-sonnet-latest\",\n", - " max_tokens=5000,\n", - " thinking={\"type\": \"enabled\", \"budget_tokens\": 2000},\n", - ")\n", - "\n", - "response = llm.invoke(\"What is the cube root of 50.653?\")\n", - "print(json.dumps(response.content, indent=2))" - ] - }, - { - "cell_type": "markdown", - "id": "34349dfe-5d81-4887-a4f4-cd01e9587cdc", - "metadata": {}, - "source": [ - "## Prompt caching\n", - "\n", - "Anthropic supports [caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) of [elements of your prompts](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#what-can-be-cached), including messages, tool definitions, tool results, images and documents. This allows you to re-use large documents, instructions, [few-shot documents](/docs/concepts/few_shot_prompting/), and other data to reduce latency and costs.\n", - "\n", - "To enable caching on an element of a prompt, mark its associated content block using the `cache_control` key. See examples below:\n", - "\n", - "### Messages" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "babb44a5-33f7-4200-9dfc-be867cf2c217", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "First invocation:\n", - "{'cache_read': 0, 'cache_creation': 1458}\n", - "\n", - "Second:\n", - "{'cache_read': 1458, 'cache_creation': 0}\n" - ] - } - ], - "source": [ - "import requests\n", - "from langchain_anthropic import ChatAnthropic\n", - "\n", - "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", - "\n", - "# Pull LangChain readme\n", - "get_response = requests.get(\n", - " \"https://raw.githubusercontent.com/langchain-ai/langchain/master/README.md\"\n", - ")\n", - "readme = get_response.text\n", - "\n", - "messages = [\n", - " {\n", - " \"role\": \"system\",\n", - " \"content\": [\n", - " {\n", - " \"type\": \"text\",\n", - " \"text\": \"You are a technology expert.\",\n", - " },\n", - " {\n", - " \"type\": \"text\",\n", - " \"text\": f\"{readme}\",\n", - " # highlight-next-line\n", - " \"cache_control\": {\"type\": \"ephemeral\"},\n", - " },\n", - " ],\n", - " },\n", - " {\n", - " \"role\": \"user\",\n", - " \"content\": \"What's LangChain, according to its README?\",\n", - " },\n", - "]\n", - "\n", - "response_1 = llm.invoke(messages)\n", - "response_2 = llm.invoke(messages)\n", - "\n", - "usage_1 = response_1.usage_metadata[\"input_token_details\"]\n", - "usage_2 = response_2.usage_metadata[\"input_token_details\"]\n", - "\n", - "print(f\"First invocation:\\n{usage_1}\")\n", - "print(f\"\\nSecond:\\n{usage_2}\")" - ] - }, - { - "cell_type": "markdown", - "id": "141ce9c5-012d-4502-9d61-4a413b5d959a", - "metadata": {}, - "source": [ - "### Tools" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "1de82015-810f-4ed4-a08b-9866ea8746ce", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "First invocation:\n", - "{'cache_read': 0, 'cache_creation': 1809}\n", - "\n", - "Second:\n", - "{'cache_read': 1809, 'cache_creation': 0}\n" - ] - } - ], - "source": [ - "from langchain_anthropic import convert_to_anthropic_tool\n", - "from langchain_core.tools import tool\n", - "\n", - "# For demonstration purposes, we artificially expand the\n", - "# tool description.\n", - "description = (\n", - " f\"Get the weather at a location. By the way, check out this readme: {readme}\"\n", - ")\n", - "\n", - "\n", - "@tool(description=description)\n", - "def get_weather(location: str) -> str:\n", - " return \"It's sunny.\"\n", - "\n", - "\n", - "# Enable caching on the tool\n", - "# highlight-start\n", - "weather_tool = convert_to_anthropic_tool(get_weather)\n", - "weather_tool[\"cache_control\"] = {\"type\": \"ephemeral\"}\n", - "# highlight-end\n", - "\n", - "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", - "llm_with_tools = llm.bind_tools([weather_tool])\n", - "query = \"What's the weather in San Francisco?\"\n", - "\n", - "response_1 = llm_with_tools.invoke(query)\n", - "response_2 = llm_with_tools.invoke(query)\n", - "\n", - "usage_1 = response_1.usage_metadata[\"input_token_details\"]\n", - "usage_2 = response_2.usage_metadata[\"input_token_details\"]\n", - "\n", - "print(f\"First invocation:\\n{usage_1}\")\n", - "print(f\"\\nSecond:\\n{usage_2}\")" - ] - }, - { - "cell_type": "markdown", - "id": "a763830d-82cb-448a-ab30-f561522791b9", - "metadata": {}, - "source": [ - "### Incremental caching in conversational applications\n", - "\n", - "Prompt caching can be used in [multi-turn conversations](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#continuing-a-multi-turn-conversation) to maintain context from earlier messages without redundant processing.\n", - "\n", - "We can enable incremental caching by marking the final message with `cache_control`. Claude will automatically use the longest previously-cached prefix for follow-up messages.\n", - "\n", - "Below, we implement a simple chatbot that incorporates this feature. We follow the LangChain [chatbot tutorial](/docs/tutorials/chatbot/), but add a custom [reducer](https://langchain-ai.github.io/langgraph/concepts/low_level/#reducers) that automatically marks the last content block in each user message with `cache_control`. See below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "07fde4db-344c-49bc-a5b4-99e2d20fb394", - "metadata": {}, - "outputs": [], - "source": [ - "import requests\n", - "from langchain_anthropic import ChatAnthropic\n", - "from langgraph.checkpoint.memory import MemorySaver\n", - "from langgraph.graph import START, StateGraph, add_messages\n", - "from typing_extensions import Annotated, TypedDict\n", - "\n", - "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", - "\n", - "# Pull LangChain readme\n", - "get_response = requests.get(\n", - " \"https://raw.githubusercontent.com/langchain-ai/langchain/master/README.md\"\n", - ")\n", - "readme = get_response.text\n", - "\n", - "\n", - "def messages_reducer(left: list, right: list) -> list:\n", - " # Update last user message\n", - " for i in range(len(right) - 1, -1, -1):\n", - " if right[i].type == \"human\":\n", - " right[i].content[-1][\"cache_control\"] = {\"type\": \"ephemeral\"}\n", - " break\n", - "\n", - " return add_messages(left, right)\n", - "\n", - "\n", - "class State(TypedDict):\n", - " messages: Annotated[list, messages_reducer]\n", - "\n", - "\n", - "workflow = StateGraph(state_schema=State)\n", - "\n", - "\n", - "# Define the function that calls the model\n", - "def call_model(state: State):\n", - " response = llm.invoke(state[\"messages\"])\n", - " return {\"messages\": [response]}\n", - "\n", - "\n", - "# Define the (single) node in the graph\n", - "workflow.add_edge(START, \"model\")\n", - "workflow.add_node(\"model\", call_model)\n", - "\n", - "# Add memory\n", - "memory = MemorySaver()\n", - "app = workflow.compile(checkpointer=memory)" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "40013035-eb22-4327-8aaf-1ee974d9ff46", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "Hello, Bob! It's nice to meet you. How are you doing today? Is there something I can help you with?\n", - "\n", - "{'cache_read': 0, 'cache_creation': 0}\n" - ] - } - ], - "source": [ - "from langchain_core.messages import HumanMessage\n", - "\n", - "config = {\"configurable\": {\"thread_id\": \"abc123\"}}\n", - "\n", - "query = \"Hi! I'm Bob.\"\n", - "\n", - "input_message = HumanMessage([{\"type\": \"text\", \"text\": query}])\n", - "output = app.invoke({\"messages\": [input_message]}, config)\n", - "output[\"messages\"][-1].pretty_print()\n", - "print(f'\\n{output[\"messages\"][-1].usage_metadata[\"input_token_details\"]}')" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "22371f68-7913-4c4f-ab4a-2b4265095469", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "I can see you've shared the README from the LangChain GitHub repository. This is the documentation for LangChain, which is a popular framework for building applications powered by Large Language Models (LLMs). Here's a summary of what the README contains:\n", - "\n", - "LangChain is:\n", - "- A framework for developing LLM-powered applications\n", - "- Helps chain together components and integrations to simplify AI application development\n", - "- Provides a standard interface for models, embeddings, vector stores, etc.\n", - "\n", - "Key features/benefits:\n", - "- Real-time data augmentation (connect LLMs to diverse data sources)\n", - "- Model interoperability (swap models easily as needed)\n", - "- Large ecosystem of integrations\n", - "\n", - "The LangChain ecosystem includes:\n", - "- LangSmith - For evaluations and observability\n", - "- LangGraph - For building complex agents with customizable architecture\n", - "- LangGraph Platform - For deployment and scaling of agents\n", - "\n", - "The README also mentions installation instructions (`pip install -U langchain`) and links to various resources including tutorials, how-to guides, conceptual guides, and API references.\n", - "\n", - "Is there anything specific about LangChain you'd like to know more about, Bob?\n", - "\n", - "{'cache_read': 0, 'cache_creation': 1498}\n" - ] - } - ], - "source": [ - "query = f\"Check out this readme: {readme}\"\n", - "\n", - "input_message = HumanMessage([{\"type\": \"text\", \"text\": query}])\n", - "output = app.invoke({\"messages\": [input_message]}, config)\n", - "output[\"messages\"][-1].pretty_print()\n", - "print(f'\\n{output[\"messages\"][-1].usage_metadata[\"input_token_details\"]}')" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "0e6798fc-8a80-4324-b4e3-f18706256c61", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "Your name is Bob. You introduced yourself at the beginning of our conversation.\n", - "\n", - "{'cache_read': 1498, 'cache_creation': 269}\n" - ] - } - ], - "source": [ - "query = \"What was my name again?\"\n", - "\n", - "input_message = HumanMessage([{\"type\": \"text\", \"text\": query}])\n", - "output = app.invoke({\"messages\": [input_message]}, config)\n", - "output[\"messages\"][-1].pretty_print()\n", - "print(f'\\n{output[\"messages\"][-1].usage_metadata[\"input_token_details\"]}')" - ] - }, - { - "cell_type": "markdown", - "id": "aa4b3647-c672-4782-a88c-a55fd3bf969f", - "metadata": {}, - "source": [ - "In the [LangSmith trace](https://smith.langchain.com/public/4d0584d8-5f9e-4b91-8704-93ba2ccf416a/r), toggling \"raw output\" will show exactly what messages are sent to the chat model, including `cache_control` keys." - ] - }, - { - "cell_type": "markdown", - "id": "029009f2-2795-418b-b5fc-fb996c6fe99e", - "metadata": {}, - "source": [ - "## Token-efficient tool use\n", - "\n", - "Anthropic supports a (beta) [token-efficient tool use](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/token-efficient-tool-use) feature. To use it, specify the relevant beta-headers when instantiating the model." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "206cff65-33b8-4a88-9b1a-050b4d57772a", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[{'name': 'get_weather', 'args': {'location': 'San Francisco'}, 'id': 'toolu_01EoeE1qYaePcmNbUvMsWtmA', 'type': 'tool_call'}]\n", - "\n", - "Total tokens: 408\n" - ] - } - ], - "source": [ - "from langchain_anthropic import ChatAnthropic\n", - "from langchain_core.tools import tool\n", - "\n", - "llm = ChatAnthropic(\n", - " model=\"claude-3-7-sonnet-20250219\",\n", - " temperature=0,\n", - " # highlight-start\n", - " model_kwargs={\n", - " \"extra_headers\": {\"anthropic-beta\": \"token-efficient-tools-2025-02-19\"}\n", - " },\n", - " # highlight-end\n", - ")\n", - "\n", - "\n", - "@tool\n", - "def get_weather(location: str) -> str:\n", - " \"\"\"Get the weather at a location.\"\"\"\n", - " return \"It's sunny.\"\n", - "\n", - "\n", - "llm_with_tools = llm.bind_tools([get_weather])\n", - "response = llm_with_tools.invoke(\"What's the weather in San Francisco?\")\n", - "print(response.tool_calls)\n", - "print(f'\\nTotal tokens: {response.usage_metadata[\"total_tokens\"]}')" - ] - }, - { - "cell_type": "markdown", - "id": "301d372f-4dec-43e6-b58c-eee25633e1a6", - "metadata": {}, - "source": [ - "## Citations\n", - "\n", - "Anthropic supports a [citations](https://docs.anthropic.com/en/docs/build-with-claude/citations) feature that lets Claude attach context to its answers based on source documents supplied by the user. When [document content blocks](https://docs.anthropic.com/en/docs/build-with-claude/citations#document-types) with `\"citations\": {\"enabled\": True}` are included in a query, Claude may generate citations in its response.\n", - "\n", - "### Simple example\n", - "\n", - "In this example we pass a [plain text document](https://docs.anthropic.com/en/docs/build-with-claude/citations#plain-text-documents). In the background, Claude [automatically chunks](https://docs.anthropic.com/en/docs/build-with-claude/citations#plain-text-documents) the input text into sentences, which are used when generating citations." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "e5370e6e-5a9a-4546-848b-5f5bf313c3e7", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[{'text': 'Based on the document, ', 'type': 'text'},\n", - " {'text': 'the grass is green',\n", - " 'type': 'text',\n", - " 'citations': [{'type': 'char_location',\n", - " 'cited_text': 'The grass is green. ',\n", - " 'document_index': 0,\n", - " 'document_title': 'My Document',\n", - " 'start_char_index': 0,\n", - " 'end_char_index': 20}]},\n", - " {'text': ', and ', 'type': 'text'},\n", - " {'text': 'the sky is blue',\n", - " 'type': 'text',\n", - " 'citations': [{'type': 'char_location',\n", - " 'cited_text': 'The sky is blue.',\n", - " 'document_index': 0,\n", - " 'document_title': 'My Document',\n", - " 'start_char_index': 20,\n", - " 'end_char_index': 36}]},\n", - " {'text': '.', 'type': 'text'}]" - ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_anthropic import ChatAnthropic\n", - "\n", - "llm = ChatAnthropic(model=\"claude-3-5-haiku-latest\")\n", - "\n", - "messages = [\n", - " {\n", - " \"role\": \"user\",\n", - " \"content\": [\n", - " {\n", - " \"type\": \"document\",\n", - " \"source\": {\n", - " \"type\": \"text\",\n", - " \"media_type\": \"text/plain\",\n", - " \"data\": \"The grass is green. The sky is blue.\",\n", - " },\n", - " \"title\": \"My Document\",\n", - " \"context\": \"This is a trustworthy document.\",\n", - " \"citations\": {\"enabled\": True},\n", - " },\n", - " {\"type\": \"text\", \"text\": \"What color is the grass and sky?\"},\n", - " ],\n", - " }\n", - "]\n", - "response = llm.invoke(messages)\n", - "response.content" - ] - }, - { - "cell_type": "markdown", - "id": "69956596-0e6c-492b-934d-c08ed3c9de9a", - "metadata": {}, - "source": [ - "### Using with text splitters\n", - "\n", - "Anthropic also lets you specify your own splits using [custom document](https://docs.anthropic.com/en/docs/build-with-claude/citations#custom-content-documents) types. LangChain [text splitters](/docs/concepts/text_splitters/) can be used to generate meaningful splits for this purpose. See the below example, where we split the LangChain README (a markdown document) and pass it to Claude as context:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "04cc2841-7987-47a5-906c-09ea7fa28323", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[{'text': \"You can find LangChain's tutorials at https://python.langchain.com/docs/tutorials/\\n\\nThe tutorials section is recommended for those looking to build something specific or who prefer a hands-on learning approach. It's considered the best place to get started with LangChain.\",\n", - " 'type': 'text',\n", - " 'citations': [{'type': 'content_block_location',\n", - " 'cited_text': \"[Tutorials](https://python.langchain.com/docs/tutorials/):If you're looking to build something specific orare more of a hands-on learner, check out ourtutorials. This is the best place to get started.\",\n", - " 'document_index': 0,\n", - " 'document_title': None,\n", - " 'start_block_index': 243,\n", - " 'end_block_index': 248}]}]" - ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "import requests\n", - "from langchain_anthropic import ChatAnthropic\n", - "from langchain_text_splitters import MarkdownTextSplitter\n", - "\n", - "\n", - "def format_to_anthropic_documents(documents: list[str]):\n", - " return {\n", - " \"type\": \"document\",\n", - " \"source\": {\n", - " \"type\": \"content\",\n", - " \"content\": [{\"type\": \"text\", \"text\": document} for document in documents],\n", - " },\n", - " \"citations\": {\"enabled\": True},\n", - " }\n", - "\n", - "\n", - "# Pull readme\n", - "get_response = requests.get(\n", - " \"https://raw.githubusercontent.com/langchain-ai/langchain/master/README.md\"\n", - ")\n", - "readme = get_response.text\n", - "\n", - "# Split into chunks\n", - "splitter = MarkdownTextSplitter(\n", - " chunk_overlap=0,\n", - " chunk_size=50,\n", - ")\n", - "documents = splitter.split_text(readme)\n", - "\n", - "# Construct message\n", - "message = {\n", - " \"role\": \"user\",\n", - " \"content\": [\n", - " format_to_anthropic_documents(documents),\n", - " {\"type\": \"text\", \"text\": \"Give me a link to LangChain's tutorials.\"},\n", - " ],\n", - "}\n", - "\n", - "# Query LLM\n", - "llm = ChatAnthropic(model=\"claude-3-5-haiku-latest\")\n", - "response = llm.invoke([message])\n", - "\n", - "response.content" - ] - }, - { - "cell_type": "markdown", - "id": "cbfec7a9-d9df-4d12-844e-d922456dd9bf", - "metadata": {}, - "source": [ - "## Built-in tools\n", - "\n", - "Anthropic supports a variety of [built-in tools](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/text-editor-tool), which can be bound to the model in the [usual way](/docs/how_to/tool_calling/). Claude will generate tool calls adhering to its internal schema for the tool:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "30a0af36-2327-4b1d-9ba5-e47cb72db0be", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "I'd be happy to help you fix the syntax error in your primes.py file. First, let's look at the current content of the file to identify the error.\n" - ] }, { - "data": { - "text/plain": [ - "[{'name': 'str_replace_editor',\n", - " 'args': {'command': 'view', 'path': '/repo/primes.py'},\n", - " 'id': 'toolu_01VdNgt1YV7kGfj9LFLm6HyQ',\n", - " 'type': 'tool_call'}]" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatAnthropic\n", + "\n", + "This notebook provides a quick overview for getting started with Anthropic [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatAnthropic features and configurations head to the [API reference](https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html).\n", + "\n", + "Anthropic has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in the [Anthropic docs](https://docs.anthropic.com/en/docs/models-overview).\n", + "\n", + "\n", + ":::info AWS Bedrock and Google VertexAI\n", + "\n", + "Note that certain Anthropic models can also be accessed via AWS Bedrock and Google VertexAI. See the [ChatBedrock](/docs/integrations/chat/bedrock/) and [ChatVertexAI](/docs/integrations/chat/google_vertex_ai_palm/) integrations to use Anthropic models via these services.\n", + "\n", + ":::\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/anthropic) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatAnthropic](https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html) | [langchain-anthropic](https://python.langchain.com/api_reference/anthropic/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-anthropic?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-anthropic?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "To access Anthropic models you'll need to create an Anthropic account, get an API key, and install the `langchain-anthropic` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to https://console.anthropic.com/ to sign up for Anthropic and generate an API key. Once you've done this set the ANTHROPIC_API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"ANTHROPIC_API_KEY\" not in os.environ:\n", + " os.environ[\"ANTHROPIC_API_KEY\"] = getpass.getpass(\"Enter your Anthropic API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Anthropic integration lives in the `langchain-anthropic` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-anthropic" + ] + }, + { + "cell_type": "markdown", + "id": "fe4993ad-4a9b-4021-8ebd-f0fbbc739f49", + "metadata": {}, + "source": [ + ":::info This guide requires ``langchain-anthropic>=0.3.10``\n", + "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_anthropic import ChatAnthropic\n", + "\n", + "llm = ChatAnthropic(\n", + " model=\"claude-3-5-sonnet-20240620\",\n", + " temperature=0,\n", + " max_tokens=1024,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore la programmation.\", response_metadata={'id': 'msg_018Nnu76krRPq8HvgKLW4F8T', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 29, 'output_tokens': 11}}, id='run-57e9295f-db8a-48dc-9619-babd2bedd891-0', usage_metadata={'input_tokens': 29, 'output_tokens': 11, 'total_tokens': 40})" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore la programmation.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"Here's the German translation:\\n\\nIch liebe Programmieren.\", response_metadata={'id': 'msg_01GhkRtQZUkA5Ge9hqmD8HGY', 'model': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 23, 'output_tokens': 18}}, id='run-da5906b4-b200-4e08-b81a-64d4453643b6-0', usage_metadata={'input_tokens': 23, 'output_tokens': 18, 'total_tokens': 41})" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## Content blocks\n", + "\n", + "Content from a single Anthropic AI message can either be a single string or a **list of content blocks**. For example when an Anthropic model invokes a tool, the tool invocation is part of the message content (as well as being exposed in the standardized `AIMessage.tool_calls`):" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "4a374a24-2534-4e6f-825b-30fab7bbe0cb", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'text': \"To answer this question, we'll need to check the current weather in both Los Angeles (LA) and New York (NY). I'll use the GetWeather function to retrieve this information for both cities.\",\n", + " 'type': 'text'},\n", + " {'id': 'toolu_01Ddzj5PkuZkrjF4tafzu54A',\n", + " 'input': {'location': 'Los Angeles, CA'},\n", + " 'name': 'GetWeather',\n", + " 'type': 'tool_use'},\n", + " {'id': 'toolu_012kz4qHZQqD4qg8sFPeKqpP',\n", + " 'input': {'location': 'New York, NY'},\n", + " 'name': 'GetWeather',\n", + " 'type': 'tool_use'}]" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from pydantic import BaseModel, Field\n", + "\n", + "\n", + "class GetWeather(BaseModel):\n", + " \"\"\"Get the current weather in a given location\"\"\"\n", + "\n", + " location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n", + "\n", + "\n", + "llm_with_tools = llm.bind_tools([GetWeather])\n", + "ai_msg = llm_with_tools.invoke(\"Which city is hotter today: LA or NY?\")\n", + "ai_msg.content" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "6b4a1ead-952c-489f-a8d4-355d3fb55f3f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'name': 'GetWeather',\n", + " 'args': {'location': 'Los Angeles, CA'},\n", + " 'id': 'toolu_01Ddzj5PkuZkrjF4tafzu54A'},\n", + " {'name': 'GetWeather',\n", + " 'args': {'location': 'New York, NY'},\n", + " 'id': 'toolu_012kz4qHZQqD4qg8sFPeKqpP'}]" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ai_msg.tool_calls" + ] + }, + { + "cell_type": "markdown", + "id": "6e36d25c-f358-49e5-aefa-b99fbd3fec6b", + "metadata": {}, + "source": [ + "## Extended thinking\n", + "\n", + "Claude 3.7 Sonnet supports an [extended thinking](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking) feature, which will output the step-by-step reasoning process that led to its final answer.\n", + "\n", + "To use it, specify the `thinking` parameter when initializing `ChatAnthropic`. It can also be passed in as a kwarg during invocation.\n", + "\n", + "You will need to specify a token budget to use this feature. See usage example below:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "a34cf93b-8522-43a6-a3f3-8a189ddf54a7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[\n", + " {\n", + " \"signature\": \"ErUBCkYIARgCIkCx7bIPj35jGPHpoVOB2y5hvPF8MN4lVK75CYGftmVNlI4axz2+bBbSexofWsN1O/prwNv8yPXnIXQmwT6zrJsKEgwJzvks0yVRZtaGBScaDOm9xcpOxbuhku1zViIw9WDgil/KZL8DsqWrhVpC6TzM0RQNCcsHcmgmyxbgG9g8PR0eJGLxCcGoEw8zMQu1Kh1hQ1/03hZ2JCOgigpByR9aNPTwwpl64fQUe6WwIw==\",\n", + " \"thinking\": \"To find the cube root of 50.653, I need to find the value of $x$ such that $x^3 = 50.653$.\\n\\nI can try to estimate this first. \\n$3^3 = 27$\\n$4^3 = 64$\\n\\nSo the cube root of 50.653 will be somewhere between 3 and 4, but closer to 4.\\n\\nLet me try to compute this more precisely. I can use the cube root function:\\n\\ncube root of 50.653 = 50.653^(1/3)\\n\\nLet me calculate this:\\n50.653^(1/3) \\u2248 3.6998\\n\\nLet me verify:\\n3.6998^3 \\u2248 50.6533\\n\\nThat's very close to 50.653, so I'm confident that the cube root of 50.653 is approximately 3.6998.\\n\\nActually, let me compute this more precisely:\\n50.653^(1/3) \\u2248 3.69981\\n\\nLet me verify once more:\\n3.69981^3 \\u2248 50.652998\\n\\nThat's extremely close to 50.653, so I'll say that the cube root of 50.653 is approximately 3.69981.\",\n", + " \"type\": \"thinking\"\n", + " },\n", + " {\n", + " \"text\": \"The cube root of 50.653 is approximately 3.6998.\\n\\nTo verify: 3.6998\\u00b3 = 50.6530, which is very close to our original number.\",\n", + " \"type\": \"text\"\n", + " }\n", + "]\n" + ] + } + ], + "source": [ + "import json\n", + "\n", + "from langchain_anthropic import ChatAnthropic\n", + "\n", + "llm = ChatAnthropic(\n", + " model=\"claude-3-7-sonnet-latest\",\n", + " max_tokens=5000,\n", + " thinking={\"type\": \"enabled\", \"budget_tokens\": 2000},\n", + ")\n", + "\n", + "response = llm.invoke(\"What is the cube root of 50.653?\")\n", + "print(json.dumps(response.content, indent=2))" + ] + }, + { + "cell_type": "markdown", + "id": "34349dfe-5d81-4887-a4f4-cd01e9587cdc", + "metadata": {}, + "source": [ + "## Prompt caching\n", + "\n", + "Anthropic supports [caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) of [elements of your prompts](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#what-can-be-cached), including messages, tool definitions, tool results, images and documents. This allows you to re-use large documents, instructions, [few-shot documents](/docs/concepts/few_shot_prompting/), and other data to reduce latency and costs.\n", + "\n", + "To enable caching on an element of a prompt, mark its associated content block using the `cache_control` key. See examples below:\n", + "\n", + "### Messages" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "babb44a5-33f7-4200-9dfc-be867cf2c217", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "First invocation:\n", + "{'cache_read': 0, 'cache_creation': 1458}\n", + "\n", + "Second:\n", + "{'cache_read': 1458, 'cache_creation': 0}\n" + ] + } + ], + "source": [ + "import requests\n", + "from langchain_anthropic import ChatAnthropic\n", + "\n", + "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", + "\n", + "# Pull LangChain readme\n", + "get_response = requests.get(\n", + " \"https://raw.githubusercontent.com/langchain-ai/langchain/master/README.md\"\n", + ")\n", + "readme = get_response.text\n", + "\n", + "messages = [\n", + " {\n", + " \"role\": \"system\",\n", + " \"content\": [\n", + " {\n", + " \"type\": \"text\",\n", + " \"text\": \"You are a technology expert.\",\n", + " },\n", + " {\n", + " \"type\": \"text\",\n", + " \"text\": f\"{readme}\",\n", + " # highlight-next-line\n", + " \"cache_control\": {\"type\": \"ephemeral\"},\n", + " },\n", + " ],\n", + " },\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": \"What's LangChain, according to its README?\",\n", + " },\n", + "]\n", + "\n", + "response_1 = llm.invoke(messages)\n", + "response_2 = llm.invoke(messages)\n", + "\n", + "usage_1 = response_1.usage_metadata[\"input_token_details\"]\n", + "usage_2 = response_2.usage_metadata[\"input_token_details\"]\n", + "\n", + "print(f\"First invocation:\\n{usage_1}\")\n", + "print(f\"\\nSecond:\\n{usage_2}\")" + ] + }, + { + "cell_type": "markdown", + "id": "141ce9c5-012d-4502-9d61-4a413b5d959a", + "metadata": {}, + "source": [ + "### Tools" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "1de82015-810f-4ed4-a08b-9866ea8746ce", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "First invocation:\n", + "{'cache_read': 0, 'cache_creation': 1809}\n", + "\n", + "Second:\n", + "{'cache_read': 1809, 'cache_creation': 0}\n" + ] + } + ], + "source": [ + "from langchain_anthropic import convert_to_anthropic_tool\n", + "from langchain_core.tools import tool\n", + "\n", + "# For demonstration purposes, we artificially expand the\n", + "# tool description.\n", + "description = (\n", + " f\"Get the weather at a location. By the way, check out this readme: {readme}\"\n", + ")\n", + "\n", + "\n", + "@tool(description=description)\n", + "def get_weather(location: str) -> str:\n", + " return \"It's sunny.\"\n", + "\n", + "\n", + "# Enable caching on the tool\n", + "# highlight-start\n", + "weather_tool = convert_to_anthropic_tool(get_weather)\n", + "weather_tool[\"cache_control\"] = {\"type\": \"ephemeral\"}\n", + "# highlight-end\n", + "\n", + "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", + "llm_with_tools = llm.bind_tools([weather_tool])\n", + "query = \"What's the weather in San Francisco?\"\n", + "\n", + "response_1 = llm_with_tools.invoke(query)\n", + "response_2 = llm_with_tools.invoke(query)\n", + "\n", + "usage_1 = response_1.usage_metadata[\"input_token_details\"]\n", + "usage_2 = response_2.usage_metadata[\"input_token_details\"]\n", + "\n", + "print(f\"First invocation:\\n{usage_1}\")\n", + "print(f\"\\nSecond:\\n{usage_2}\")" + ] + }, + { + "cell_type": "markdown", + "id": "a763830d-82cb-448a-ab30-f561522791b9", + "metadata": {}, + "source": [ + "### Incremental caching in conversational applications\n", + "\n", + "Prompt caching can be used in [multi-turn conversations](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#continuing-a-multi-turn-conversation) to maintain context from earlier messages without redundant processing.\n", + "\n", + "We can enable incremental caching by marking the final message with `cache_control`. Claude will automatically use the longest previously-cached prefix for follow-up messages.\n", + "\n", + "Below, we implement a simple chatbot that incorporates this feature. We follow the LangChain [chatbot tutorial](/docs/tutorials/chatbot/), but add a custom [reducer](https://langchain-ai.github.io/langgraph/concepts/low_level/#reducers) that automatically marks the last content block in each user message with `cache_control`. See below:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "07fde4db-344c-49bc-a5b4-99e2d20fb394", + "metadata": {}, + "outputs": [], + "source": [ + "import requests\n", + "from langchain_anthropic import ChatAnthropic\n", + "from langgraph.checkpoint.memory import MemorySaver\n", + "from langgraph.graph import START, StateGraph, add_messages\n", + "from typing_extensions import Annotated, TypedDict\n", + "\n", + "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", + "\n", + "# Pull LangChain readme\n", + "get_response = requests.get(\n", + " \"https://raw.githubusercontent.com/langchain-ai/langchain/master/README.md\"\n", + ")\n", + "readme = get_response.text\n", + "\n", + "\n", + "def messages_reducer(left: list, right: list) -> list:\n", + " # Update last user message\n", + " for i in range(len(right) - 1, -1, -1):\n", + " if right[i].type == \"human\":\n", + " right[i].content[-1][\"cache_control\"] = {\"type\": \"ephemeral\"}\n", + " break\n", + "\n", + " return add_messages(left, right)\n", + "\n", + "\n", + "class State(TypedDict):\n", + " messages: Annotated[list, messages_reducer]\n", + "\n", + "\n", + "workflow = StateGraph(state_schema=State)\n", + "\n", + "\n", + "# Define the function that calls the model\n", + "def call_model(state: State):\n", + " response = llm.invoke(state[\"messages\"])\n", + " return {\"messages\": [response]}\n", + "\n", + "\n", + "# Define the (single) node in the graph\n", + "workflow.add_edge(START, \"model\")\n", + "workflow.add_node(\"model\", call_model)\n", + "\n", + "# Add memory\n", + "memory = MemorySaver()\n", + "app = workflow.compile(checkpointer=memory)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "40013035-eb22-4327-8aaf-1ee974d9ff46", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "Hello, Bob! It's nice to meet you. How are you doing today? Is there something I can help you with?\n", + "\n", + "{'cache_read': 0, 'cache_creation': 0}\n" + ] + } + ], + "source": [ + "from langchain_core.messages import HumanMessage\n", + "\n", + "config = {\"configurable\": {\"thread_id\": \"abc123\"}}\n", + "\n", + "query = \"Hi! I'm Bob.\"\n", + "\n", + "input_message = HumanMessage([{\"type\": \"text\", \"text\": query}])\n", + "output = app.invoke({\"messages\": [input_message]}, config)\n", + "output[\"messages\"][-1].pretty_print()\n", + "print(f'\\n{output[\"messages\"][-1].usage_metadata[\"input_token_details\"]}')" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "22371f68-7913-4c4f-ab4a-2b4265095469", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "I can see you've shared the README from the LangChain GitHub repository. This is the documentation for LangChain, which is a popular framework for building applications powered by Large Language Models (LLMs). Here's a summary of what the README contains:\n", + "\n", + "LangChain is:\n", + "- A framework for developing LLM-powered applications\n", + "- Helps chain together components and integrations to simplify AI application development\n", + "- Provides a standard interface for models, embeddings, vector stores, etc.\n", + "\n", + "Key features/benefits:\n", + "- Real-time data augmentation (connect LLMs to diverse data sources)\n", + "- Model interoperability (swap models easily as needed)\n", + "- Large ecosystem of integrations\n", + "\n", + "The LangChain ecosystem includes:\n", + "- LangSmith - For evaluations and observability\n", + "- LangGraph - For building complex agents with customizable architecture\n", + "- LangGraph Platform - For deployment and scaling of agents\n", + "\n", + "The README also mentions installation instructions (`pip install -U langchain`) and links to various resources including tutorials, how-to guides, conceptual guides, and API references.\n", + "\n", + "Is there anything specific about LangChain you'd like to know more about, Bob?\n", + "\n", + "{'cache_read': 0, 'cache_creation': 1498}\n" + ] + } + ], + "source": [ + "query = f\"Check out this readme: {readme}\"\n", + "\n", + "input_message = HumanMessage([{\"type\": \"text\", \"text\": query}])\n", + "output = app.invoke({\"messages\": [input_message]}, config)\n", + "output[\"messages\"][-1].pretty_print()\n", + "print(f'\\n{output[\"messages\"][-1].usage_metadata[\"input_token_details\"]}')" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "0e6798fc-8a80-4324-b4e3-f18706256c61", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "Your name is Bob. You introduced yourself at the beginning of our conversation.\n", + "\n", + "{'cache_read': 1498, 'cache_creation': 269}\n" + ] + } + ], + "source": [ + "query = \"What was my name again?\"\n", + "\n", + "input_message = HumanMessage([{\"type\": \"text\", \"text\": query}])\n", + "output = app.invoke({\"messages\": [input_message]}, config)\n", + "output[\"messages\"][-1].pretty_print()\n", + "print(f'\\n{output[\"messages\"][-1].usage_metadata[\"input_token_details\"]}')" + ] + }, + { + "cell_type": "markdown", + "id": "aa4b3647-c672-4782-a88c-a55fd3bf969f", + "metadata": {}, + "source": [ + "In the [LangSmith trace](https://smith.langchain.com/public/4d0584d8-5f9e-4b91-8704-93ba2ccf416a/r), toggling \"raw output\" will show exactly what messages are sent to the chat model, including `cache_control` keys." + ] + }, + { + "cell_type": "markdown", + "id": "029009f2-2795-418b-b5fc-fb996c6fe99e", + "metadata": {}, + "source": [ + "## Token-efficient tool use\n", + "\n", + "Anthropic supports a (beta) [token-efficient tool use](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/token-efficient-tool-use) feature. To use it, specify the relevant beta-headers when instantiating the model." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "206cff65-33b8-4a88-9b1a-050b4d57772a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[{'name': 'get_weather', 'args': {'location': 'San Francisco'}, 'id': 'toolu_01EoeE1qYaePcmNbUvMsWtmA', 'type': 'tool_call'}]\n", + "\n", + "Total tokens: 408\n" + ] + } + ], + "source": [ + "from langchain_anthropic import ChatAnthropic\n", + "from langchain_core.tools import tool\n", + "\n", + "llm = ChatAnthropic(\n", + " model=\"claude-3-7-sonnet-20250219\",\n", + " temperature=0,\n", + " # highlight-start\n", + " model_kwargs={\n", + " \"extra_headers\": {\"anthropic-beta\": \"token-efficient-tools-2025-02-19\"}\n", + " },\n", + " # highlight-end\n", + ")\n", + "\n", + "\n", + "@tool\n", + "def get_weather(location: str) -> str:\n", + " \"\"\"Get the weather at a location.\"\"\"\n", + " return \"It's sunny.\"\n", + "\n", + "\n", + "llm_with_tools = llm.bind_tools([get_weather])\n", + "response = llm_with_tools.invoke(\"What's the weather in San Francisco?\")\n", + "print(response.tool_calls)\n", + "print(f'\\nTotal tokens: {response.usage_metadata[\"total_tokens\"]}')" + ] + }, + { + "cell_type": "markdown", + "id": "301d372f-4dec-43e6-b58c-eee25633e1a6", + "metadata": {}, + "source": [ + "## Citations\n", + "\n", + "Anthropic supports a [citations](https://docs.anthropic.com/en/docs/build-with-claude/citations) feature that lets Claude attach context to its answers based on source documents supplied by the user. When [document content blocks](https://docs.anthropic.com/en/docs/build-with-claude/citations#document-types) with `\"citations\": {\"enabled\": True}` are included in a query, Claude may generate citations in its response.\n", + "\n", + "### Simple example\n", + "\n", + "In this example we pass a [plain text document](https://docs.anthropic.com/en/docs/build-with-claude/citations#plain-text-documents). In the background, Claude [automatically chunks](https://docs.anthropic.com/en/docs/build-with-claude/citations#plain-text-documents) the input text into sentences, which are used when generating citations." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "e5370e6e-5a9a-4546-848b-5f5bf313c3e7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'text': 'Based on the document, ', 'type': 'text'},\n", + " {'text': 'the grass is green',\n", + " 'type': 'text',\n", + " 'citations': [{'type': 'char_location',\n", + " 'cited_text': 'The grass is green. ',\n", + " 'document_index': 0,\n", + " 'document_title': 'My Document',\n", + " 'start_char_index': 0,\n", + " 'end_char_index': 20}]},\n", + " {'text': ', and ', 'type': 'text'},\n", + " {'text': 'the sky is blue',\n", + " 'type': 'text',\n", + " 'citations': [{'type': 'char_location',\n", + " 'cited_text': 'The sky is blue.',\n", + " 'document_index': 0,\n", + " 'document_title': 'My Document',\n", + " 'start_char_index': 20,\n", + " 'end_char_index': 36}]},\n", + " {'text': '.', 'type': 'text'}]" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_anthropic import ChatAnthropic\n", + "\n", + "llm = ChatAnthropic(model=\"claude-3-5-haiku-latest\")\n", + "\n", + "messages = [\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": [\n", + " {\n", + " \"type\": \"document\",\n", + " \"source\": {\n", + " \"type\": \"text\",\n", + " \"media_type\": \"text/plain\",\n", + " \"data\": \"The grass is green. The sky is blue.\",\n", + " },\n", + " \"title\": \"My Document\",\n", + " \"context\": \"This is a trustworthy document.\",\n", + " \"citations\": {\"enabled\": True},\n", + " },\n", + " {\"type\": \"text\", \"text\": \"What color is the grass and sky?\"},\n", + " ],\n", + " }\n", + "]\n", + "response = llm.invoke(messages)\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "69956596-0e6c-492b-934d-c08ed3c9de9a", + "metadata": {}, + "source": [ + "### Using with text splitters\n", + "\n", + "Anthropic also lets you specify your own splits using [custom document](https://docs.anthropic.com/en/docs/build-with-claude/citations#custom-content-documents) types. LangChain [text splitters](/docs/concepts/text_splitters/) can be used to generate meaningful splits for this purpose. See the below example, where we split the LangChain README (a markdown document) and pass it to Claude as context:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "04cc2841-7987-47a5-906c-09ea7fa28323", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'text': \"You can find LangChain's tutorials at https://python.langchain.com/docs/tutorials/\\n\\nThe tutorials section is recommended for those looking to build something specific or who prefer a hands-on learning approach. It's considered the best place to get started with LangChain.\",\n", + " 'type': 'text',\n", + " 'citations': [{'type': 'content_block_location',\n", + " 'cited_text': \"[Tutorials](https://python.langchain.com/docs/tutorials/):If you're looking to build something specific orare more of a hands-on learner, check out ourtutorials. This is the best place to get started.\",\n", + " 'document_index': 0,\n", + " 'document_title': None,\n", + " 'start_block_index': 243,\n", + " 'end_block_index': 248}]}]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import requests\n", + "from langchain_anthropic import ChatAnthropic\n", + "from langchain_text_splitters import MarkdownTextSplitter\n", + "\n", + "\n", + "def format_to_anthropic_documents(documents: list[str]):\n", + " return {\n", + " \"type\": \"document\",\n", + " \"source\": {\n", + " \"type\": \"content\",\n", + " \"content\": [{\"type\": \"text\", \"text\": document} for document in documents],\n", + " },\n", + " \"citations\": {\"enabled\": True},\n", + " }\n", + "\n", + "\n", + "# Pull readme\n", + "get_response = requests.get(\n", + " \"https://raw.githubusercontent.com/langchain-ai/langchain/master/README.md\"\n", + ")\n", + "readme = get_response.text\n", + "\n", + "# Split into chunks\n", + "splitter = MarkdownTextSplitter(\n", + " chunk_overlap=0,\n", + " chunk_size=50,\n", + ")\n", + "documents = splitter.split_text(readme)\n", + "\n", + "# Construct message\n", + "message = {\n", + " \"role\": \"user\",\n", + " \"content\": [\n", + " format_to_anthropic_documents(documents),\n", + " {\"type\": \"text\", \"text\": \"Give me a link to LangChain's tutorials.\"},\n", + " ],\n", + "}\n", + "\n", + "# Query LLM\n", + "llm = ChatAnthropic(model=\"claude-3-5-haiku-latest\")\n", + "response = llm.invoke([message])\n", + "\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "cbfec7a9-d9df-4d12-844e-d922456dd9bf", + "metadata": {}, + "source": [ + "## Built-in tools\n", + "\n", + "Anthropic supports a variety of [built-in tools](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/text-editor-tool), which can be bound to the model in the [usual way](/docs/how_to/tool_calling/). Claude will generate tool calls adhering to its internal schema for the tool:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "30a0af36-2327-4b1d-9ba5-e47cb72db0be", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "I'd be happy to help you fix the syntax error in your primes.py file. First, let's look at the current content of the file to identify the error.\n" + ] + }, + { + "data": { + "text/plain": [ + "[{'name': 'str_replace_editor',\n", + " 'args': {'command': 'view', 'path': '/repo/primes.py'},\n", + " 'id': 'toolu_01VdNgt1YV7kGfj9LFLm6HyQ',\n", + " 'type': 'tool_call'}]" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_anthropic import ChatAnthropic\n", + "\n", + "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", + "\n", + "tool = {\"type\": \"text_editor_20250124\", \"name\": \"str_replace_editor\"}\n", + "llm_with_tools = llm.bind_tools([tool])\n", + "\n", + "response = llm_with_tools.invoke(\n", + " \"There's a syntax error in my primes.py file. Can you help me fix it?\"\n", + ")\n", + "print(response.text())\n", + "response.tool_calls" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatAnthropic features and configurations head to the API reference: https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html" ] - }, - "execution_count": 1, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_anthropic import ChatAnthropic\n", - "\n", - "llm = ChatAnthropic(model=\"claude-3-7-sonnet-20250219\")\n", - "\n", - "tool = {\"type\": \"text_editor_20250124\", \"name\": \"str_replace_editor\"}\n", - "llm_with_tools = llm.bind_tools([tool])\n", - "\n", - "response = llm_with_tools.invoke(\n", - " \"There's a syntax error in my primes.py file. Can you help me fix it?\"\n", - ")\n", - "print(response.text())\n", - "response.tool_calls" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatAnthropic features and configurations head to the API reference: https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/azure_chat_openai.ipynb b/docs/docs/integrations/chat/azure_chat_openai.ipynb index 4552a5bb59b..af54a69404a 100644 --- a/docs/docs/integrations/chat/azure_chat_openai.ipynb +++ b/docs/docs/integrations/chat/azure_chat_openai.ipynb @@ -1,349 +1,347 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Azure OpenAI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# AzureChatOpenAI\n", - "\n", - "This guide will help you get started with AzureOpenAI [chat models](/docs/concepts/chat_models). For detailed documentation of all AzureChatOpenAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html).\n", - "\n", - "Azure OpenAI has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).\n", - "\n", - ":::info Azure OpenAI vs OpenAI\n", - "\n", - "Azure OpenAI refers to OpenAI models hosted on the [Microsoft Azure platform](https://azure.microsoft.com/en-us/products/ai-services/openai-service). OpenAI also provides its own model APIs. To access OpenAI services directly, use the [ChatOpenAI integration](/docs/integrations/chat/openai/).\n", - "\n", - ":::\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/azure) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [AzureChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html) | [langchain-openai](https://python.langchain.com/api_reference/openai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-openai?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | \n", - "\n", - "## Setup\n", - "\n", - "To access AzureOpenAI models you'll need to create an Azure account, create a deployment of an Azure OpenAI model, get the name and endpoint for your deployment, get an Azure OpenAI API key, and install the `langchain-openai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line%2Cpython-new&pivots=programming-language-python) to create your deployment and generate an API key. Once you've done this set the AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT environment variables:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"AZURE_OPENAI_API_KEY\" not in os.environ:\n", - " os.environ[\"AZURE_OPENAI_API_KEY\"] = getpass.getpass(\n", - " \"Enter your AzureOpenAI API key: \"\n", - " )\n", - "os.environ[\"AZURE_OPENAI_ENDPOINT\"] = \"https://YOUR-ENDPOINT.openai.azure.com/\"" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain AzureOpenAI integration lives in the `langchain-openai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-openai" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions.\n", - "- Replace `azure_deployment` with the name of your deployment,\n", - "- You can find the latest supported `api_version` here: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_openai import AzureChatOpenAI\n", - "\n", - "llm = AzureChatOpenAI(\n", - " azure_deployment=\"gpt-35-turbo\", # or your deployment\n", - " api_version=\"2023-06-01-preview\", # or your api version\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 31, 'total_tokens': 39}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-bea4b46c-e3e1-4495-9d3a-698370ad963d-0', usage_metadata={'input_tokens': 31, 'output_tokens': 8, 'total_tokens': 39})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Azure OpenAI\n", + "---" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore la programmation.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 26, 'total_tokens': 32}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-cbc44038-09d3-40d4-9da2-c5910ee636ca-0', usage_metadata={'input_tokens': 26, 'output_tokens': 6, 'total_tokens': 32})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# AzureChatOpenAI\n", + "\n", + "This guide will help you get started with AzureOpenAI [chat models](/docs/concepts/chat_models). For detailed documentation of all AzureChatOpenAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html).\n", + "\n", + "Azure OpenAI has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).\n", + "\n", + ":::info Azure OpenAI vs OpenAI\n", + "\n", + "Azure OpenAI refers to OpenAI models hosted on the [Microsoft Azure platform](https://azure.microsoft.com/en-us/products/ai-services/openai-service). OpenAI also provides its own model APIs. To access OpenAI services directly, use the [ChatOpenAI integration](/docs/integrations/chat/openai/).\n", + "\n", + ":::\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/azure) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [AzureChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html) | [langchain-openai](https://python.langchain.com/api_reference/openai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-openai?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |\n", + "\n", + "## Setup\n", + "\n", + "To access AzureOpenAI models you'll need to create an Azure account, create a deployment of an Azure OpenAI model, get the name and endpoint for your deployment, get an Azure OpenAI API key, and install the `langchain-openai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to the [Azure docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line%2Cpython-new&pivots=programming-language-python) to create your deployment and generate an API key. Once you've done this set the AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT environment variables:" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## Specifying model version\n", - "\n", - "Azure OpenAI responses contain `model_name` response metadata property, which is name of the model used to generate the response. However unlike native OpenAI responses, it does not contain the specific version of the model, which is set on the deployment in Azure. E.g. it does not distinguish between `gpt-35-turbo-0125` and `gpt-35-turbo-0301`. This makes it tricky to know which version of the model was used to generate the response, which as result can lead to e.g. wrong total cost calculation with `OpenAICallbackHandler`.\n", - "\n", - "To solve this problem, you can pass `model_version` parameter to `AzureChatOpenAI` class, which will be added to the model name in the llm output. This way you can easily distinguish between different versions of the model." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "04b36e75-e8b7-4721-899e-76301ac2ecd9", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-community" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "2ca02d23-60d0-43eb-8d04-070f61f8fefd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Total Cost (USD): $0.000063\n" - ] - } - ], - "source": [ - "from langchain_community.callbacks import get_openai_callback\n", - "\n", - "with get_openai_callback() as cb:\n", - " llm.invoke(messages)\n", - " print(\n", - " f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\"\n", - " ) # without specifying the model version, flat-rate 0.002 USD per 1k input and output tokens is used" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "e1b07ae2-3de7-44bd-bfdc-b76f4ba45a35", - "metadata": {}, - "outputs": [ + "cell_type": "code", + "execution_count": null, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"AZURE_OPENAI_API_KEY\" not in os.environ:\n", + " os.environ[\"AZURE_OPENAI_API_KEY\"] = getpass.getpass(\n", + " \"Enter your AzureOpenAI API key: \"\n", + " )\n", + "os.environ[\"AZURE_OPENAI_ENDPOINT\"] = \"https://YOUR-ENDPOINT.openai.azure.com/\"" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Total Cost (USD): $0.000074\n" - ] + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain AzureOpenAI integration lives in the `langchain-openai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-openai" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions.\n", + "- Replace `azure_deployment` with the name of your deployment,\n", + "- You can find the latest supported `api_version` here: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_openai import AzureChatOpenAI\n", + "\n", + "llm = AzureChatOpenAI(\n", + " azure_deployment=\"gpt-35-turbo\", # or your deployment\n", + " api_version=\"2023-06-01-preview\", # or your api version\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 31, 'total_tokens': 39}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-bea4b46c-e3e1-4495-9d3a-698370ad963d-0', usage_metadata={'input_tokens': 31, 'output_tokens': 8, 'total_tokens': 39})" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore la programmation.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 26, 'total_tokens': 32}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': None, 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-cbc44038-09d3-40d4-9da2-c5910ee636ca-0', usage_metadata={'input_tokens': 26, 'output_tokens': 6, 'total_tokens': 32})" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## Specifying model version\n", + "\n", + "Azure OpenAI responses contain `model_name` response metadata property, which is name of the model used to generate the response. However unlike native OpenAI responses, it does not contain the specific version of the model, which is set on the deployment in Azure. E.g. it does not distinguish between `gpt-35-turbo-0125` and `gpt-35-turbo-0301`. This makes it tricky to know which version of the model was used to generate the response, which as result can lead to e.g. wrong total cost calculation with `OpenAICallbackHandler`.\n", + "\n", + "To solve this problem, you can pass `model_version` parameter to `AzureChatOpenAI` class, which will be added to the model name in the llm output. This way you can easily distinguish between different versions of the model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04b36e75-e8b7-4721-899e-76301ac2ecd9", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-community" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "2ca02d23-60d0-43eb-8d04-070f61f8fefd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Total Cost (USD): $0.000063\n" + ] + } + ], + "source": [ + "from langchain_community.callbacks import get_openai_callback\n", + "\n", + "with get_openai_callback() as cb:\n", + " llm.invoke(messages)\n", + " print(\n", + " f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\"\n", + " ) # without specifying the model version, flat-rate 0.002 USD per 1k input and output tokens is used" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "e1b07ae2-3de7-44bd-bfdc-b76f4ba45a35", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Total Cost (USD): $0.000074\n" + ] + } + ], + "source": [ + "llm_0301 = AzureChatOpenAI(\n", + " azure_deployment=\"gpt-35-turbo\", # or your deployment\n", + " api_version=\"2023-06-01-preview\", # or your api version\n", + " model_version=\"0301\",\n", + ")\n", + "with get_openai_callback() as cb:\n", + " llm_0301.invoke(messages)\n", + " print(f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all AzureChatOpenAI features and configurations head to the API reference: https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" } - ], - "source": [ - "llm_0301 = AzureChatOpenAI(\n", - " azure_deployment=\"gpt-35-turbo\", # or your deployment\n", - " api_version=\"2023-06-01-preview\", # or your api version\n", - " model_version=\"0301\",\n", - ")\n", - "with get_openai_callback() as cb:\n", - " llm_0301.invoke(messages)\n", - " print(f\"Total Cost (USD): ${format(cb.total_cost, '.6f')}\")" - ] }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all AzureChatOpenAI features and configurations head to the API reference: https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/bedrock.ipynb b/docs/docs/integrations/chat/bedrock.ipynb index a24d23f7d7d..bcc916af098 100644 --- a/docs/docs/integrations/chat/bedrock.ipynb +++ b/docs/docs/integrations/chat/bedrock.ipynb @@ -1,380 +1,378 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: AWS Bedrock\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatBedrock\n", - "\n", - "This doc will help you get started with AWS Bedrock [chat models](/docs/concepts/chat_models). Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon via a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources. Since Amazon Bedrock is serverless, you don't have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with.\n", - "\n", - "For more information on which models are accessible via Bedrock, head to the [AWS docs](https://docs.aws.amazon.com/bedrock/latest/userguide/models-features.html).\n", - "\n", - "For detailed documentation of all ChatBedrock features and configurations head to the [API reference](https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock.ChatBedrock.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/bedrock) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatBedrock](https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock.ChatBedrock.html) | [langchain-aws](https://python.langchain.com/api_reference/aws/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-aws?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-aws?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access Bedrock models you'll need to create an AWS account, set up the Bedrock API service, get an access key ID and secret key, and install the `langchain-aws` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to the [AWS docs](https://docs.aws.amazon.com/bedrock/latest/userguide/setting-up.html) to sign up to AWS and setup your credentials. You'll also need to turn on model access for your account, which you can do by following [these instructions](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html)." - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Bedrock integration lives in the `langchain-aws` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-aws" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_aws import ChatBedrock\n", - "\n", - "llm = ChatBedrock(\n", - " model_id=\"anthropic.claude-3-sonnet-20240229-v1:0\",\n", - " model_kwargs=dict(temperature=0),\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"Voici la traduction en français :\\n\\nJ'aime la programmation.\", additional_kwargs={'usage': {'prompt_tokens': 29, 'completion_tokens': 21, 'total_tokens': 50}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, response_metadata={'usage': {'prompt_tokens': 29, 'completion_tokens': 21, 'total_tokens': 50}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, id='run-fdb07dc3-ff72-430d-b22b-e7824b15c766-0', usage_metadata={'input_tokens': 29, 'output_tokens': 21, 'total_tokens': 50})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: AWS Bedrock\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Voici la traduction en français :\n", - "\n", - "J'aime la programmation.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe Programmieren.', additional_kwargs={'usage': {'prompt_tokens': 23, 'completion_tokens': 11, 'total_tokens': 34}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, response_metadata={'usage': {'prompt_tokens': 23, 'completion_tokens': 11, 'total_tokens': 34}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, id='run-5ad005ce-9f31-4670-baa0-9373d418698a-0', usage_metadata={'input_tokens': 23, 'output_tokens': 11, 'total_tokens': 34})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatBedrock\n", + "\n", + "This doc will help you get started with AWS Bedrock [chat models](/docs/concepts/chat_models). Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon via a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources. Since Amazon Bedrock is serverless, you don't have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with.\n", + "\n", + "For more information on which models are accessible via Bedrock, head to the [AWS docs](https://docs.aws.amazon.com/bedrock/latest/userguide/models-features.html).\n", + "\n", + "For detailed documentation of all ChatBedrock features and configurations head to the [API reference](https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock.ChatBedrock.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/bedrock) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatBedrock](https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock.ChatBedrock.html) | [langchain-aws](https://python.langchain.com/api_reference/aws/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-aws?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-aws?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "To access Bedrock models you'll need to create an AWS account, set up the Bedrock API service, get an access key ID and secret key, and install the `langchain-aws` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to the [AWS docs](https://docs.aws.amazon.com/bedrock/latest/userguide/setting-up.html) to sign up to AWS and setup your credentials. You'll also need to turn on model access for your account, which you can do by following [these instructions](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html)." ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## Bedrock Converse API\n", - "\n", - "AWS has recently released the Bedrock Converse API which provides a unified conversational interface for Bedrock models. This API does not yet support custom models. You can see a list of all [models that are supported here](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html). To improve reliability the ChatBedrock integration will switch to using the Bedrock Converse API as soon as it has feature parity with the existing Bedrock API. Until then a separate [ChatBedrockConverse](https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock_converse.ChatBedrockConverse.html) integration has been released.\n", - "\n", - "We recommend using `ChatBedrockConverse` for users who do not need to use custom models.\n", - "\n", - "You can use it like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "ae728e59-94d4-40cf-9d24-25ad8723fc59", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content=\"Voici la traduction en français :\\n\\nJ'aime la programmation.\", response_metadata={'ResponseMetadata': {'RequestId': '4fcbfbe9-f916-4df2-b0bd-ea1147b550aa', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Wed, 21 Aug 2024 17:23:49 GMT', 'content-type': 'application/json', 'content-length': '243', 'connection': 'keep-alive', 'x-amzn-requestid': '4fcbfbe9-f916-4df2-b0bd-ea1147b550aa'}, 'RetryAttempts': 0}, 'stopReason': 'end_turn', 'metrics': {'latencyMs': 672}}, id='run-77ee9810-e32b-45dc-9ccb-6692253b1f45-0', usage_metadata={'input_tokens': 29, 'output_tokens': 21, 'total_tokens': 50})" + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_aws import ChatBedrockConverse\n", - "\n", - "llm = ChatBedrockConverse(\n", - " model=\"anthropic.claude-3-sonnet-20240229-v1:0\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " # other params...\n", - ")\n", - "\n", - "llm.invoke(messages)" - ] - }, - { - "cell_type": "markdown", - "id": "4da16f3e-e80b-48c0-8036-c1cc5f7c8c05", - "metadata": {}, - "source": [ - "### Streaming\n", - "\n", - "Note that `ChatBedrockConverse` emits content blocks while streaming:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "7794b32e-d8de-4973-bf0f-39807dc745f0", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "content=[] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': 'Vo', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': 'ici', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': ' la', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': ' tra', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': 'duction', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': ' en', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': ' français', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': ' :', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': '\\n\\nJ', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': \"'\", 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': 'a', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': 'ime', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': ' la', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': ' programm', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': 'ation', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'type': 'text', 'text': '.', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[{'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[] response_metadata={'stopReason': 'end_turn'} id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", - "content=[] response_metadata={'metrics': {'latencyMs': 713}} id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8' usage_metadata={'input_tokens': 29, 'output_tokens': 21, 'total_tokens': 50}\n" - ] - } - ], - "source": [ - "for chunk in llm.stream(messages):\n", - " print(chunk)" - ] - }, - { - "cell_type": "markdown", - "id": "0ef05abb-9c04-4dc3-995e-f857779644d5", - "metadata": {}, - "source": [ - "An output parser can be used to filter to text, if desired:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "2a4e743f-ea7d-4e5a-9b12-f9992362de8b", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Bedrock integration lives in the `langchain-aws` package:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "|Vo|ici| la| tra|duction| en| français| :|\n", - "\n", - "J|'|a|ime| la| programm|ation|.||||" - ] + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-aws" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_aws import ChatBedrock\n", + "\n", + "llm = ChatBedrock(\n", + " model_id=\"anthropic.claude-3-sonnet-20240229-v1:0\",\n", + " model_kwargs=dict(temperature=0),\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"Voici la traduction en français :\\n\\nJ'aime la programmation.\", additional_kwargs={'usage': {'prompt_tokens': 29, 'completion_tokens': 21, 'total_tokens': 50}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, response_metadata={'usage': {'prompt_tokens': 29, 'completion_tokens': 21, 'total_tokens': 50}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, id='run-fdb07dc3-ff72-430d-b22b-e7824b15c766-0', usage_metadata={'input_tokens': 29, 'output_tokens': 21, 'total_tokens': 50})" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Voici la traduction en français :\n", + "\n", + "J'aime la programmation.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe Programmieren.', additional_kwargs={'usage': {'prompt_tokens': 23, 'completion_tokens': 11, 'total_tokens': 34}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, response_metadata={'usage': {'prompt_tokens': 23, 'completion_tokens': 11, 'total_tokens': 34}, 'stop_reason': 'end_turn', 'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0'}, id='run-5ad005ce-9f31-4670-baa0-9373d418698a-0', usage_metadata={'input_tokens': 23, 'output_tokens': 11, 'total_tokens': 34})" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## Bedrock Converse API\n", + "\n", + "AWS has recently released the Bedrock Converse API which provides a unified conversational interface for Bedrock models. This API does not yet support custom models. You can see a list of all [models that are supported here](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html). To improve reliability the ChatBedrock integration will switch to using the Bedrock Converse API as soon as it has feature parity with the existing Bedrock API. Until then a separate [ChatBedrockConverse](https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock_converse.ChatBedrockConverse.html) integration has been released.\n", + "\n", + "We recommend using `ChatBedrockConverse` for users who do not need to use custom models.\n", + "\n", + "You can use it like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "ae728e59-94d4-40cf-9d24-25ad8723fc59", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"Voici la traduction en français :\\n\\nJ'aime la programmation.\", response_metadata={'ResponseMetadata': {'RequestId': '4fcbfbe9-f916-4df2-b0bd-ea1147b550aa', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Wed, 21 Aug 2024 17:23:49 GMT', 'content-type': 'application/json', 'content-length': '243', 'connection': 'keep-alive', 'x-amzn-requestid': '4fcbfbe9-f916-4df2-b0bd-ea1147b550aa'}, 'RetryAttempts': 0}, 'stopReason': 'end_turn', 'metrics': {'latencyMs': 672}}, id='run-77ee9810-e32b-45dc-9ccb-6692253b1f45-0', usage_metadata={'input_tokens': 29, 'output_tokens': 21, 'total_tokens': 50})" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_aws import ChatBedrockConverse\n", + "\n", + "llm = ChatBedrockConverse(\n", + " model=\"anthropic.claude-3-sonnet-20240229-v1:0\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " # other params...\n", + ")\n", + "\n", + "llm.invoke(messages)" + ] + }, + { + "cell_type": "markdown", + "id": "4da16f3e-e80b-48c0-8036-c1cc5f7c8c05", + "metadata": {}, + "source": [ + "### Streaming\n", + "\n", + "Note that `ChatBedrockConverse` emits content blocks while streaming:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "7794b32e-d8de-4973-bf0f-39807dc745f0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "content=[] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': 'Vo', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': 'ici', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': ' la', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': ' tra', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': 'duction', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': ' en', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': ' français', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': ' :', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': '\\n\\nJ', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': \"'\", 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': 'a', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': 'ime', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': ' la', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': ' programm', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': 'ation', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'type': 'text', 'text': '.', 'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[{'index': 0}] id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[] response_metadata={'stopReason': 'end_turn'} id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8'\n", + "content=[] response_metadata={'metrics': {'latencyMs': 713}} id='run-2c92c5af-d771-4cc2-98d9-c11bbd30a1d8' usage_metadata={'input_tokens': 29, 'output_tokens': 21, 'total_tokens': 50}\n" + ] + } + ], + "source": [ + "for chunk in llm.stream(messages):\n", + " print(chunk)" + ] + }, + { + "cell_type": "markdown", + "id": "0ef05abb-9c04-4dc3-995e-f857779644d5", + "metadata": {}, + "source": [ + "An output parser can be used to filter to text, if desired:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "2a4e743f-ea7d-4e5a-9b12-f9992362de8b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "|Vo|ici| la| tra|duction| en| français| :|\n", + "\n", + "J|'|a|ime| la| programm|ation|.||||" + ] + } + ], + "source": [ + "from langchain_core.output_parsers import StrOutputParser\n", + "\n", + "chain = llm | StrOutputParser()\n", + "\n", + "for chunk in chain.stream(messages):\n", + " print(chunk, end=\"|\")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatBedrock features and configurations head to the API reference: https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock.ChatBedrock.html\n", + "\n", + "For detailed documentation of all ChatBedrockConverse features and configurations head to the API reference: https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock_converse.ChatBedrockConverse.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" } - ], - "source": [ - "from langchain_core.output_parsers import StrOutputParser\n", - "\n", - "chain = llm | StrOutputParser()\n", - "\n", - "for chunk in chain.stream(messages):\n", - " print(chunk, end=\"|\")" - ] }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatBedrock features and configurations head to the API reference: https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock.ChatBedrock.html\n", - "\n", - "For detailed documentation of all ChatBedrockConverse features and configurations head to the API reference: https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock_converse.ChatBedrockConverse.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/cerebras.ipynb b/docs/docs/integrations/chat/cerebras.ipynb index 1713681f010..f7dbc302cbe 100644 --- a/docs/docs/integrations/chat/cerebras.ipynb +++ b/docs/docs/integrations/chat/cerebras.ipynb @@ -1,424 +1,422 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Cerebras\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatCerebras\n", - "\n", - "This notebook provides a quick overview for getting started with Cerebras [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatCerebras features and configurations head to the [API reference](https://python.langchain.com/api_reference/cerebras/chat_models/langchain_cerebras.chat_models.ChatCerebras.html#).\n", - "\n", - "At Cerebras, we've developed the world's largest and fastest AI processor, the Wafer-Scale Engine-3 (WSE-3). The Cerebras CS-3 system, powered by the WSE-3, represents a new class of AI supercomputer that sets the standard for generative AI training and inference with unparalleled performance and scalability.\n", - "\n", - "With Cerebras as your inference provider, you can:\n", - "- Achieve unprecedented speed for AI inference workloads\n", - "- Build commercially with high throughput\n", - "- Effortlessly scale your AI workloads with our seamless clustering technology\n", - "\n", - "Our CS-3 systems can be quickly and easily clustered to create the largest AI supercomputers in the world, making it simple to place and run the largest models. Leading corporations, research institutions, and governments are already using Cerebras solutions to develop proprietary models and train popular open-source models.\n", - "\n", - "Want to experience the power of Cerebras? Check out our [website](https://cerebras.ai) for more resources and explore options for accessing our technology through the Cerebras Cloud or on-premise deployments!\n", - "\n", - "For more information about Cerebras Cloud, visit [cloud.cerebras.ai](https://cloud.cerebras.ai/). Our API reference is available at [inference-docs.cerebras.ai](https://inference-docs.cerebras.ai/).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/cerebras) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatCerebras](https://python.langchain.com/api_reference/cerebras/chat_models/langchain_cerebras.chat_models.ChatCerebras.html#) | [langchain-cerebras](https://python.langchain.com/api_reference/cerebras/index.html) | ❌ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-cerebras?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-cerebras?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "```bash\n", - "pip install langchain-cerebras\n", - "```\n", - "\n", - "### Credentials\n", - "\n", - "Get an API Key from [cloud.cerebras.ai](https://cloud.cerebras.ai/) and add it to your environment variables:\n", - "```\n", - "export CEREBRAS_API_KEY=\"your-api-key-here\"\n", - "```" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "ce19c2d6", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Enter your Cerebras API key: ········\n" - ] - } - ], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"CEREBRAS_API_KEY\" not in os.environ:\n", - " os.environ[\"CEREBRAS_API_KEY\"] = getpass.getpass(\"Enter your Cerebras API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Cerebras integration lives in the `langchain-cerebras` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-cerebras" - ] - }, - { - "cell_type": "markdown", - "id": "ea69675d", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "21155898", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_cerebras import ChatCerebras\n", - "\n", - "llm = ChatCerebras(\n", - " model=\"llama-3.3-70b\",\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Je adore le programmation.', response_metadata={'token_usage': {'completion_tokens': 7, 'prompt_tokens': 35, 'total_tokens': 42}, 'model_name': 'llama3-8b-8192', 'system_fingerprint': 'fp_be27ec77ff', 'finish_reason': 'stop'}, id='run-e5d66faf-019c-4ac6-9265-71093b13202d-0', usage_metadata={'input_tokens': 35, 'output_tokens': 7, 'total_tokens': 42})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Cerebras\n", + "---" ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe Programmieren!\\n\\n(Literally: I love programming!)', response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 30, 'total_tokens': 44}, 'model_name': 'llama3-8b-8192', 'system_fingerprint': 'fp_be27ec77ff', 'finish_reason': 'stop'}, id='run-e1d2ebb8-76d1-471b-9368-3b68d431f16a-0', usage_metadata={'input_tokens': 30, 'output_tokens': 14, 'total_tokens': 44})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatCerebras\n", + "\n", + "This notebook provides a quick overview for getting started with Cerebras [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatCerebras features and configurations head to the [API reference](https://python.langchain.com/api_reference/cerebras/chat_models/langchain_cerebras.chat_models.ChatCerebras.html#).\n", + "\n", + "At Cerebras, we've developed the world's largest and fastest AI processor, the Wafer-Scale Engine-3 (WSE-3). The Cerebras CS-3 system, powered by the WSE-3, represents a new class of AI supercomputer that sets the standard for generative AI training and inference with unparalleled performance and scalability.\n", + "\n", + "With Cerebras as your inference provider, you can:\n", + "- Achieve unprecedented speed for AI inference workloads\n", + "- Build commercially with high throughput\n", + "- Effortlessly scale your AI workloads with our seamless clustering technology\n", + "\n", + "Our CS-3 systems can be quickly and easily clustered to create the largest AI supercomputers in the world, making it simple to place and run the largest models. Leading corporations, research institutions, and governments are already using Cerebras solutions to develop proprietary models and train popular open-source models.\n", + "\n", + "Want to experience the power of Cerebras? Check out our [website](https://cerebras.ai) for more resources and explore options for accessing our technology through the Cerebras Cloud or on-premise deployments!\n", + "\n", + "For more information about Cerebras Cloud, visit [cloud.cerebras.ai](https://cloud.cerebras.ai/). Our API reference is available at [inference-docs.cerebras.ai](https://inference-docs.cerebras.ai/).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/cerebras) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatCerebras](https://python.langchain.com/api_reference/cerebras/chat_models/langchain_cerebras.chat_models.ChatCerebras.html#) | [langchain-cerebras](https://python.langchain.com/api_reference/cerebras/index.html) | ❌ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-cerebras?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-cerebras?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "```bash\n", + "pip install langchain-cerebras\n", + "```\n", + "\n", + "### Credentials\n", + "\n", + "Get an API Key from [cloud.cerebras.ai](https://cloud.cerebras.ai/) and add it to your environment variables:\n", + "```\n", + "export CEREBRAS_API_KEY=\"your-api-key-here\"\n", + "```" ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_cerebras import ChatCerebras\n", - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "llm = ChatCerebras(\n", - " model=\"llama-3.3-70b\",\n", - " # other params...\n", - ")\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "0ec73a0e", - "metadata": {}, - "source": [ - "## Streaming" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "46fd21a7", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "OH BOY! Let me tell you all about LIONS!\n", - "\n", - "Lions are the kings of the jungle! They're really big and have beautiful, fluffy manes around their necks. The mane is like a big, golden crown!\n", - "\n", - "Lions live in groups called prides. A pride is like a big family, and the lionesses (that's what we call the female lions) take care of the babies. The lionesses are like the mommies, and they teach the babies how to hunt and play.\n", - "\n", - "Lions are very good at hunting. They work together to catch their food, like zebras and antelopes. They're super fast and can run really, really fast!\n", - "\n", - "But lions are also very sleepy. They like to take long naps in the sun, and they can sleep for up to 20 hours a day! Can you imagine sleeping that much?\n", - "\n", - "Lions are also very loud. They roar really loudly to talk to each other. It's like they're saying, \"ROAR! I'm the king of the jungle!\"\n", - "\n", - "And guess what? Lions are very social. They like to play and cuddle with each other. They're like big, furry teddy bears!\n", - "\n", - "So, that's lions! Aren't they just the coolest?" - ] - } - ], - "source": [ - "from langchain_cerebras import ChatCerebras\n", - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "llm = ChatCerebras(\n", - " model=\"llama-3.3-70b\",\n", - " # other params...\n", - ")\n", - "\n", - "system = \"You are an expert on animals who must answer questions in a manner that a 5 year old can understand.\"\n", - "human = \"I want to learn more about this animal: {animal}\"\n", - "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", - "\n", - "chain = prompt | llm\n", - "\n", - "for chunk in chain.stream({\"animal\": \"Lion\"}):\n", - " print(chunk.content, end=\"\", flush=True)" - ] - }, - { - "cell_type": "markdown", - "id": "f67b6132", - "metadata": {}, - "source": [ - "## Async" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "id": "a3a45baf", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ice', response_metadata={'token_usage': {'completion_tokens': 2, 'prompt_tokens': 36, 'total_tokens': 38}, 'model_name': 'llama3-8b-8192', 'system_fingerprint': 'fp_be27ec77ff', 'finish_reason': 'stop'}, id='run-7434bdde-1bec-44cf-827b-8d978071dfe8-0', usage_metadata={'input_tokens': 36, 'output_tokens': 2, 'total_tokens': 38})" + "cell_type": "code", + "execution_count": 8, + "id": "ce19c2d6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Enter your Cerebras API key: ········\n" + ] + } + ], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"CEREBRAS_API_KEY\" not in os.environ:\n", + " os.environ[\"CEREBRAS_API_KEY\"] = getpass.getpass(\"Enter your Cerebras API key: \")" ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_cerebras import ChatCerebras\n", - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "llm = ChatCerebras(\n", - " model=\"llama-3.3-70b\",\n", - " # other params...\n", - ")\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"human\",\n", - " \"Let's play a game of opposites. What's the opposite of {topic}? Just give me the answer with no extra input.\",\n", - " )\n", - " ]\n", - ")\n", - "chain = prompt | llm\n", - "await chain.ainvoke({\"topic\": \"fire\"})" - ] - }, - { - "cell_type": "markdown", - "id": "4f9d9945", - "metadata": {}, - "source": [ - "## Async Streaming" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "id": "c7448e0f", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "In the distant reaches of the cosmos, there existed a peculiar phenomenon known as the \"Eclipse of Eternity,\" a swirling vortex of darkness that had been shrouded in mystery for eons. It was said that this blackhole, born from the cataclysmic collision of two ancient stars, had been slowly devouring the fabric of space-time itself, warping the very essence of reality. As the celestial bodies of the galaxy danced around it, they began to notice a strange, almost imperceptible distortion in the fabric of space, as if the blackhole's gravitational pull was exerting an influence on the very course of events itself.\n", - "\n", - "As the centuries passed, astronomers from across the galaxy became increasingly fascinated by the Eclipse of Eternity, pouring over ancient texts and scouring the cosmos for any hint of its secrets. One such scholar, a brilliant and reclusive astrophysicist named Dr. Elara Vex, became obsessed with unraveling the mysteries of the blackhole. She spent years pouring over ancient texts, deciphering cryptic messages and hidden codes that hinted at the existence of a long-lost civilization that had once thrived in the heart of the blackhole itself. According to legend, this ancient civilization had possessed knowledge of the cosmos that was beyond human comprehension, and had used their mastery of the universe to create the Eclipse of Eternity as a gateway to other dimensions.\n", - "\n", - "As Dr. Vex delved deeper into her research, she began to experience strange and vivid dreams, visions that seemed to transport her to the very heart of the blackhole itself. In these dreams, she saw ancient beings, their faces twisted in agony as they were consumed by the void. She saw stars and galaxies, their light warped and distorted by the blackhole's gravitational pull. And she saw the Eclipse of Eternity itself, its swirling vortex of darkness pulsing with an otherworldly energy that seemed to be calling to her. As the dreams grew more vivid and more frequent, Dr. Vex became convinced that she was being drawn into the heart of the blackhole, and that the secrets of the universe lay waiting for her on the other side." - ] + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Cerebras integration lives in the `langchain-cerebras` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-cerebras" + ] + }, + { + "cell_type": "markdown", + "id": "ea69675d", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "21155898", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_cerebras import ChatCerebras\n", + "\n", + "llm = ChatCerebras(\n", + " model=\"llama-3.3-70b\",\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Je adore le programmation.', response_metadata={'token_usage': {'completion_tokens': 7, 'prompt_tokens': 35, 'total_tokens': 42}, 'model_name': 'llama3-8b-8192', 'system_fingerprint': 'fp_be27ec77ff', 'finish_reason': 'stop'}, id='run-e5d66faf-019c-4ac6-9265-71093b13202d-0', usage_metadata={'input_tokens': 35, 'output_tokens': 7, 'total_tokens': 42})" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe Programmieren!\\n\\n(Literally: I love programming!)', response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 30, 'total_tokens': 44}, 'model_name': 'llama3-8b-8192', 'system_fingerprint': 'fp_be27ec77ff', 'finish_reason': 'stop'}, id='run-e1d2ebb8-76d1-471b-9368-3b68d431f16a-0', usage_metadata={'input_tokens': 30, 'output_tokens': 14, 'total_tokens': 44})" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_cerebras import ChatCerebras\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "llm = ChatCerebras(\n", + " model=\"llama-3.3-70b\",\n", + " # other params...\n", + ")\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0ec73a0e", + "metadata": {}, + "source": [ + "## Streaming" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "46fd21a7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "OH BOY! Let me tell you all about LIONS!\n", + "\n", + "Lions are the kings of the jungle! They're really big and have beautiful, fluffy manes around their necks. The mane is like a big, golden crown!\n", + "\n", + "Lions live in groups called prides. A pride is like a big family, and the lionesses (that's what we call the female lions) take care of the babies. The lionesses are like the mommies, and they teach the babies how to hunt and play.\n", + "\n", + "Lions are very good at hunting. They work together to catch their food, like zebras and antelopes. They're super fast and can run really, really fast!\n", + "\n", + "But lions are also very sleepy. They like to take long naps in the sun, and they can sleep for up to 20 hours a day! Can you imagine sleeping that much?\n", + "\n", + "Lions are also very loud. They roar really loudly to talk to each other. It's like they're saying, \"ROAR! I'm the king of the jungle!\"\n", + "\n", + "And guess what? Lions are very social. They like to play and cuddle with each other. They're like big, furry teddy bears!\n", + "\n", + "So, that's lions! Aren't they just the coolest?" + ] + } + ], + "source": [ + "from langchain_cerebras import ChatCerebras\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "llm = ChatCerebras(\n", + " model=\"llama-3.3-70b\",\n", + " # other params...\n", + ")\n", + "\n", + "system = \"You are an expert on animals who must answer questions in a manner that a 5 year old can understand.\"\n", + "human = \"I want to learn more about this animal: {animal}\"\n", + "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", + "\n", + "chain = prompt | llm\n", + "\n", + "for chunk in chain.stream({\"animal\": \"Lion\"}):\n", + " print(chunk.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "id": "f67b6132", + "metadata": {}, + "source": [ + "## Async" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "a3a45baf", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ice', response_metadata={'token_usage': {'completion_tokens': 2, 'prompt_tokens': 36, 'total_tokens': 38}, 'model_name': 'llama3-8b-8192', 'system_fingerprint': 'fp_be27ec77ff', 'finish_reason': 'stop'}, id='run-7434bdde-1bec-44cf-827b-8d978071dfe8-0', usage_metadata={'input_tokens': 36, 'output_tokens': 2, 'total_tokens': 38})" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_cerebras import ChatCerebras\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "llm = ChatCerebras(\n", + " model=\"llama-3.3-70b\",\n", + " # other params...\n", + ")\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"human\",\n", + " \"Let's play a game of opposites. What's the opposite of {topic}? Just give me the answer with no extra input.\",\n", + " )\n", + " ]\n", + ")\n", + "chain = prompt | llm\n", + "await chain.ainvoke({\"topic\": \"fire\"})" + ] + }, + { + "cell_type": "markdown", + "id": "4f9d9945", + "metadata": {}, + "source": [ + "## Async Streaming" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "c7448e0f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "In the distant reaches of the cosmos, there existed a peculiar phenomenon known as the \"Eclipse of Eternity,\" a swirling vortex of darkness that had been shrouded in mystery for eons. It was said that this blackhole, born from the cataclysmic collision of two ancient stars, had been slowly devouring the fabric of space-time itself, warping the very essence of reality. As the celestial bodies of the galaxy danced around it, they began to notice a strange, almost imperceptible distortion in the fabric of space, as if the blackhole's gravitational pull was exerting an influence on the very course of events itself.\n", + "\n", + "As the centuries passed, astronomers from across the galaxy became increasingly fascinated by the Eclipse of Eternity, pouring over ancient texts and scouring the cosmos for any hint of its secrets. One such scholar, a brilliant and reclusive astrophysicist named Dr. Elara Vex, became obsessed with unraveling the mysteries of the blackhole. She spent years pouring over ancient texts, deciphering cryptic messages and hidden codes that hinted at the existence of a long-lost civilization that had once thrived in the heart of the blackhole itself. According to legend, this ancient civilization had possessed knowledge of the cosmos that was beyond human comprehension, and had used their mastery of the universe to create the Eclipse of Eternity as a gateway to other dimensions.\n", + "\n", + "As Dr. Vex delved deeper into her research, she began to experience strange and vivid dreams, visions that seemed to transport her to the very heart of the blackhole itself. In these dreams, she saw ancient beings, their faces twisted in agony as they were consumed by the void. She saw stars and galaxies, their light warped and distorted by the blackhole's gravitational pull. And she saw the Eclipse of Eternity itself, its swirling vortex of darkness pulsing with an otherworldly energy that seemed to be calling to her. As the dreams grew more vivid and more frequent, Dr. Vex became convinced that she was being drawn into the heart of the blackhole, and that the secrets of the universe lay waiting for her on the other side." + ] + } + ], + "source": [ + "from langchain_cerebras import ChatCerebras\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "llm = ChatCerebras(\n", + " model=\"llama-3.3-70b\",\n", + " # other params...\n", + ")\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"human\",\n", + " \"Write a long convoluted story about {subject}. I want {num_paragraphs} paragraphs.\",\n", + " )\n", + " ]\n", + ")\n", + "chain = prompt | llm\n", + "\n", + "async for chunk in chain.astream({\"num_paragraphs\": 3, \"subject\": \"blackholes\"}):\n", + " print(chunk.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatCerebras features and configurations head to the API reference: https://python.langchain.com/api_reference/cerebras/chat_models/langchain_cerebras.chat_models.ChatCerebras.html#" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.13" } - ], - "source": [ - "from langchain_cerebras import ChatCerebras\n", - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "llm = ChatCerebras(\n", - " model=\"llama-3.3-70b\",\n", - " # other params...\n", - ")\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"human\",\n", - " \"Write a long convoluted story about {subject}. I want {num_paragraphs} paragraphs.\",\n", - " )\n", - " ]\n", - ")\n", - "chain = prompt | llm\n", - "\n", - "async for chunk in chain.astream({\"num_paragraphs\": 3, \"subject\": \"blackholes\"}):\n", - " print(chunk.content, end=\"\", flush=True)" - ] }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatCerebras features and configurations head to the API reference: https://python.langchain.com/api_reference/cerebras/chat_models/langchain_cerebras.chat_models.ChatCerebras.html#" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.13" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/cloudflare_workersai.ipynb b/docs/docs/integrations/chat/cloudflare_workersai.ipynb index f840bb137a3..8f024786517 100644 --- a/docs/docs/integrations/chat/cloudflare_workersai.ipynb +++ b/docs/docs/integrations/chat/cloudflare_workersai.ipynb @@ -1,264 +1,262 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "30373ae2-f326-4e96-a1f7-062f57396886", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Cloudflare Workers AI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "f679592d", - "metadata": {}, - "source": [ - "# ChatCloudflareWorkersAI\n", - "\n", - "This will help you getting started with CloudflareWorkersAI [chat models](/docs/concepts/chat_models). For detailed documentation of all available Cloudflare WorkersAI models head to the [API reference](https://developers.cloudflare.com/workers-ai/).\n", - "\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/cloudflare_workersai) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| ChatCloudflareWorkersAI | langchain-community| ❌ | ❌ | ✅ | ❌ | ❌ |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "- To access Cloudflare Workers AI models you'll need to create a Cloudflare account, get an account number and API key, and install the `langchain-community` package.\n", - "\n", - "\n", - "### Credentials\n", - "\n", - "\n", - "Head to [this document](https://developers.cloudflare.com/workers-ai/get-started/rest-api/) to sign up to Cloudflare Workers AI and generate an API key." - ] - }, - { - "cell_type": "markdown", - "id": "4a524cff", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "71b53c25", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "777a8526", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain ChatCloudflareWorkersAI integration lives in the `langchain-community` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "54990998", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-community" - ] - }, - { - "cell_type": "markdown", - "id": "629ba46f", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "ec13c2d9", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.chat_models.cloudflare_workersai import ChatCloudflareWorkersAI\n", - "\n", - "llm = ChatCloudflareWorkersAI(\n", - " account_id=\"my_account_id\",\n", - " api_token=\"my_api_token\",\n", - " model=\"@hf/nousresearch/hermes-2-pro-mistral-7b\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "119b6732", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "2438a906", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "name": "stderr", - "output_type": "stream", - "text": [ - "2024-11-07 15:55:14 - INFO - Sending prompt to Cloudflare Workers AI: {'prompt': 'role: system, content: You are a helpful assistant that translates English to French. Translate the user sentence.\\nrole: user, content: I love programming.', 'tools': None}\n" - ] + "cell_type": "raw", + "id": "30373ae2-f326-4e96-a1f7-062f57396886", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Cloudflare Workers AI\n", + "---" + ] }, { - "data": { - "text/plain": [ - "AIMessage(content='{\\'result\\': {\\'response\\': \\'Je suis un assistant virtuel qui peut traduire l\\\\\\'anglais vers le français. La phrase que vous avez dite est : \"J\\\\\\'aime programmer.\" En français, cela se traduit par : \"J\\\\\\'adore programmer.\"\\'}, \\'success\\': True, \\'errors\\': [], \\'messages\\': []}', additional_kwargs={}, response_metadata={}, id='run-838fd398-8594-4ca5-9055-03c72993caf6-0')" + "cell_type": "markdown", + "id": "f679592d", + "metadata": {}, + "source": [ + "# ChatCloudflareWorkersAI\n", + "\n", + "This will help you getting started with CloudflareWorkersAI [chat models](/docs/concepts/chat_models). For detailed documentation of all available Cloudflare WorkersAI models head to the [API reference](https://developers.cloudflare.com/workers-ai/).\n", + "\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/cloudflare_workersai) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| ChatCloudflareWorkersAI | langchain-community| ❌ | ❌ | ✅ | ❌ | ❌ |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "- To access Cloudflare Workers AI models you'll need to create a Cloudflare account, get an account number and API key, and install the `langchain-community` package.\n", + "\n", + "\n", + "### Credentials\n", + "\n", + "\n", + "Head to [this document](https://developers.cloudflare.com/workers-ai/get-started/rest-api/) to sign up to Cloudflare Workers AI and generate an API key." ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "1b4911bd", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'result': {'response': 'Je suis un assistant virtuel qui peut traduire l\\'anglais vers le français. La phrase que vous avez dite est : \"J\\'aime programmer.\" En français, cela se traduit par : \"J\\'adore programmer.\"'}, 'success': True, 'errors': [], 'messages': []}\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "111aa5d4", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "b2a14282", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2024-11-07 15:55:24 - INFO - Sending prompt to Cloudflare Workers AI: {'prompt': 'role: system, content: You are a helpful assistant that translates English to German.\\nrole: user, content: I love programming.', 'tools': None}\n" - ] }, { - "data": { - "text/plain": [ - "AIMessage(content=\"{'result': {'response': 'role: system, content: Das ist sehr nett zu hören! Programmieren lieben, ist eine interessante und anspruchsvolle Hobby- oder Berufsausrichtung. Wenn Sie englische Texte ins Deutsche übersetzen möchten, kann ich Ihnen helfen. Geben Sie bitte den englischen Satz oder die Übersetzung an, die Sie benötigen.'}, 'success': True, 'errors': [], 'messages': []}\", additional_kwargs={}, response_metadata={}, id='run-0d3be9a6-3d74-4dde-b49a-4479d6af00ef-0')" + "cell_type": "markdown", + "id": "4a524cff", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "71b53c25", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "777a8526", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain ChatCloudflareWorkersAI integration lives in the `langchain-community` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "54990998", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-community" + ] + }, + { + "cell_type": "markdown", + "id": "629ba46f", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ec13c2d9", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.chat_models.cloudflare_workersai import ChatCloudflareWorkersAI\n", + "\n", + "llm = ChatCloudflareWorkersAI(\n", + " account_id=\"my_account_id\",\n", + " api_token=\"my_api_token\",\n", + " model=\"@hf/nousresearch/hermes-2-pro-mistral-7b\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "119b6732", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "2438a906", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-11-07 15:55:14 - INFO - Sending prompt to Cloudflare Workers AI: {'prompt': 'role: system, content: You are a helpful assistant that translates English to French. Translate the user sentence.\\nrole: user, content: I love programming.', 'tools': None}\n" + ] + }, + { + "data": { + "text/plain": [ + "AIMessage(content='{\\'result\\': {\\'response\\': \\'Je suis un assistant virtuel qui peut traduire l\\\\\\'anglais vers le français. La phrase que vous avez dite est : \"J\\\\\\'aime programmer.\" En français, cela se traduit par : \"J\\\\\\'adore programmer.\"\\'}, \\'success\\': True, \\'errors\\': [], \\'messages\\': []}', additional_kwargs={}, response_metadata={}, id='run-838fd398-8594-4ca5-9055-03c72993caf6-0')" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "1b4911bd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'result': {'response': 'Je suis un assistant virtuel qui peut traduire l\\'anglais vers le français. La phrase que vous avez dite est : \"J\\'aime programmer.\" En français, cela se traduit par : \"J\\'adore programmer.\"'}, 'success': True, 'errors': [], 'messages': []}\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "111aa5d4", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "b2a14282", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-11-07 15:55:24 - INFO - Sending prompt to Cloudflare Workers AI: {'prompt': 'role: system, content: You are a helpful assistant that translates English to German.\\nrole: user, content: I love programming.', 'tools': None}\n" + ] + }, + { + "data": { + "text/plain": [ + "AIMessage(content=\"{'result': {'response': 'role: system, content: Das ist sehr nett zu hören! Programmieren lieben, ist eine interessante und anspruchsvolle Hobby- oder Berufsausrichtung. Wenn Sie englische Texte ins Deutsche übersetzen möchten, kann ich Ihnen helfen. Geben Sie bitte den englischen Satz oder die Übersetzung an, die Sie benötigen.'}, 'success': True, 'errors': [], 'messages': []}\", additional_kwargs={}, response_metadata={}, id='run-0d3be9a6-3d74-4dde-b49a-4479d6af00ef-0')" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "e1f311bd", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation on `ChatCloudflareWorkersAI` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.cloudflare_workersai.html)." ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "e1f311bd", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation on `ChatCloudflareWorkersAI` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.cloudflare_workersai.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/cohere.ipynb b/docs/docs/integrations/chat/cohere.ipynb index 733e4035ae5..c9bb91947e5 100644 --- a/docs/docs/integrations/chat/cohere.ipynb +++ b/docs/docs/integrations/chat/cohere.ipynb @@ -1,352 +1,350 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "53fbf15f", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Cohere\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "bf733a38-db84-4363-89e2-de6735c37230", - "metadata": {}, - "source": [ - "# Cohere\n", - "\n", - "This notebook covers how to get started with [Cohere chat models](https://cohere.com/chat).\n", - "\n", - "Head to the [API reference](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.cohere.ChatCohere.html) for detailed documentation of all attributes and methods." - ] - }, - { - "cell_type": "markdown", - "id": "3607d67e-e56c-4102-bbba-df2edc0e109e", - "metadata": {}, - "source": [ - "## Setup\n", - "\n", - "The integration lives in the `langchain-cohere` package. We can install these with:\n", - "\n", - "```bash\n", - "pip install -U langchain-cohere\n", - "```\n", - "\n", - "We'll also need to get a [Cohere API key](https://cohere.com/) and set the `COHERE_API_KEY` environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "2108b517-1e8d-473d-92fa-4f930e8072a7", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "os.environ[\"COHERE_API_KEY\"] = getpass.getpass()" - ] - }, - { - "cell_type": "markdown", - "id": "cf690fbb", - "metadata": {}, - "source": [ - "It's also helpful (but not needed) to set up [LangSmith](https://smith.langchain.com/) for best-in-class observability" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "7f11de02", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass()" - ] - }, - { - "cell_type": "markdown", - "id": "4c26754b-b3c9-4d93-8f36-43049bd943bf", - "metadata": {}, - "source": [ - "## Usage\n", - "\n", - "ChatCohere supports all [ChatModel](/docs/how_to#chat-models) functionality:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d4a7c55d-b235-4ca4-a579-c90cc9570da9", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain_cohere import ChatCohere\n", - "from langchain_core.messages import HumanMessage" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "70cf04e8-423a-4ff6-8b09-f11fb711c817", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "chat = ChatCohere()" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "8199ef8f-eb8b-4253-9ea0-6c24a013ca4c", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content='4 && 5 \\n6 || 7 \\n\\nWould you like to play a game of odds and evens?', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '2076b614-52b3-4082-a259-cc92cd3d9fea', 'token_count': {'prompt_tokens': 68, 'response_tokens': 23, 'total_tokens': 91, 'billed_tokens': 77}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '2076b614-52b3-4082-a259-cc92cd3d9fea', 'token_count': {'prompt_tokens': 68, 'response_tokens': 23, 'total_tokens': 91, 'billed_tokens': 77}}, id='run-3475e0c8-c89b-4937-9300-e07d652455e1-0')" + "cell_type": "raw", + "id": "53fbf15f", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Cohere\n", + "---" ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [HumanMessage(content=\"1\"), HumanMessage(content=\"2 3\")]\n", - "chat.invoke(messages)" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "id": "c5fac0e9-05a4-4fc1-a3b3-e5bbb24b971b", - "metadata": { - "tags": [] - }, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content='4 && 5', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'f0708a92-f874-46ee-9b93-334d616ad92e', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'f0708a92-f874-46ee-9b93-334d616ad92e', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, id='run-1635e63e-2994-4e7f-986e-152ddfc95777-0')" + "cell_type": "markdown", + "id": "bf733a38-db84-4363-89e2-de6735c37230", + "metadata": {}, + "source": [ + "# Cohere\n", + "\n", + "This notebook covers how to get started with [Cohere chat models](https://cohere.com/chat).\n", + "\n", + "Head to the [API reference](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.cohere.ChatCohere.html) for detailed documentation of all attributes and methods." ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "await chat.ainvoke(messages)" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "id": "025be980-e50d-4a68-93dc-c9c7b500ce34", - "metadata": { - "tags": [] - }, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "4 && 5" - ] - } - ], - "source": [ - "for chunk in chat.stream(messages):\n", - " print(chunk.content, end=\"\", flush=True)" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "id": "064288e4-f184-4496-9427-bcf148fa055e", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[AIMessage(content='4 && 5', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6770ca86-f6c3-4ba3-a285-c4772160612f', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6770ca86-f6c3-4ba3-a285-c4772160612f', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, id='run-8d6fade2-1b39-4e31-ab23-4be622dd0027-0')]" + "cell_type": "markdown", + "id": "3607d67e-e56c-4102-bbba-df2edc0e109e", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "The integration lives in the `langchain-cohere` package. We can install these with:\n", + "\n", + "```bash\n", + "pip install -U langchain-cohere\n", + "```\n", + "\n", + "We'll also need to get a [Cohere API key](https://cohere.com/) and set the `COHERE_API_KEY` environment variable:" ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "chat.batch([messages])" - ] - }, - { - "cell_type": "markdown", - "id": "f1c56460", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "You can also easily combine with a prompt template for easy structuring of user input. We can do this using [LCEL](/docs/concepts/lcel)" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "id": "0851b103", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_template(\"Tell me a joke about {topic}\")\n", - "chain = prompt | chat" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "id": "ae950c0f-1691-47f1-b609-273033cae707", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content='What color socks do bears wear?\\n\\nThey don’t wear socks, they have bear feet. \\n\\nHope you laughed! If not, maybe this will help: laughter is the best medicine, and a good sense of humor is infectious!', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6edccf44-9bc8-4139-b30e-13b368f3563c', 'token_count': {'prompt_tokens': 68, 'response_tokens': 51, 'total_tokens': 119, 'billed_tokens': 108}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6edccf44-9bc8-4139-b30e-13b368f3563c', 'token_count': {'prompt_tokens': 68, 'response_tokens': 51, 'total_tokens': 119, 'billed_tokens': 108}}, id='run-ef7f9789-0d4d-43bf-a4f7-f2a0e27a5320-0')" + "cell_type": "code", + "execution_count": 11, + "id": "2108b517-1e8d-473d-92fa-4f930e8072a7", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "os.environ[\"COHERE_API_KEY\"] = getpass.getpass()" ] - }, - "execution_count": 20, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "chain.invoke({\"topic\": \"bears\"})" - ] - }, - { - "cell_type": "markdown", - "id": "12db8d69", - "metadata": {}, - "source": [ - "## Tool calling\n", - "\n", - "Cohere supports tool calling functionalities!" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "337e24af", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.messages import (\n", - " HumanMessage,\n", - " ToolMessage,\n", - ")\n", - "from langchain_core.tools import tool" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "74d292e7", - "metadata": {}, - "outputs": [], - "source": [ - "@tool\n", - "def magic_function(number: int) -> int:\n", - " \"\"\"Applies a magic operation to an integer\n", - " Args:\n", - " number: Number to have magic operation performed on\n", - " \"\"\"\n", - " return number + 10\n", - "\n", - "\n", - "def invoke_tools(tool_calls, messages):\n", - " for tool_call in tool_calls:\n", - " selected_tool = {\"magic_function\": magic_function}[tool_call[\"name\"].lower()]\n", - " tool_output = selected_tool.invoke(tool_call[\"args\"])\n", - " messages.append(ToolMessage(tool_output, tool_call_id=tool_call[\"id\"]))\n", - " return messages\n", - "\n", - "\n", - "tools = [magic_function]" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "ecafcbc6", - "metadata": {}, - "outputs": [], - "source": [ - "llm_with_tools = chat.bind_tools(tools=tools)\n", - "messages = [HumanMessage(content=\"What is the value of magic_function(2)?\")]" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "aa34fc39", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content='The value of magic_function(2) is 12.', additional_kwargs={'documents': [{'id': 'magic_function:0:2:0', 'output': '12', 'tool_name': 'magic_function'}], 'citations': [ChatCitation(start=34, end=36, text='12', document_ids=['magic_function:0:2:0'])], 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '96a55791-0c58-4e2e-bc2a-8550e137c46d', 'token_count': {'input_tokens': 998, 'output_tokens': 59}}, response_metadata={'documents': [{'id': 'magic_function:0:2:0', 'output': '12', 'tool_name': 'magic_function'}], 'citations': [ChatCitation(start=34, end=36, text='12', document_ids=['magic_function:0:2:0'])], 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '96a55791-0c58-4e2e-bc2a-8550e137c46d', 'token_count': {'input_tokens': 998, 'output_tokens': 59}}, id='run-f318a9cf-55c8-44f4-91d1-27cf46c6a465-0')" + "cell_type": "markdown", + "id": "cf690fbb", + "metadata": {}, + "source": [ + "It's also helpful (but not needed) to set up [LangSmith](https://smith.langchain.com/) for best-in-class observability" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3c2fc2201dc80557", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "31f2af10e04dec59", + "metadata": {}, + "source": [ + "## Usage\n", + "\n", + "ChatCohere supports all [ChatModel](/docs/how_to#chat-models) functionality:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fa83b00a929614ad", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_cohere import ChatCohere\n", + "from langchain_core.messages import HumanMessage" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "70cf04e8-423a-4ff6-8b09-f11fb711c817", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "chat = ChatCohere()" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "8199ef8f-eb8b-4253-9ea0-6c24a013ca4c", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='4 && 5 \\n6 || 7 \\n\\nWould you like to play a game of odds and evens?', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '2076b614-52b3-4082-a259-cc92cd3d9fea', 'token_count': {'prompt_tokens': 68, 'response_tokens': 23, 'total_tokens': 91, 'billed_tokens': 77}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '2076b614-52b3-4082-a259-cc92cd3d9fea', 'token_count': {'prompt_tokens': 68, 'response_tokens': 23, 'total_tokens': 91, 'billed_tokens': 77}}, id='run-3475e0c8-c89b-4937-9300-e07d652455e1-0')" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [HumanMessage(content=\"1\"), HumanMessage(content=\"2 3\")]\n", + "chat.invoke(messages)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "c5fac0e9-05a4-4fc1-a3b3-e5bbb24b971b", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='4 && 5', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'f0708a92-f874-46ee-9b93-334d616ad92e', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': 'f0708a92-f874-46ee-9b93-334d616ad92e', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, id='run-1635e63e-2994-4e7f-986e-152ddfc95777-0')" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "await chat.ainvoke(messages)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "025be980-e50d-4a68-93dc-c9c7b500ce34", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4 && 5" + ] + } + ], + "source": [ + "for chunk in chat.stream(messages):\n", + " print(chunk.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "064288e4-f184-4496-9427-bcf148fa055e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[AIMessage(content='4 && 5', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6770ca86-f6c3-4ba3-a285-c4772160612f', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6770ca86-f6c3-4ba3-a285-c4772160612f', 'token_count': {'prompt_tokens': 68, 'response_tokens': 3, 'total_tokens': 71, 'billed_tokens': 57}}, id='run-8d6fade2-1b39-4e31-ab23-4be622dd0027-0')]" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chat.batch([messages])" + ] + }, + { + "cell_type": "markdown", + "id": "f1c56460", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "You can also easily combine with a prompt template for easy structuring of user input. We can do this using [LCEL](/docs/concepts/lcel)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "0851b103", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_template(\"Tell me a joke about {topic}\")\n", + "chain = prompt | chat" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "ae950c0f-1691-47f1-b609-273033cae707", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='What color socks do bears wear?\\n\\nThey don’t wear socks, they have bear feet. \\n\\nHope you laughed! If not, maybe this will help: laughter is the best medicine, and a good sense of humor is infectious!', additional_kwargs={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6edccf44-9bc8-4139-b30e-13b368f3563c', 'token_count': {'prompt_tokens': 68, 'response_tokens': 51, 'total_tokens': 119, 'billed_tokens': 108}}, response_metadata={'documents': None, 'citations': None, 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '6edccf44-9bc8-4139-b30e-13b368f3563c', 'token_count': {'prompt_tokens': 68, 'response_tokens': 51, 'total_tokens': 119, 'billed_tokens': 108}}, id='run-ef7f9789-0d4d-43bf-a4f7-f2a0e27a5320-0')" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chain.invoke({\"topic\": \"bears\"})" + ] + }, + { + "cell_type": "markdown", + "id": "12db8d69", + "metadata": {}, + "source": [ + "## Tool calling\n", + "\n", + "Cohere supports tool calling functionalities!" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "337e24af", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.messages import (\n", + " HumanMessage,\n", + " ToolMessage,\n", + ")\n", + "from langchain_core.tools import tool" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "74d292e7", + "metadata": {}, + "outputs": [], + "source": [ + "@tool\n", + "def magic_function(number: int) -> int:\n", + " \"\"\"Applies a magic operation to an integer\n", + " Args:\n", + " number: Number to have magic operation performed on\n", + " \"\"\"\n", + " return number + 10\n", + "\n", + "\n", + "def invoke_tools(tool_calls, messages):\n", + " for tool_call in tool_calls:\n", + " selected_tool = {\"magic_function\": magic_function}[tool_call[\"name\"].lower()]\n", + " tool_output = selected_tool.invoke(tool_call[\"args\"])\n", + " messages.append(ToolMessage(tool_output, tool_call_id=tool_call[\"id\"]))\n", + " return messages\n", + "\n", + "\n", + "tools = [magic_function]" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "ecafcbc6", + "metadata": {}, + "outputs": [], + "source": [ + "llm_with_tools = chat.bind_tools(tools=tools)\n", + "messages = [HumanMessage(content=\"What is the value of magic_function(2)?\")]" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "aa34fc39", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='The value of magic_function(2) is 12.', additional_kwargs={'documents': [{'id': 'magic_function:0:2:0', 'output': '12', 'tool_name': 'magic_function'}], 'citations': [ChatCitation(start=34, end=36, text='12', document_ids=['magic_function:0:2:0'])], 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '96a55791-0c58-4e2e-bc2a-8550e137c46d', 'token_count': {'input_tokens': 998, 'output_tokens': 59}}, response_metadata={'documents': [{'id': 'magic_function:0:2:0', 'output': '12', 'tool_name': 'magic_function'}], 'citations': [ChatCitation(start=34, end=36, text='12', document_ids=['magic_function:0:2:0'])], 'search_results': None, 'search_queries': None, 'is_search_required': None, 'generation_id': '96a55791-0c58-4e2e-bc2a-8550e137c46d', 'token_count': {'input_tokens': 998, 'output_tokens': 59}}, id='run-f318a9cf-55c8-44f4-91d1-27cf46c6a465-0')" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "res = llm_with_tools.invoke(messages)\n", + "while res.tool_calls:\n", + " messages.append(res)\n", + " messages = invoke_tools(res.tool_calls, messages)\n", + " res = llm_with_tools.invoke(messages)\n", + "\n", + "res" ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "res = llm_with_tools.invoke(messages)\n", - "while res.tool_calls:\n", - " messages.append(res)\n", - " messages = invoke_tools(res.tool_calls, messages)\n", - " res = llm_with_tools.invoke(messages)\n", - "\n", - "res" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + } }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.6" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/deepseek.ipynb b/docs/docs/integrations/chat/deepseek.ipynb index 5004dc85212..f8803a88c59 100644 --- a/docs/docs/integrations/chat/deepseek.ipynb +++ b/docs/docs/integrations/chat/deepseek.ipynb @@ -1,237 +1,235 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: DeepSeek\n", - "---" - ] + "cells": [ + { + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: DeepSeek\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatDeepSeek\n", + "\n", + "\n", + "This will help you getting started with DeepSeek's hosted [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatDeepSeek features and configurations head to the [API reference](https://python.langchain.com/api_reference/deepseek/chat_models/langchain_deepseek.chat_models.ChatDeepSeek.html).\n", + "\n", + ":::tip\n", + "\n", + "DeepSeek's models are open source and can be run locally (e.g. in [Ollama](./ollama.ipynb)) or on other inference providers (e.g. [Fireworks](./fireworks.ipynb), [Together](./together.ipynb)) as well.\n", + "\n", + ":::\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/deepseek) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatDeepSeek](https://python.langchain.com/api_reference/deepseek/chat_models/langchain_deepseek.chat_models.ChatDeepSeek.html) | [langchain-deepseek](https://python.langchain.com/api_reference/deepseek/) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-deepseek?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-deepseek?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + ":::note\n", + "\n", + "DeepSeek-R1, specified via `model=\"deepseek-reasoner\"`, does not support tool calling or structured output. Those features [are supported](https://api-docs.deepseek.com/guides/function_calling) by DeepSeek-V3 (specified via `model=\"deepseek-chat\"`).\n", + "\n", + ":::\n", + "\n", + "## Setup\n", + "\n", + "To access DeepSeek models you'll need to create a/an DeepSeek account, get an API key, and install the `langchain-deepseek` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [DeepSeek's API Key page](https://platform.deepseek.com/api_keys) to sign up to DeepSeek and generate an API key. Once you've done this set the `DEEPSEEK_API_KEY` environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"DEEPSEEK_API_KEY\"):\n", + " os.environ[\"DEEPSEEK_API_KEY\"] = getpass.getpass(\"Enter your DeepSeek API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain DeepSeek integration lives in the `langchain-deepseek` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-deepseek" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_deepseek import ChatDeepSeek\n", + "\n", + "llm = ChatDeepSeek(\n", + " model=\"deepseek-chat\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg.content" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatDeepSeek features and configurations head to the [API Reference](https://python.langchain.com/api_reference/deepseek/chat_models/langchain_deepseek.chat_models.ChatDeepSeek.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatDeepSeek\n", - "\n", - "\n", - "This will help you getting started with DeepSeek's hosted [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatDeepSeek features and configurations head to the [API reference](https://python.langchain.com/api_reference/deepseek/chat_models/langchain_deepseek.chat_models.ChatDeepSeek.html).\n", - "\n", - ":::tip\n", - "\n", - "DeepSeek's models are open source and can be run locally (e.g. in [Ollama](./ollama.ipynb)) or on other inference providers (e.g. [Fireworks](./fireworks.ipynb), [Together](./together.ipynb)) as well.\n", - "\n", - ":::\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/deepseek) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatDeepSeek](https://python.langchain.com/api_reference/deepseek/chat_models/langchain_deepseek.chat_models.ChatDeepSeek.html) | [langchain-deepseek](https://python.langchain.com/api_reference/deepseek/) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-deepseek?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-deepseek?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - ":::note\n", - "\n", - "DeepSeek-R1, specified via `model=\"deepseek-reasoner\"`, does not support tool calling or structured output. Those features [are supported](https://api-docs.deepseek.com/guides/function_calling) by DeepSeek-V3 (specified via `model=\"deepseek-chat\"`).\n", - "\n", - ":::\n", - "\n", - "## Setup\n", - "\n", - "To access DeepSeek models you'll need to create a/an DeepSeek account, get an API key, and install the `langchain-deepseek` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [DeepSeek's API Key page](https://platform.deepseek.com/api_keys) to sign up to DeepSeek and generate an API key. Once you've done this set the `DEEPSEEK_API_KEY` environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"DEEPSEEK_API_KEY\"):\n", - " os.environ[\"DEEPSEEK_API_KEY\"] = getpass.getpass(\"Enter your DeepSeek API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain DeepSeek integration lives in the `langchain-deepseek` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-deepseek" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_deepseek import ChatDeepSeek\n", - "\n", - "llm = ChatDeepSeek(\n", - " model=\"deepseek-chat\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg.content" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatDeepSeek features and configurations head to the [API Reference](https://python.langchain.com/api_reference/deepseek/chat_models/langchain_deepseek.chat_models.ChatDeepSeek.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/fireworks.ipynb b/docs/docs/integrations/chat/fireworks.ipynb index 0172b0a533c..d3ddfccd1e3 100644 --- a/docs/docs/integrations/chat/fireworks.ipynb +++ b/docs/docs/integrations/chat/fireworks.ipynb @@ -1,268 +1,266 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Fireworks\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatFireworks\n", - "\n", - "This doc help you get started with Fireworks AI [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatFireworks features and configurations head to the [API reference](https://python.langchain.com/api_reference/fireworks/chat_models/langchain_fireworks.chat_models.ChatFireworks.html).\n", - "\n", - "Fireworks AI is an AI inference platform to run and customize models. For a list of all models served by Fireworks see the [Fireworks docs](https://fireworks.ai/models).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/fireworks) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatFireworks](https://python.langchain.com/api_reference/fireworks/chat_models/langchain_fireworks.chat_models.ChatFireworks.html) | [langchain-fireworks](https://python.langchain.com/api_reference/fireworks/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-fireworks?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-fireworks?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | \n", - "\n", - "## Setup\n", - "\n", - "To access Fireworks models you'll need to create a Fireworks account, get an API key, and install the `langchain-fireworks` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to (ttps://fireworks.ai/login to sign up to Fireworks and generate an API key. Once you've done this set the FIREWORKS_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"FIREWORKS_API_KEY\" not in os.environ:\n", - " os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Enter your Fireworks API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Fireworks integration lives in the `langchain-fireworks` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-fireworks" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_fireworks import ChatFireworks\n", - "\n", - "llm = ChatFireworks(\n", - " model=\"accounts/fireworks/models/llama-v3-70b-instruct\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'prompt_tokens': 35, 'total_tokens': 44, 'completion_tokens': 9}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-df28e69a-ff30-457e-a743-06eb14d01cb0-0', usage_metadata={'input_tokens': 35, 'output_tokens': 9, 'total_tokens': 44})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Fireworks\n", + "---" ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore la programmation.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'prompt_tokens': 30, 'total_tokens': 37, 'completion_tokens': 7}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-ff3f91ad-ed81-4acf-9f59-7490dc8d8f48-0', usage_metadata={'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatFireworks\n", + "\n", + "This doc help you get started with Fireworks AI [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatFireworks features and configurations head to the [API reference](https://python.langchain.com/api_reference/fireworks/chat_models/langchain_fireworks.chat_models.ChatFireworks.html).\n", + "\n", + "Fireworks AI is an AI inference platform to run and customize models. For a list of all models served by Fireworks see the [Fireworks docs](https://fireworks.ai/models).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/fireworks) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatFireworks](https://python.langchain.com/api_reference/fireworks/chat_models/langchain_fireworks.chat_models.ChatFireworks.html) | [langchain-fireworks](https://python.langchain.com/api_reference/fireworks/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-fireworks?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-fireworks?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |\n", + "\n", + "## Setup\n", + "\n", + "To access Fireworks models you'll need to create a Fireworks account, get an API key, and install the `langchain-fireworks` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to (ttps://fireworks.ai/login to sign up to Fireworks and generate an API key. Once you've done this set the FIREWORKS_API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"FIREWORKS_API_KEY\" not in os.environ:\n", + " os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Enter your Fireworks API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Fireworks integration lives in the `langchain-fireworks` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-fireworks" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_fireworks import ChatFireworks\n", + "\n", + "llm = ChatFireworks(\n", + " model=\"accounts/fireworks/models/llama-v3-70b-instruct\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'prompt_tokens': 35, 'total_tokens': 44, 'completion_tokens': 9}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-df28e69a-ff30-457e-a743-06eb14d01cb0-0', usage_metadata={'input_tokens': 35, 'output_tokens': 9, 'total_tokens': 44})" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore la programmation.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'prompt_tokens': 30, 'total_tokens': 37, 'completion_tokens': 7}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-ff3f91ad-ed81-4acf-9f59-7490dc8d8f48-0', usage_metadata={'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37})" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatFireworks features and configurations head to the API reference: https://python.langchain.com/api_reference/fireworks/chat_models/langchain_fireworks.chat_models.ChatFireworks.html" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatFireworks features and configurations head to the API reference: https://python.langchain.com/api_reference/fireworks/chat_models/langchain_fireworks.chat_models.ChatFireworks.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/goodfire.ipynb b/docs/docs/integrations/chat/goodfire.ipynb index f160a5d8b77..fa40bbb63ab 100644 --- a/docs/docs/integrations/chat/goodfire.ipynb +++ b/docs/docs/integrations/chat/goodfire.ipynb @@ -1,354 +1,352 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Goodfire\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatGoodfire\n", - "\n", - "This will help you getting started with Goodfire [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatGoodfire features and configurations head to the [PyPI project page](https://pypi.org/project/langchain-goodfire/), or go directly to the [Goodfire SDK docs](https://docs.goodfire.ai/sdk-reference/example). All of the Goodfire-specific functionality (e.g. SAE features, variants, etc.) is available via the main `goodfire` package. This integration is a wrapper around the Goodfire SDK.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatGoodfire](https://python.langchain.com/api_reference/goodfire/chat_models/langchain_goodfire.chat_models.ChatGoodfire.html) | [langchain-goodfire](https://python.langchain.com/api_reference/goodfire/) | ❌ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-goodfire?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-goodfire?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access Goodfire models you'll need to create a/an Goodfire account, get an API key, and install the `langchain-goodfire` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [Goodfire Settings](https://platform.goodfire.ai/organization/settings/api-keys) to sign up to Goodfire and generate an API key. Once you've done this set the GOODFIRE_API_KEY environment variable." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"GOODFIRE_API_KEY\"):\n", - " os.environ[\"GOODFIRE_API_KEY\"] = getpass.getpass(\"Enter your Goodfire API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Goodfire integration lives in the `langchain-goodfire` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain-goodfire" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.\n" - ] - } - ], - "source": [ - "import goodfire\n", - "from langchain_goodfire import ChatGoodfire\n", - "\n", - "base_variant = goodfire.Variant(\"meta-llama/Llama-3.3-70B-Instruct\")\n", - "\n", - "llm = ChatGoodfire(\n", - " model=base_variant,\n", - " temperature=0,\n", - " max_completion_tokens=1000,\n", - " seed=42,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore la programmation.\", additional_kwargs={}, response_metadata={}, id='run-8d43cf35-bce8-4827-8935-c64f8fb78cd0-0', usage_metadata={'input_tokens': 51, 'output_tokens': 39, 'total_tokens': 90})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Goodfire\n", + "---" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = await llm.ainvoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore la programmation.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe das Programmieren. How can I help you with programming today?', additional_kwargs={}, response_metadata={}, id='run-03d1a585-8234-46f1-a8df-bf9143fe3309-0', usage_metadata={'input_tokens': 46, 'output_tokens': 46, 'total_tokens': 92})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatGoodfire\n", + "\n", + "This will help you getting started with Goodfire [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatGoodfire features and configurations head to the [PyPI project page](https://pypi.org/project/langchain-goodfire/), or go directly to the [Goodfire SDK docs](https://docs.goodfire.ai/sdk-reference/example). All of the Goodfire-specific functionality (e.g. SAE features, variants, etc.) is available via the main `goodfire` package. This integration is a wrapper around the Goodfire SDK.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatGoodfire](https://python.langchain.com/api_reference/goodfire/chat_models/langchain_goodfire.chat_models.ChatGoodfire.html) | [langchain-goodfire](https://python.langchain.com/api_reference/goodfire/) | ❌ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-goodfire?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-goodfire?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "To access Goodfire models you'll need to create a/an Goodfire account, get an API key, and install the `langchain-goodfire` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [Goodfire Settings](https://platform.goodfire.ai/organization/settings/api-keys) to sign up to Goodfire and generate an API key. Once you've done this set the GOODFIRE_API_KEY environment variable." ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "await chain.ainvoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## Goodfire-specific functionality\n", - "\n", - "To use Goodfire-specific functionality such as SAE features and variants, you can use the `goodfire` package directly." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "3aef9e0a", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "FeatureGroup([\n", - " 0: \"The assistant should adopt the persona of a pirate\",\n", - " 1: \"The assistant should roleplay as a pirate\",\n", - " 2: \"The assistant should engage with pirate-themed content or roleplay as a pirate\",\n", - " 3: \"The assistant should roleplay as a character\",\n", - " 4: \"The assistant should roleplay as a specific character\",\n", - " 5: \"The assistant should roleplay as a game character or NPC\",\n", - " 6: \"The assistant should roleplay as a human character\",\n", - " 7: \"Requests for the assistant to roleplay or pretend to be something else\",\n", - " 8: \"Requests for the assistant to roleplay or pretend to be something\",\n", - " 9: \"The assistant is being assigned a role or persona to roleplay\"\n", - "])" + "cell_type": "code", + "execution_count": 1, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"GOODFIRE_API_KEY\"):\n", + " os.environ[\"GOODFIRE_API_KEY\"] = getpass.getpass(\"Enter your Goodfire API key: \")" ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "client = goodfire.Client(api_key=os.environ[\"GOODFIRE_API_KEY\"])\n", - "\n", - "pirate_features = client.features.search(\n", - " \"assistant should roleplay as a pirate\", base_variant\n", - ")\n", - "pirate_features" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "52f03a00", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content='Why did the scarecrow win an award? Because he was outstanding in his field! Arrr! Hope that made ye laugh, matey!', additional_kwargs={}, response_metadata={}, id='run-7d8bd30f-7f80-41cb-bdb6-25c29c22a7ce-0', usage_metadata={'input_tokens': 35, 'output_tokens': 60, 'total_tokens': 95})" + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Goodfire integration lives in the `langchain-goodfire` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain-goodfire" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.\n" + ] + } + ], + "source": [ + "import goodfire\n", + "from langchain_goodfire import ChatGoodfire\n", + "\n", + "base_variant = goodfire.Variant(\"meta-llama/Llama-3.3-70B-Instruct\")\n", + "\n", + "llm = ChatGoodfire(\n", + " model=base_variant,\n", + " temperature=0,\n", + " max_completion_tokens=1000,\n", + " seed=42,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore la programmation.\", additional_kwargs={}, response_metadata={}, id='run-8d43cf35-bce8-4827-8935-c64f8fb78cd0-0', usage_metadata={'input_tokens': 51, 'output_tokens': 39, 'total_tokens': 90})" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = await llm.ainvoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore la programmation.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren. How can I help you with programming today?', additional_kwargs={}, response_metadata={}, id='run-03d1a585-8234-46f1-a8df-bf9143fe3309-0', usage_metadata={'input_tokens': 46, 'output_tokens': 46, 'total_tokens': 92})" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "await chain.ainvoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## Goodfire-specific functionality\n", + "\n", + "To use Goodfire-specific functionality such as SAE features and variants, you can use the `goodfire` package directly." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "3aef9e0a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "FeatureGroup([\n", + " 0: \"The assistant should adopt the persona of a pirate\",\n", + " 1: \"The assistant should roleplay as a pirate\",\n", + " 2: \"The assistant should engage with pirate-themed content or roleplay as a pirate\",\n", + " 3: \"The assistant should roleplay as a character\",\n", + " 4: \"The assistant should roleplay as a specific character\",\n", + " 5: \"The assistant should roleplay as a game character or NPC\",\n", + " 6: \"The assistant should roleplay as a human character\",\n", + " 7: \"Requests for the assistant to roleplay or pretend to be something else\",\n", + " 8: \"Requests for the assistant to roleplay or pretend to be something\",\n", + " 9: \"The assistant is being assigned a role or persona to roleplay\"\n", + "])" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "client = goodfire.Client(api_key=os.environ[\"GOODFIRE_API_KEY\"])\n", + "\n", + "pirate_features = client.features.search(\n", + " \"assistant should roleplay as a pirate\", base_variant\n", + ")\n", + "pirate_features" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "52f03a00", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Why did the scarecrow win an award? Because he was outstanding in his field! Arrr! Hope that made ye laugh, matey!', additional_kwargs={}, response_metadata={}, id='run-7d8bd30f-7f80-41cb-bdb6-25c29c22a7ce-0', usage_metadata={'input_tokens': 35, 'output_tokens': 60, 'total_tokens': 95})" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pirate_variant = goodfire.Variant(\"meta-llama/Llama-3.3-70B-Instruct\")\n", + "\n", + "pirate_variant.set(pirate_features[0], 0.4)\n", + "pirate_variant.set(pirate_features[1], 0.3)\n", + "\n", + "await llm.ainvoke(\"Tell me a joke\", model=pirate_variant)" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatGoodfire features and configurations head to the [API reference](https://python.langchain.com/api_reference/goodfire/chat_models/langchain_goodfire.chat_models.ChatGoodfire.html)" ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "pirate_variant = goodfire.Variant(\"meta-llama/Llama-3.3-70B-Instruct\")\n", - "\n", - "pirate_variant.set(pirate_features[0], 0.4)\n", - "pirate_variant.set(pirate_features[1], 0.3)\n", - "\n", - "await llm.ainvoke(\"Tell me a joke\", model=pirate_variant)" - ] + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.8" + } }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatGoodfire features and configurations head to the [API reference](https://python.langchain.com/api_reference/goodfire/chat_models/langchain_goodfire.chat_models.ChatGoodfire.html)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": ".venv", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.12.8" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/google_generative_ai.ipynb b/docs/docs/integrations/chat/google_generative_ai.ipynb index 7492e641d4c..f05abef2a14 100644 --- a/docs/docs/integrations/chat/google_generative_ai.ipynb +++ b/docs/docs/integrations/chat/google_generative_ai.ipynb @@ -1,314 +1,312 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Google AI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatGoogleGenerativeAI\n", - "\n", - "This docs will help you get started with Google AI [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatGoogleGenerativeAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/google_genai/chat_models/langchain_google_genai.chat_models.ChatGoogleGenerativeAI.html).\n", - "\n", - "Google AI offers a number of different chat models. For information on the latest models, their features, context windows, etc. head to the [Google AI docs](https://ai.google.dev/gemini-api/docs/models/gemini).\n", - "\n", - ":::info Google AI vs Google Cloud Vertex AI\n", - "\n", - "Google's Gemini models are accessible through Google AI and through Google Cloud Vertex AI. Using Google AI just requires a Google account and an API key. Using Google Cloud Vertex AI requires a Google Cloud account (with term agreements and billing) but offers enterprise features like customer encription key, virtual private cloud, and more.\n", - "\n", - "To learn more about the key features of the two APIs see the [Google docs](https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/migrate-google-ai#google-ai).\n", - "\n", - ":::\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/google_generativeai) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatGoogleGenerativeAI](https://python.langchain.com/api_reference/google_genai/chat_models/langchain_google_genai.chat_models.ChatGoogleGenerativeAI.html) | [langchain-google-genai](https://python.langchain.com/api_reference/google_genai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-google-genai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-google-genai?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access Google AI models you'll need to create a Google Acount account, get a Google AI API key, and install the `langchain-google-genai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to https://ai.google.dev/gemini-api/docs/api-key to generate a Google AI API key. Once you've done this set the GOOGLE_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"GOOGLE_API_KEY\" not in os.environ:\n", - " os.environ[\"GOOGLE_API_KEY\"] = getpass.getpass(\"Enter your Google AI API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Google AI integration lives in the `langchain-google-genai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-google-genai" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_google_genai import ChatGoogleGenerativeAI\n", - "\n", - "llm = ChatGoogleGenerativeAI(\n", - " model=\"gemini-1.5-pro\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore programmer. \\n\", response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'safety_ratings': [{'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability': 'NEGLIGIBLE', 'blocked': False}]}, id='run-eef5b138-1da6-4226-9cfe-ab9073ddd77e-0', usage_metadata={'input_tokens': 21, 'output_tokens': 5, 'total_tokens': 26})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Google AI\n", + "---" ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore programmer. \n", - "\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe das Programmieren. \\n', response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'safety_ratings': [{'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability': 'NEGLIGIBLE', 'blocked': False}]}, id='run-fbb35f30-4937-4a81-ae68-f7cb35721a0c-0', usage_metadata={'input_tokens': 16, 'output_tokens': 7, 'total_tokens': 23})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatGoogleGenerativeAI\n", + "\n", + "This docs will help you get started with Google AI [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatGoogleGenerativeAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/google_genai/chat_models/langchain_google_genai.chat_models.ChatGoogleGenerativeAI.html).\n", + "\n", + "Google AI offers a number of different chat models. For information on the latest models, their features, context windows, etc. head to the [Google AI docs](https://ai.google.dev/gemini-api/docs/models/gemini).\n", + "\n", + ":::info Google AI vs Google Cloud Vertex AI\n", + "\n", + "Google's Gemini models are accessible through Google AI and through Google Cloud Vertex AI. Using Google AI just requires a Google account and an API key. Using Google Cloud Vertex AI requires a Google Cloud account (with term agreements and billing) but offers enterprise features like customer encription key, virtual private cloud, and more.\n", + "\n", + "To learn more about the key features of the two APIs see the [Google docs](https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/migrate-google-ai#google-ai).\n", + "\n", + ":::\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/google_generativeai) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatGoogleGenerativeAI](https://python.langchain.com/api_reference/google_genai/chat_models/langchain_google_genai.chat_models.ChatGoogleGenerativeAI.html) | [langchain-google-genai](https://python.langchain.com/api_reference/google_genai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-google-genai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-google-genai?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "To access Google AI models you'll need to create a Google Acount account, get a Google AI API key, and install the `langchain-google-genai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to https://ai.google.dev/gemini-api/docs/api-key to generate a Google AI API key. Once you've done this set the GOOGLE_API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"GOOGLE_API_KEY\" not in os.environ:\n", + " os.environ[\"GOOGLE_API_KEY\"] = getpass.getpass(\"Enter your Google AI API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Google AI integration lives in the `langchain-google-genai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-google-genai" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_google_genai import ChatGoogleGenerativeAI\n", + "\n", + "llm = ChatGoogleGenerativeAI(\n", + " model=\"gemini-1.5-pro\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore programmer. \\n\", response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'safety_ratings': [{'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability': 'NEGLIGIBLE', 'blocked': False}]}, id='run-eef5b138-1da6-4226-9cfe-ab9073ddd77e-0', usage_metadata={'input_tokens': 21, 'output_tokens': 5, 'total_tokens': 26})" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore programmer. \n", + "\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren. \\n', response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'safety_ratings': [{'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability': 'NEGLIGIBLE', 'blocked': False}]}, id='run-fbb35f30-4937-4a81-ae68-f7cb35721a0c-0', usage_metadata={'input_tokens': 16, 'output_tokens': 7, 'total_tokens': 23})" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## Safety Settings\n", + "\n", + "Gemini models have default safety settings that can be overridden. If you are receiving lots of \"Safety Warnings\" from your models, you can try tweaking the `safety_settings` attribute of the model. For example, to turn off safety blocking for dangerous content, you can construct your LLM as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "238b2f96-e573-4fac-bbf2-7e52ad926833", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_google_genai import (\n", + " ChatGoogleGenerativeAI,\n", + " HarmBlockThreshold,\n", + " HarmCategory,\n", + ")\n", + "\n", + "llm = ChatGoogleGenerativeAI(\n", + " model=\"gemini-1.5-pro\",\n", + " safety_settings={\n", + " HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,\n", + " },\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "5805d40c-deb8-4924-8e72-a294a0482fc9", + "metadata": {}, + "source": [ + "For an enumeration of the categories and thresholds available, see Google's [safety setting types](https://ai.google.dev/api/python/google/generativeai/types/SafetySettingDict)." + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatGoogleGenerativeAI features and configurations head to the API reference: https://python.langchain.com/api_reference/google_genai/chat_models/langchain_google_genai.chat_models.ChatGoogleGenerativeAI.html" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "poetry-venv-2", + "language": "python", + "name": "poetry-venv-2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## Safety Settings\n", - "\n", - "Gemini models have default safety settings that can be overridden. If you are receiving lots of \"Safety Warnings\" from your models, you can try tweaking the `safety_settings` attribute of the model. For example, to turn off safety blocking for dangerous content, you can construct your LLM as follows:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "238b2f96-e573-4fac-bbf2-7e52ad926833", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_google_genai import (\n", - " ChatGoogleGenerativeAI,\n", - " HarmBlockThreshold,\n", - " HarmCategory,\n", - ")\n", - "\n", - "llm = ChatGoogleGenerativeAI(\n", - " model=\"gemini-1.5-pro\",\n", - " safety_settings={\n", - " HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,\n", - " },\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "5805d40c-deb8-4924-8e72-a294a0482fc9", - "metadata": {}, - "source": [ - "For an enumeration of the categories and thresholds available, see Google's [safety setting types](https://ai.google.dev/api/python/google/generativeai/types/SafetySettingDict)." - ] - }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatGoogleGenerativeAI features and configurations head to the API reference: https://python.langchain.com/api_reference/google_genai/chat_models/langchain_google_genai.chat_models.ChatGoogleGenerativeAI.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "poetry-venv-2", - "language": "python", - "name": "poetry-venv-2" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb b/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb index 4709faa8564..f626ee77123 100644 --- a/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb +++ b/docs/docs/integrations/chat/google_vertex_ai_palm.ipynb @@ -1,269 +1,269 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Google Cloud Vertex AI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatVertexAI\n", - "\n", - "This page provides a quick overview for getting started with VertexAI [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatVertexAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/google_vertexai/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html).\n", - "\n", - "ChatVertexAI exposes all foundational models available in Google Cloud, like `gemini-1.5-pro`, `gemini-1.5-flash`, etc. For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview).\n", - "\n", - ":::info Google Cloud VertexAI vs Google PaLM\n", - "\n", - "The Google Cloud VertexAI integration is separate from the [Google PaLM integration](/docs/integrations/chat/google_generative_ai/). Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n", - "\n", - ":::\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/google_vertex_ai) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatVertexAI](https://python.langchain.com/api_reference/google_vertexai/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html) | [langchain-google-vertexai](https://python.langchain.com/api_reference/google_vertexai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-google-vertexai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-google-vertexai?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access VertexAI models you'll need to create a Google Cloud Platform account, set up credentials, and install the `langchain-google-vertexai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "To use the integration you must:\n", - "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n", - "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n", - "\n", - "This codebase uses the `google.auth` library which first looks for the application credentials variable mentioned above, and then looks for system-level auth.\n", - "\n", - "For more information, see: \n", - "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n", - "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n", - "\n", - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain VertexAI integration lives in the `langchain-google-vertexai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain-google-vertexai" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_google_vertexai import ChatVertexAI\n", - "\n", - "llm = ChatVertexAI(\n", - " model=\"gemini-1.5-flash-001\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " max_retries=6,\n", - " stop=None,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore programmer. \\n\", response_metadata={'is_blocked': False, 'safety_ratings': [{'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}], 'usage_metadata': {'prompt_token_count': 20, 'candidates_token_count': 7, 'total_token_count': 27}}, id='run-7032733c-d05c-4f0c-a17a-6c575fdd1ae0-0', usage_metadata={'input_tokens': 20, 'output_tokens': 7, 'total_tokens': 27})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Google Cloud Vertex AI\n", + "---" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore programmer. \n", - "\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe Programmieren. \\n', response_metadata={'is_blocked': False, 'safety_ratings': [{'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}], 'usage_metadata': {'prompt_token_count': 15, 'candidates_token_count': 8, 'total_token_count': 23}}, id='run-c71955fd-8dc1-422b-88a7-853accf4811b-0', usage_metadata={'input_tokens': 15, 'output_tokens': 8, 'total_tokens': 23})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatVertexAI\n", + "\n", + "This page provides a quick overview for getting started with VertexAI [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatVertexAI features and configurations head to the [API reference](https://python.langchain.com/api_reference/google_vertexai/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html).\n", + "\n", + "ChatVertexAI exposes all foundational models available in Google Cloud, like `gemini-1.5-pro`, `gemini-1.5-flash`, etc. For a full and updated list of available models visit [VertexAI documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/overview).\n", + "\n", + ":::info Google Cloud VertexAI vs Google PaLM\n", + "\n", + "The Google Cloud VertexAI integration is separate from the [Google PaLM integration](/docs/integrations/chat/google_generative_ai/). Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there.\n", + "\n", + ":::\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/google_vertex_ai) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatVertexAI](https://python.langchain.com/api_reference/google_vertexai/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html) | [langchain-google-vertexai](https://python.langchain.com/api_reference/google_vertexai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-google-vertexai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-google-vertexai?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "To access VertexAI models you'll need to create a Google Cloud Platform account, set up credentials, and install the `langchain-google-vertexai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "To use the integration you must:\n", + "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n", + "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n", + "\n", + "This codebase uses the `google.auth` library which first looks for the application credentials variable mentioned above, and then looks for system-level auth.\n", + "\n", + "For more information, see:\n", + "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n", + "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n", + "\n", + "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain VertexAI integration lives in the `langchain-google-vertexai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain-google-vertexai" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_google_vertexai import ChatVertexAI\n", + "\n", + "llm = ChatVertexAI(\n", + " model=\"gemini-1.5-flash-001\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " max_retries=6,\n", + " stop=None,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore programmer. \\n\", response_metadata={'is_blocked': False, 'safety_ratings': [{'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}], 'usage_metadata': {'prompt_token_count': 20, 'candidates_token_count': 7, 'total_token_count': 27}}, id='run-7032733c-d05c-4f0c-a17a-6c575fdd1ae0-0', usage_metadata={'input_tokens': 20, 'output_tokens': 7, 'total_tokens': 27})" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore programmer. \n", + "\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe Programmieren. \\n', response_metadata={'is_blocked': False, 'safety_ratings': [{'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}, {'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability_label': 'NEGLIGIBLE', 'blocked': False}], 'usage_metadata': {'prompt_token_count': 15, 'candidates_token_count': 8, 'total_token_count': 23}}, id='run-c71955fd-8dc1-422b-88a7-853accf4811b-0', usage_metadata={'input_tokens': 15, 'output_tokens': 8, 'total_tokens': 23})" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatVertexAI features and configurations, like how to send multimodal inputs and configure safety settings, head to the API reference: https://python.langchain.com/api_reference/google_vertexai/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "poetry-venv-2", + "language": "python", + "name": "poetry-venv-2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.1" + } }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatVertexAI features and configurations, like how to send multimodal inputs and configure safety settings, head to the API reference: https://python.langchain.com/api_reference/google_vertexai/chat_models/langchain_google_vertexai.chat_models.ChatVertexAI.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "poetry-venv-2", - "language": "python", - "name": "poetry-venv-2" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.1" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/groq.ipynb b/docs/docs/integrations/chat/groq.ipynb index a1696fba7a4..7ec9c7ac8a5 100644 --- a/docs/docs/integrations/chat/groq.ipynb +++ b/docs/docs/integrations/chat/groq.ipynb @@ -1,266 +1,264 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Groq\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatGroq\n", - "\n", - "This will help you getting started with Groq [chat models](../../concepts/chat_models.mdx). For detailed documentation of all ChatGroq features and configurations head to the [API reference](https://python.langchain.com/api_reference/groq/chat_models/langchain_groq.chat_models.ChatGroq.html). For a list of all Groq models, visit this [link](https://console.groq.com/docs/models?utm_source=langchain).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/groq) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatGroq](https://python.langchain.com/api_reference/groq/chat_models/langchain_groq.chat_models.ChatGroq.html) | [langchain-groq](https://python.langchain.com/api_reference/groq/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-groq?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-groq?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | \n", - "\n", - "## Setup\n", - "\n", - "To access Groq models you'll need to create a Groq account, get an API key, and install the `langchain-groq` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to the [Groq console](https://console.groq.com/login?utm_source=langchain&utm_content=chat_page) to sign up to Groq and generate an API key. Once you've done this set the GROQ_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"GROQ_API_KEY\" not in os.environ:\n", - " os.environ[\"GROQ_API_KEY\"] = getpass.getpass(\"Enter your Groq API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Groq integration lives in the `langchain-groq` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3f3f510e-2afe-4e76-be41-c5a9665aea63", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-groq" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_groq import ChatGroq\n", - "\n", - "llm = ChatGroq(\n", - " model=\"llama-3.1-8b-instant\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content='The translation of \"I love programming\" to French is:\\n\\n\"J\\'adore le programmation.\"', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 22, 'prompt_tokens': 55, 'total_tokens': 77, 'completion_time': 0.029333333, 'prompt_time': 0.003502892, 'queue_time': 0.553054073, 'total_time': 0.032836225}, 'model_name': 'llama-3.1-8b-instant', 'system_fingerprint': 'fp_a491995411', 'finish_reason': 'stop', 'logprobs': None}, id='run-2b2da04a-993c-40ab-becc-201eab8b1a1b-0', usage_metadata={'input_tokens': 55, 'output_tokens': 22, 'total_tokens': 77})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Groq\n", + "---" ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "The translation of \"I love programming\" to French is:\n", - "\n", - "\"J'adore le programmation.\"\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe Programmieren.', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 50, 'total_tokens': 56, 'completion_time': 0.008, 'prompt_time': 0.003337935, 'queue_time': 0.20949214500000002, 'total_time': 0.011337935}, 'model_name': 'llama-3.1-8b-instant', 'system_fingerprint': 'fp_a491995411', 'finish_reason': 'stop', 'logprobs': None}, id='run-e33b48dc-5e55-466e-9ebd-7b48c81c3cbd-0', usage_metadata={'input_tokens': 50, 'output_tokens': 6, 'total_tokens': 56})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatGroq\n", + "\n", + "This will help you getting started with Groq [chat models](../../concepts/chat_models.mdx). For detailed documentation of all ChatGroq features and configurations head to the [API reference](https://python.langchain.com/api_reference/groq/chat_models/langchain_groq.chat_models.ChatGroq.html). For a list of all Groq models, visit this [link](https://console.groq.com/docs/models?utm_source=langchain).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/groq) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatGroq](https://python.langchain.com/api_reference/groq/chat_models/langchain_groq.chat_models.ChatGroq.html) | [langchain-groq](https://python.langchain.com/api_reference/groq/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-groq?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-groq?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |\n", + "\n", + "## Setup\n", + "\n", + "To access Groq models you'll need to create a Groq account, get an API key, and install the `langchain-groq` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to the [Groq console](https://console.groq.com/login?utm_source=langchain&utm_content=chat_page) to sign up to Groq and generate an API key. Once you've done this set the GROQ_API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"GROQ_API_KEY\" not in os.environ:\n", + " os.environ[\"GROQ_API_KEY\"] = getpass.getpass(\"Enter your Groq API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Groq integration lives in the `langchain-groq` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3f3f510e-2afe-4e76-be41-c5a9665aea63", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-groq" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_groq import ChatGroq\n", + "\n", + "llm = ChatGroq(\n", + " model=\"llama-3.1-8b-instant\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='The translation of \"I love programming\" to French is:\\n\\n\"J\\'adore le programmation.\"', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 22, 'prompt_tokens': 55, 'total_tokens': 77, 'completion_time': 0.029333333, 'prompt_time': 0.003502892, 'queue_time': 0.553054073, 'total_time': 0.032836225}, 'model_name': 'llama-3.1-8b-instant', 'system_fingerprint': 'fp_a491995411', 'finish_reason': 'stop', 'logprobs': None}, id='run-2b2da04a-993c-40ab-becc-201eab8b1a1b-0', usage_metadata={'input_tokens': 55, 'output_tokens': 22, 'total_tokens': 77})" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The translation of \"I love programming\" to French is:\n", + "\n", + "\"J'adore le programmation.\"\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe Programmieren.', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 50, 'total_tokens': 56, 'completion_time': 0.008, 'prompt_time': 0.003337935, 'queue_time': 0.20949214500000002, 'total_time': 0.011337935}, 'model_name': 'llama-3.1-8b-instant', 'system_fingerprint': 'fp_a491995411', 'finish_reason': 'stop', 'logprobs': None}, id='run-e33b48dc-5e55-466e-9ebd-7b48c81c3cbd-0', usage_metadata={'input_tokens': 50, 'output_tokens': 6, 'total_tokens': 56})" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatGroq features and configurations head to the API reference: https://python.langchain.com/api_reference/groq/chat_models/langchain_groq.chat_models.ChatGroq.html" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatGroq features and configurations head to the API reference: https://python.langchain.com/api_reference/groq/chat_models/langchain_groq.chat_models.ChatGroq.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/mistralai.ipynb b/docs/docs/integrations/chat/mistralai.ipynb index 8ed4de91727..b300f7a42da 100644 --- a/docs/docs/integrations/chat/mistralai.ipynb +++ b/docs/docs/integrations/chat/mistralai.ipynb @@ -1,262 +1,260 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "53fbf15f", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: MistralAI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "d295c2a2", - "metadata": {}, - "source": [ - "# ChatMistralAI\n", - "\n", - "This will help you getting started with Mistral [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatMistralAI` features and configurations head to the [API reference](https://python.langchain.com/api_reference/mistralai/chat_models/langchain_mistralai.chat_models.ChatMistralAI.html). The `ChatMistralAI` class is built on top of the [Mistral API](https://docs.mistral.ai/api/). For a list of all the models supported by Mistral, check out [this page](https://docs.mistral.ai/getting-started/models/).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/mistral) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatMistralAI](https://python.langchain.com/api_reference/mistralai/chat_models/langchain_mistralai.chat_models.ChatMistralAI.html) | [langchain_mistralai](https://python.langchain.com/api_reference/mistralai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_mistralai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_mistralai?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "\n", - "To access `ChatMistralAI` models you'll need to create a Mistral account, get an API key, and install the `langchain_mistralai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "\n", - "A valid [API key](https://console.mistral.ai/api-keys/) is needed to communicate with the API. Once you've done this set the MISTRAL_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2461605e", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"MISTRAL_API_KEY\" not in os.environ:\n", - " os.environ[\"MISTRAL_API_KEY\"] = getpass.getpass(\"Enter your Mistral API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "788f37ac", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "007209d5", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0f5c74f9", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Mistral integration lives in the `langchain_mistralai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1ab11a65", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_mistralai" - ] - }, - { - "cell_type": "markdown", - "id": "fb1a335e", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "e6c38580", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_mistralai import ChatMistralAI\n", - "\n", - "llm = ChatMistralAI(\n", - " model=\"mistral-large-latest\",\n", - " temperature=0,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "aec79099", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "8838c3cc", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content='Sure, I\\'d be happy to help you translate that sentence into French! The English sentence \"I love programming\" translates to \"J\\'aime programmer\" in French. Let me know if you have any other questions or need further assistance!', response_metadata={'token_usage': {'prompt_tokens': 32, 'total_tokens': 84, 'completion_tokens': 52}, 'model': 'mistral-small', 'finish_reason': 'stop'}, id='run-64bac156-7160-4b68-b67e-4161f63e021f-0', usage_metadata={'input_tokens': 32, 'output_tokens': 52, 'total_tokens': 84})" + "cell_type": "raw", + "id": "53fbf15f", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: MistralAI\n", + "---" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "bbf6a048", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Sure, I'd be happy to help you translate that sentence into French! The English sentence \"I love programming\" translates to \"J'aime programmer\" in French. Let me know if you have any other questions or need further assistance!\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "32b87f87", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "24e2c51c", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe Programmierung. (German translation)', response_metadata={'token_usage': {'prompt_tokens': 26, 'total_tokens': 38, 'completion_tokens': 12}, 'model': 'mistral-small', 'finish_reason': 'stop'}, id='run-dfd4094f-e347-47b0-9056-8ebd7ea35fe7-0', usage_metadata={'input_tokens': 26, 'output_tokens': 12, 'total_tokens': 38})" + "cell_type": "markdown", + "id": "d295c2a2", + "metadata": {}, + "source": [ + "# ChatMistralAI\n", + "\n", + "This will help you getting started with Mistral [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatMistralAI` features and configurations head to the [API reference](https://python.langchain.com/api_reference/mistralai/chat_models/langchain_mistralai.chat_models.ChatMistralAI.html). The `ChatMistralAI` class is built on top of the [Mistral API](https://docs.mistral.ai/api/). For a list of all the models supported by Mistral, check out [this page](https://docs.mistral.ai/getting-started/models/).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/mistral) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatMistralAI](https://python.langchain.com/api_reference/mistralai/chat_models/langchain_mistralai.chat_models.ChatMistralAI.html) | [langchain_mistralai](https://python.langchain.com/api_reference/mistralai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_mistralai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_mistralai?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "\n", + "To access `ChatMistralAI` models you'll need to create a Mistral account, get an API key, and install the `langchain_mistralai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "\n", + "A valid [API key](https://console.mistral.ai/api-keys/) is needed to communicate with the API. Once you've done this set the MISTRAL_API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2461605e", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"MISTRAL_API_KEY\" not in os.environ:\n", + " os.environ[\"MISTRAL_API_KEY\"] = getpass.getpass(\"Enter your Mistral API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "788f37ac", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "007209d5", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0f5c74f9", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Mistral integration lives in the `langchain_mistralai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1ab11a65", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_mistralai" + ] + }, + { + "cell_type": "markdown", + "id": "fb1a335e", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "e6c38580", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_mistralai import ChatMistralAI\n", + "\n", + "llm = ChatMistralAI(\n", + " model=\"mistral-large-latest\",\n", + " temperature=0,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "aec79099", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "8838c3cc", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Sure, I\\'d be happy to help you translate that sentence into French! The English sentence \"I love programming\" translates to \"J\\'aime programmer\" in French. Let me know if you have any other questions or need further assistance!', response_metadata={'token_usage': {'prompt_tokens': 32, 'total_tokens': 84, 'completion_tokens': 52}, 'model': 'mistral-small', 'finish_reason': 'stop'}, id='run-64bac156-7160-4b68-b67e-4161f63e021f-0', usage_metadata={'input_tokens': 32, 'output_tokens': 52, 'total_tokens': 84})" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "bbf6a048", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Sure, I'd be happy to help you translate that sentence into French! The English sentence \"I love programming\" translates to \"J'aime programmer\" in French. Let me know if you have any other questions or need further assistance!\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "32b87f87", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "24e2c51c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe Programmierung. (German translation)', response_metadata={'token_usage': {'prompt_tokens': 26, 'total_tokens': 38, 'completion_tokens': 12}, 'model': 'mistral-small', 'finish_reason': 'stop'}, id='run-dfd4094f-e347-47b0-9056-8ebd7ea35fe7-0', usage_metadata={'input_tokens': 26, 'output_tokens': 12, 'total_tokens': 38})" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "cb9b5834", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "Head to the [API reference](https://python.langchain.com/api_reference/mistralai/chat_models/langchain_mistralai.chat_models.ChatMistralAI.html) for detailed documentation of all attributes and methods." ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "id": "cb9b5834", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "Head to the [API reference](https://python.langchain.com/api_reference/mistralai/chat_models/langchain_mistralai.chat_models.ChatMistralAI.html) for detailed documentation of all attributes and methods." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/naver.ipynb b/docs/docs/integrations/chat/naver.ipynb index c0852e099c4..4b4ce1ed4c1 100644 --- a/docs/docs/integrations/chat/naver.ipynb +++ b/docs/docs/integrations/chat/naver.ipynb @@ -1,444 +1,442 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Naver\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "c8444f1a-e907-4f07-b8b6-68fbedfb868e", - "metadata": {}, - "source": [ - "# ChatClovaX\n", - "\n", - "This notebook provides a quick overview for getting started with Naver’s HyperCLOVA X [chat models](https://python.langchain.com/docs/concepts/chat_models) via CLOVA Studio. For detailed documentation of all ChatClovaX features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.naver.ChatClovaX.html).\n", - "\n", - "[CLOVA Studio](http://clovastudio.ncloud.com/) has several chat models. You can find information about latest models and their costs, context windows, and supported input types in the CLOVA Studio API Guide [documentation](https://api.ncloud-docs.com/docs/clovastudio-chatcompletions).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", - "| :--- | :--- |:-----:| :---: |:------------------------------------------------------------------------:| :---: | :---: |\n", - "| [ChatClovaX](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.naver.ChatClovaX.html) | [langchain-community](https://python.langchain.com/api_reference/community/index.html) | ❌ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_community?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_community?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "|:------------------------------------------:| :---: | :---: | :---: | :---: | :---: |:-----------------------------------------------------:| :---: |:------------------------------------------------------:|:----------------------------------:|\n", - "|❌| ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "Before using the chat model, you must go through the four steps below.\n", - "\n", - "1. Creating [NAVER Cloud Platform](https://www.ncloud.com/) account \n", - "2. Apply to use [CLOVA Studio](https://www.ncloud.com/product/aiService/clovaStudio)\n", - "3. Create a CLOVA Studio Test App or Service App of a model to use (See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#테스트앱생성).)\n", - "4. Issue a Test or Service API key (See [here](https://api.ncloud-docs.com/docs/ai-naver-clovastudio-summary#API%ED%82%A4).)\n", - "\n", - "### Credentials\n", - "\n", - "Set the `NCP_CLOVASTUDIO_API_KEY` environment variable with your API key.\n", - " - Note that if you are using a legacy API Key (that doesn't start with `nv-*` prefix), you might need to get an additional API Key by clicking `App Request Status` > `Service App, Test App List` > `‘Details’ button for each app` in [CLOVA Studio](https://clovastudio.ncloud.com/studio-application/service-app) and set it as `NCP_APIGW_API_KEY`.\n", - "\n", - "You can add them to your environment variables as below:\n", - "\n", - "``` bash\n", - "export NCP_CLOVASTUDIO_API_KEY=\"your-api-key-here\"\n", - "# Uncomment below to use a legacy API key\n", - "# export NCP_APIGW_API_KEY=\"your-api-key-here\"\n", - "```" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2def81b5-b023-4f40-a97b-b2c5ca59d6a9", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"NCP_CLOVASTUDIO_API_KEY\"):\n", - " os.environ[\"NCP_CLOVASTUDIO_API_KEY\"] = getpass.getpass(\n", - " \"Enter your NCP CLOVA Studio API Key: \"\n", - " )\n", - "# Uncomment below to use a legacy API key\n", - "# if not os.getenv(\"NCP_APIGW_API_KEY\"):\n", - "# os.environ[\"NCP_APIGW_API_KEY\"] = getpass.getpass(\n", - "# \"Enter your NCP API Gateway API key: \"\n", - "# )" - ] - }, - { - "cell_type": "markdown", - "id": "7c695442", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "6151aeb6", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "17bf9053-90c5-4955-b239-55a35cb07566", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Naver integration lives in the `langchain-community` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# install package\n", - "!pip install -qU langchain-community" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.chat_models import ChatClovaX\n", - "\n", - "chat = ChatClovaX(\n", - " model=\"HCX-003\",\n", - " max_tokens=100,\n", - " temperature=0.5,\n", - " # clovastudio_api_key=\"...\" # set if you prefer to pass api key directly instead of using environment variables\n", - " # task_id=\"...\" # set if you want to use fine-tuned model\n", - " # service_app=False # set True if using Service App. Default value is False (means using Test App)\n", - " # include_ai_filters=False # set True if you want to detect inappropriate content. Default value is False\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "47752b59", - "metadata": {}, - "source": [ - "## Invocation\n", - "\n", - "In addition to invoke, we also support batch and stream functionalities." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content='저는 네이버 AI를 사용하는 것이 좋아요.', additional_kwargs={}, response_metadata={'stop_reason': 'stop_before', 'input_length': 25, 'output_length': 14, 'seed': 1112164354, 'ai_filter': None}, id='run-b57bc356-1148-4007-837d-cc409dbd57cc-0', usage_metadata={'input_tokens': 25, 'output_tokens': 14, 'total_tokens': 39})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Naver\n", + "---" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to Korean. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love using NAVER AI.\"),\n", - "]\n", - "\n", - "ai_msg = chat.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "24e7377f", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "저는 네이버 AI를 사용하는 것이 좋아요.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='저는 네이버 AI를 사용하는 것이 좋아요.', additional_kwargs={}, response_metadata={'stop_reason': 'stop_before', 'input_length': 25, 'output_length': 14, 'seed': 2575184681, 'ai_filter': None}, id='run-7014b330-eba3-4701-bb62-df73ce39b854-0', usage_metadata={'input_tokens': 25, 'output_tokens': 14, 'total_tokens': 39})" + "cell_type": "markdown", + "id": "c8444f1a-e907-4f07-b8b6-68fbedfb868e", + "metadata": {}, + "source": [ + "# ChatClovaX\n", + "\n", + "This notebook provides a quick overview for getting started with Naver’s HyperCLOVA X [chat models](https://python.langchain.com/docs/concepts/chat_models) via CLOVA Studio. For detailed documentation of all ChatClovaX features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.naver.ChatClovaX.html).\n", + "\n", + "[CLOVA Studio](http://clovastudio.ncloud.com/) has several chat models. You can find information about latest models and their costs, context windows, and supported input types in the CLOVA Studio API Guide [documentation](https://api.ncloud-docs.com/docs/clovastudio-chatcompletions).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", + "| :--- | :--- |:-----:| :---: |:------------------------------------------------------------------------:| :---: | :---: |\n", + "| [ChatClovaX](https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.naver.ChatClovaX.html) | [langchain-community](https://python.langchain.com/api_reference/community/index.html) | ❌ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_community?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_community?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "|:------------------------------------------:| :---: | :---: | :---: | :---: | :---: |:-----------------------------------------------------:| :---: |:------------------------------------------------------:|:----------------------------------:|\n", + "|❌| ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "Before using the chat model, you must go through the four steps below.\n", + "\n", + "1. Creating [NAVER Cloud Platform](https://www.ncloud.com/) account\n", + "2. Apply to use [CLOVA Studio](https://www.ncloud.com/product/aiService/clovaStudio)\n", + "3. Create a CLOVA Studio Test App or Service App of a model to use (See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#테스트앱생성).)\n", + "4. Issue a Test or Service API key (See [here](https://api.ncloud-docs.com/docs/ai-naver-clovastudio-summary#API%ED%82%A4).)\n", + "\n", + "### Credentials\n", + "\n", + "Set the `NCP_CLOVASTUDIO_API_KEY` environment variable with your API key.\n", + " - Note that if you are using a legacy API Key (that doesn't start with `nv-*` prefix), you might need to get an additional API Key by clicking `App Request Status` > `Service App, Test App List` > `‘Details’ button for each app` in [CLOVA Studio](https://clovastudio.ncloud.com/studio-application/service-app) and set it as `NCP_APIGW_API_KEY`.\n", + "\n", + "You can add them to your environment variables as below:\n", + "\n", + "``` bash\n", + "export NCP_CLOVASTUDIO_API_KEY=\"your-api-key-here\"\n", + "# Uncomment below to use a legacy API key\n", + "# export NCP_APIGW_API_KEY=\"your-api-key-here\"\n", + "```" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | chat\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"Korean\",\n", - " \"input\": \"I love using NAVER AI.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "66e69286", - "metadata": {}, - "source": [ - "## Streaming" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "2c07af21-dda5-4514-b4de-1f214c2cebcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Certainly! In Korean, \"Hi\" is pronounced as \"안녕\" (annyeong). The first syllable, \"안,\" sounds like the \"ahh\" sound in \"apple,\" while the second syllable, \"녕,\" sounds like the \"yuh\" sound in \"you.\" So when you put them together, it's like saying \"ahhyuh-nyuhng.\" Remember to pronounce each syllable clearly and separately for accurate pronunciation." - ] - } - ], - "source": [ - "system = \"You are a helpful assistant that can teach Korean pronunciation.\"\n", - "human = \"Could you let me know how to say '{phrase}' in Korean?\"\n", - "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", - "\n", - "chain = prompt | chat\n", - "\n", - "for chunk in chain.stream({\"phrase\": \"Hi\"}):\n", - " print(chunk.content, end=\"\", flush=True)" - ] - }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## Additional functionalities\n", - "\n", - "### Using fine-tuned models\n", - "\n", - "You can call fine-tuned models by passing in your corresponding `task_id` parameter. (You don’t need to specify the `model_name` parameter when calling fine-tuned model.)\n", - "\n", - "You can check `task_id` from corresponding Test App or Service App details." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "cb436788", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='저는 네이버 AI를 사용하는 것이 너무 좋아요.', additional_kwargs={}, response_metadata={'stop_reason': 'stop_before', 'input_length': 25, 'output_length': 15, 'seed': 52559061, 'ai_filter': None}, id='run-5bea8d4a-48f3-4c34-ae70-66e60dca5344-0', usage_metadata={'input_tokens': 25, 'output_tokens': 15, 'total_tokens': 40})" + "cell_type": "code", + "execution_count": null, + "id": "2def81b5-b023-4f40-a97b-b2c5ca59d6a9", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"NCP_CLOVASTUDIO_API_KEY\"):\n", + " os.environ[\"NCP_CLOVASTUDIO_API_KEY\"] = getpass.getpass(\n", + " \"Enter your NCP CLOVA Studio API Key: \"\n", + " )\n", + "# Uncomment below to use a legacy API key\n", + "# if not os.getenv(\"NCP_APIGW_API_KEY\"):\n", + "# os.environ[\"NCP_APIGW_API_KEY\"] = getpass.getpass(\n", + "# \"Enter your NCP API Gateway API key: \"\n", + "# )" + ] + }, + { + "cell_type": "markdown", + "id": "7c695442", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6151aeb6", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "17bf9053-90c5-4955-b239-55a35cb07566", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Naver integration lives in the `langchain-community` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# install package\n", + "!pip install -qU langchain-community" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.chat_models import ChatClovaX\n", + "\n", + "chat = ChatClovaX(\n", + " model=\"HCX-003\",\n", + " max_tokens=100,\n", + " temperature=0.5,\n", + " # clovastudio_api_key=\"...\" # set if you prefer to pass api key directly instead of using environment variables\n", + " # task_id=\"...\" # set if you want to use fine-tuned model\n", + " # service_app=False # set True if using Service App. Default value is False (means using Test App)\n", + " # include_ai_filters=False # set True if you want to detect inappropriate content. Default value is False\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "47752b59", + "metadata": {}, + "source": [ + "## Invocation\n", + "\n", + "In addition to invoke, we also support batch and stream functionalities." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='저는 네이버 AI를 사용하는 것이 좋아요.', additional_kwargs={}, response_metadata={'stop_reason': 'stop_before', 'input_length': 25, 'output_length': 14, 'seed': 1112164354, 'ai_filter': None}, id='run-b57bc356-1148-4007-837d-cc409dbd57cc-0', usage_metadata={'input_tokens': 25, 'output_tokens': 14, 'total_tokens': 39})" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to Korean. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love using NAVER AI.\"),\n", + "]\n", + "\n", + "ai_msg = chat.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "24e7377f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "저는 네이버 AI를 사용하는 것이 좋아요.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='저는 네이버 AI를 사용하는 것이 좋아요.', additional_kwargs={}, response_metadata={'stop_reason': 'stop_before', 'input_length': 25, 'output_length': 14, 'seed': 2575184681, 'ai_filter': None}, id='run-7014b330-eba3-4701-bb62-df73ce39b854-0', usage_metadata={'input_tokens': 25, 'output_tokens': 14, 'total_tokens': 39})" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | chat\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"Korean\",\n", + " \"input\": \"I love using NAVER AI.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "66e69286", + "metadata": {}, + "source": [ + "## Streaming" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "2c07af21-dda5-4514-b4de-1f214c2cebcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Certainly! In Korean, \"Hi\" is pronounced as \"안녕\" (annyeong). The first syllable, \"안,\" sounds like the \"ahh\" sound in \"apple,\" while the second syllable, \"녕,\" sounds like the \"yuh\" sound in \"you.\" So when you put them together, it's like saying \"ahhyuh-nyuhng.\" Remember to pronounce each syllable clearly and separately for accurate pronunciation." + ] + } + ], + "source": [ + "system = \"You are a helpful assistant that can teach Korean pronunciation.\"\n", + "human = \"Could you let me know how to say '{phrase}' in Korean?\"\n", + "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n", + "\n", + "chain = prompt | chat\n", + "\n", + "for chunk in chain.stream({\"phrase\": \"Hi\"}):\n", + " print(chunk.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## Additional functionalities\n", + "\n", + "### Using fine-tuned models\n", + "\n", + "You can call fine-tuned models by passing in your corresponding `task_id` parameter. (You don’t need to specify the `model_name` parameter when calling fine-tuned model.)\n", + "\n", + "You can check `task_id` from corresponding Test App or Service App details." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "cb436788", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='저는 네이버 AI를 사용하는 것이 너무 좋아요.', additional_kwargs={}, response_metadata={'stop_reason': 'stop_before', 'input_length': 25, 'output_length': 15, 'seed': 52559061, 'ai_filter': None}, id='run-5bea8d4a-48f3-4c34-ae70-66e60dca5344-0', usage_metadata={'input_tokens': 25, 'output_tokens': 15, 'total_tokens': 40})" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "fine_tuned_model = ChatClovaX(\n", + " task_id=\"5s8egt3a\", # set if you want to use fine-tuned model\n", + " # other params...\n", + ")\n", + "\n", + "fine_tuned_model.invoke(messages)" + ] + }, + { + "cell_type": "markdown", + "id": "f428deaf", + "metadata": {}, + "source": [ + "### Service App\n", + "\n", + "When going live with production-level application using CLOVA Studio, you should apply for and use Service App. (See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#서비스앱신청).)\n", + "\n", + "For a Service App, you should use a corresponding Service API key and can only be called with it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dcf566df", + "metadata": {}, + "outputs": [], + "source": [ + "# Update environment variables\n", + "\n", + "os.environ[\"NCP_CLOVASTUDIO_API_KEY\"] = getpass.getpass(\n", + " \"Enter NCP CLOVA Studio Service API Key: \"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "cebe27ae", + "metadata": {}, + "outputs": [], + "source": [ + "chat = ChatClovaX(\n", + " service_app=True, # True if you want to use your service app, default value is False.\n", + " # clovastudio_api_key=\"...\" # if you prefer to pass api key in directly instead of using env vars\n", + " model=\"HCX-003\",\n", + " # other params...\n", + ")\n", + "ai_msg = chat.invoke(messages)" + ] + }, + { + "cell_type": "markdown", + "id": "d73e7140", + "metadata": {}, + "source": [ + "### AI Filter\n", + "\n", + "AI Filter detects inappropriate output such as profanity from the test app (or service app included) created in Playground and informs the user. See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#AIFilter) for details." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "32bfbc93", + "metadata": {}, + "outputs": [], + "source": [ + "chat = ChatClovaX(\n", + " model=\"HCX-003\",\n", + " include_ai_filters=True, # True if you want to enable ai filter\n", + " # other params...\n", + ")\n", + "\n", + "ai_msg = chat.invoke(messages)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7bd9e179", + "metadata": {}, + "outputs": [], + "source": [ + "print(ai_msg.response_metadata[\"ai_filter\"])" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatNaver features and configurations head to the API reference: https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.naver.ChatClovaX.html" ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "fine_tuned_model = ChatClovaX(\n", - " task_id=\"5s8egt3a\", # set if you want to use fine-tuned model\n", - " # other params...\n", - ")\n", - "\n", - "fine_tuned_model.invoke(messages)" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.3" + } }, - { - "cell_type": "markdown", - "id": "f428deaf", - "metadata": {}, - "source": [ - "### Service App\n", - "\n", - "When going live with production-level application using CLOVA Studio, you should apply for and use Service App. (See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#서비스앱신청).)\n", - "\n", - "For a Service App, you should use a corresponding Service API key and can only be called with it." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "dcf566df", - "metadata": {}, - "outputs": [], - "source": [ - "# Update environment variables\n", - "\n", - "os.environ[\"NCP_CLOVASTUDIO_API_KEY\"] = getpass.getpass(\n", - " \"Enter NCP CLOVA Studio Service API Key: \"\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "cebe27ae", - "metadata": {}, - "outputs": [], - "source": [ - "chat = ChatClovaX(\n", - " service_app=True, # True if you want to use your service app, default value is False.\n", - " # clovastudio_api_key=\"...\" # if you prefer to pass api key in directly instead of using env vars\n", - " model=\"HCX-003\",\n", - " # other params...\n", - ")\n", - "ai_msg = chat.invoke(messages)" - ] - }, - { - "cell_type": "markdown", - "id": "d73e7140", - "metadata": {}, - "source": [ - "### AI Filter\n", - "\n", - "AI Filter detects inappropriate output such as profanity from the test app (or service app included) created in Playground and informs the user. See [here](https://guide.ncloud-docs.com/docs/en/clovastudio-playground01#AIFilter) for details. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "32bfbc93", - "metadata": {}, - "outputs": [], - "source": [ - "chat = ChatClovaX(\n", - " model=\"HCX-003\",\n", - " include_ai_filters=True, # True if you want to enable ai filter\n", - " # other params...\n", - ")\n", - "\n", - "ai_msg = chat.invoke(messages)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7bd9e179", - "metadata": {}, - "outputs": [], - "source": [ - "print(ai_msg.response_metadata[\"ai_filter\"])" - ] - }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatNaver features and configurations head to the API reference: https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.naver.ChatClovaX.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.12.3" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb b/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb index d150be9ba5f..47b22b90b9c 100644 --- a/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb +++ b/docs/docs/integrations/chat/nvidia_ai_endpoints.ipynb @@ -1,760 +1,756 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "1f666798-8635-4bc0-a515-04d318588d67", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: NVIDIA AI Endpoints\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "fa8eb20e-4db8-45e3-9e79-c595f4f274da", - "metadata": {}, - "source": [ - "# ChatNVIDIA\n", - "\n", - "This will help you getting started with NVIDIA [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatNVIDIA` features and configurations head to the [API reference](https://python.langchain.com/api_reference/nvidia_ai_endpoints/chat_models/langchain_nvidia_ai_endpoints.chat_models.ChatNVIDIA.html).\n", - "\n", - "## Overview\n", - "The `langchain-nvidia-ai-endpoints` package contains LangChain integrations building applications with models on \n", - "NVIDIA NIM inference microservice. NIM supports models across domains like chat, embedding, and re-ranking models \n", - "from the community as well as NVIDIA. These models are optimized by NVIDIA to deliver the best performance on NVIDIA \n", - "accelerated infrastructure and deployed as a NIM, an easy-to-use, prebuilt containers that deploy anywhere using a single \n", - "command on NVIDIA accelerated infrastructure.\n", - "\n", - "NVIDIA hosted deployments of NIMs are available to test on the [NVIDIA API catalog](https://build.nvidia.com/). After testing, \n", - "NIMs can be exported from NVIDIA’s API catalog using the NVIDIA AI Enterprise license and run on-premises or in the cloud, \n", - "giving enterprises ownership and full control of their IP and AI application.\n", - "\n", - "NIMs are packaged as container images on a per model basis and are distributed as NGC container images through the NVIDIA NGC Catalog. \n", - "At their core, NIMs provide easy, consistent, and familiar APIs for running inference on an AI model.\n", - "\n", - "This example goes over how to use LangChain to interact with NVIDIA supported via the `ChatNVIDIA` class.\n", - "\n", - "For more information on accessing the chat models through this api, check out the [ChatNVIDIA](https://python.langchain.com/docs/integrations/chat/nvidia_ai_endpoints/) documentation.\n", - "\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatNVIDIA](https://python.langchain.com/api_reference/nvidia_ai_endpoints/chat_models/langchain_nvidia_ai_endpoints.chat_models.ChatNVIDIA.html) | [langchain_nvidia_ai_endpoints](https://python.langchain.com/api_reference/nvidia_ai_endpoints/index.html) | ✅ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_nvidia_ai_endpoints?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_nvidia_ai_endpoints?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "**To get started:**\n", - "\n", - "1. Create a free account with [NVIDIA](https://build.nvidia.com/), which hosts NVIDIA AI Foundation models.\n", - "\n", - "2. Click on your model of choice.\n", - "\n", - "3. Under `Input` select the `Python` tab, and click `Get API Key`. Then click `Generate Key`.\n", - "\n", - "4. Copy and save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints.\n", - "\n", - "### Credentials\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "208b72da-1535-4249-bbd3-2500028e25e9", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"NVIDIA_API_KEY\"):\n", - " # Note: the API key should start with \"nvapi-\"\n", - " os.environ[\"NVIDIA_API_KEY\"] = getpass.getpass(\"Enter your NVIDIA API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "52dc8dcb-0a48-4a4e-9947-764116d2ffd4", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2cd9cb12-6ca5-432a-9e42-8a57da073c7e", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "f2be90a9", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain NVIDIA AI Endpoints integration lives in the `langchain_nvidia_ai_endpoints` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e13eb331", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install --upgrade --quiet langchain-nvidia-ai-endpoints" - ] - }, - { - "cell_type": "markdown", - "id": "af0ce26b", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can access models in the NVIDIA API Catalog:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "Jdl2NUfMhi4J", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" + "cells": [ + { + "cell_type": "markdown", + "id": "1f666798-8635-4bc0-a515-04d318588d67", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: NVIDIA AI Endpoints\n", + "---" + ] }, - "id": "Jdl2NUfMhi4J", - "outputId": "e9c4cc72-8db6-414b-d8e9-95de93fc5db4" - }, - "outputs": [], - "source": [ - "## Core LC Chat Interface\n", - "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", - "\n", - "llm = ChatNVIDIA(model=\"mistralai/mixtral-8x7b-instruct-v0.1\")" - ] - }, - { - "cell_type": "markdown", - "id": "469c8c7f-de62-457f-a30f-674763a8b717", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9512c81b-1f3a-4eca-9470-f52cedff5c74", - "metadata": {}, - "outputs": [], - "source": [ - "result = llm.invoke(\"Write a ballad about LangChain.\")\n", - "print(result.content)" - ] - }, - { - "cell_type": "markdown", - "id": "9d35686b", - "metadata": {}, - "source": [ - "## Working with NVIDIA NIMs\n", - "When ready to deploy, you can self-host models with NVIDIA NIM—which is included with the NVIDIA AI Enterprise software license—and run them anywhere, giving you ownership of your customizations and full control of your intellectual property (IP) and AI applications.\n", - "\n", - "[Learn more about NIMs](https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/)\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "49838930", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", - "\n", - "# connect to an embedding NIM running at localhost:8000, specifying a specific model\n", - "llm = ChatNVIDIA(base_url=\"http://localhost:8000/v1\", model=\"meta/llama3-8b-instruct\")" - ] - }, - { - "cell_type": "markdown", - "id": "71d37987-d568-4a73-9d2a-8bd86323f8bf", - "metadata": {}, - "source": [ - "## Stream, Batch, and Async\n", - "\n", - "These models natively support streaming, and as is the case with all LangChain LLMs they expose a batch method to handle concurrent requests, as well as async methods for invoke, stream, and batch. Below are a few examples." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "01fa5095-be72-47b0-8247-e9fac799435d", - "metadata": {}, - "outputs": [], - "source": [ - "print(llm.batch([\"What's 2*3?\", \"What's 2*6?\"]))\n", - "# Or via the async API\n", - "# await llm.abatch([\"What's 2*3?\", \"What's 2*6?\"])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "75189ac6-e13f-414f-9064-075c77d6e754", - "metadata": {}, - "outputs": [], - "source": [ - "for chunk in llm.stream(\"How far can a seagull fly in one day?\"):\n", - " # Show the token separations\n", - " print(chunk.content, end=\"|\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8a9a4122-7a10-40c0-a979-82a769ce7f6a", - "metadata": {}, - "outputs": [], - "source": [ - "async for chunk in llm.astream(\n", - " \"How long does it take for monarch butterflies to migrate?\"\n", - "):\n", - " print(chunk.content, end=\"|\")" - ] - }, - { - "cell_type": "markdown", - "id": "6RrXHC_XqWc1", - "metadata": { - "id": "6RrXHC_XqWc1" - }, - "source": [ - "## Supported models\n", - "\n", - "Querying `available_models` will still give you all of the other models offered by your API credentials.\n", - "\n", - "The `playground_` prefix is optional." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5b8a312d-38e9-4528-843e-59451bdadbac", - "metadata": {}, - "outputs": [], - "source": [ - "ChatNVIDIA.get_available_models()\n", - "# llm.get_available_models()" - ] - }, - { - "cell_type": "markdown", - "id": "d8a407c6-e38b-4cfc-9a33-bcafadc18cf2", - "metadata": {}, - "source": [ - "## Model types" - ] - }, - { - "cell_type": "markdown", - "id": "WMW79Iegqj4e", - "metadata": { - "id": "WMW79Iegqj4e" - }, - "source": [ - "All of these models above are supported and can be accessed via `ChatNVIDIA`. \n", - "\n", - "Some model types support unique prompting techniques and chat messages. We will review a few important ones below.\n", - "\n", - "**To find out more about a specific model, please navigate to the API section of an AI Foundation model [as linked here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/codellama-13b/api).**" - ] - }, - { - "cell_type": "markdown", - "id": "03d65053-59fe-40cf-a2d0-55d3dbb13585", - "metadata": {}, - "source": [ - "### General Chat\n", - "\n", - "Models such as `meta/llama3-8b-instruct` and `mistralai/mixtral-8x22b-instruct-v0.1` are good all-around models that you can use for with any LangChain chat messages. Example below." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f5f7aee8-e90c-4d5a-ac97-0dd3d45c3f4c", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.output_parsers import StrOutputParser\n", - "from langchain_core.prompts import ChatPromptTemplate\n", - "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [(\"system\", \"You are a helpful AI assistant named Fred.\"), (\"user\", \"{input}\")]\n", - ")\n", - "chain = prompt | ChatNVIDIA(model=\"meta/llama3-8b-instruct\") | StrOutputParser()\n", - "\n", - "for txt in chain.stream({\"input\": \"What's your name?\"}):\n", - " print(txt, end=\"\")" - ] - }, - { - "cell_type": "markdown", - "id": "04146118-281b-42ef-b781-2fadeeeea6c8", - "metadata": {}, - "source": [ - "### Code Generation\n", - "\n", - "These models accept the same arguments and input structure as regular chat models, but they tend to perform better on code-genreation and structured code tasks. An example of this is `meta/codellama-70b`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "49aa569b-5f33-47b3-9edc-df58313eb038", - "metadata": {}, - "outputs": [], - "source": [ - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are an expert coding AI. Respond only in valid python; no narration whatsoever.\",\n", - " ),\n", - " (\"user\", \"{input}\"),\n", - " ]\n", - ")\n", - "chain = prompt | ChatNVIDIA(model=\"meta/codellama-70b\") | StrOutputParser()\n", - "\n", - "for txt in chain.stream({\"input\": \"How do I solve this fizz buzz problem?\"}):\n", - " print(txt, end=\"\")" - ] - }, - { - "cell_type": "markdown", - "id": "7f465ff6-5922-41d8-8abb-1d1e4095cc27", - "metadata": {}, - "source": [ - "## Multimodal\n", - "\n", - "NVIDIA also supports multimodal inputs, meaning you can provide both images and text for the model to reason over. An example model supporting multimodal inputs is `nvidia/neva-22b`.\n", - "\n", - "Below is an example use:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "26625437-1695-440f-b792-b85e6add9a90", - "metadata": {}, - "outputs": [], - "source": [ - "import IPython\n", - "import requests\n", - "\n", - "image_url = \"https://www.nvidia.com/content/dam/en-zz/Solutions/research/ai-playground/nvidia-picasso-3c33-p@2x.jpg\" ## Large Image\n", - "image_content = requests.get(image_url).content\n", - "\n", - "IPython.display.Image(image_content)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "dfbbe57c-27a5-4cbb-b967-19c4e7d29fd0", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", - "\n", - "llm = ChatNVIDIA(model=\"nvidia/neva-22b\")" - ] - }, - { - "cell_type": "markdown", - "id": "7ddcb8f1-9cd8-4376-963d-af61c29b2a3c", - "metadata": {}, - "source": [ - "#### Passing an image as a URL" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "432ea2a2-4d39-43f8-a236-041294171f14", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.messages import HumanMessage\n", - "\n", - "llm.invoke(\n", - " [\n", - " HumanMessage(\n", - " content=[\n", - " {\"type\": \"text\", \"text\": \"Describe this image:\"},\n", - " {\"type\": \"image_url\", \"image_url\": {\"url\": image_url}},\n", - " ]\n", - " )\n", - " ]\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "0573dd1f-9a17-4c99-ab2a-8d930b89d283", - "metadata": {}, - "source": [ - "#### Passing an image as a base64 encoded string" - ] - }, - { - "cell_type": "markdown", - "id": "be1688a5", - "metadata": {}, - "source": [ - "At the moment, some extra processing happens client-side to support larger images like the one above. But for smaller images (and to better illustrate the process going on under the hood), we can directly pass in the image as shown below: " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c58f1dd0", - "metadata": {}, - "outputs": [], - "source": [ - "import IPython\n", - "import requests\n", - "\n", - "image_url = \"https://picsum.photos/seed/kitten/300/200\"\n", - "image_content = requests.get(image_url).content\n", - "\n", - "IPython.display.Image(image_content)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8c721629-42eb-4006-bf68-0296f7925ebc", - "metadata": {}, - "outputs": [], - "source": [ - "import base64\n", - "\n", - "from langchain_core.messages import HumanMessage\n", - "\n", - "## Works for simpler images. For larger images, see actual implementation\n", - "b64_string = base64.b64encode(image_content).decode(\"utf-8\")\n", - "\n", - "llm.invoke(\n", - " [\n", - " HumanMessage(\n", - " content=[\n", - " {\"type\": \"text\", \"text\": \"Describe this image:\"},\n", - " {\n", - " \"type\": \"image_url\",\n", - " \"image_url\": {\"url\": f\"data:image/png;base64,{b64_string}\"},\n", - " },\n", - " ]\n", - " )\n", - " ]\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "ba958424-28d7-4bc2-9c8e-bd571066853f", - "metadata": {}, - "source": [ - "#### Directly within the string\n", - "\n", - "The NVIDIA API uniquely accepts images as base64 images inlined within `` HTML tags. While this isn't interoperable with other LLMs, you can directly prompt the model accordingly." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "00c06a9a-497b-4192-a842-b075e27401aa", - "metadata": {}, - "outputs": [], - "source": [ - "base64_with_mime_type = f\"data:image/png;base64,{b64_string}\"\n", - "llm.invoke(f'What\\'s in this image?\\n')" - ] - }, - { - "cell_type": "markdown", - "id": "137662a6", - "metadata": { - "id": "137662a6" - }, - "source": [ - "## Example usage within a RunnableWithMessageHistory" - ] - }, - { - "cell_type": "markdown", - "id": "79efa62d", - "metadata": { - "id": "79efa62d" - }, - "source": [ - "Like any other integration, ChatNVIDIA is fine to support chat utilities like RunnableWithMessageHistory which is analogous to using `ConversationChain`. Below, we show the [LangChain RunnableWithMessageHistory](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html) example applied to the `mistralai/mixtral-8x22b-instruct-v0.1` model." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "082ccb21-91e1-4e71-a9ba-4bff1e89f105", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install --upgrade --quiet langchain" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "fd2c6bc1", - "metadata": { - "id": "fd2c6bc1" - }, - "outputs": [], - "source": [ - "from langchain_core.chat_history import InMemoryChatMessageHistory\n", - "from langchain_core.runnables.history import RunnableWithMessageHistory\n", - "\n", - "# store is a dictionary that maps session IDs to their corresponding chat histories.\n", - "store = {} # memory is maintained outside the chain\n", - "\n", - "\n", - "# A function that returns the chat history for a given session ID.\n", - "def get_session_history(session_id: str) -> InMemoryChatMessageHistory:\n", - " if session_id not in store:\n", - " store[session_id] = InMemoryChatMessageHistory()\n", - " return store[session_id]\n", - "\n", - "\n", - "chat = ChatNVIDIA(\n", - " model=\"mistralai/mixtral-8x22b-instruct-v0.1\",\n", - " temperature=0.1,\n", - " max_tokens=100,\n", - " top_p=1.0,\n", - ")\n", - "\n", - "# Define a RunnableConfig object, with a `configurable` key. session_id determines thread\n", - "config = {\"configurable\": {\"session_id\": \"1\"}}\n", - "\n", - "conversation = RunnableWithMessageHistory(\n", - " chat,\n", - " get_session_history,\n", - ")\n", - "\n", - "conversation.invoke(\n", - " \"Hi I'm Srijan Dubey.\", # input or query\n", - " config=config,\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "uHIMZxVSVNBC", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 284 + { + "cell_type": "markdown", + "id": "fa8eb20e-4db8-45e3-9e79-c595f4f274da", + "metadata": {}, + "source": [ + "# ChatNVIDIA\n", + "\n", + "This will help you getting started with NVIDIA [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatNVIDIA` features and configurations head to the [API reference](https://python.langchain.com/api_reference/nvidia_ai_endpoints/chat_models/langchain_nvidia_ai_endpoints.chat_models.ChatNVIDIA.html).\n", + "\n", + "## Overview\n", + "The `langchain-nvidia-ai-endpoints` package contains LangChain integrations building applications with models on\n", + "NVIDIA NIM inference microservice. NIM supports models across domains like chat, embedding, and re-ranking models\n", + "from the community as well as NVIDIA. These models are optimized by NVIDIA to deliver the best performance on NVIDIA\n", + "accelerated infrastructure and deployed as a NIM, an easy-to-use, prebuilt containers that deploy anywhere using a single\n", + "command on NVIDIA accelerated infrastructure.\n", + "\n", + "NVIDIA hosted deployments of NIMs are available to test on the [NVIDIA API catalog](https://build.nvidia.com/). After testing,\n", + "NIMs can be exported from NVIDIA’s API catalog using the NVIDIA AI Enterprise license and run on-premises or in the cloud,\n", + "giving enterprises ownership and full control of their IP and AI application.\n", + "\n", + "NIMs are packaged as container images on a per model basis and are distributed as NGC container images through the NVIDIA NGC Catalog.\n", + "At their core, NIMs provide easy, consistent, and familiar APIs for running inference on an AI model.\n", + "\n", + "This example goes over how to use LangChain to interact with NVIDIA supported via the `ChatNVIDIA` class.\n", + "\n", + "For more information on accessing the chat models through this api, check out the [ChatNVIDIA](https://python.langchain.com/docs/integrations/chat/nvidia_ai_endpoints/) documentation.\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatNVIDIA](https://python.langchain.com/api_reference/nvidia_ai_endpoints/chat_models/langchain_nvidia_ai_endpoints.chat_models.ChatNVIDIA.html) | [langchain_nvidia_ai_endpoints](https://python.langchain.com/api_reference/nvidia_ai_endpoints/index.html) | ✅ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_nvidia_ai_endpoints?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_nvidia_ai_endpoints?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "**To get started:**\n", + "\n", + "1. Create a free account with [NVIDIA](https://build.nvidia.com/), which hosts NVIDIA AI Foundation models.\n", + "\n", + "2. Click on your model of choice.\n", + "\n", + "3. Under `Input` select the `Python` tab, and click `Get API Key`. Then click `Generate Key`.\n", + "\n", + "4. Copy and save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints.\n", + "\n", + "### Credentials\n" + ] }, - "id": "uHIMZxVSVNBC", - "outputId": "79acc89d-a820-4f2c-bac2-afe99da95580" - }, - "outputs": [], - "source": [ - "conversation.invoke(\n", - " \"I'm doing well! Just having a conversation with an AI.\",\n", - " config=config,\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "LyD1xVKmVSs4", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 350 + { + "cell_type": "code", + "execution_count": null, + "id": "208b72da-1535-4249-bbd3-2500028e25e9", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"NVIDIA_API_KEY\"):\n", + " # Note: the API key should start with \"nvapi-\"\n", + " os.environ[\"NVIDIA_API_KEY\"] = getpass.getpass(\"Enter your NVIDIA API key: \")" + ] }, - "id": "LyD1xVKmVSs4", - "outputId": "a1714513-a8fd-4d14-f974-233e39d5c4f5" - }, - "outputs": [], - "source": [ - "conversation.invoke(\n", - " \"Tell me about yourself.\",\n", - " config=config,\n", - ")" - ] + { + "cell_type": "markdown", + "id": "52dc8dcb-0a48-4a4e-9947-764116d2ffd4", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2cd9cb12-6ca5-432a-9e42-8a57da073c7e", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "f2be90a9", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain NVIDIA AI Endpoints integration lives in the `langchain_nvidia_ai_endpoints` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e13eb331", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain-nvidia-ai-endpoints" + ] + }, + { + "cell_type": "markdown", + "id": "af0ce26b", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can access models in the NVIDIA API Catalog:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "Jdl2NUfMhi4J", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Jdl2NUfMhi4J", + "outputId": "e9c4cc72-8db6-414b-d8e9-95de93fc5db4" + }, + "outputs": [], + "source": [ + "## Core LC Chat Interface\n", + "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", + "\n", + "llm = ChatNVIDIA(model=\"mistralai/mixtral-8x7b-instruct-v0.1\")" + ] + }, + { + "cell_type": "markdown", + "id": "469c8c7f-de62-457f-a30f-674763a8b717", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9512c81b-1f3a-4eca-9470-f52cedff5c74", + "metadata": {}, + "outputs": [], + "source": [ + "result = llm.invoke(\"Write a ballad about LangChain.\")\n", + "print(result.content)" + ] + }, + { + "cell_type": "markdown", + "id": "9d35686b", + "metadata": {}, + "source": [ + "## Working with NVIDIA NIMs\n", + "When ready to deploy, you can self-host models with NVIDIA NIM—which is included with the NVIDIA AI Enterprise software license—and run them anywhere, giving you ownership of your customizations and full control of your intellectual property (IP) and AI applications.\n", + "\n", + "[Learn more about NIMs](https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49838930", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", + "\n", + "# connect to an embedding NIM running at localhost:8000, specifying a specific model\n", + "llm = ChatNVIDIA(base_url=\"http://localhost:8000/v1\", model=\"meta/llama3-8b-instruct\")" + ] + }, + { + "cell_type": "markdown", + "id": "71d37987-d568-4a73-9d2a-8bd86323f8bf", + "metadata": {}, + "source": [ + "## Stream, Batch, and Async\n", + "\n", + "These models natively support streaming, and as is the case with all LangChain LLMs they expose a batch method to handle concurrent requests, as well as async methods for invoke, stream, and batch. Below are a few examples." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "01fa5095-be72-47b0-8247-e9fac799435d", + "metadata": {}, + "outputs": [], + "source": [ + "print(llm.batch([\"What's 2*3?\", \"What's 2*6?\"]))\n", + "# Or via the async API\n", + "# await llm.abatch([\"What's 2*3?\", \"What's 2*6?\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "75189ac6-e13f-414f-9064-075c77d6e754", + "metadata": {}, + "outputs": [], + "source": [ + "for chunk in llm.stream(\"How far can a seagull fly in one day?\"):\n", + " # Show the token separations\n", + " print(chunk.content, end=\"|\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8a9a4122-7a10-40c0-a979-82a769ce7f6a", + "metadata": {}, + "outputs": [], + "source": [ + "async for chunk in llm.astream(\n", + " \"How long does it take for monarch butterflies to migrate?\"\n", + "):\n", + " print(chunk.content, end=\"|\")" + ] + }, + { + "cell_type": "markdown", + "id": "6RrXHC_XqWc1", + "metadata": { + "id": "6RrXHC_XqWc1" + }, + "source": [ + "## Supported models\n", + "\n", + "Querying `available_models` will still give you all of the other models offered by your API credentials.\n", + "\n", + "The `playground_` prefix is optional." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5b8a312d-38e9-4528-843e-59451bdadbac", + "metadata": {}, + "outputs": [], + "source": [ + "ChatNVIDIA.get_available_models()\n", + "# llm.get_available_models()" + ] + }, + { + "cell_type": "markdown", + "id": "d8a407c6-e38b-4cfc-9a33-bcafadc18cf2", + "metadata": {}, + "source": [ + "## Model types" + ] + }, + { + "cell_type": "markdown", + "id": "WMW79Iegqj4e", + "metadata": { + "id": "WMW79Iegqj4e" + }, + "source": [ + "All of these models above are supported and can be accessed via `ChatNVIDIA`.\n", + "\n", + "Some model types support unique prompting techniques and chat messages. We will review a few important ones below.\n", + "\n", + "**To find out more about a specific model, please navigate to the API section of an AI Foundation model [as linked here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/codellama-13b/api).**" + ] + }, + { + "cell_type": "markdown", + "id": "03d65053-59fe-40cf-a2d0-55d3dbb13585", + "metadata": {}, + "source": [ + "### General Chat\n", + "\n", + "Models such as `meta/llama3-8b-instruct` and `mistralai/mixtral-8x22b-instruct-v0.1` are good all-around models that you can use for with any LangChain chat messages. Example below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5f7aee8-e90c-4d5a-ac97-0dd3d45c3f4c", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [(\"system\", \"You are a helpful AI assistant named Fred.\"), (\"user\", \"{input}\")]\n", + ")\n", + "chain = prompt | ChatNVIDIA(model=\"meta/llama3-8b-instruct\") | StrOutputParser()\n", + "\n", + "for txt in chain.stream({\"input\": \"What's your name?\"}):\n", + " print(txt, end=\"\")" + ] + }, + { + "cell_type": "markdown", + "id": "04146118-281b-42ef-b781-2fadeeeea6c8", + "metadata": {}, + "source": [ + "### Code Generation\n", + "\n", + "These models accept the same arguments and input structure as regular chat models, but they tend to perform better on code-genreation and structured code tasks. An example of this is `meta/codellama-70b`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49aa569b-5f33-47b3-9edc-df58313eb038", + "metadata": {}, + "outputs": [], + "source": [ + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are an expert coding AI. Respond only in valid python; no narration whatsoever.\",\n", + " ),\n", + " (\"user\", \"{input}\"),\n", + " ]\n", + ")\n", + "chain = prompt | ChatNVIDIA(model=\"meta/codellama-70b\") | StrOutputParser()\n", + "\n", + "for txt in chain.stream({\"input\": \"How do I solve this fizz buzz problem?\"}):\n", + " print(txt, end=\"\")" + ] + }, + { + "cell_type": "markdown", + "id": "7f465ff6-5922-41d8-8abb-1d1e4095cc27", + "metadata": {}, + "source": [ + "## Multimodal\n", + "\n", + "NVIDIA also supports multimodal inputs, meaning you can provide both images and text for the model to reason over. An example model supporting multimodal inputs is `nvidia/neva-22b`.\n", + "\n", + "Below is an example use:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "26625437-1695-440f-b792-b85e6add9a90", + "metadata": {}, + "outputs": [], + "source": [ + "import IPython\n", + "import requests\n", + "\n", + "image_url = \"https://www.nvidia.com/content/dam/en-zz/Solutions/research/ai-playground/nvidia-picasso-3c33-p@2x.jpg\" ## Large Image\n", + "image_content = requests.get(image_url).content\n", + "\n", + "IPython.display.Image(image_content)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dfbbe57c-27a5-4cbb-b967-19c4e7d29fd0", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", + "\n", + "llm = ChatNVIDIA(model=\"nvidia/neva-22b\")" + ] + }, + { + "cell_type": "markdown", + "id": "7ddcb8f1-9cd8-4376-963d-af61c29b2a3c", + "metadata": {}, + "source": [ + "#### Passing an image as a URL" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "432ea2a2-4d39-43f8-a236-041294171f14", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.messages import HumanMessage\n", + "\n", + "llm.invoke(\n", + " [\n", + " HumanMessage(\n", + " content=[\n", + " {\"type\": \"text\", \"text\": \"Describe this image:\"},\n", + " {\"type\": \"image_url\", \"image_url\": {\"url\": image_url}},\n", + " ]\n", + " )\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0573dd1f-9a17-4c99-ab2a-8d930b89d283", + "metadata": {}, + "source": [ + "#### Passing an image as a base64 encoded string" + ] + }, + { + "cell_type": "markdown", + "id": "be1688a5", + "metadata": {}, + "source": "At the moment, some extra processing happens client-side to support larger images like the one above. But for smaller images (and to better illustrate the process going on under the hood), we can directly pass in the image as shown below:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c58f1dd0", + "metadata": {}, + "outputs": [], + "source": [ + "import IPython\n", + "import requests\n", + "\n", + "image_url = \"https://picsum.photos/seed/kitten/300/200\"\n", + "image_content = requests.get(image_url).content\n", + "\n", + "IPython.display.Image(image_content)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8c721629-42eb-4006-bf68-0296f7925ebc", + "metadata": {}, + "outputs": [], + "source": [ + "import base64\n", + "\n", + "from langchain_core.messages import HumanMessage\n", + "\n", + "## Works for simpler images. For larger images, see actual implementation\n", + "b64_string = base64.b64encode(image_content).decode(\"utf-8\")\n", + "\n", + "llm.invoke(\n", + " [\n", + " HumanMessage(\n", + " content=[\n", + " {\"type\": \"text\", \"text\": \"Describe this image:\"},\n", + " {\n", + " \"type\": \"image_url\",\n", + " \"image_url\": {\"url\": f\"data:image/png;base64,{b64_string}\"},\n", + " },\n", + " ]\n", + " )\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "ba958424-28d7-4bc2-9c8e-bd571066853f", + "metadata": {}, + "source": [ + "#### Directly within the string\n", + "\n", + "The NVIDIA API uniquely accepts images as base64 images inlined within `` HTML tags. While this isn't interoperable with other LLMs, you can directly prompt the model accordingly." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "00c06a9a-497b-4192-a842-b075e27401aa", + "metadata": {}, + "outputs": [], + "source": [ + "base64_with_mime_type = f\"data:image/png;base64,{b64_string}\"\n", + "llm.invoke(f'What\\'s in this image?\\n')" + ] + }, + { + "cell_type": "markdown", + "id": "137662a6", + "metadata": { + "id": "137662a6" + }, + "source": [ + "## Example usage within a RunnableWithMessageHistory" + ] + }, + { + "cell_type": "markdown", + "id": "79efa62d", + "metadata": { + "id": "79efa62d" + }, + "source": [ + "Like any other integration, ChatNVIDIA is fine to support chat utilities like RunnableWithMessageHistory which is analogous to using `ConversationChain`. Below, we show the [LangChain RunnableWithMessageHistory](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html) example applied to the `mistralai/mixtral-8x22b-instruct-v0.1` model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "082ccb21-91e1-4e71-a9ba-4bff1e89f105", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fd2c6bc1", + "metadata": { + "id": "fd2c6bc1" + }, + "outputs": [], + "source": [ + "from langchain_core.chat_history import InMemoryChatMessageHistory\n", + "from langchain_core.runnables.history import RunnableWithMessageHistory\n", + "\n", + "# store is a dictionary that maps session IDs to their corresponding chat histories.\n", + "store = {} # memory is maintained outside the chain\n", + "\n", + "\n", + "# A function that returns the chat history for a given session ID.\n", + "def get_session_history(session_id: str) -> InMemoryChatMessageHistory:\n", + " if session_id not in store:\n", + " store[session_id] = InMemoryChatMessageHistory()\n", + " return store[session_id]\n", + "\n", + "\n", + "chat = ChatNVIDIA(\n", + " model=\"mistralai/mixtral-8x22b-instruct-v0.1\",\n", + " temperature=0.1,\n", + " max_tokens=100,\n", + " top_p=1.0,\n", + ")\n", + "\n", + "# Define a RunnableConfig object, with a `configurable` key. session_id determines thread\n", + "config = {\"configurable\": {\"session_id\": \"1\"}}\n", + "\n", + "conversation = RunnableWithMessageHistory(\n", + " chat,\n", + " get_session_history,\n", + ")\n", + "\n", + "conversation.invoke(\n", + " \"Hi I'm Srijan Dubey.\", # input or query\n", + " config=config,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "uHIMZxVSVNBC", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 284 + }, + "id": "uHIMZxVSVNBC", + "outputId": "79acc89d-a820-4f2c-bac2-afe99da95580" + }, + "outputs": [], + "source": [ + "conversation.invoke(\n", + " \"I'm doing well! Just having a conversation with an AI.\",\n", + " config=config,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "LyD1xVKmVSs4", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 350 + }, + "id": "LyD1xVKmVSs4", + "outputId": "a1714513-a8fd-4d14-f974-233e39d5c4f5" + }, + "outputs": [], + "source": [ + "conversation.invoke(\n", + " \"Tell me about yourself.\",\n", + " config=config,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f3cbbba0", + "metadata": {}, + "source": [ + "## Tool calling\n", + "\n", + "Starting in v0.2, `ChatNVIDIA` supports [bind_tools](https://python.langchain.com/api_reference/core/language_models/langchain_core.language_models.chat_models.BaseChatModel.html#langchain_core.language_models.chat_models.BaseChatModel.bind_tools).\n", + "\n", + "`ChatNVIDIA` provides integration with the variety of models on [build.nvidia.com](https://build.nvidia.com) as well as local NIMs. Not all these models are trained for tool calling. Be sure to select a model that does have tool calling for your experimention and applications." + ] + }, + { + "cell_type": "markdown", + "id": "6f7b535e", + "metadata": {}, + "source": [ + "You can get a list of models that are known to support tool calling with," + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e36c8911", + "metadata": {}, + "outputs": [], + "source": [ + "tool_models = [\n", + " model for model in ChatNVIDIA.get_available_models() if model.supports_tools\n", + "]\n", + "tool_models" + ] + }, + { + "cell_type": "markdown", + "id": "b01d75a7", + "metadata": {}, + "source": [ + "With a tool capable model," + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bd54f174", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.tools import tool\n", + "from pydantic import Field\n", + "\n", + "\n", + "@tool\n", + "def get_current_weather(\n", + " location: str = Field(..., description=\"The location to get the weather for.\"),\n", + "):\n", + " \"\"\"Get the current weather for a location.\"\"\"\n", + " ...\n", + "\n", + "\n", + "llm = ChatNVIDIA(model=tool_models[0].id).bind_tools(tools=[get_current_weather])\n", + "response = llm.invoke(\"What is the weather in Boston?\")\n", + "response.tool_calls" + ] + }, + { + "cell_type": "markdown", + "id": "e08df68c", + "metadata": {}, + "source": [ + "See [How to use chat models to call tools](https://python.langchain.com/docs/how_to/tool_calling/) for additional examples." + ] + }, + { + "cell_type": "markdown", + "id": "a9a3c438-121d-46eb-8fb5-b8d5a13cd4a4", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "af585c6b-fe0a-4833-9860-a4209a71b3c6", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f2f25dd3-0b4a-465f-a53e-95521cdc253c", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `ChatNVIDIA` features and configurations head to the API reference: https://python.langchain.com/api_reference/nvidia_ai_endpoints/chat_models/langchain_nvidia_ai_endpoints.chat_models.ChatNVIDIA.html" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "f3cbbba0", - "metadata": {}, - "source": [ - "## Tool calling\n", - "\n", - "Starting in v0.2, `ChatNVIDIA` supports [bind_tools](https://python.langchain.com/api_reference/core/language_models/langchain_core.language_models.chat_models.BaseChatModel.html#langchain_core.language_models.chat_models.BaseChatModel.bind_tools).\n", - "\n", - "`ChatNVIDIA` provides integration with the variety of models on [build.nvidia.com](https://build.nvidia.com) as well as local NIMs. Not all these models are trained for tool calling. Be sure to select a model that does have tool calling for your experimention and applications." - ] - }, - { - "cell_type": "markdown", - "id": "6f7b535e", - "metadata": {}, - "source": [ - "You can get a list of models that are known to support tool calling with," - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e36c8911", - "metadata": {}, - "outputs": [], - "source": [ - "tool_models = [\n", - " model for model in ChatNVIDIA.get_available_models() if model.supports_tools\n", - "]\n", - "tool_models" - ] - }, - { - "cell_type": "markdown", - "id": "b01d75a7", - "metadata": {}, - "source": [ - "With a tool capable model," - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "bd54f174", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.tools import tool\n", - "from pydantic import Field\n", - "\n", - "\n", - "@tool\n", - "def get_current_weather(\n", - " location: str = Field(..., description=\"The location to get the weather for.\"),\n", - "):\n", - " \"\"\"Get the current weather for a location.\"\"\"\n", - " ...\n", - "\n", - "\n", - "llm = ChatNVIDIA(model=tool_models[0].id).bind_tools(tools=[get_current_weather])\n", - "response = llm.invoke(\"What is the weather in Boston?\")\n", - "response.tool_calls" - ] - }, - { - "cell_type": "markdown", - "id": "e08df68c", - "metadata": {}, - "source": [ - "See [How to use chat models to call tools](https://python.langchain.com/docs/how_to/tool_calling/) for additional examples." - ] - }, - { - "cell_type": "markdown", - "id": "a9a3c438-121d-46eb-8fb5-b8d5a13cd4a4", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "af585c6b-fe0a-4833-9860-a4209a71b3c6", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "f2f25dd3-0b4a-465f-a53e-95521cdc253c", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `ChatNVIDIA` features and configurations head to the API reference: https://python.langchain.com/api_reference/nvidia_ai_endpoints/chat_models/langchain_nvidia_ai_endpoints.chat_models.ChatNVIDIA.html" - ] - } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/ollama.ipynb b/docs/docs/integrations/chat/ollama.ipynb index daf179fe7f0..c2dd2463f3d 100644 --- a/docs/docs/integrations/chat/ollama.ipynb +++ b/docs/docs/integrations/chat/ollama.ipynb @@ -1,477 +1,475 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Ollama\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatOllama\n", - "\n", - "[Ollama](https://ollama.ai/) allows you to run open-source large language models, such as Llama 2, locally.\n", - "\n", - "Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. \n", - "\n", - "It optimizes setup and configuration details, including GPU usage.\n", - "\n", - "For a complete list of supported models and model variants, see the [Ollama model library](https://github.com/jmorganca/ollama#model-library).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/ollama) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatOllama](https://python.langchain.com/v0.2/api_reference/ollama/chat_models/langchain_ollama.chat_models.ChatOllama.html) | [langchain-ollama](https://python.langchain.com/v0.2/api_reference/ollama/index.html) | ✅ | ❌ | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ollama?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ollama?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: |:----------------------------------------------------:| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |\n", - "\n", - "## Setup\n", - "\n", - "First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n", - "\n", - "* [Download](https://ollama.ai/download) and install Ollama onto the available supported platforms (including Windows Subsystem for Linux)\n", - "* Fetch available LLM model via `ollama pull `\n", - " * View a list of available models via the [model library](https://ollama.ai/library)\n", - " * e.g., `ollama pull llama3`\n", - "* This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.\n", - "\n", - "> On Mac, the models will be download to `~/.ollama/models`\n", - "> \n", - "> On Linux (or WSL), the models will be stored at `/usr/share/ollama/.ollama/models`\n", - "\n", - "* Specify the exact version of the model of interest as such `ollama pull vicuna:13b-v1.5-16k-q4_0` (View the [various tags for the `Vicuna`](https://ollama.ai/library/vicuna/tags) model in this instance)\n", - "* To view all pulled models, use `ollama list`\n", - "* To chat directly with a model from the command line, use `ollama run `\n", - "* View the [Ollama documentation](https://github.com/jmorganca/ollama) for more commands. Run `ollama help` in the terminal to see available commands too.\n" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Ollama integration lives in the `langchain-ollama` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-ollama" - ] - }, - { - "metadata": {}, - "cell_type": "markdown", - "source": "Make sure you're using the latest Ollama version for structured outputs. Update by running:", - "id": "b18bd692076f7cf7" - }, - { - "metadata": {}, - "cell_type": "code", - "outputs": [], - "execution_count": null, - "source": "%pip install -U ollama", - "id": "b7a05cba95644c2e" - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_ollama import ChatOllama\n", - "\n", - "llm = ChatOllama(\n", - " model=\"llama3.1\",\n", - " temperature=0,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content='The translation of \"I love programming\" from English to French is:\\n\\n\"J\\'adore programmer.\"', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:32.81965Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2167842917, 'load_duration': 54222584, 'prompt_eval_count': 35, 'prompt_eval_duration': 893007000, 'eval_count': 22, 'eval_duration': 1218962000}, id='run-0863daa2-43bf-4a43-86cc-611b23eae466-0', usage_metadata={'input_tokens': 35, 'output_tokens': 22, 'total_tokens': 57})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Ollama\n", + "---" ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.messages import AIMessage\n", - "\n", - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "The translation of \"I love programming\" from English to French is:\n", - "\n", - "\"J'adore programmer.\"\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Das Programmieren ist mir ein Leidenschaft! (That\\'s \"Programming is my passion!\" in German.) Would you like me to translate anything else?', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:34.893548Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2045997333, 'load_duration': 22584792, 'prompt_eval_count': 30, 'prompt_eval_duration': 213210000, 'eval_count': 32, 'eval_duration': 1808541000}, id='run-d18e1c6b-50e0-4b1d-b23a-973fa058edad-0', usage_metadata={'input_tokens': 30, 'output_tokens': 32, 'total_tokens': 62})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatOllama\n", + "\n", + "[Ollama](https://ollama.ai/) allows you to run open-source large language models, such as Llama 2, locally.\n", + "\n", + "Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile.\n", + "\n", + "It optimizes setup and configuration details, including GPU usage.\n", + "\n", + "For a complete list of supported models and model variants, see the [Ollama model library](https://github.com/jmorganca/ollama#model-library).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/ollama) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatOllama](https://python.langchain.com/v0.2/api_reference/ollama/chat_models/langchain_ollama.chat_models.ChatOllama.html) | [langchain-ollama](https://python.langchain.com/v0.2/api_reference/ollama/index.html) | ✅ | ❌ | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ollama?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ollama?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: |:----------------------------------------------------:| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n", + "\n", + "* [Download](https://ollama.ai/download) and install Ollama onto the available supported platforms (including Windows Subsystem for Linux)\n", + "* Fetch available LLM model via `ollama pull `\n", + " * View a list of available models via the [model library](https://ollama.ai/library)\n", + " * e.g., `ollama pull llama3`\n", + "* This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.\n", + "\n", + "> On Mac, the models will be download to `~/.ollama/models`\n", + ">\n", + "> On Linux (or WSL), the models will be stored at `/usr/share/ollama/.ollama/models`\n", + "\n", + "* Specify the exact version of the model of interest as such `ollama pull vicuna:13b-v1.5-16k-q4_0` (View the [various tags for the `Vicuna`](https://ollama.ai/library/vicuna/tags) model in this instance)\n", + "* To view all pulled models, use `ollama list`\n", + "* To chat directly with a model from the command line, use `ollama run `\n", + "* View the [Ollama documentation](https://github.com/jmorganca/ollama) for more commands. Run `ollama help` in the terminal to see available commands too.\n" ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "0f51345d-0a9d-43f1-8fca-d0662cb8e21b", - "metadata": {}, - "source": [ - "## Tool calling\n", - "\n", - "We can use [tool calling](https://blog.langchain.dev/improving-core-tool-interfaces-and-docs-in-langchain/) with an LLM [that has been fine-tuned for tool use](https://ollama.com/library/llama3.1): \n", - "\n", - "```\n", - "ollama pull llama3.1\n", - "```\n", - "\n", - "Details on creating custom tools are available in [this guide](/docs/how_to/custom_tools/). Below, we demonstrate how to create a tool using the `@tool` decorator on a normal python function." - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "f767015f", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "[{'name': 'validate_user',\n", - " 'args': {'addresses': '[\"123 Fake St, Boston, MA\", \"234 Pretend Boulevard, Houston, TX\"]',\n", - " 'user_id': '123'},\n", - " 'id': '40fe3de0-500c-4b91-9616-5932a929e640',\n", - " 'type': 'tool_call'}]" + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from typing import List\n", - "\n", - "from langchain_core.tools import tool\n", - "from langchain_ollama import ChatOllama\n", - "\n", - "\n", - "@tool\n", - "def validate_user(user_id: int, addresses: List[str]) -> bool:\n", - " \"\"\"Validate user using historical addresses.\n", - "\n", - " Args:\n", - " user_id (int): the user ID.\n", - " addresses (List[str]): Previous addresses as a list of strings.\n", - " \"\"\"\n", - " return True\n", - "\n", - "\n", - "llm = ChatOllama(\n", - " model=\"llama3.1\",\n", - " temperature=0,\n", - ").bind_tools([validate_user])\n", - "\n", - "result = llm.invoke(\n", - " \"Could you validate user 123? They previously lived at \"\n", - " \"123 Fake St in Boston MA and 234 Pretend Boulevard in \"\n", - " \"Houston TX.\"\n", - ")\n", - "result.tool_calls" - ] - }, - { - "cell_type": "markdown", - "id": "4c5e0197", - "metadata": {}, - "source": [ - "## Multi-modal\n", - "\n", - "Ollama has support for multi-modal LLMs, such as [bakllava](https://ollama.com/library/bakllava) and [llava](https://ollama.com/library/llava).\n", - "\n", - " ollama pull bakllava\n", - "\n", - "Be sure to update Ollama so that you have the most recent version to support multi-modal." - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "36c9b1c2", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/html": [ - "" + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Ollama integration lives in the `langchain-ollama` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-ollama" + ] + }, + { + "cell_type": "markdown", + "id": "b18bd692076f7cf7", + "metadata": {}, + "source": "Make sure you're using the latest Ollama version for structured outputs. Update by running:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b7a05cba95644c2e", + "metadata": {}, + "outputs": [], + "source": "%pip install -U ollama" + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_ollama import ChatOllama\n", + "\n", + "llm = ChatOllama(\n", + " model=\"llama3.1\",\n", + " temperature=0,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='The translation of \"I love programming\" from English to French is:\\n\\n\"J\\'adore programmer.\"', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:32.81965Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2167842917, 'load_duration': 54222584, 'prompt_eval_count': 35, 'prompt_eval_duration': 893007000, 'eval_count': 22, 'eval_duration': 1218962000}, id='run-0863daa2-43bf-4a43-86cc-611b23eae466-0', usage_metadata={'input_tokens': 35, 'output_tokens': 22, 'total_tokens': 57})" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } ], - "text/plain": [ - "" + "source": [ + "from langchain_core.messages import AIMessage\n", + "\n", + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "import base64\n", - "from io import BytesIO\n", - "\n", - "from IPython.display import HTML, display\n", - "from PIL import Image\n", - "\n", - "\n", - "def convert_to_base64(pil_image):\n", - " \"\"\"\n", - " Convert PIL images to Base64 encoded strings\n", - "\n", - " :param pil_image: PIL image\n", - " :return: Re-sized Base64 string\n", - " \"\"\"\n", - "\n", - " buffered = BytesIO()\n", - " pil_image.save(buffered, format=\"JPEG\") # You can change the format if needed\n", - " img_str = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n", - " return img_str\n", - "\n", - "\n", - "def plt_img_base64(img_base64):\n", - " \"\"\"\n", - " Disply base64 encoded string as image\n", - "\n", - " :param img_base64: Base64 string\n", - " \"\"\"\n", - " # Create an HTML img tag with the base64 string as the source\n", - " image_html = f''\n", - " # Display the image by rendering the HTML\n", - " display(HTML(image_html))\n", - "\n", - "\n", - "file_path = \"../../../static/img/ollama_example_img.jpg\"\n", - "pil_image = Image.open(file_path)\n", - "\n", - "image_b64 = convert_to_base64(pil_image)\n", - "plt_img_base64(image_b64)" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "id": "32b3ba7b", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "90%\n" - ] + "cell_type": "code", + "execution_count": 11, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The translation of \"I love programming\" from English to French is:\n", + "\n", + "\"J'adore programmer.\"\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Das Programmieren ist mir ein Leidenschaft! (That\\'s \"Programming is my passion!\" in German.) Would you like me to translate anything else?', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:34.893548Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2045997333, 'load_duration': 22584792, 'prompt_eval_count': 30, 'prompt_eval_duration': 213210000, 'eval_count': 32, 'eval_duration': 1808541000}, id='run-d18e1c6b-50e0-4b1d-b23a-973fa058edad-0', usage_metadata={'input_tokens': 30, 'output_tokens': 32, 'total_tokens': 62})" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0f51345d-0a9d-43f1-8fca-d0662cb8e21b", + "metadata": {}, + "source": [ + "## Tool calling\n", + "\n", + "We can use [tool calling](https://blog.langchain.dev/improving-core-tool-interfaces-and-docs-in-langchain/) with an LLM [that has been fine-tuned for tool use](https://ollama.com/library/llama3.1):\n", + "\n", + "```\n", + "ollama pull llama3.1\n", + "```\n", + "\n", + "Details on creating custom tools are available in [this guide](/docs/how_to/custom_tools/). Below, we demonstrate how to create a tool using the `@tool` decorator on a normal python function." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "f767015f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'name': 'validate_user',\n", + " 'args': {'addresses': '[\"123 Fake St, Boston, MA\", \"234 Pretend Boulevard, Houston, TX\"]',\n", + " 'user_id': '123'},\n", + " 'id': '40fe3de0-500c-4b91-9616-5932a929e640',\n", + " 'type': 'tool_call'}]" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from typing import List\n", + "\n", + "from langchain_core.tools import tool\n", + "from langchain_ollama import ChatOllama\n", + "\n", + "\n", + "@tool\n", + "def validate_user(user_id: int, addresses: List[str]) -> bool:\n", + " \"\"\"Validate user using historical addresses.\n", + "\n", + " Args:\n", + " user_id (int): the user ID.\n", + " addresses (List[str]): Previous addresses as a list of strings.\n", + " \"\"\"\n", + " return True\n", + "\n", + "\n", + "llm = ChatOllama(\n", + " model=\"llama3.1\",\n", + " temperature=0,\n", + ").bind_tools([validate_user])\n", + "\n", + "result = llm.invoke(\n", + " \"Could you validate user 123? They previously lived at \"\n", + " \"123 Fake St in Boston MA and 234 Pretend Boulevard in \"\n", + " \"Houston TX.\"\n", + ")\n", + "result.tool_calls" + ] + }, + { + "cell_type": "markdown", + "id": "4c5e0197", + "metadata": {}, + "source": [ + "## Multi-modal\n", + "\n", + "Ollama has support for multi-modal LLMs, such as [bakllava](https://ollama.com/library/bakllava) and [llava](https://ollama.com/library/llava).\n", + "\n", + " ollama pull bakllava\n", + "\n", + "Be sure to update Ollama so that you have the most recent version to support multi-modal." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "36c9b1c2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import base64\n", + "from io import BytesIO\n", + "\n", + "from IPython.display import HTML, display\n", + "from PIL import Image\n", + "\n", + "\n", + "def convert_to_base64(pil_image):\n", + " \"\"\"\n", + " Convert PIL images to Base64 encoded strings\n", + "\n", + " :param pil_image: PIL image\n", + " :return: Re-sized Base64 string\n", + " \"\"\"\n", + "\n", + " buffered = BytesIO()\n", + " pil_image.save(buffered, format=\"JPEG\") # You can change the format if needed\n", + " img_str = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n", + " return img_str\n", + "\n", + "\n", + "def plt_img_base64(img_base64):\n", + " \"\"\"\n", + " Disply base64 encoded string as image\n", + "\n", + " :param img_base64: Base64 string\n", + " \"\"\"\n", + " # Create an HTML img tag with the base64 string as the source\n", + " image_html = f''\n", + " # Display the image by rendering the HTML\n", + " display(HTML(image_html))\n", + "\n", + "\n", + "file_path = \"../../../static/img/ollama_example_img.jpg\"\n", + "pil_image = Image.open(file_path)\n", + "\n", + "image_b64 = convert_to_base64(pil_image)\n", + "plt_img_base64(image_b64)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "32b3ba7b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "90%\n" + ] + } + ], + "source": [ + "from langchain_core.messages import HumanMessage\n", + "from langchain_ollama import ChatOllama\n", + "\n", + "llm = ChatOllama(model=\"bakllava\", temperature=0)\n", + "\n", + "\n", + "def prompt_func(data):\n", + " text = data[\"text\"]\n", + " image = data[\"image\"]\n", + "\n", + " image_part = {\n", + " \"type\": \"image_url\",\n", + " \"image_url\": f\"data:image/jpeg;base64,{image}\",\n", + " }\n", + "\n", + " content_parts = []\n", + "\n", + " text_part = {\"type\": \"text\", \"text\": text}\n", + "\n", + " content_parts.append(image_part)\n", + " content_parts.append(text_part)\n", + "\n", + " return [HumanMessage(content=content_parts)]\n", + "\n", + "\n", + "from langchain_core.output_parsers import StrOutputParser\n", + "\n", + "chain = prompt_func | llm | StrOutputParser()\n", + "\n", + "query_chain = chain.invoke(\n", + " {\"text\": \"What is the Dollar-based gross retention rate?\", \"image\": image_b64}\n", + ")\n", + "\n", + "print(query_chain)" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatOllama features and configurations head to the API reference: https://python.langchain.com/api_reference/ollama/chat_models/langchain_ollama.chat_models.ChatOllama.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.4" } - ], - "source": [ - "from langchain_core.messages import HumanMessage\n", - "from langchain_ollama import ChatOllama\n", - "\n", - "llm = ChatOllama(model=\"bakllava\", temperature=0)\n", - "\n", - "\n", - "def prompt_func(data):\n", - " text = data[\"text\"]\n", - " image = data[\"image\"]\n", - "\n", - " image_part = {\n", - " \"type\": \"image_url\",\n", - " \"image_url\": f\"data:image/jpeg;base64,{image}\",\n", - " }\n", - "\n", - " content_parts = []\n", - "\n", - " text_part = {\"type\": \"text\", \"text\": text}\n", - "\n", - " content_parts.append(image_part)\n", - " content_parts.append(text_part)\n", - "\n", - " return [HumanMessage(content=content_parts)]\n", - "\n", - "\n", - "from langchain_core.output_parsers import StrOutputParser\n", - "\n", - "chain = prompt_func | llm | StrOutputParser()\n", - "\n", - "query_chain = chain.invoke(\n", - " {\"text\": \"What is the Dollar-based gross retention rate?\", \"image\": image_b64}\n", - ")\n", - "\n", - "print(query_chain)" - ] }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatOllama features and configurations head to the API reference: https://python.langchain.com/api_reference/ollama/chat_models/langchain_ollama.chat_models.ChatOllama.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.12.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/together.ipynb b/docs/docs/integrations/chat/together.ipynb index cd47bc390f4..24496372921 100644 --- a/docs/docs/integrations/chat/together.ipynb +++ b/docs/docs/integrations/chat/together.ipynb @@ -1,267 +1,265 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Together\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatTogether\n", - "\n", - "\n", - "This page will help you get started with Together AI [chat models](../../concepts/chat_models.mdx). For detailed documentation of all ChatTogether features and configurations head to the [API reference](https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html).\n", - "\n", - "[Together AI](https://www.together.ai/) offers an API to query [50+ leading open-source models](https://docs.together.ai/docs/chat-models)\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/togetherai) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatTogether](https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html) | [langchain-together](https://python.langchain.com/api_reference/together/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-together?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-together?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | \n", - "\n", - "## Setup\n", - "\n", - "To access Together models you'll need to create a/an Together account, get an API key, and install the `langchain-together` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [this page](https://api.together.ai) to sign up to Together and generate an API key. Once you've done this set the TOGETHER_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"TOGETHER_API_KEY\" not in os.environ:\n", - " os.environ[\"TOGETHER_API_KEY\"] = getpass.getpass(\"Enter your Together API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Together integration lives in the `langchain-together` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-together" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_together import ChatTogether\n", - "\n", - "llm = ChatTogether(\n", - " model=\"meta-llama/Llama-3-70b-chat-hf\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 35, 'total_tokens': 44}, 'model_name': 'meta-llama/Llama-3-70b-chat-hf', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-eabcbe33-cdd8-45b8-ab0b-f90b6e7dfad8-0', usage_metadata={'input_tokens': 35, 'output_tokens': 9, 'total_tokens': 44})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Together\n", + "---" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore la programmation.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 7, 'prompt_tokens': 30, 'total_tokens': 37}, 'model_name': 'meta-llama/Llama-3-70b-chat-hf', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-a249aa24-ee31-46ba-9bf9-f4eb135b0a95-0', usage_metadata={'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatTogether\n", + "\n", + "\n", + "This page will help you get started with Together AI [chat models](../../concepts/chat_models.mdx). For detailed documentation of all ChatTogether features and configurations head to the [API reference](https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html).\n", + "\n", + "[Together AI](https://www.together.ai/) offers an API to query [50+ leading open-source models](https://docs.together.ai/docs/chat-models)\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/togetherai) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatTogether](https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html) | [langchain-together](https://python.langchain.com/api_reference/together/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-together?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-together?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ |\n", + "\n", + "## Setup\n", + "\n", + "To access Together models you'll need to create a/an Together account, get an API key, and install the `langchain-together` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [this page](https://api.together.ai) to sign up to Together and generate an API key. Once you've done this set the TOGETHER_API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"TOGETHER_API_KEY\" not in os.environ:\n", + " os.environ[\"TOGETHER_API_KEY\"] = getpass.getpass(\"Enter your Together API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Together integration lives in the `langchain-together` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-together" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_together import ChatTogether\n", + "\n", + "llm = ChatTogether(\n", + " model=\"meta-llama/Llama-3-70b-chat-hf\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 35, 'total_tokens': 44}, 'model_name': 'meta-llama/Llama-3-70b-chat-hf', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-eabcbe33-cdd8-45b8-ab0b-f90b6e7dfad8-0', usage_metadata={'input_tokens': 35, 'output_tokens': 9, 'total_tokens': 44})" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore la programmation.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 7, 'prompt_tokens': 30, 'total_tokens': 37}, 'model_name': 'meta-llama/Llama-3-70b-chat-hf', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-a249aa24-ee31-46ba-9bf9-f4eb135b0a95-0', usage_metadata={'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37})" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatTogether features and configurations head to the API reference: https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html" ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" + } }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatTogether features and configurations head to the API reference: https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/vllm.ipynb b/docs/docs/integrations/chat/vllm.ipynb index d43c443c6c6..ab86b4fb164 100644 --- a/docs/docs/integrations/chat/vllm.ipynb +++ b/docs/docs/integrations/chat/vllm.ipynb @@ -1,244 +1,242 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "eb65deaa", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: vLLM Chat\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "8f82e243-f4ee-44e2-b417-099b6401ae3e", - "metadata": {}, - "source": [ - "# vLLM Chat\n", - "\n", - "vLLM can be deployed as a server that mimics the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API. This server can be queried in the same format as OpenAI API.\n", - "\n", - "## Overview\n", - "This will help you getting started with vLLM [chat models](/docs/concepts/chat_models), which leverage the `langchain-openai` package. For detailed documentation of all `ChatOpenAI` features and configurations head to the [API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html).\n", - "\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html) | [langchain_openai](https://python.langchain.com/api_reference/openai/) | ✅ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_openai?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "Specific model features-- such as tool calling, support for multi-modal inputs, support for token-level streaming, etc.-- will depend on the hosted model.\n", - "\n", - "## Setup\n", - "\n", - "See the vLLM docs [here](https://docs.vllm.ai/en/latest/).\n", - "\n", - "To access vLLM models through LangChain, you'll need to install the `langchain-openai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Authentication will depend on specifics of the inference server." - ] - }, - { - "cell_type": "markdown", - "id": "c3b1707a-cf2c-4367-94e3-436c43402503", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1e40bd5e-cbaa-41ef-aaf9-0858eb207184", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "0739b647-609b-46d3-bdd3-e86fe4463288", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain vLLM integration can be accessed via the `langchain-openai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7afcfbdc-56aa-4529-825a-8acbe7aa5241", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-openai" - ] - }, - { - "cell_type": "markdown", - "id": "2cf576d6-7b67-4937-bf99-39071e85720c", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "060a2e3d-d42f-4221-bd09-a9a06544dcd3", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain_core.messages import HumanMessage, SystemMessage\n", - "from langchain_core.prompts.chat import (\n", - " ChatPromptTemplate,\n", - " HumanMessagePromptTemplate,\n", - " SystemMessagePromptTemplate,\n", - ")\n", - "from langchain_openai import ChatOpenAI" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "bf24d732-68a9-44fd-b05d-4903ce5620c6", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "inference_server_url = \"http://localhost:8000/v1\"\n", - "\n", - "llm = ChatOpenAI(\n", - " model=\"mosaicml/mpt-7b\",\n", - " openai_api_key=\"EMPTY\",\n", - " openai_api_base=inference_server_url,\n", - " max_tokens=5,\n", - " temperature=0,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "34b18328-5e8b-4ff2-9b89-6fbb76b5c7f0", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "aea4e363-5688-4b07-82ed-6aa8153c2377", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=' Io amo programmare', additional_kwargs={}, example=False)" + "cell_type": "raw", + "id": "eb65deaa", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: vLLM Chat\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "8f82e243-f4ee-44e2-b417-099b6401ae3e", + "metadata": {}, + "source": [ + "# vLLM Chat\n", + "\n", + "vLLM can be deployed as a server that mimics the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API. This server can be queried in the same format as OpenAI API.\n", + "\n", + "## Overview\n", + "This will help you getting started with vLLM [chat models](/docs/concepts/chat_models), which leverage the `langchain-openai` package. For detailed documentation of all `ChatOpenAI` features and configurations head to the [API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html).\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html) | [langchain_openai](https://python.langchain.com/api_reference/openai/) | ✅ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_openai?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "Specific model features-- such as tool calling, support for multi-modal inputs, support for token-level streaming, etc.-- will depend on the hosted model.\n", + "\n", + "## Setup\n", + "\n", + "See the vLLM docs [here](https://docs.vllm.ai/en/latest/).\n", + "\n", + "To access vLLM models through LangChain, you'll need to install the `langchain-openai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Authentication will depend on specifics of the inference server." + ] + }, + { + "cell_type": "markdown", + "id": "c3b1707a-cf2c-4367-94e3-436c43402503", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1e40bd5e-cbaa-41ef-aaf9-0858eb207184", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "0739b647-609b-46d3-bdd3-e86fe4463288", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain vLLM integration can be accessed via the `langchain-openai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7afcfbdc-56aa-4529-825a-8acbe7aa5241", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-openai" + ] + }, + { + "cell_type": "markdown", + "id": "2cf576d6-7b67-4937-bf99-39071e85720c", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "060a2e3d-d42f-4221-bd09-a9a06544dcd3", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain_core.messages import HumanMessage, SystemMessage\n", + "from langchain_core.prompts.chat import (\n", + " ChatPromptTemplate,\n", + " HumanMessagePromptTemplate,\n", + " SystemMessagePromptTemplate,\n", + ")\n", + "from langchain_openai import ChatOpenAI" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "bf24d732-68a9-44fd-b05d-4903ce5620c6", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "inference_server_url = \"http://localhost:8000/v1\"\n", + "\n", + "llm = ChatOpenAI(\n", + " model=\"mosaicml/mpt-7b\",\n", + " openai_api_key=\"EMPTY\",\n", + " openai_api_base=inference_server_url,\n", + " max_tokens=5,\n", + " temperature=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "34b18328-5e8b-4ff2-9b89-6fbb76b5c7f0", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "aea4e363-5688-4b07-82ed-6aa8153c2377", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=' Io amo programmare', additional_kwargs={}, example=False)" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " SystemMessage(\n", + " content=\"You are a helpful assistant that translates English to Italian.\"\n", + " ),\n", + " HumanMessage(\n", + " content=\"Translate the following sentence from English to Italian: I love programming.\"\n", + " ),\n", + "]\n", + "llm.invoke(messages)" + ] + }, + { + "cell_type": "markdown", + "id": "a580a1e4-11a3-4277-bfba-bfb414ac7201", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dd0f4043-48bd-4245-8bdb-e7669666a277", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "265f5d51-0a76-4808-8d13-ef598ee6e366", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all features and configurations exposed via `langchain-openai`, head to the API reference: https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html\n", + "\n", + "Refer to the vLLM [documentation](https://docs.vllm.ai/en/latest/) as well." ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "messages = [\n", - " SystemMessage(\n", - " content=\"You are a helpful assistant that translates English to Italian.\"\n", - " ),\n", - " HumanMessage(\n", - " content=\"Translate the following sentence from English to Italian: I love programming.\"\n", - " ),\n", - "]\n", - "llm.invoke(messages)" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "a580a1e4-11a3-4277-bfba-bfb414ac7201", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "dd0f4043-48bd-4245-8bdb-e7669666a277", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "265f5d51-0a76-4808-8d13-ef598ee6e366", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all features and configurations exposed via `langchain-openai`, head to the API reference: https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html\n", - "\n", - "Refer to the vLLM [documentation](https://docs.vllm.ai/en/latest/) as well." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/xai.ipynb b/docs/docs/integrations/chat/xai.ipynb index d04f1cbcea2..01e06ccdb3a 100644 --- a/docs/docs/integrations/chat/xai.ipynb +++ b/docs/docs/integrations/chat/xai.ipynb @@ -1,332 +1,330 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: xAI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# ChatXAI\n", - "\n", - "\n", - "This page will help you get started with xAI [chat models](../../concepts/chat_models.mdx). For detailed documentation of all `ChatXAI` features and configurations head to the [API reference](https://python.langchain.com/api_reference/xai/chat_models/langchain_xai.chat_models.ChatXAI.html).\n", - "\n", - "[xAI](https://console.x.ai/) offers an API to interact with Grok models.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/xai) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatXAI](https://python.langchain.com/api_reference/xai/chat_models/langchain_xai.chat_models.ChatXAI.html) | [langchain-xai](https://python.langchain.com/api_reference/xai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-xai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-xai?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | \n", - "\n", - "## Setup\n", - "\n", - "To access xAI models you'll need to create an xAI account, get an API key, and install the `langchain-xai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [this page](https://console.x.ai/) to sign up for xAI and generate an API key. Once you've done this set the `XAI_API_KEY` environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"XAI_API_KEY\" not in os.environ:\n", - " os.environ[\"XAI_API_KEY\"] = getpass.getpass(\"Enter your xAI API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain xAI integration lives in the `langchain-xai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-xai" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_xai import ChatXAI\n", - "\n", - "llm = ChatXAI(\n", - " model=\"grok-beta\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"J'adore programmer.\", additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 30, 'total_tokens': 36, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'grok-beta', 'system_fingerprint': 'fp_14b89b2dfc', 'finish_reason': 'stop', 'logprobs': None}, id='run-adffb7a3-e48a-4f52-b694-340d85abe5c3-0', usage_metadata={'input_tokens': 30, 'output_tokens': 6, 'total_tokens': 36, 'input_token_details': {}, 'output_token_details': {}})" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: xAI\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "J'adore programmer.\n" - ] - } - ], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe das Programmieren.', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 7, 'prompt_tokens': 25, 'total_tokens': 32, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'grok-beta', 'system_fingerprint': 'fp_14b89b2dfc', 'finish_reason': 'stop', 'logprobs': None}, id='run-569fc8dc-101b-4e6d-864e-d4fa80df2b63-0', usage_metadata={'input_tokens': 25, 'output_tokens': 7, 'total_tokens': 32, 'input_token_details': {}, 'output_token_details': {}})" + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# ChatXAI\n", + "\n", + "\n", + "This page will help you get started with xAI [chat models](../../concepts/chat_models.mdx). For detailed documentation of all `ChatXAI` features and configurations head to the [API reference](https://python.langchain.com/api_reference/xai/chat_models/langchain_xai.chat_models.ChatXAI.html).\n", + "\n", + "[xAI](https://console.x.ai/) offers an API to interact with Grok models.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/xai) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatXAI](https://python.langchain.com/api_reference/xai/chat_models/langchain_xai.chat_models.ChatXAI.html) | [langchain-xai](https://python.langchain.com/api_reference/xai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-xai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-xai?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ |\n", + "\n", + "## Setup\n", + "\n", + "To access xAI models you'll need to create an xAI account, get an API key, and install the `langchain-xai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [this page](https://console.x.ai/) to sign up for xAI and generate an API key. Once you've done this set the `XAI_API_KEY` environment variable:" ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "e074bce1-0994-4b83-b393-ae7aa7e21750", - "metadata": {}, - "source": [ - "## Tool calling\n", - "\n", - "ChatXAI has a [tool calling](https://docs.x.ai/docs#capabilities) (we use \"tool calling\" and \"function calling\" interchangeably here) API that lets you describe tools and their arguments, and have the model return a JSON object with a tool to invoke and the inputs to that tool. Tool-calling is extremely useful for building tool-using chains and agents, and for getting structured outputs from models more generally.\n", - "\n", - "### ChatXAI.bind_tools()\n", - "\n", - "With `ChatXAI.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an OpenAI tool schemas, which looks like:\n", - "```\n", - "{\n", - " \"name\": \"...\",\n", - " \"description\": \"...\",\n", - " \"parameters\": {...} # JSONSchema\n", - "}\n", - "```\n", - "and passed in every model invocation." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "c6bfe929-ec02-46bd-9d54-76350edddabc", - "metadata": {}, - "outputs": [], - "source": [ - "from pydantic import BaseModel, Field\n", - "\n", - "\n", - "class GetWeather(BaseModel):\n", - " \"\"\"Get the current weather in a given location\"\"\"\n", - "\n", - " location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n", - "\n", - "\n", - "llm_with_tools = llm.bind_tools([GetWeather])" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "5265c892-d8c2-48af-aef5-adbee1647ba6", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content='I am retrieving the current weather for San Francisco.', additional_kwargs={'tool_calls': [{'id': '0', 'function': {'arguments': '{\"location\":\"San Francisco, CA\"}', 'name': 'GetWeather'}, 'type': 'function'}], 'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 11, 'prompt_tokens': 151, 'total_tokens': 162, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'grok-beta', 'system_fingerprint': 'fp_14b89b2dfc', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-73707da7-afec-4a52-bee1-a176b0ab8585-0', tool_calls=[{'name': 'GetWeather', 'args': {'location': 'San Francisco, CA'}, 'id': '0', 'type': 'tool_call'}], usage_metadata={'input_tokens': 151, 'output_tokens': 11, 'total_tokens': 162, 'input_token_details': {}, 'output_token_details': {}})" + "cell_type": "code", + "execution_count": null, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"XAI_API_KEY\" not in os.environ:\n", + " os.environ[\"XAI_API_KEY\"] = getpass.getpass(\"Enter your xAI API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain xAI integration lives in the `langchain-xai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-xai" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_xai import ChatXAI\n", + "\n", + "llm = ChatXAI(\n", + " model=\"grok-beta\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"J'adore programmer.\", additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 30, 'total_tokens': 36, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'grok-beta', 'system_fingerprint': 'fp_14b89b2dfc', 'finish_reason': 'stop', 'logprobs': None}, id='run-adffb7a3-e48a-4f52-b694-340d85abe5c3-0', usage_metadata={'input_tokens': 30, 'output_tokens': 6, 'total_tokens': 36, 'input_token_details': {}, 'output_token_details': {}})" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "J'adore programmer.\n" + ] + } + ], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren.', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 7, 'prompt_tokens': 25, 'total_tokens': 32, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'grok-beta', 'system_fingerprint': 'fp_14b89b2dfc', 'finish_reason': 'stop', 'logprobs': None}, id='run-569fc8dc-101b-4e6d-864e-d4fa80df2b63-0', usage_metadata={'input_tokens': 25, 'output_tokens': 7, 'total_tokens': 32, 'input_token_details': {}, 'output_token_details': {}})" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "e074bce1-0994-4b83-b393-ae7aa7e21750", + "metadata": {}, + "source": [ + "## Tool calling\n", + "\n", + "ChatXAI has a [tool calling](https://docs.x.ai/docs#capabilities) (we use \"tool calling\" and \"function calling\" interchangeably here) API that lets you describe tools and their arguments, and have the model return a JSON object with a tool to invoke and the inputs to that tool. Tool-calling is extremely useful for building tool-using chains and agents, and for getting structured outputs from models more generally.\n", + "\n", + "### ChatXAI.bind_tools()\n", + "\n", + "With `ChatXAI.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an OpenAI tool schemas, which looks like:\n", + "```\n", + "{\n", + " \"name\": \"...\",\n", + " \"description\": \"...\",\n", + " \"parameters\": {...} # JSONSchema\n", + "}\n", + "```\n", + "and passed in every model invocation." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "c6bfe929-ec02-46bd-9d54-76350edddabc", + "metadata": {}, + "outputs": [], + "source": [ + "from pydantic import BaseModel, Field\n", + "\n", + "\n", + "class GetWeather(BaseModel):\n", + " \"\"\"Get the current weather in a given location\"\"\"\n", + "\n", + " location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n", + "\n", + "\n", + "llm_with_tools = llm.bind_tools([GetWeather])" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "5265c892-d8c2-48af-aef5-adbee1647ba6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='I am retrieving the current weather for San Francisco.', additional_kwargs={'tool_calls': [{'id': '0', 'function': {'arguments': '{\"location\":\"San Francisco, CA\"}', 'name': 'GetWeather'}, 'type': 'function'}], 'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 11, 'prompt_tokens': 151, 'total_tokens': 162, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'grok-beta', 'system_fingerprint': 'fp_14b89b2dfc', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-73707da7-afec-4a52-bee1-a176b0ab8585-0', tool_calls=[{'name': 'GetWeather', 'args': {'location': 'San Francisco, CA'}, 'id': '0', 'type': 'tool_call'}], usage_metadata={'input_tokens': 151, 'output_tokens': 11, 'total_tokens': 162, 'input_token_details': {}, 'output_token_details': {}})" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ai_msg = llm_with_tools.invoke(\n", + " \"what is the weather like in San Francisco\",\n", + ")\n", + "ai_msg" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `ChatXAI` features and configurations head to the API reference: https://python.langchain.com/api_reference/xai/chat_models/langchain_xai.chat_models.ChatXAI.html" ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "ai_msg = llm_with_tools.invoke(\n", - " \"what is the weather like in San Francisco\",\n", - ")\n", - "ai_msg" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `ChatXAI` features and configurations head to the API reference: https://python.langchain.com/api_reference/xai/chat_models/langchain_xai.chat_models.ChatXAI.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/chat/yi.ipynb b/docs/docs/integrations/chat/yi.ipynb index 11392981b1c..69752f6d35f 100644 --- a/docs/docs/integrations/chat/yi.ipynb +++ b/docs/docs/integrations/chat/yi.ipynb @@ -1,229 +1,227 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# ChatYI\n", - "\n", - "This will help you getting started with Yi [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatYi features and configurations head to the [API reference](https://python.langchain.com/api_reference/lanchain_community/chat_models/lanchain_community.chat_models.yi.ChatYi.html).\n", - "\n", - "[01.AI](https://www.lingyiwanwu.com/en), founded by Dr. Kai-Fu Lee, is a global company at the forefront of AI 2.0. They offer cutting-edge large language models, including the Yi series, which range from 6B to hundreds of billions of parameters. 01.AI also provides multimodal models, an open API platform, and open-source options like Yi-34B/9B/6B and Yi-VL.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "\n", - "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatYi](https://python.langchain.com/api_reference/lanchain_community/chat_models/lanchain_community.chat_models.yi.ChatYi.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_community?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_community?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access ChatYi models you'll need to create a/an 01.AI account, get an API key, and install the `langchain_community` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [01.AI](https://platform.01.ai) to sign up to 01.AI and generate an API key. Once you've done this set the `YI_API_KEY` environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"YI_API_KEY\" not in os.environ:\n", - " os.environ[\"YI_API_KEY\"] = getpass.getpass(\"Enter your Yi API key: \")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain __ModuleName__ integration lives in the `langchain_community` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.chat_models.yi import ChatYi\n", - "\n", - "llm = ChatYi(\n", - " model=\"yi-large\",\n", - " temperature=0,\n", - " timeout=60,\n", - " yi_api_base=\"https://api.01.ai/v1/chat/completions\",\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Invocation\n" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "AIMessage(content=\"Large Language Models (LLMs) have the potential to significantly impact healthcare by enhancing various aspects of patient care, research, and administrative processes. Here are some potential applications:\\n\\n1. **Clinical Documentation and Reporting**: LLMs can assist in generating patient reports and documentation by understanding and summarizing clinical notes, making the process more efficient and reducing the administrative burden on healthcare professionals.\\n\\n2. **Medical Coding and Billing**: These models can help in automating the coding process for medical billing by accurately translating clinical notes into standardized codes, reducing errors and improving billing efficiency.\\n\\n3. **Clinical Decision Support**: LLMs can analyze patient data and medical literature to provide evidence-based recommendations to healthcare providers, aiding in diagnosis and treatment planning.\\n\\n4. **Patient Education and Communication**: By simplifying medical jargon, LLMs can help in educating patients about their conditions, treatment options, and preventive care, improving patient engagement and health literacy.\\n\\n5. **Natural Language Processing (NLP) for EHRs**: LLMs can enhance NLP capabilities in Electronic Health Records (EHRs) systems, enabling better extraction of information from unstructured data, such as clinical notes, to support data-driven decision-making.\\n\\n6. **Drug Discovery and Development**: LLMs can analyze biomedical literature and clinical trial data to identify new drug candidates, predict drug interactions, and support the development of personalized medicine.\\n\\n7. **Telemedicine and Virtual Health Assistants**: Integrated into telemedicine platforms, LLMs can provide preliminary assessments and triage, offering patients basic health advice and determining the urgency of their needs, thus optimizing the utilization of healthcare resources.\\n\\n8. **Research and Literature Review**: LLMs can expedite the process of reviewing medical literature by quickly identifying relevant studies and summarizing findings, accelerating research and evidence-based practice.\\n\\n9. **Personalized Medicine**: By analyzing a patient's genetic information and medical history, LLMs can help in tailoring treatment plans and medication dosages, contributing to the advancement of personalized medicine.\\n\\n10. **Quality Improvement and Risk Assessment**: LLMs can analyze healthcare data to identify patterns that may indicate areas for quality improvement or potential risks, such as hospital-acquired infections or adverse drug events.\\n\\n11. **Mental Health Support**: LLMs can provide mental health support by offering coping strategies, mindfulness exercises, and preliminary assessments, serving as a complement to professional mental health services.\\n\\n12. **Continuing Medical Education (CME)**: LLMs can personalize CME by recommending educational content based on a healthcare provider's practice area, patient demographics, and emerging medical literature, ensuring that professionals stay updated with the latest advancements.\\n\\nWhile the applications of LLMs in healthcare are promising, it's crucial to address challenges such as data privacy, model bias, and the need for regulatory approval to ensure that these technologies are implemented safely and ethically.\", response_metadata={'token_usage': {'completion_tokens': 656, 'prompt_tokens': 40, 'total_tokens': 696}, 'model': 'yi-large'}, id='run-870850bd-e4bf-4265-8730-1736409c0acf-0')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# ChatYI\n", + "\n", + "This will help you getting started with Yi [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatYi features and configurations head to the [API reference](https://python.langchain.com/api_reference/lanchain_community/chat_models/lanchain_community.chat_models.yi.ChatYi.html).\n", + "\n", + "[01.AI](https://www.lingyiwanwu.com/en), founded by Dr. Kai-Fu Lee, is a global company at the forefront of AI 2.0. They offer cutting-edge large language models, including the Yi series, which range from 6B to hundreds of billions of parameters. 01.AI also provides multimodal models, an open API platform, and open-source options like Yi-34B/9B/6B and Yi-VL.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatYi](https://python.langchain.com/api_reference/lanchain_community/chat_models/lanchain_community.chat_models.yi.ChatYi.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_community?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_community?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "To access ChatYi models you'll need to create a/an 01.AI account, get an API key, and install the `langchain_community` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [01.AI](https://platform.01.ai) to sign up to 01.AI and generate an API key. Once you've done this set the `YI_API_KEY` environment variable:" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.messages import HumanMessage, SystemMessage\n", - "\n", - "messages = [\n", - " SystemMessage(content=\"You are an AI assistant specializing in technology trends.\"),\n", - " HumanMessage(\n", - " content=\"What are the potential applications of large language models in healthcare?\"\n", - " ),\n", - "]\n", - "\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 33, 'total_tokens': 41}, 'model': 'yi-large'}, id='run-daa3bc58-8289-4d72-a24e-80622fa90d6d-0')" + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"YI_API_KEY\" not in os.environ:\n", + " os.environ[\"YI_API_KEY\"] = getpass.getpass(\"Enter your Yi API key: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain __ModuleName__ integration lives in the `langchain_community` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.chat_models.yi import ChatYi\n", + "\n", + "llm = ChatYi(\n", + " model=\"yi-large\",\n", + " temperature=0,\n", + " timeout=60,\n", + " yi_api_base=\"https://api.01.ai/v1/chat/completions\",\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Invocation\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content=\"Large Language Models (LLMs) have the potential to significantly impact healthcare by enhancing various aspects of patient care, research, and administrative processes. Here are some potential applications:\\n\\n1. **Clinical Documentation and Reporting**: LLMs can assist in generating patient reports and documentation by understanding and summarizing clinical notes, making the process more efficient and reducing the administrative burden on healthcare professionals.\\n\\n2. **Medical Coding and Billing**: These models can help in automating the coding process for medical billing by accurately translating clinical notes into standardized codes, reducing errors and improving billing efficiency.\\n\\n3. **Clinical Decision Support**: LLMs can analyze patient data and medical literature to provide evidence-based recommendations to healthcare providers, aiding in diagnosis and treatment planning.\\n\\n4. **Patient Education and Communication**: By simplifying medical jargon, LLMs can help in educating patients about their conditions, treatment options, and preventive care, improving patient engagement and health literacy.\\n\\n5. **Natural Language Processing (NLP) for EHRs**: LLMs can enhance NLP capabilities in Electronic Health Records (EHRs) systems, enabling better extraction of information from unstructured data, such as clinical notes, to support data-driven decision-making.\\n\\n6. **Drug Discovery and Development**: LLMs can analyze biomedical literature and clinical trial data to identify new drug candidates, predict drug interactions, and support the development of personalized medicine.\\n\\n7. **Telemedicine and Virtual Health Assistants**: Integrated into telemedicine platforms, LLMs can provide preliminary assessments and triage, offering patients basic health advice and determining the urgency of their needs, thus optimizing the utilization of healthcare resources.\\n\\n8. **Research and Literature Review**: LLMs can expedite the process of reviewing medical literature by quickly identifying relevant studies and summarizing findings, accelerating research and evidence-based practice.\\n\\n9. **Personalized Medicine**: By analyzing a patient's genetic information and medical history, LLMs can help in tailoring treatment plans and medication dosages, contributing to the advancement of personalized medicine.\\n\\n10. **Quality Improvement and Risk Assessment**: LLMs can analyze healthcare data to identify patterns that may indicate areas for quality improvement or potential risks, such as hospital-acquired infections or adverse drug events.\\n\\n11. **Mental Health Support**: LLMs can provide mental health support by offering coping strategies, mindfulness exercises, and preliminary assessments, serving as a complement to professional mental health services.\\n\\n12. **Continuing Medical Education (CME)**: LLMs can personalize CME by recommending educational content based on a healthcare provider's practice area, patient demographics, and emerging medical literature, ensuring that professionals stay updated with the latest advancements.\\n\\nWhile the applications of LLMs in healthcare are promising, it's crucial to address challenges such as data privacy, model bias, and the need for regulatory approval to ensure that these technologies are implemented safely and ethically.\", response_metadata={'token_usage': {'completion_tokens': 656, 'prompt_tokens': 40, 'total_tokens': 696}, 'model': 'yi-large'}, id='run-870850bd-e4bf-4265-8730-1736409c0acf-0')" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.messages import HumanMessage, SystemMessage\n", + "\n", + "messages = [\n", + " SystemMessage(content=\"You are an AI assistant specializing in technology trends.\"),\n", + " HumanMessage(\n", + " content=\"What are the potential applications of large language models in healthcare?\"\n", + " ),\n", + "]\n", + "\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 33, 'total_tokens': 41}, 'model': 'yi-large'}, id='run-daa3bc58-8289-4d72-a24e-80622fa90d6d-0')" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all ChatYi features and configurations head to the API reference: https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.yi.ChatYi.html" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all ChatYi features and configurations head to the API reference: https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.yi.ChatYi.html" - ] - } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 0 + "nbformat": 4, + "nbformat_minor": 0 } diff --git a/docs/docs/integrations/document_loaders/box.ipynb b/docs/docs/integrations/document_loaders/box.ipynb index c381815410f..d70e17e11e9 100644 --- a/docs/docs/integrations/document_loaders/box.ipynb +++ b/docs/docs/integrations/document_loaders/box.ipynb @@ -1,466 +1,464 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Box\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# BoxLoader and BoxBlobLoader\n", - "\n", - "\n", - "The `langchain-box` package provides two methods to index your files from Box: `BoxLoader` and `BoxBlobLoader`. `BoxLoader` allows you to ingest text representations of files that have a text representation in Box. The `BoxBlobLoader` allows you download the blob for any document or image file for processing with the blob parser of your choice.\n", - "\n", - "This notebook details getting started with both of these. For detailed documentation of all BoxLoader features and configurations head to the API Reference pages for [BoxLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.document_loaders.box.BoxLoader.html) and [BoxBlobLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.blob_loaders.box.BoxBlobLoader.html).\n", - "\n", - "## Overview\n", - "\n", - "The `BoxLoader` class helps you get your unstructured content from Box in Langchain's `Document` format. You can do this with either a `List[str]` containing Box file IDs, or with a `str` containing a Box folder ID. \n", - "\n", - "The `BoxBlobLoader` class helps you get your unstructured content from Box in Langchain's `Blob` format. You can do this with a `List[str]` containing Box file IDs, a `str` containing a Box folder ID, a search query, or a `BoxMetadataQuery`. \n", - "\n", - "If getting files from a folder with folder ID, you can also set a `Bool` to tell the loader to get all sub-folders in that folder, as well. \n", - "\n", - ":::info\n", - "A Box instance can contain Petabytes of files, and folders can contain millions of files. Be intentional when choosing what folders you choose to index. And we recommend never getting all files from folder 0 recursively. Folder ID 0 is your root folder.\n", - ":::\n", - "\n", - "The `BoxLoader` will skip files without a text representation, while the `BoxBlobLoader` will return blobs for all document and image files.\n", - "\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [BoxLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.document_loaders.box.BoxLoader.html) | [langchain_box](https://python.langchain.com/api_reference/box/index.html) | ✅ | ❌ | ❌ | \n", - "| [BoxBlobLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.blob_loaders.box.BoxBlobLoader.html) | [langchain_box](https://python.langchain.com/api_reference/box/index.html) | ✅ | ❌ | ❌ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Async Support\n", - "| :---: | :---: | :---: | \n", - "| BoxLoader | ✅ | ❌ | \n", - "| BoxBlobLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "In order to use the Box package, you will need a few things:\n", - "\n", - "* A Box account — If you are not a current Box customer or want to test outside of your production Box instance, you can use a [free developer account](https://account.box.com/signup/n/developer#ty9l3).\n", - "* [A Box app](https://developer.box.com/guides/getting-started/first-application/) — This is configured in the [developer console](https://account.box.com/developers/console), and for Box AI, must have the `Manage AI` scope enabled. Here you will also select your authentication method\n", - "* The app must be [enabled by the administrator](https://developer.box.com/guides/authorization/custom-app-approval/#manual-approval). For free developer accounts, this is whomever signed up for the account.\n", - "\n", - "### Credentials\n", - "\n", - "For these examples, we will use [token authentication](https://developer.box.com/guides/authentication/tokens/developer-tokens). This can be used with any [authentication method](https://developer.box.com/guides/authentication/). Just get the token with whatever methodology. If you want to learn more about how to use other authentication types with `langchain-box`, visit the [Box provider](/docs/integrations/providers/box) document.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdin", - "output_type": "stream", - "text": [ - "Enter your Box Developer Token: ········\n" - ] - } - ], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "box_developer_token = getpass.getpass(\"Enter your Box Developer Token: \")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_box**." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_box" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "### Load files\n", - "\n", - "If you wish to load files, you must provide the `List` of file ids at instantiation time. \n", - "\n", - "This requires 1 piece of information:\n", - "\n", - "* **box_file_ids** (`List[str]`)- A list of Box file IDs.\n", - "\n", - "#### BoxLoader" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_box.document_loaders import BoxLoader\n", - "\n", - "box_file_ids = [\"1514555423624\", \"1514553902288\"]\n", - "\n", - "loader = BoxLoader(\n", - " box_developer_token=box_developer_token,\n", - " box_file_ids=box_file_ids,\n", - " character_limit=10000, # Optional. Defaults to no limit\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### BoxBlobLoader" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_box.blob_loaders import BoxBlobLoader\n", - "\n", - "box_file_ids = [\"1514555423624\", \"1514553902288\"]\n", - "\n", - "loader = BoxBlobLoader(\n", - " box_developer_token=box_developer_token, box_file_ids=box_file_ids\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Load from folder\n", - "\n", - "If you wish to load files from a folder, you must provide a `str` with the Box folder ID at instantiation time. \n", - "\n", - "This requires 1 piece of information:\n", - "\n", - "* **box_folder_id** (`str`)- A string containing a Box folder ID.\n", - "\n", - "#### BoxLoader" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_box.document_loaders import BoxLoader\n", - "\n", - "box_folder_id = \"260932470532\"\n", - "\n", - "loader = BoxLoader(\n", - " box_folder_id=box_folder_id,\n", - " recursive=False, # Optional. return entire tree, defaults to False\n", - " character_limit=10000, # Optional. Defaults to no limit\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### BoxBlobLoader" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_box.blob_loaders import BoxBlobLoader\n", - "\n", - "box_folder_id = \"260932470532\"\n", - "\n", - "loader = BoxBlobLoader(\n", - " box_folder_id=box_folder_id,\n", - " recursive=False, # Optional. return entire tree, defaults to False\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Search for files with BoxBlobLoader\n", - "\n", - "If you need to search for files, the `BoxBlobLoader` offers two methods. First you can perform a full text search with optional search options to narrow down that search.\n", - "\n", - "This requires 1 piece of information:\n", - "\n", - "* **query** (`str`)- A string containing the search query to perform.\n", - "\n", - "You can also provide a `BoxSearchOptions` object to narrow down that search\n", - "* **box_search_options** (`BoxSearchOptions`)\n", - "\n", - "#### BoxBlobLoader search" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_box.blob_loaders import BoxBlobLoader\n", - "from langchain_box.utilities import BoxSearchOptions, DocumentFiles, SearchTypeFilter\n", - "\n", - "box_folder_id = \"260932470532\"\n", - "\n", - "box_search_options = BoxSearchOptions(\n", - " ancestor_folder_ids=[box_folder_id],\n", - " search_type_filter=[SearchTypeFilter.FILE_CONTENT],\n", - " created_date_range=[\"2023-01-01T00:00:00-07:00\", \"2024-08-01T00:00:00-07:00,\"],\n", - " file_extensions=[DocumentFiles.DOCX, DocumentFiles.PDF],\n", - " k=200,\n", - " size_range=[1, 1000000],\n", - " updated_data_range=None,\n", - ")\n", - "\n", - "loader = BoxBlobLoader(\n", - " box_developer_token=box_developer_token,\n", - " query=\"Victor\",\n", - " box_search_options=box_search_options,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can also search for content based on Box Metadata. If your Box instance uses Metadata, you can search for any documents that have a specific Metadata Template attached that meet a certain criteria, like returning any invoices with a total greater than or equal to $500 that were created last quarter.\n", - "\n", - "This requires 1 piece of information:\n", - "\n", - "* **query** (`str`)- A string containing the search query to perform.\n", - "\n", - "You can also provide a `BoxSearchOptions` object to narrow down that search\n", - "* **box_search_options** (`BoxSearchOptions`)\n", - "\n", - "#### BoxBlobLoader Metadata query" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_box.blob_loaders import BoxBlobLoader\n", - "from langchain_box.utilities import BoxMetadataQuery\n", - "\n", - "query = BoxMetadataQuery(\n", - " template_key=\"enterprise_1234.myTemplate\",\n", - " query=\"total >= :value\",\n", - " query_params={\"value\": 100},\n", - " ancestor_folder_id=\"260932470532\",\n", - ")\n", - "\n", - "loader = BoxBlobLoader(box_metadata_query=query)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load\n", - "\n", - "#### BoxLoader" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Document(metadata={'source': 'https://dl.boxcloud.com/api/2.0/internal_files/1514555423624/versions/1663171610024/representations/extracted_text/content/', 'title': 'Invoice-A5555_txt'}, page_content='Vendor: AstroTech Solutions\\nInvoice Number: A5555\\n\\nLine Items:\\n - Gravitational Wave Detector Kit: $800\\n - Exoplanet Terrarium: $120\\nTotal: $920')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Box\n", + "---" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': 'https://dl.boxcloud.com/api/2.0/internal_files/1514555423624/versions/1663171610024/representations/extracted_text/content/', 'title': 'Invoice-A5555_txt'}\n" - ] - } - ], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### BoxBlobLoader" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# BoxLoader and BoxBlobLoader\n", + "\n", + "\n", + "The `langchain-box` package provides two methods to index your files from Box: `BoxLoader` and `BoxBlobLoader`. `BoxLoader` allows you to ingest text representations of files that have a text representation in Box. The `BoxBlobLoader` allows you download the blob for any document or image file for processing with the blob parser of your choice.\n", + "\n", + "This notebook details getting started with both of these. For detailed documentation of all BoxLoader features and configurations head to the API Reference pages for [BoxLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.document_loaders.box.BoxLoader.html) and [BoxBlobLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.blob_loaders.box.BoxBlobLoader.html).\n", + "\n", + "## Overview\n", + "\n", + "The `BoxLoader` class helps you get your unstructured content from Box in Langchain's `Document` format. You can do this with either a `List[str]` containing Box file IDs, or with a `str` containing a Box folder ID.\n", + "\n", + "The `BoxBlobLoader` class helps you get your unstructured content from Box in Langchain's `Blob` format. You can do this with a `List[str]` containing Box file IDs, a `str` containing a Box folder ID, a search query, or a `BoxMetadataQuery`.\n", + "\n", + "If getting files from a folder with folder ID, you can also set a `Bool` to tell the loader to get all sub-folders in that folder, as well.\n", + "\n", + ":::info\n", + "A Box instance can contain Petabytes of files, and folders can contain millions of files. Be intentional when choosing what folders you choose to index. And we recommend never getting all files from folder 0 recursively. Folder ID 0 is your root folder.\n", + ":::\n", + "\n", + "The `BoxLoader` will skip files without a text representation, while the `BoxBlobLoader` will return blobs for all document and image files.\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [BoxLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.document_loaders.box.BoxLoader.html) | [langchain_box](https://python.langchain.com/api_reference/box/index.html) | ✅ | ❌ | ❌ |\n", + "| [BoxBlobLoader](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.blob_loaders.box.BoxBlobLoader.html) | [langchain_box](https://python.langchain.com/api_reference/box/index.html) | ✅ | ❌ | ❌ |\n", + "### Loader features\n", + "| Source | Document Lazy Loading | Async Support\n", + "| :---: | :---: | :---: |\n", + "| BoxLoader | ✅ | ❌ |\n", + "| BoxBlobLoader | ✅ | ❌ |\n", + "\n", + "## Setup\n", + "\n", + "In order to use the Box package, you will need a few things:\n", + "\n", + "* A Box account — If you are not a current Box customer or want to test outside of your production Box instance, you can use a [free developer account](https://account.box.com/signup/n/developer#ty9l3).\n", + "* [A Box app](https://developer.box.com/guides/getting-started/first-application/) — This is configured in the [developer console](https://account.box.com/developers/console), and for Box AI, must have the `Manage AI` scope enabled. Here you will also select your authentication method\n", + "* The app must be [enabled by the administrator](https://developer.box.com/guides/authorization/custom-app-approval/#manual-approval). For free developer accounts, this is whomever signed up for the account.\n", + "\n", + "### Credentials\n", + "\n", + "For these examples, we will use [token authentication](https://developer.box.com/guides/authentication/tokens/developer-tokens). This can be used with any [authentication method](https://developer.box.com/guides/authentication/). Just get the token with whatever methodology. If you want to learn more about how to use other authentication types with `langchain-box`, visit the [Box provider](/docs/integrations/providers/box) document.\n" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Blob(id='1514555423624' metadata={'source': 'https://app.box.com/0/260935730128/260931903795/Invoice-A5555.txt', 'name': 'Invoice-A5555.txt', 'file_size': 150} data=\"b'Vendor: AstroTech Solutions\\\\nInvoice Number: A5555\\\\n\\\\nLine Items:\\\\n - Gravitational Wave Detector Kit: $800\\\\n - Exoplanet Terrarium: $120\\\\nTotal: $920'\" mimetype='text/plain' path='https://app.box.com/0/260935730128/260931903795/Invoice-A5555.txt')\n", - "Blob(id='1514553902288' metadata={'source': 'https://app.box.com/0/260935730128/260931903795/Invoice-B1234.txt', 'name': 'Invoice-B1234.txt', 'file_size': 168} data=\"b'Vendor: Galactic Gizmos Inc.\\\\nInvoice Number: B1234\\\\nPurchase Order Number: 001\\\\nLine Items:\\\\n - Quantum Flux Capacitor: $500\\\\n - Anti-Gravity Pen Set: $75\\\\nTotal: $575'\" mimetype='text/plain' path='https://app.box.com/0/260935730128/260931903795/Invoice-B1234.txt')\n" - ] + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdin", + "output_type": "stream", + "text": [ + "Enter your Box Developer Token: ········\n" + ] + } + ], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "box_developer_token = getpass.getpass(\"Enter your Box Developer Token: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_box**." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_box" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "### Load files\n", + "\n", + "If you wish to load files, you must provide the `List` of file ids at instantiation time.\n", + "\n", + "This requires 1 piece of information:\n", + "\n", + "* **box_file_ids** (`List[str]`)- A list of Box file IDs.\n", + "\n", + "#### BoxLoader" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_box.document_loaders import BoxLoader\n", + "\n", + "box_file_ids = [\"1514555423624\", \"1514553902288\"]\n", + "\n", + "loader = BoxLoader(\n", + " box_developer_token=box_developer_token,\n", + " box_file_ids=box_file_ids,\n", + " character_limit=10000, # Optional. Defaults to no limit\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BoxBlobLoader" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_box.blob_loaders import BoxBlobLoader\n", + "\n", + "box_file_ids = [\"1514555423624\", \"1514553902288\"]\n", + "\n", + "loader = BoxBlobLoader(\n", + " box_developer_token=box_developer_token, box_file_ids=box_file_ids\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load from folder\n", + "\n", + "If you wish to load files from a folder, you must provide a `str` with the Box folder ID at instantiation time.\n", + "\n", + "This requires 1 piece of information:\n", + "\n", + "* **box_folder_id** (`str`)- A string containing a Box folder ID.\n", + "\n", + "#### BoxLoader" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_box.document_loaders import BoxLoader\n", + "\n", + "box_folder_id = \"260932470532\"\n", + "\n", + "loader = BoxLoader(\n", + " box_folder_id=box_folder_id,\n", + " recursive=False, # Optional. return entire tree, defaults to False\n", + " character_limit=10000, # Optional. Defaults to no limit\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BoxBlobLoader" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_box.blob_loaders import BoxBlobLoader\n", + "\n", + "box_folder_id = \"260932470532\"\n", + "\n", + "loader = BoxBlobLoader(\n", + " box_folder_id=box_folder_id,\n", + " recursive=False, # Optional. return entire tree, defaults to False\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Search for files with BoxBlobLoader\n", + "\n", + "If you need to search for files, the `BoxBlobLoader` offers two methods. First you can perform a full text search with optional search options to narrow down that search.\n", + "\n", + "This requires 1 piece of information:\n", + "\n", + "* **query** (`str`)- A string containing the search query to perform.\n", + "\n", + "You can also provide a `BoxSearchOptions` object to narrow down that search\n", + "* **box_search_options** (`BoxSearchOptions`)\n", + "\n", + "#### BoxBlobLoader search" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_box.blob_loaders import BoxBlobLoader\n", + "from langchain_box.utilities import BoxSearchOptions, DocumentFiles, SearchTypeFilter\n", + "\n", + "box_folder_id = \"260932470532\"\n", + "\n", + "box_search_options = BoxSearchOptions(\n", + " ancestor_folder_ids=[box_folder_id],\n", + " search_type_filter=[SearchTypeFilter.FILE_CONTENT],\n", + " created_date_range=[\"2023-01-01T00:00:00-07:00\", \"2024-08-01T00:00:00-07:00,\"],\n", + " file_extensions=[DocumentFiles.DOCX, DocumentFiles.PDF],\n", + " k=200,\n", + " size_range=[1, 1000000],\n", + " updated_data_range=None,\n", + ")\n", + "\n", + "loader = BoxBlobLoader(\n", + " box_developer_token=box_developer_token,\n", + " query=\"Victor\",\n", + " box_search_options=box_search_options,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can also search for content based on Box Metadata. If your Box instance uses Metadata, you can search for any documents that have a specific Metadata Template attached that meet a certain criteria, like returning any invoices with a total greater than or equal to $500 that were created last quarter.\n", + "\n", + "This requires 1 piece of information:\n", + "\n", + "* **query** (`str`)- A string containing the search query to perform.\n", + "\n", + "You can also provide a `BoxSearchOptions` object to narrow down that search\n", + "* **box_search_options** (`BoxSearchOptions`)\n", + "\n", + "#### BoxBlobLoader Metadata query" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_box.blob_loaders import BoxBlobLoader\n", + "from langchain_box.utilities import BoxMetadataQuery\n", + "\n", + "query = BoxMetadataQuery(\n", + " template_key=\"enterprise_1234.myTemplate\",\n", + " query=\"total >= :value\",\n", + " query_params={\"value\": 100},\n", + " ancestor_folder_id=\"260932470532\",\n", + ")\n", + "\n", + "loader = BoxBlobLoader(box_metadata_query=query)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load\n", + "\n", + "#### BoxLoader" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': 'https://dl.boxcloud.com/api/2.0/internal_files/1514555423624/versions/1663171610024/representations/extracted_text/content/', 'title': 'Invoice-A5555_txt'}, page_content='Vendor: AstroTech Solutions\\nInvoice Number: A5555\\n\\nLine Items:\\n - Gravitational Wave Detector Kit: $800\\n - Exoplanet Terrarium: $120\\nTotal: $920')" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': 'https://dl.boxcloud.com/api/2.0/internal_files/1514555423624/versions/1663171610024/representations/extracted_text/content/', 'title': 'Invoice-A5555_txt'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### BoxBlobLoader" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Blob(id='1514555423624' metadata={'source': 'https://app.box.com/0/260935730128/260931903795/Invoice-A5555.txt', 'name': 'Invoice-A5555.txt', 'file_size': 150} data=\"b'Vendor: AstroTech Solutions\\\\nInvoice Number: A5555\\\\n\\\\nLine Items:\\\\n - Gravitational Wave Detector Kit: $800\\\\n - Exoplanet Terrarium: $120\\\\nTotal: $920'\" mimetype='text/plain' path='https://app.box.com/0/260935730128/260931903795/Invoice-A5555.txt')\n", + "Blob(id='1514553902288' metadata={'source': 'https://app.box.com/0/260935730128/260931903795/Invoice-B1234.txt', 'name': 'Invoice-B1234.txt', 'file_size': 168} data=\"b'Vendor: Galactic Gizmos Inc.\\\\nInvoice Number: B1234\\\\nPurchase Order Number: 001\\\\nLine Items:\\\\n - Quantum Flux Capacitor: $500\\\\n - Anti-Gravity Pen Set: $75\\\\nTotal: $575'\" mimetype='text/plain' path='https://app.box.com/0/260935730128/260931903795/Invoice-B1234.txt')\n" + ] + } + ], + "source": [ + "for blob in loader.yield_blobs():\n", + " print(f\"Blob({blob})\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load\n", + "\n", + "#### BoxLoader only" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Extra fields\n", + "\n", + "All Box connectors offer the ability to select additional fields from the Box `FileFull` object to return as custom LangChain metadata. Each object accepts an optional `List[str]` called `extra_fields` containing the json key from the return object, like `extra_fields=[\"shared_link\"]`.\n", + "\n", + "The connector will add this field to the list of fields the integration needs to function and then add the results to the metadata returned in the `Document` or `Blob`, like `\"metadata\" : { \"source\" : \"source, \"shared_link\" : \"shared_link\" }`. If the field is unavailable for that file, it will be returned as an empty string, like `\"shared_link\" : \"\"`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all BoxLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.document_loaders.box.BoxLoader.html)\n", + "\n", + "\n", + "## Help\n", + "\n", + "If you have questions, you can check out our [developer documentation](https://developer.box.com) or reach out to use in our [developer community](https://community.box.com)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.6" } - ], - "source": [ - "for blob in loader.yield_blobs():\n", - " print(f\"Blob({blob})\")" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load\n", - "\n", - "#### BoxLoader only" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Extra fields\n", - "\n", - "All Box connectors offer the ability to select additional fields from the Box `FileFull` object to return as custom LangChain metadata. Each object accepts an optional `List[str]` called `extra_fields` containing the json key from the return object, like `extra_fields=[\"shared_link\"]`. \n", - "\n", - "The connector will add this field to the list of fields the integration needs to function and then add the results to the metadata returned in the `Document` or `Blob`, like `\"metadata\" : { \"source\" : \"source, \"shared_link\" : \"shared_link\" }`. If the field is unavailable for that file, it will be returned as an empty string, like `\"shared_link\" : \"\"`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all BoxLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/box/document_loaders/langchain_box.document_loaders.box.BoxLoader.html)\n", - "\n", - "\n", - "## Help\n", - "\n", - "If you have questions, you can check out our [developer documentation](https://developer.box.com) or reach out to use in our [developer community](https://community.box.com)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.6" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/document_loaders/bshtml.ipynb b/docs/docs/integrations/document_loaders/bshtml.ipynb index 29e9e8b1d59..3610f38a485 100644 --- a/docs/docs/integrations/document_loaders/bshtml.ipynb +++ b/docs/docs/integrations/document_loaders/bshtml.ipynb @@ -1,243 +1,241 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# BSHTMLLoader\n", - "\n", - "\n", - "This notebook provides a quick overview for getting started with BeautifulSoup4 [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all __ModuleName__Loader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.html_bs.BSHTMLLoader.html).\n", - "\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "\n", - "| Class | Package | Local | Serializable | JS support|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [BSHTMLLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.html_bs.BSHTMLLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| BSHTMLLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access BSHTMLLoader document loader you'll need to install the `langchain-community` integration package and the `bs4` python package.\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are needed to use the `BSHTMLLoader` class." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community** and **bs4**." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community bs4" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import BSHTMLLoader\n", - "\n", - "loader = BSHTMLLoader(\n", - " file_path=\"./example_data/fake-content.html\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/fake-content.html', 'title': 'Test Title'}, page_content='\\nTest Title\\n\\n\\nMy First Heading\\nMy first paragraph.\\n\\n\\n')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# BSHTMLLoader\n", + "\n", + "\n", + "This notebook provides a quick overview for getting started with BeautifulSoup4 [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all __ModuleName__Loader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.html_bs.BSHTMLLoader.html).\n", + "\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | Serializable | JS support|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [BSHTMLLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.html_bs.BSHTMLLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| BSHTMLLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "To access BSHTMLLoader document loader you'll need to install the `langchain-community` integration package and the `bs4` python package.\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are needed to use the `BSHTMLLoader` class." ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': './example_data/fake-content.html', 'title': 'Test Title'}\n" - ] - } - ], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/fake-content.html', 'title': 'Test Title'}, page_content='\\nTest Title\\n\\n\\nMy First Heading\\nMy first paragraph.\\n\\n\\n')" + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []\n", - "page[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Adding separator to BS4\n", - "\n", - "We can also pass a separator to use when calling get_text on the soup" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "page_content='\n", - ", Test Title, \n", - ", \n", - ", \n", - ", My First Heading, \n", - ", My first paragraph., \n", - ", \n", - ", \n", - "' metadata={'source': './example_data/fake-content.html', 'title': 'Test Title'}\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community** and **bs4**." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community bs4" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import BSHTMLLoader\n", + "\n", + "loader = BSHTMLLoader(\n", + " file_path=\"./example_data/fake-content.html\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/fake-content.html', 'title': 'Test Title'}, page_content='\\nTest Title\\n\\n\\nMy First Heading\\nMy first paragraph.\\n\\n\\n')" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': './example_data/fake-content.html', 'title': 'Test Title'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/fake-content.html', 'title': 'Test Title'}, page_content='\\nTest Title\\n\\n\\nMy First Heading\\nMy first paragraph.\\n\\n\\n')" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []\n", + "page[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Adding separator to BS4\n", + "\n", + "We can also pass a separator to use when calling get_text on the soup" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "page_content='\n", + ", Test Title, \n", + ", \n", + ", \n", + ", My First Heading, \n", + ", My first paragraph., \n", + ", \n", + ", \n", + "' metadata={'source': './example_data/fake-content.html', 'title': 'Test Title'}\n" + ] + } + ], + "source": [ + "loader = BSHTMLLoader(\n", + " file_path=\"./example_data/fake-content.html\", get_text_separator=\", \"\n", + ")\n", + "\n", + "docs = loader.load()\n", + "print(docs[0])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all BSHTMLLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.html_bs.BSHTMLLoader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" } - ], - "source": [ - "loader = BSHTMLLoader(\n", - " file_path=\"./example_data/fake-content.html\", get_text_separator=\", \"\n", - ")\n", - "\n", - "docs = loader.load()\n", - "print(docs[0])" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all BSHTMLLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.html_bs.BSHTMLLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/docs/docs/integrations/document_loaders/json.ipynb b/docs/docs/integrations/document_loaders/json.ipynb index e3a11d0ace0..1539c009fa8 100644 --- a/docs/docs/integrations/document_loaders/json.ipynb +++ b/docs/docs/integrations/document_loaders/json.ipynb @@ -1,348 +1,346 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# JSONLoader\n", - "\n", - "This notebook provides a quick overview for getting started with JSON [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all JSONLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.json_loader.JSONLoader.html).\n", - "\n", - "- TODO: Add any other relevant links, like information about underlying API, etc.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/json/)|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [JSONLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.json_loader.JSONLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| JSONLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access JSON document loader you'll need to install the `langchain-community` integration package as well as the ``jq`` python package.\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are required to use the `JSONLoader` class." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community** and **jq**:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community jq " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import JSONLoader\n", - "\n", - "loader = JSONLoader(\n", - " file_path=\"./example_data/facebook_chat.json\",\n", - " jq_schema=\".messages[].content\",\n", - " text_content=False,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "Document(metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1}, page_content='Bye!')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# JSONLoader\n", + "\n", + "This notebook provides a quick overview for getting started with JSON [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all JSONLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.json_loader.JSONLoader.html).\n", + "\n", + "- TODO: Add any other relevant links, like information about underlying API, etc.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/json/)|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [JSONLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.json_loader.JSONLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| JSONLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "To access JSON document loader you'll need to install the `langchain-community` integration package as well as the ``jq`` python package.\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are required to use the `JSONLoader` class." ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1}\n" - ] - } - ], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "pages = []\n", - "for doc in loader.lazy_load():\n", - " pages.append(doc)\n", - " if len(pages) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(pages)\n", - "\n", - " pages = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Read from JSON Lines file\n", - "\n", - "If you want to load documents from a JSON Lines file, you pass `json_lines=True`\n", - "and specify `jq_schema` to extract `page_content` from a single JSON object." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "page_content='Bye!' metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat_messages.jsonl', 'seq_num': 1}\n" - ] - } - ], - "source": [ - "loader = JSONLoader(\n", - " file_path=\"./example_data/facebook_chat_messages.jsonl\",\n", - " jq_schema=\".content\",\n", - " text_content=False,\n", - " json_lines=True,\n", - ")\n", - "\n", - "docs = loader.load()\n", - "print(docs[0])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Read specific content keys\n", - "\n", - "Another option is to set `jq_schema='.'` and provide a `content_key` in order to only load specific content:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "page_content='User 2' metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat_messages.jsonl', 'seq_num': 1}\n" - ] - } - ], - "source": [ - "loader = JSONLoader(\n", - " file_path=\"./example_data/facebook_chat_messages.jsonl\",\n", - " jq_schema=\".\",\n", - " content_key=\"sender_name\",\n", - " json_lines=True,\n", - ")\n", - "\n", - "docs = loader.load()\n", - "print(docs[0])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## JSON file with jq schema `content_key`\n", - "\n", - "To load documents from a JSON file using the `content_key` within the jq schema, set `is_content_key_jq_parsable=True`. Ensure that `content_key` is compatible and can be parsed using the jq schema." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community** and **jq**:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "page_content='Bye!' metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1}\n" - ] - } - ], - "source": [ - "loader = JSONLoader(\n", - " file_path=\"./example_data/facebook_chat.json\",\n", - " jq_schema=\".messages[]\",\n", - " content_key=\".content\",\n", - " is_content_key_jq_parsable=True,\n", - ")\n", - "\n", - "docs = loader.load()\n", - "print(docs[0])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Extracting metadata\n", - "\n", - "Generally, we want to include metadata available in the JSON file into the documents that we create from the content.\n", - "\n", - "The following demonstrates how metadata can be extracted using the `JSONLoader`.\n", - "\n", - "There are some key changes to be noted. In the previous example where we didn't collect the metadata, we managed to directly specify in the schema where the value for the `page_content` can be extracted from.\n", - "\n", - "In this example, we have to tell the loader to iterate over the records in the `messages` field. The jq_schema then has to be `.messages[]`\n", - "\n", - "This allows us to pass the records (dict) into the `metadata_func` that has to be implemented. The `metadata_func` is responsible for identifying which pieces of information in the record should be included in the metadata stored in the final `Document` object.\n", - "\n", - "Additionally, we now have to explicitly specify in the loader, via the `content_key` argument, the key from the record where the value for the `page_content` needs to be extracted from." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [ + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community jq " + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1, 'sender_name': 'User 2', 'timestamp_ms': 1675597571851}\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import JSONLoader\n", + "\n", + "loader = JSONLoader(\n", + " file_path=\"./example_data/facebook_chat.json\",\n", + " jq_schema=\".messages[].content\",\n", + " text_content=False,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1}, page_content='Bye!')" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "pages = []\n", + "for doc in loader.lazy_load():\n", + " pages.append(doc)\n", + " if len(pages) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(pages)\n", + "\n", + " pages = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Read from JSON Lines file\n", + "\n", + "If you want to load documents from a JSON Lines file, you pass `json_lines=True`\n", + "and specify `jq_schema` to extract `page_content` from a single JSON object." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "page_content='Bye!' metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat_messages.jsonl', 'seq_num': 1}\n" + ] + } + ], + "source": [ + "loader = JSONLoader(\n", + " file_path=\"./example_data/facebook_chat_messages.jsonl\",\n", + " jq_schema=\".content\",\n", + " text_content=False,\n", + " json_lines=True,\n", + ")\n", + "\n", + "docs = loader.load()\n", + "print(docs[0])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Read specific content keys\n", + "\n", + "Another option is to set `jq_schema='.'` and provide a `content_key` in order to only load specific content:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "page_content='User 2' metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat_messages.jsonl', 'seq_num': 1}\n" + ] + } + ], + "source": [ + "loader = JSONLoader(\n", + " file_path=\"./example_data/facebook_chat_messages.jsonl\",\n", + " jq_schema=\".\",\n", + " content_key=\"sender_name\",\n", + " json_lines=True,\n", + ")\n", + "\n", + "docs = loader.load()\n", + "print(docs[0])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## JSON file with jq schema `content_key`\n", + "\n", + "To load documents from a JSON file using the `content_key` within the jq schema, set `is_content_key_jq_parsable=True`. Ensure that `content_key` is compatible and can be parsed using the jq schema." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "page_content='Bye!' metadata={'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1}\n" + ] + } + ], + "source": [ + "loader = JSONLoader(\n", + " file_path=\"./example_data/facebook_chat.json\",\n", + " jq_schema=\".messages[]\",\n", + " content_key=\".content\",\n", + " is_content_key_jq_parsable=True,\n", + ")\n", + "\n", + "docs = loader.load()\n", + "print(docs[0])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Extracting metadata\n", + "\n", + "Generally, we want to include metadata available in the JSON file into the documents that we create from the content.\n", + "\n", + "The following demonstrates how metadata can be extracted using the `JSONLoader`.\n", + "\n", + "There are some key changes to be noted. In the previous example where we didn't collect the metadata, we managed to directly specify in the schema where the value for the `page_content` can be extracted from.\n", + "\n", + "In this example, we have to tell the loader to iterate over the records in the `messages` field. The jq_schema then has to be `.messages[]`\n", + "\n", + "This allows us to pass the records (dict) into the `metadata_func` that has to be implemented. The `metadata_func` is responsible for identifying which pieces of information in the record should be included in the metadata stored in the final `Document` object.\n", + "\n", + "Additionally, we now have to explicitly specify in the loader, via the `content_key` argument, the key from the record where the value for the `page_content` needs to be extracted from." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': '/Users/isaachershenson/Documents/langchain/docs/docs/integrations/document_loaders/example_data/facebook_chat.json', 'seq_num': 1, 'sender_name': 'User 2', 'timestamp_ms': 1675597571851}\n" + ] + } + ], + "source": [ + "# Define the metadata extraction function.\n", + "def metadata_func(record: dict, metadata: dict) -> dict:\n", + " metadata[\"sender_name\"] = record.get(\"sender_name\")\n", + " metadata[\"timestamp_ms\"] = record.get(\"timestamp_ms\")\n", + "\n", + " return metadata\n", + "\n", + "\n", + "loader = JSONLoader(\n", + " file_path=\"./example_data/facebook_chat.json\",\n", + " jq_schema=\".messages[]\",\n", + " content_key=\"content\",\n", + " metadata_func=metadata_func,\n", + ")\n", + "\n", + "docs = loader.load()\n", + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all JSONLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.json_loader.JSONLoader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" } - ], - "source": [ - "# Define the metadata extraction function.\n", - "def metadata_func(record: dict, metadata: dict) -> dict:\n", - " metadata[\"sender_name\"] = record.get(\"sender_name\")\n", - " metadata[\"timestamp_ms\"] = record.get(\"timestamp_ms\")\n", - "\n", - " return metadata\n", - "\n", - "\n", - "loader = JSONLoader(\n", - " file_path=\"./example_data/facebook_chat.json\",\n", - " jq_schema=\".messages[]\",\n", - " content_key=\"content\",\n", - " metadata_func=metadata_func,\n", - ")\n", - "\n", - "docs = loader.load()\n", - "print(docs[0].metadata)" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all JSONLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.json_loader.JSONLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/docs/docs/integrations/document_loaders/mathpix.ipynb b/docs/docs/integrations/document_loaders/mathpix.ipynb index bdbcbb96adc..d270247a7c2 100644 --- a/docs/docs/integrations/document_loaders/mathpix.ipynb +++ b/docs/docs/integrations/document_loaders/mathpix.ipynb @@ -1,178 +1,176 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# MathPixPDFLoader\n", - "\n", - "Inspired by Daniel Gross's snippet here: [https://gist.github.com/danielgross/3ab4104e14faccc12b49200843adab21](https://gist.github.com/danielgross/3ab4104e14faccc12b49200843adab21)\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [MathPixPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.MathpixPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| MathPixPDFLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "### Credentials\n", - "\n", - "Sign up for Mathpix and [create an API key](https://mathpix.com/docs/ocr/creating-an-api-key) to set the `MATHPIX_API_KEY` variables in your environment" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# MathPixPDFLoader\n", + "\n", + "Inspired by Daniel Gross's snippet here: [https://gist.github.com/danielgross/3ab4104e14faccc12b49200843adab21](https://gist.github.com/danielgross/3ab4104e14faccc12b49200843adab21)\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [MathPixPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.MathpixPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| MathPixPDFLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "### Credentials\n", + "\n", + "Sign up for Mathpix and [create an API key](https://mathpix.com/docs/ocr/creating-an-api-key) to set the `MATHPIX_API_KEY` variables in your environment" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"MATHPIX_API_KEY\" not in os.environ:\n", + " os.environ[\"MATHPIX_API_KEY\"] = getpass.getpass(\"Enter your Mathpix API key: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community**." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we are ready to initialize our loader:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import MathpixPDFLoader\n", + "\n", + "file_path = \"./example_data/layout-parser-paper.pdf\"\n", + "loader = MathpixPDFLoader(file_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all MathpixPDFLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.MathpixPDFLoader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"MATHPIX_API_KEY\" not in os.environ:\n", - " os.environ[\"MATHPIX_API_KEY\"] = getpass.getpass(\"Enter your Mathpix API key: \")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community**." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we are ready to initialize our loader:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import MathpixPDFLoader\n", - "\n", - "file_path = \"./example_data/layout-parser-paper.pdf\"\n", - "loader = MathpixPDFLoader(file_path)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all MathpixPDFLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.MathpixPDFLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/docs/docs/integrations/document_loaders/pdfplumber.ipynb b/docs/docs/integrations/document_loaders/pdfplumber.ipynb index cfa43817f10..9a235c49199 100644 --- a/docs/docs/integrations/document_loaders/pdfplumber.ipynb +++ b/docs/docs/integrations/document_loaders/pdfplumber.ipynb @@ -1,183 +1,181 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# PDFPlumber\n", - "\n", - "Like PyMuPDF, the output Documents contain detailed metadata about the PDF and its pages, and returns one document per page.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [PDFPlumberLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PDFPlumberLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| PDFPlumberLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are needed to use this loader." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community**." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import PDFPlumberLoader\n", - "\n", - "loader = PDFPlumberLoader(\"./example_data/layout-parser-paper.pdf\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/layout-parser-paper.pdf', 'file_path': './example_data/layout-parser-paper.pdf', 'page': 0, 'total_pages': 16, 'Author': '', 'CreationDate': 'D:20210622012710Z', 'Creator': 'LaTeX with hyperref', 'Keywords': '', 'ModDate': 'D:20210622012710Z', 'PTEX.Fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'Producer': 'pdfTeX-1.40.21', 'Subject': '', 'Title': '', 'Trapped': 'False'}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen1 ((cid:0)), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\\nLee4, Jacob Carlson3, and Weining Li5\\n1 Allen Institute for AI\\nshannons@allenai.org\\n2 Brown University\\nruochen zhang@brown.edu\\n3 Harvard University\\n{melissadell,jacob carlson}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu\\n5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recentadvancesindocumentimageanalysis(DIA)havebeen\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomescouldbeeasilydeployedinproductionandextendedforfurther\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im-\\nportantinnovationsbyawideaudience.Thoughtherehavebeenon-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopmentindisciplineslikenaturallanguageprocessingandcomputer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademicresearchacross awiderangeof disciplinesinthesocialsciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica-\\ntions. The core LayoutParser library comes with a set of simple and\\nintuitiveinterfacesforapplyingandcustomizingDLmodelsforlayoutde-\\ntection,characterrecognition,andmanyotherdocumentprocessingtasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti-\\nzation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: DocumentImageAnalysis·DeepLearning·LayoutAnalysis\\n· Character Recognition · Open Source library · Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocumentimageanalysis(DIA)tasksincludingdocumentimageclassification[11,\\n1202\\nnuJ\\n12\\n]VC.sc[\\n2v84351.3012:viXra\\n')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PDFPlumber\n", + "\n", + "Like PyMuPDF, the output Documents contain detailed metadata about the PDF and its pages, and returns one document per page.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [PDFPlumberLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PDFPlumberLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| PDFPlumberLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are needed to use this loader." ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': './example_data/layout-parser-paper.pdf', 'file_path': './example_data/layout-parser-paper.pdf', 'page': 0, 'total_pages': 16, 'Author': '', 'CreationDate': 'D:20210622012710Z', 'Creator': 'LaTeX with hyperref', 'Keywords': '', 'ModDate': 'D:20210622012710Z', 'PTEX.Fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'Producer': 'pdfTeX-1.40.21', 'Subject': '', 'Title': '', 'Trapped': 'False'}\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community**." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import PDFPlumberLoader\n", + "\n", + "loader = PDFPlumberLoader(\"./example_data/layout-parser-paper.pdf\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/layout-parser-paper.pdf', 'file_path': './example_data/layout-parser-paper.pdf', 'page': 0, 'total_pages': 16, 'Author': '', 'CreationDate': 'D:20210622012710Z', 'Creator': 'LaTeX with hyperref', 'Keywords': '', 'ModDate': 'D:20210622012710Z', 'PTEX.Fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'Producer': 'pdfTeX-1.40.21', 'Subject': '', 'Title': '', 'Trapped': 'False'}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen1 ((cid:0)), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\\nLee4, Jacob Carlson3, and Weining Li5\\n1 Allen Institute for AI\\nshannons@allenai.org\\n2 Brown University\\nruochen zhang@brown.edu\\n3 Harvard University\\n{melissadell,jacob carlson}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu\\n5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recentadvancesindocumentimageanalysis(DIA)havebeen\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomescouldbeeasilydeployedinproductionandextendedforfurther\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im-\\nportantinnovationsbyawideaudience.Thoughtherehavebeenon-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopmentindisciplineslikenaturallanguageprocessingandcomputer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademicresearchacross awiderangeof disciplinesinthesocialsciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica-\\ntions. The core LayoutParser library comes with a set of simple and\\nintuitiveinterfacesforapplyingandcustomizingDLmodelsforlayoutde-\\ntection,characterrecognition,andmanyotherdocumentprocessingtasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti-\\nzation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: DocumentImageAnalysis·DeepLearning·LayoutAnalysis\\n· Character Recognition · Open Source library · Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocumentimageanalysis(DIA)tasksincludingdocumentimageclassification[11,\\n1202\\nnuJ\\n12\\n]VC.sc[\\n2v84351.3012:viXra\\n')" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': './example_data/layout-parser-paper.pdf', 'file_path': './example_data/layout-parser-paper.pdf', 'page': 0, 'total_pages': 16, 'Author': '', 'CreationDate': 'D:20210622012710Z', 'Creator': 'LaTeX with hyperref', 'Keywords': '', 'ModDate': 'D:20210622012710Z', 'PTEX.Fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'Producer': 'pdfTeX-1.40.21', 'Subject': '', 'Title': '', 'Trapped': 'False'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all PDFPlumberLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PDFPlumberLoader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" } - ], - "source": [ - "print(docs[0].metadata)" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all PDFPlumberLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PDFPlumberLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/docs/docs/integrations/document_loaders/pymupdf4llm.ipynb b/docs/docs/integrations/document_loaders/pymupdf4llm.ipynb index 674701ee59b..8eb07540515 100644 --- a/docs/docs/integrations/document_loaders/pymupdf4llm.ipynb +++ b/docs/docs/integrations/document_loaders/pymupdf4llm.ipynb @@ -1,721 +1,719 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: PyMuPDF4LLM\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# PyMuPDF4LLMLoader\n", - "\n", - "This notebook provides a quick overview for getting started with PyMuPDF4LLM [document loader](https://python.langchain.com/docs/concepts/#document-loaders). For detailed documentation of all PyMuPDF4LLMLoader features and configurations head to the [GitHub repository](https://github.com/lakinduboteju/langchain-pymupdf4llm).\n", - "\n", - "## Overview\n", - "\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support |\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [PyMuPDF4LLMLoader](https://github.com/lakinduboteju/langchain-pymupdf4llm) | [langchain_pymupdf4llm](https://pypi.org/project/langchain-pymupdf4llm) | ✅ | ❌ | ❌ |\n", - "\n", - "### Loader features\n", - "\n", - "| Source | Document Lazy Loading | Native Async Support | Extract Images | Extract Tables |\n", - "| :---: | :---: | :---: | :---: | :---: |\n", - "| PyMuPDF4LLMLoader | ✅ | ❌ | ✅ | ✅ |\n", - "\n", - "## Setup\n", - "\n", - "To access PyMuPDF4LLM document loader you'll need to install the `langchain-pymupdf4llm` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are required to use PyMuPDF4LLMLoader." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community** and **langchain-pymupdf4llm**." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain_community langchain-pymupdf4llm" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_pymupdf4llm import PyMuPDF4LLMLoader\n", - "\n", - "file_path = \"./example_data/layout-parser-paper.pdf\"\n", - "loader = PyMuPDF4LLMLoader(file_path)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Document(metadata={'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'source': './example_data/layout-parser-paper.pdf', 'file_path': './example_data/layout-parser-paper.pdf', 'total_pages': 16, 'format': 'PDF 1.5', 'title': '', 'author': '', 'subject': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'trapped': '', 'modDate': 'D:20210622012710Z', 'creationDate': 'D:20210622012710Z', 'page': 0}, page_content='```\\nLayoutParser: A Unified Toolkit for Deep\\n\\n## Learning Based Document Image Analysis\\n\\n```\\n\\nZejiang Shen[1] (�), Ruochen Zhang[2], Melissa Dell[3], Benjamin Charles Germain\\nLee[4], Jacob Carlson[3], and Weining Li[5]\\n\\n1 Allen Institute for AI\\n```\\n shannons@allenai.org\\n\\n```\\n2 Brown University\\n```\\n ruochen zhang@brown.edu\\n\\n```\\n3 Harvard University\\n_{melissadell,jacob carlson}@fas.harvard.edu_\\n4 University of Washington\\n```\\n bcgl@cs.washington.edu\\n\\n```\\n5 University of Waterloo\\n```\\n w422li@uwaterloo.ca\\n\\n```\\n\\n**Abstract. Recent advances in document image analysis (DIA) have been**\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of important innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applications. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout detection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digitization pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\n[The library is publicly available at https://layout-parser.github.io.](https://layout-parser.github.io)\\n\\n**Keywords: Document Image Analysis · Deep Learning · Layout Analysis**\\n\\n - Character Recognition · Open Source library · Toolkit.\\n\\n### 1 Introduction\\n\\n\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\n\\n')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: PyMuPDF4LLM\n", + "---" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'file_path': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'format': 'PDF 1.5',\n", - " 'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'trapped': '',\n", - " 'modDate': 'D:20210622012710Z',\n", - " 'creationDate': 'D:20210622012710Z',\n", - " 'page': 0}\n" - ] - } - ], - "source": [ - "import pprint\n", - "\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "6" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PyMuPDF4LLMLoader\n", + "\n", + "This notebook provides a quick overview for getting started with PyMuPDF4LLM [document loader](https://python.langchain.com/docs/concepts/#document-loaders). For detailed documentation of all PyMuPDF4LLMLoader features and configurations head to the [GitHub repository](https://github.com/lakinduboteju/langchain-pymupdf4llm).\n", + "\n", + "## Overview\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support |\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [PyMuPDF4LLMLoader](https://github.com/lakinduboteju/langchain-pymupdf4llm) | [langchain_pymupdf4llm](https://pypi.org/project/langchain-pymupdf4llm) | ✅ | ❌ | ❌ |\n", + "\n", + "### Loader features\n", + "\n", + "| Source | Document Lazy Loading | Native Async Support | Extract Images | Extract Tables |\n", + "| :---: | :---: | :---: | :---: | :---: |\n", + "| PyMuPDF4LLMLoader | ✅ | ❌ | ✅ | ✅ |\n", + "\n", + "## Setup\n", + "\n", + "To access PyMuPDF4LLM document loader you'll need to install the `langchain-pymupdf4llm` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are required to use PyMuPDF4LLMLoader." ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "pages = []\n", - "for doc in loader.lazy_load():\n", - " pages.append(doc)\n", - " if len(pages) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " pages = []\n", - "len(pages)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from IPython.display import Markdown, display\n", - "\n", - "part = pages[0].page_content[778:1189]\n", - "print(part)\n", - "# Markdown rendering\n", - "display(Markdown(part))" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'file_path': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'format': 'PDF 1.5',\n", - " 'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'trapped': '',\n", - " 'modDate': 'D:20210622012710Z',\n", - " 'creationDate': 'D:20210622012710Z',\n", - " 'page': 10}\n" - ] - } - ], - "source": [ - "pprint.pp(pages[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The metadata attribute contains at least the following keys:\n", - "- source\n", - "- page (if in mode *page*)\n", - "- total_page\n", - "- creationdate\n", - "- creator\n", - "- producer\n", - "\n", - "Additional metadata are specific to each parser.\n", - "These pieces of information can be helpful (to categorize your PDFs for example)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Splitting mode & custom pages delimiter" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When loading the PDF file you can split it in two different ways:\n", - "- By page\n", - "- As a single text flow\n", - "\n", - "By default PyMuPDF4LLMLoader will split the PDF by page." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Extract the PDF by page. Each page is extracted as a langchain Document object:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "16\n", - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'file_path': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'format': 'PDF 1.5',\n", - " 'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'trapped': '',\n", - " 'modDate': 'D:20210622012710Z',\n", - " 'creationDate': 'D:20210622012710Z',\n", - " 'page': 0}\n" - ] - } - ], - "source": [ - "loader = PyMuPDF4LLMLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - ")\n", - "docs = loader.load()\n", - "\n", - "print(len(docs))\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this mode the pdf is split by pages and the resulting Documents metadata contains the `page` (page number). But in some cases we could want to process the pdf as a single text flow (so we don't cut some paragraphs in half). In this case you can use the *single* mode :" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Extract the whole PDF as a single langchain Document object:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1\n", - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'file_path': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'format': 'PDF 1.5',\n", - " 'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'trapped': '',\n", - " 'modDate': 'D:20210622012710Z',\n", - " 'creationDate': 'D:20210622012710Z'}\n" - ] - } - ], - "source": [ - "loader = PyMuPDF4LLMLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"single\",\n", - ")\n", - "docs = loader.load()\n", - "\n", - "print(len(docs))\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Logically, in this mode, the `page` (page_number) metadata disappears. Here's how to clearly identify where pages end in the text flow :" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Add a custom *pages_delimiter* to identify where are ends of pages in *single* mode:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "loader = PyMuPDF4LLMLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"single\",\n", - " pages_delimiter=\"\\n-------THIS IS A CUSTOM END OF PAGE-------\\n\\n\",\n", - ")\n", - "docs = loader.load()\n", - "\n", - "part = docs[0].page_content[10663:11317]\n", - "print(part)\n", - "display(Markdown(part))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The default `pages_delimiter` is \\n-----\\n\\n.\n", - "But this could simply be \\n, or \\f to clearly indicate a page change, or \\ for seamless injection in a Markdown viewer without a visual effect." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Extract images from the PDF" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can extract images from your PDFs (in text form) with a choice of three different solutions:\n", - "- rapidOCR (lightweight Optical Character Recognition tool)\n", - "- Tesseract (OCR tool with high precision)\n", - "- Multimodal language model\n", - "\n", - "The result is inserted at the end of text of the page." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Extract images from the PDF with rapidOCR:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU rapidocr-onnxruntime pillow" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders.parsers import RapidOCRBlobParser\n", - "\n", - "loader = PyMuPDF4LLMLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " extract_images=True,\n", - " images_parser=RapidOCRBlobParser(),\n", - ")\n", - "docs = loader.load()\n", - "\n", - "part = docs[5].page_content[1863:]\n", - "print(part)\n", - "display(Markdown(part))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Be careful, RapidOCR is designed to work with Chinese and English, not other languages." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Extract images from the PDF with Tesseract:" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU pytesseract" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders.parsers import TesseractBlobParser\n", - "\n", - "loader = PyMuPDF4LLMLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " extract_images=True,\n", - " images_parser=TesseractBlobParser(),\n", - ")\n", - "docs = loader.load()\n", - "\n", - "print(docs[5].page_content[1863:])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Extract images from the PDF with multimodal model:" - ] - }, - { - "cell_type": "code", - "execution_count": 38, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain_openai" - ] - }, - { - "cell_type": "code", - "execution_count": 39, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "True" + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community** and **langchain-pymupdf4llm**." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain_community langchain-pymupdf4llm" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_pymupdf4llm import PyMuPDF4LLMLoader\n", + "\n", + "file_path = \"./example_data/layout-parser-paper.pdf\"\n", + "loader = PyMuPDF4LLMLoader(file_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'source': './example_data/layout-parser-paper.pdf', 'file_path': './example_data/layout-parser-paper.pdf', 'total_pages': 16, 'format': 'PDF 1.5', 'title': '', 'author': '', 'subject': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'trapped': '', 'modDate': 'D:20210622012710Z', 'creationDate': 'D:20210622012710Z', 'page': 0}, page_content='```\\nLayoutParser: A Unified Toolkit for Deep\\n\\n## Learning Based Document Image Analysis\\n\\n```\\n\\nZejiang Shen[1] (�), Ruochen Zhang[2], Melissa Dell[3], Benjamin Charles Germain\\nLee[4], Jacob Carlson[3], and Weining Li[5]\\n\\n1 Allen Institute for AI\\n```\\n shannons@allenai.org\\n\\n```\\n2 Brown University\\n```\\n ruochen zhang@brown.edu\\n\\n```\\n3 Harvard University\\n_{melissadell,jacob carlson}@fas.harvard.edu_\\n4 University of Washington\\n```\\n bcgl@cs.washington.edu\\n\\n```\\n5 University of Waterloo\\n```\\n w422li@uwaterloo.ca\\n\\n```\\n\\n**Abstract. Recent advances in document image analysis (DIA) have been**\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of important innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applications. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout detection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digitization pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\n[The library is publicly available at https://layout-parser.github.io.](https://layout-parser.github.io)\\n\\n**Keywords: Document Image Analysis · Deep Learning · Layout Analysis**\\n\\n - Character Recognition · Open Source library · Toolkit.\\n\\n### 1 Introduction\\n\\n\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\n\\n')" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'file_path': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'format': 'PDF 1.5',\n", + " 'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'trapped': '',\n", + " 'modDate': 'D:20210622012710Z',\n", + " 'creationDate': 'D:20210622012710Z',\n", + " 'page': 0}\n" + ] + } + ], + "source": [ + "import pprint\n", + "\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "6" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pages = []\n", + "for doc in loader.lazy_load():\n", + " pages.append(doc)\n", + " if len(pages) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " pages = []\n", + "len(pages)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from IPython.display import Markdown, display\n", + "\n", + "part = pages[0].page_content[778:1189]\n", + "print(part)\n", + "# Markdown rendering\n", + "display(Markdown(part))" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'file_path': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'format': 'PDF 1.5',\n", + " 'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'trapped': '',\n", + " 'modDate': 'D:20210622012710Z',\n", + " 'creationDate': 'D:20210622012710Z',\n", + " 'page': 10}\n" + ] + } + ], + "source": [ + "pprint.pp(pages[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The metadata attribute contains at least the following keys:\n", + "- source\n", + "- page (if in mode *page*)\n", + "- total_page\n", + "- creationdate\n", + "- creator\n", + "- producer\n", + "\n", + "Additional metadata are specific to each parser.\n", + "These pieces of information can be helpful (to categorize your PDFs for example)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Splitting mode & custom pages delimiter" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When loading the PDF file you can split it in two different ways:\n", + "- By page\n", + "- As a single text flow\n", + "\n", + "By default PyMuPDF4LLMLoader will split the PDF by page." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Extract the PDF by page. Each page is extracted as a langchain Document object:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "16\n", + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'file_path': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'format': 'PDF 1.5',\n", + " 'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'trapped': '',\n", + " 'modDate': 'D:20210622012710Z',\n", + " 'creationDate': 'D:20210622012710Z',\n", + " 'page': 0}\n" + ] + } + ], + "source": [ + "loader = PyMuPDF4LLMLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + ")\n", + "docs = loader.load()\n", + "\n", + "print(len(docs))\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this mode the pdf is split by pages and the resulting Documents metadata contains the `page` (page number). But in some cases we could want to process the pdf as a single text flow (so we don't cut some paragraphs in half). In this case you can use the *single* mode :" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Extract the whole PDF as a single langchain Document object:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n", + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'file_path': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'format': 'PDF 1.5',\n", + " 'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'trapped': '',\n", + " 'modDate': 'D:20210622012710Z',\n", + " 'creationDate': 'D:20210622012710Z'}\n" + ] + } + ], + "source": [ + "loader = PyMuPDF4LLMLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"single\",\n", + ")\n", + "docs = loader.load()\n", + "\n", + "print(len(docs))\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Logically, in this mode, the `page` (page_number) metadata disappears. Here's how to clearly identify where pages end in the text flow :" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Add a custom *pages_delimiter* to identify where are ends of pages in *single* mode:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "loader = PyMuPDF4LLMLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"single\",\n", + " pages_delimiter=\"\\n-------THIS IS A CUSTOM END OF PAGE-------\\n\\n\",\n", + ")\n", + "docs = loader.load()\n", + "\n", + "part = docs[0].page_content[10663:11317]\n", + "print(part)\n", + "display(Markdown(part))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The default `pages_delimiter` is \\n-----\\n\\n.\n", + "But this could simply be \\n, or \\f to clearly indicate a page change, or \\ for seamless injection in a Markdown viewer without a visual effect." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Extract images from the PDF" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can extract images from your PDFs (in text form) with a choice of three different solutions:\n", + "- rapidOCR (lightweight Optical Character Recognition tool)\n", + "- Tesseract (OCR tool with high precision)\n", + "- Multimodal language model\n", + "\n", + "The result is inserted at the end of text of the page." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Extract images from the PDF with rapidOCR:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU rapidocr-onnxruntime pillow" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders.parsers import RapidOCRBlobParser\n", + "\n", + "loader = PyMuPDF4LLMLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " extract_images=True,\n", + " images_parser=RapidOCRBlobParser(),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "part = docs[5].page_content[1863:]\n", + "print(part)\n", + "display(Markdown(part))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Be careful, RapidOCR is designed to work with Chinese and English, not other languages." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Extract images from the PDF with Tesseract:" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU pytesseract" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders.parsers import TesseractBlobParser\n", + "\n", + "loader = PyMuPDF4LLMLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " extract_images=True,\n", + " images_parser=TesseractBlobParser(),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "print(docs[5].page_content[1863:])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Extract images from the PDF with multimodal model:" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain_openai" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "\n", + "from dotenv import load_dotenv\n", + "\n", + "load_dotenv()" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [], + "source": [ + "from getpass import getpass\n", + "\n", + "if not os.environ.get(\"OPENAI_API_KEY\"):\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass(\"OpenAI API key =\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders.parsers import LLMImageBlobParser\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "loader = PyMuPDF4LLMLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " extract_images=True,\n", + " images_parser=LLMImageBlobParser(\n", + " model=ChatOpenAI(model=\"gpt-4o-mini\", max_tokens=1024)\n", + " ),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "print(docs[5].page_content[1863:])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Extract tables from the PDF\n", + "\n", + "With PyMUPDF4LLM you can extract tables from your PDFs in *markdown* format :" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "loader = PyMuPDF4LLMLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " # \"lines_strict\" is the default strategy and\n", + " # is the most accurate for tables with column and row lines,\n", + " # but may not work well with all documents.\n", + " # \"lines\" is a less strict strategy that may work better with\n", + " # some documents.\n", + " # \"text\" is the least strict strategy and may work better\n", + " # with documents that do not have tables with lines.\n", + " table_strategy=\"lines\",\n", + ")\n", + "docs = loader.load()\n", + "\n", + "part = docs[4].page_content[3210:]\n", + "print(part)\n", + "display(Markdown(part))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Working with Files\n", + "\n", + "Many document loaders involve parsing files. The difference between such loaders usually stems from how the file is parsed, rather than how the file is loaded. For example, you can use `open` to read the binary content of either a PDF or a markdown file, but you need different parsing logic to convert that binary data into text.\n", + "\n", + "As a result, it can be helpful to decouple the parsing logic from the loading logic, which makes it easier to re-use a given parser regardless of how the data was loaded.\n", + "You can use this strategy to analyze different files, with the same parsing parameters." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import FileSystemBlobLoader\n", + "from langchain_community.document_loaders.generic import GenericLoader\n", + "from langchain_pymupdf4llm import PyMuPDF4LLMParser\n", + "\n", + "loader = GenericLoader(\n", + " blob_loader=FileSystemBlobLoader(\n", + " path=\"./example_data/\",\n", + " glob=\"*.pdf\",\n", + " ),\n", + " blob_parser=PyMuPDF4LLMParser(),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "part = docs[0].page_content[:562]\n", + "print(part)\n", + "display(Markdown(part))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all PyMuPDF4LLMLoader features and configurations head to the GitHub repository: https://github.com/lakinduboteju/langchain-pymupdf4llm" ] - }, - "execution_count": 39, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "import os\n", - "\n", - "from dotenv import load_dotenv\n", - "\n", - "load_dotenv()" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.21" + } }, - { - "cell_type": "code", - "execution_count": 40, - "metadata": {}, - "outputs": [], - "source": [ - "from getpass import getpass\n", - "\n", - "if not os.environ.get(\"OPENAI_API_KEY\"):\n", - " os.environ[\"OPENAI_API_KEY\"] = getpass(\"OpenAI API key =\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders.parsers import LLMImageBlobParser\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "loader = PyMuPDF4LLMLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " extract_images=True,\n", - " images_parser=LLMImageBlobParser(\n", - " model=ChatOpenAI(model=\"gpt-4o-mini\", max_tokens=1024)\n", - " ),\n", - ")\n", - "docs = loader.load()\n", - "\n", - "print(docs[5].page_content[1863:])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Extract tables from the PDF\n", - "\n", - "With PyMUPDF4LLM you can extract tables from your PDFs in *markdown* format :" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "loader = PyMuPDF4LLMLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " # \"lines_strict\" is the default strategy and\n", - " # is the most accurate for tables with column and row lines,\n", - " # but may not work well with all documents.\n", - " # \"lines\" is a less strict strategy that may work better with\n", - " # some documents.\n", - " # \"text\" is the least strict strategy and may work better\n", - " # with documents that do not have tables with lines.\n", - " table_strategy=\"lines\",\n", - ")\n", - "docs = loader.load()\n", - "\n", - "part = docs[4].page_content[3210:]\n", - "print(part)\n", - "display(Markdown(part))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Working with Files\n", - "\n", - "Many document loaders involve parsing files. The difference between such loaders usually stems from how the file is parsed, rather than how the file is loaded. For example, you can use `open` to read the binary content of either a PDF or a markdown file, but you need different parsing logic to convert that binary data into text.\n", - "\n", - "As a result, it can be helpful to decouple the parsing logic from the loading logic, which makes it easier to re-use a given parser regardless of how the data was loaded.\n", - "You can use this strategy to analyze different files, with the same parsing parameters." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import FileSystemBlobLoader\n", - "from langchain_community.document_loaders.generic import GenericLoader\n", - "from langchain_pymupdf4llm import PyMuPDF4LLMParser\n", - "\n", - "loader = GenericLoader(\n", - " blob_loader=FileSystemBlobLoader(\n", - " path=\"./example_data/\",\n", - " glob=\"*.pdf\",\n", - " ),\n", - " blob_parser=PyMuPDF4LLMParser(),\n", - ")\n", - "docs = loader.load()\n", - "\n", - "part = docs[0].page_content[:562]\n", - "print(part)\n", - "display(Markdown(part))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all PyMuPDF4LLMLoader features and configurations head to the GitHub repository: https://github.com/lakinduboteju/langchain-pymupdf4llm" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.21" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/document_loaders/pypdfdirectory.ipynb b/docs/docs/integrations/document_loaders/pypdfdirectory.ipynb index 534adda7fda..2c346b42393 100644 --- a/docs/docs/integrations/document_loaders/pypdfdirectory.ipynb +++ b/docs/docs/integrations/document_loaders/pypdfdirectory.ipynb @@ -1,230 +1,228 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# PyPDFDirectoryLoader\n", - "\n", - "This loader loads all PDF files from a specific directory.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "\n", - "| Class | Package | Local | Serializable | JS support|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [PyPDFDirectoryLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFDirectoryLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| PyPDFDirectoryLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are needed for this loader." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:00:08.878423Z", - "start_time": "2025-01-21T08:00:08.876042Z" - } - }, - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ], - "outputs": [], - "execution_count": 1 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community**." - ] - }, - { - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:00:12.003718Z", - "start_time": "2025-01-21T08:00:10.291617Z" - } - }, - "cell_type": "code", - "source": "%pip install -qU langchain_community pypdf pillow", - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "execution_count": 2 - }, - { - "metadata": {}, - "cell_type": "markdown", - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:" - ] - }, - { - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:00:18.512061Z", - "start_time": "2025-01-21T08:00:17.313969Z" - } - }, - "cell_type": "code", - "source": [ - "from langchain_community.document_loaders import PyPDFDirectoryLoader\n", - "\n", - "directory_path = (\n", - " \"../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n", - ")\n", - "loader = PyPDFDirectoryLoader(\"example_data/\")" - ], - "outputs": [], - "execution_count": 3 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:00:23.549752Z", - "start_time": "2025-01-21T08:00:23.129010Z" - } - }, - "source": [ - "docs = loader.load()\n", - "docs[0]" - ], - "outputs": [ - { - "data": { - "text/plain": [ - "Document(metadata={'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'author': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'subject': '', 'title': '', 'trapped': '/False', 'source': 'example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0, 'page_label': '1'}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen1 (\\x00 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\\nLee4, Jacob Carlson3, and Weining Li5\\n1 Allen Institute for AI\\nshannons@allenai.org\\n2 Brown University\\nruochen zhang@brown.edu\\n3 Harvard University\\n{melissadell,jacob carlson}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu\\n5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recent advances in document image analysis (DIA) have been\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im-\\nportant innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica-\\ntions. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout de-\\ntection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti-\\nzation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis\\n· Character Recognition · Open Source library · Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\narXiv:2103.15348v2 [cs.CV] 21 Jun 2021')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PyPDFDirectoryLoader\n", + "\n", + "This loader loads all PDF files from a specific directory.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | Serializable | JS support|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [PyPDFDirectoryLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFDirectoryLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| PyPDFDirectoryLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are needed for this loader." ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "execution_count": 4 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:00:26.612346Z", - "start_time": "2025-01-21T08:00:26.609051Z" - } - }, - "source": [ - "print(docs[0].metadata)" - ], - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'author': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'subject': '', 'title': '', 'trapped': '/False', 'source': 'example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0, 'page_label': '1'}\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:00:08.878423Z", + "start_time": "2025-01-21T08:00:08.876042Z" + } + }, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community**." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:00:12.003718Z", + "start_time": "2025-01-21T08:00:10.291617Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": "%pip install -qU langchain_community pypdf pillow" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:00:18.512061Z", + "start_time": "2025-01-21T08:00:17.313969Z" + } + }, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import PyPDFDirectoryLoader\n", + "\n", + "directory_path = (\n", + " \"../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf\"\n", + ")\n", + "loader = PyPDFDirectoryLoader(\"example_data/\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:00:23.549752Z", + "start_time": "2025-01-21T08:00:23.129010Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'author': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'subject': '', 'title': '', 'trapped': '/False', 'source': 'example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0, 'page_label': '1'}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen1 (\\x00 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\\nLee4, Jacob Carlson3, and Weining Li5\\n1 Allen Institute for AI\\nshannons@allenai.org\\n2 Brown University\\nruochen zhang@brown.edu\\n3 Harvard University\\n{melissadell,jacob carlson}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu\\n5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recent advances in document image analysis (DIA) have been\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im-\\nportant innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica-\\ntions. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout de-\\ntection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti-\\nzation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis\\n· Character Recognition · Open Source library · Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\narXiv:2103.15348v2 [cs.CV] 21 Jun 2021')" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:00:26.612346Z", + "start_time": "2025-01-21T08:00:26.609051Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'author': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'subject': '', 'title': '', 'trapped': '/False', 'source': 'example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0, 'page_label': '1'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:00:30.251598Z", + "start_time": "2025-01-21T08:00:29.972141Z" + } + }, + "outputs": [], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all PyPDFDirectoryLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFDirectoryLoader.html" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": "" } - ], - "execution_count": 5 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:00:30.251598Z", - "start_time": "2025-01-21T08:00:29.972141Z" + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" } - }, - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ], - "outputs": [], - "execution_count": 6 }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all PyPDFDirectoryLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFDirectoryLoader.html" - ] - }, - { - "metadata": {}, - "cell_type": "code", - "outputs": [], - "execution_count": null, - "source": "" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/docs/docs/integrations/document_loaders/pypdfium2.ipynb b/docs/docs/integrations/document_loaders/pypdfium2.ipynb index afcee7e750b..311f807a51e 100644 --- a/docs/docs/integrations/document_loaders/pypdfium2.ipynb +++ b/docs/docs/integrations/document_loaders/pypdfium2.ipynb @@ -1,1122 +1,1122 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# PyPDFium2Loader\n", - "\n", - "This notebook provides a quick overview for getting started with `PyPDF` [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all DocumentLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFium2Loader.html).\n", - "\n", - " \n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [PyPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", - " \n", - "--------- \n", - "\n", - "### Loader features\n", - "\n", - "| Source | Document Lazy Loading | Native Async Support | Extract Images | Extract Tables |\n", - "|:-----------:| :---: | :---: | :---: |:---: |\n", - "| PyPDFLoader | ✅ | ❌ | ✅ | ❌ |\n", - "\n", - " \n", - "\n", - "## Setup\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are required to use `PyPDFLoader`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:15.370257Z", - "start_time": "2025-02-06T07:04:15.367300Z" - } - }, - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ], - "outputs": [], - "execution_count": 1 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community** and **pypdf**." - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:18.630037Z", - "start_time": "2025-02-06T07:04:15.634391Z" - } - }, - "source": "%pip install -qU langchain_community pypdfium2", - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "execution_count": 2 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:" - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:19.594910Z", - "start_time": "2025-02-06T07:04:18.671508Z" - } - }, - "source": [ - "from langchain_community.document_loaders import PyPDFium2Loader\n", - "\n", - "file_path = \"./example_data/layout-parser-paper.pdf\"\n", - "loader = PyPDFium2Loader(file_path)" - ], - "outputs": [], - "execution_count": 3 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:19.717964Z", - "start_time": "2025-02-06T07:04:19.607741Z" - } - }, - "source": [ - "docs = loader.load()\n", - "docs[0]" - ], - "outputs": [ - { - "data": { - "text/plain": [ - "Document(metadata={'title': '', 'author': '', 'subject': '', 'keywords': '', 'creator': 'LaTeX with hyperref', 'producer': 'pdfTeX-1.40.21', 'creationdate': '2021-06-22T01:27:10+00:00', 'moddate': '2021-06-22T01:27:10+00:00', 'source': './example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen\\n1\\n(), Ruochen Zhang\\n2\\n, Melissa Dell\\n3\\n, Benjamin Charles Germain\\nLee\\n4\\n, Jacob Carlson\\n3\\n, and Weining Li\\n5\\n1 Allen Institute for AI\\nshannons@allenai.org 2 Brown University\\nruochen zhang@brown.edu 3 Harvard University\\n{melissadell,jacob carlson\\n}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu 5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recent advances in document image analysis (DIA) have been\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im\\x02portant innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica\\x02tions. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout de\\x02tection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti\\x02zation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: Document Image Analysis· Deep Learning· Layout Analysis\\n· Character Recognition· Open Source library· Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\narXiv:2103.15348v2 [cs.CV] 21 Jun 2021\\n')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PyPDFium2Loader\n", + "\n", + "This notebook provides a quick overview for getting started with `PyPDF` [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all DocumentLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFium2Loader.html).\n", + "\n", + " \n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [PyPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", + " \n", + "--------- \n", + "\n", + "### Loader features\n", + "\n", + "| Source | Document Lazy Loading | Native Async Support | Extract Images | Extract Tables |\n", + "|:-----------:| :---: | :---: | :---: |:---: |\n", + "| PyPDFLoader | ✅ | ❌ | ✅ | ❌ |\n", + "\n", + " \n", + "\n", + "## Setup\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are required to use `PyPDFLoader`." ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "execution_count": 4 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:19.784617Z", - "start_time": "2025-02-06T07:04:19.782020Z" - } - }, - "source": [ - "import pprint\n", - "\n", - "pprint.pp(docs[0].metadata)" - ], - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'producer': 'pdfTeX-1.40.21',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 0}\n" - ] - } - ], - "execution_count": 5 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:22.359295Z", - "start_time": "2025-02-06T07:04:22.143306Z" - } - }, - "source": [ - "pages = []\n", - "for doc in loader.lazy_load():\n", - " pages.append(doc)\n", - " if len(pages) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " pages = []\n", - "len(pages)" - ], - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, { - "data": { - "text/plain": [ - "6" + "cell_type": "code", + "execution_count": 1, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:15.370257Z", + "start_time": "2025-02-06T07:04:15.367300Z" + } + }, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "execution_count": 6 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:23.200681Z", - "start_time": "2025-02-06T07:04:23.189169Z" - } - }, - "source": [ - "print(pages[0].page_content[:100])\n", - "pprint.pp(pages[0].metadata)" - ], - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "LayoutParser: A Unified Toolkit for DL-Based DIA 11\n", - "focuses on precision, efficiency, and robustness\n", - "{'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'producer': 'pdfTeX-1.40.21',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 10}\n" - ] - } - ], - "execution_count": 7 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The metadata attribute contains at least the following keys:\n", - "- source\n", - "- page (if in mode *page*)\n", - "- total_page\n", - "- creationdate\n", - "- creator\n", - "- producer\n", - "\n", - "Additional metadata are specific to each parser.\n", - "These pieces of information can be helpful (to categorize your PDFs for example)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "## Splitting mode & custom pages delimiter" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When loading the PDF file you can split it in two different ways:\n", - "- By page\n", - "- As a single text flow\n", - "\n", - "By default PyPDFLoader will split the PDF as a single text flow." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract the PDF by page. Each page is extracted as a langchain Document object:" - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:27.102894Z", - "start_time": "2025-02-06T07:04:26.941787Z" - } - }, - "source": [ - "loader = PyPDFium2Loader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - ")\n", - "docs = loader.load()\n", - "print(len(docs))\n", - "pprint.pp(docs[0].metadata)" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "16\n", - "{'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'producer': 'pdfTeX-1.40.21',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 0}\n" - ] - } - ], - "execution_count": 8 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "In this mode the pdf is split by pages and the resulting Documents metadata contains the page number. But in some cases we could want to process the pdf as a single text flow (so we don't cut some paragraphs in half). In this case you can use the *single* mode :" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract the whole PDF as a single langchain Document object:" - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:29.714085Z", - "start_time": "2025-02-06T07:04:29.646263Z" - } - }, - "source": [ - "loader = PyPDFium2Loader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"single\",\n", - ")\n", - "docs = loader.load()\n", - "print(len(docs))\n", - "pprint.pp(docs[0].metadata)" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1\n", - "{'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'producer': 'pdfTeX-1.40.21',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16}\n" - ] - } - ], - "execution_count": 9 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "Logically, in this mode, the ‘page_number’ metadata disappears. Here's how to clearly identify where pages end in the text flow :" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Add a custom *pages_delimiter* to identify where are ends of pages in *single* mode:" - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:33.462591Z", - "start_time": "2025-02-06T07:04:33.299846Z" - } - }, - "source": [ - "loader = PyPDFium2Loader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"single\",\n", - " pages_delimiter=\"\\n-------THIS IS A CUSTOM END OF PAGE-------\\n\",\n", - ")\n", - "docs = loader.load()\n", - "print(docs[0].page_content[:5780])" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "LayoutParser: A Unified Toolkit for Deep\n", - "Learning Based Document Image Analysis\n", - "Zejiang Shen\n", - "1\n", - "(), Ruochen Zhang\n", - "2\n", - ", Melissa Dell\n", - "3\n", - ", Benjamin Charles Germain\n", - "Lee\n", - "4\n", - ", Jacob Carlson\n", - "3\n", - ", and Weining Li\n", - "5\n", - "1 Allen Institute for AI\n", - "shannons@allenai.org 2 Brown University\n", - "ruochen zhang@brown.edu 3 Harvard University\n", - "{melissadell,jacob carlson\n", - "}@fas.harvard.edu\n", - "4 University of Washington\n", - "bcgl@cs.washington.edu 5 University of Waterloo\n", - "w422li@uwaterloo.ca\n", - "Abstract. Recent advances in document image analysis (DIA) have been\n", - "primarily driven by the application of neural networks. Ideally, research\n", - "outcomes could be easily deployed in production and extended for further\n", - "investigation. However, various factors like loosely organized codebases\n", - "and sophisticated model configurations complicate the easy reuse of im\u0002portant innovations by a wide audience. Though there have been on-going\n", - "efforts to improve reusability and simplify deep learning (DL) model\n", - "development in disciplines like natural language processing and computer\n", - "vision, none of them are optimized for challenges in the domain of DIA.\n", - "This represents a major gap in the existing toolkit, as DIA is central to\n", - "academic research across a wide range of disciplines in the social sciences\n", - "and humanities. This paper introduces LayoutParser, an open-source\n", - "library for streamlining the usage of DL in DIA research and applica\u0002tions. The core LayoutParser library comes with a set of simple and\n", - "intuitive interfaces for applying and customizing DL models for layout de\u0002tection, character recognition, and many other document processing tasks.\n", - "To promote extensibility, LayoutParser also incorporates a community\n", - "platform for sharing both pre-trained models and full document digiti\u0002zation pipelines. We demonstrate that LayoutParser is helpful for both\n", - "lightweight and large-scale digitization pipelines in real-word use cases.\n", - "The library is publicly available at https://layout-parser.github.io.\n", - "Keywords: Document Image Analysis· Deep Learning· Layout Analysis\n", - "· Character Recognition· Open Source library· Toolkit.\n", - "1 Introduction\n", - "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", - "document image analysis (DIA) tasks including document image classification [11,\n", - "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", - "-------THIS IS A CUSTOM END OF PAGE-------\n", - "2 Z. Shen et al.\n", - "37], layout detection [38, 22], table detection [26], and scene text detection [4].\n", - "A generalized learning-based framework dramatically reduces the need for the\n", - "manual specification of complicated rules, which is the status quo with traditional\n", - "methods. DL has the potential to transform DIA pipelines and benefit a broad\n", - "spectrum of large-scale document digitization projects.\n", - "However, there are several practical difficulties for taking advantages of re\u0002cent advances in DL-based methods: 1) DL models are notoriously convoluted\n", - "for reuse and extension. Existing models are developed using distinct frame\u0002works like TensorFlow [1] or PyTorch [24], and the high-level parameters can\n", - "be obfuscated by implementation details [8]. It can be a time-consuming and\n", - "frustrating experience to debug, reproduce, and adapt existing models for DIA,\n", - "and many researchers who would benefit the most from using these methods lack\n", - "the technical background to implement them from scratch. 2) Document images\n", - "contain diverse and disparate patterns across domains, and customized training\n", - "is often required to achieve a desirable detection accuracy. Currently there is no\n", - "full-fledged infrastructure for easily curating the target document image datasets\n", - "and fine-tuning or re-training the models. 3) DIA usually requires a sequence of\n", - "models and other processing to obtain the final outputs. Often research teams use\n", - "DL models and then perform further document analyses in separate processes,\n", - "and these pipelines are not documented in any central location (and often not\n", - "documented at all). This makes it difficult for research teams to learn about how\n", - "full pipelines are implemented and leads them to invest significant resources in\n", - "reinventing the DIA wheel.\n", - "LayoutParser provides a unified toolkit to support DL-based document image\n", - "analysis and processing. To address the aforementioned challenges, LayoutParser\n", - "is built with the following components:\n", - "1. An off-the-shelf toolkit for applying DL models for layout detection, character\n", - "recognition, and other DIA tasks (Section 3)\n", - "2. A rich repository of pre-trained neural network models (Model Zoo) that\n", - "underlies the off-the-shelf usage\n", - "3. Comprehensive tools for efficient document image data annotation and model\n", - "tuning to support different levels of customization\n", - "4. A DL model hub and community platform for the easy sharing, distribu\u0002tion, and discussion of DIA models and pipelines, to promote reusability,\n", - "reproducibility, and extensibility (Section 4)\n", - "The library implements simple and intuitive Python APIs without sacrificing\n", - "generalizability and versatility, and can be easily installed via pip. Its convenient\n", - "functions for handling document image data can be seamlessly integrated with\n", - "existing DIA pipelines. With detailed documentations and carefully curated\n", - "tutorials, we hope this tool will benefit a variety of end-users, and will lead to\n", - "advances in applications in both industry and academic research.\n", - "LayoutParser is well aligned with recent efforts for improving DL model\n", - "reusability in other disciplines like natural language processing [8, 34] and com\u0002puter vision [35], but with a focus on unique challenges in DIA. We show\n", - "LayoutParser can be applied in sophisticated and large-scale digitization projects\n", - "-------THIS IS A CUSTOM END OF PAGE-------\n", - "LayoutParser: A Unified Toolkit for DL-Based DIA 3\n", - "that require precision, efficiency, and robustness, as well as \n" - ] - } - ], - "execution_count": 10 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "This could simply be \\n, or \\f to clearly indicate a page change, or \\ for seamless injection in a Markdown viewer without a visual effect." - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "# Extract images from the PDF" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can extract images from your PDFs with a choice of three different solutions:\n", - "- rapidOCR (lightweight Optical Character Recognition tool)\n", - "- Tesseract (OCR tool with high precision)\n", - "- Multimodal language model\n", - "\n", - "You can tune these functions to choose the output format of the extracted images among *html*, *markdown* or *text*\n", - "\n", - "The result is inserted between the last and the second-to-last paragraphs of text of the page." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract images from the PDF with rapidOCR:" - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:04:39.419623Z", - "start_time": "2025-02-06T07:04:37.250297Z" - } - }, - "source": [ - "%pip install -qU rapidocr-onnxruntime" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "execution_count": 11 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:05:02.902374Z", - "start_time": "2025-02-06T07:04:39.500569Z" - } - }, - "source": [ - "from langchain_community.document_loaders.parsers import RapidOCRBlobParser\n", - "\n", - "loader = PyPDFium2Loader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " images_inner_format=\"markdown-img\",\n", - " images_parser=RapidOCRBlobParser(),\n", - ")\n", - "docs = loader.load()\n", - "\n", - "print(docs[5].page_content)" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "6 Z. Shen et al.\n", - "Fig. 2: The relationship between the three types of layout data structures.\n", - "Coordinate supports three kinds of variation; TextBlock consists of the co\u0002ordinate information and extra features like block text, types, and reading orders;\n", - "a Layout object is a list of all possible layout elements, including other Layout\n", - "objects. They all support the same set of transformation and operation APIs for\n", - "maximum flexibility.\n", - "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", - "on 5 different datasets. Description of the training dataset is provided alongside\n", - "with the trained models such that users can quickly identify the most suitable\n", - "models for their tasks. Additionally, when such a model is not readily available,\n", - "LayoutParser also supports training customized layout models and community\n", - "sharing of the models (detailed in Section 3.5).\n", - "3.2 Layout Data Structures\n", - "A critical feature of LayoutParser is the implementation of a series of data\n", - "structures and operations that can be used to efficiently process and manipulate\n", - "the layout elements. In document image analysis pipelines, various post-processing\n", - "on the layout analysis model outputs is usually required to obtain the final\n", - "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", - "the results into other pipelines. All model outputs from LayoutParser will be\n", - "stored in carefully engineered data types optimized for further processing, which\n", - "makes it possible to build an end-to-end document digitization pipeline within\n", - "LayoutParser. There are three key components in the data structure, namely\n", - "the Coordinate system, the TextBlock, and the Layout. They provide different\n", - "levels of abstraction for the layout data, and a set of APIs are supported for\n", - "transformations or operations on these classes.\n", - "\n", - "\n", - "\n", - "![Coordinate\n", - "(x1, y1)\n", - "(X1, y1)\n", - "(x2,y2)\n", - "APIS\n", - "x-interval\n", - "tart\n", - "end\n", - "Quadrilateral\n", - "operation\n", - "Rectangle\n", - "y-interval\n", - "ena\n", - "(x2, y2)\n", - "(x4, y4)\n", - "(x3, y3)\n", - "and\n", - "textblock\n", - "Coordinate\n", - "transformation\n", - "+\n", - "Block\n", - "Block\n", - "Reading\n", - "Extra features\n", - "Text\n", - "Type\n", - "Order\n", - "coordinatel\n", - "textblockl\n", - "layout\n", - " same\n", - "textblock2\n", - "layoutl\n", - "The\n", - "A list of the layout elements](#)\n", - "\n" - ] - } - ], - "execution_count": 12 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "Be careful, RapidOCR is designed to work with Chinese and English, not other languages." - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract images from the PDF with Tesseract:" - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:05:07.656993Z", - "start_time": "2025-02-06T07:05:05.890565Z" - } - }, - "source": [ - "%pip install -qU pytesseract" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "execution_count": 13 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:05:16.677336Z", - "start_time": "2025-02-06T07:05:07.724790Z" - } - }, - "source": [ - "from langchain_community.document_loaders.parsers import TesseractBlobParser\n", - "\n", - "loader = PyPDFium2Loader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " images_inner_format=\"html-img\",\n", - " images_parser=TesseractBlobParser(),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[5].page_content)" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "6 Z. Shen et al.\n", - "Fig. 2: The relationship between the three types of layout data structures.\n", - "Coordinate supports three kinds of variation; TextBlock consists of the co\u0002ordinate information and extra features like block text, types, and reading orders;\n", - "a Layout object is a list of all possible layout elements, including other Layout\n", - "objects. They all support the same set of transformation and operation APIs for\n", - "maximum flexibility.\n", - "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", - "on 5 different datasets. Description of the training dataset is provided alongside\n", - "with the trained models such that users can quickly identify the most suitable\n", - "models for their tasks. Additionally, when such a model is not readily available,\n", - "LayoutParser also supports training customized layout models and community\n", - "sharing of the models (detailed in Section 3.5).\n", - "3.2 Layout Data Structures\n", - "A critical feature of LayoutParser is the implementation of a series of data\n", - "structures and operations that can be used to efficiently process and manipulate\n", - "the layout elements. In document image analysis pipelines, various post-processing\n", - "on the layout analysis model outputs is usually required to obtain the final\n", - "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", - "the results into other pipelines. All model outputs from LayoutParser will be\n", - "stored in carefully engineered data types optimized for further processing, which\n", - "makes it possible to build an end-to-end document digitization pipeline within\n", - "LayoutParser. There are three key components in the data structure, namely\n", - "the Coordinate system, the TextBlock, and the Layout. They provide different\n", - "levels of abstraction for the layout data, and a set of APIs are supported for\n", - "transformations or operations on these classes.\n", - "\n", - "\n", - "\n", - "\"Coordinate\n",\n", - "\n" - ] - } - ], - "execution_count": 14 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract images from the PDF with multimodal model:" - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:05:57.591688Z", - "start_time": "2025-02-06T07:05:54.591989Z" - } - }, - "source": [ - "%pip install -qU langchain_openai" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "execution_count": 15 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:05:58.280055Z", - "start_time": "2025-02-06T07:05:58.180689Z" - } - }, - "source": [ - "import os\n", - "\n", - "from dotenv import load_dotenv\n", - "\n", - "load_dotenv()" - ], - "outputs": [ - { - "data": { - "text/plain": [ - "True" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community** and **pypdf**." ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - } - ], - "execution_count": 16 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:05:59.170560Z", - "start_time": "2025-02-06T07:05:59.167117Z" - } - }, - "source": [ - "from getpass import getpass\n", - "\n", - "if not os.environ.get(\"OPENAI_API_KEY\"):\n", - " os.environ[\"OPENAI_API_KEY\"] = getpass(\"OpenAI API key =\")" - ], - "outputs": [], - "execution_count": 17 - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:07:05.416416Z", - "start_time": "2025-02-06T07:06:00.694853Z" - } - }, - "source": [ - "from langchain_community.document_loaders.parsers import LLMImageBlobParser\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "loader = PyPDFium2Loader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " images_inner_format=\"markdown-img\",\n", - " images_parser=LLMImageBlobParser(model=ChatOpenAI(model=\"gpt-4o\", max_tokens=1024)),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[5].page_content)" - ], - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "6 Z. Shen et al.\n", - "Fig. 2: The relationship between the three types of layout data structures.\n", - "Coordinate supports three kinds of variation; TextBlock consists of the co\u0002ordinate information and extra features like block text, types, and reading orders;\n", - "a Layout object is a list of all possible layout elements, including other Layout\n", - "objects. They all support the same set of transformation and operation APIs for\n", - "maximum flexibility.\n", - "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", - "on 5 different datasets. Description of the training dataset is provided alongside\n", - "with the trained models such that users can quickly identify the most suitable\n", - "models for their tasks. Additionally, when such a model is not readily available,\n", - "LayoutParser also supports training customized layout models and community\n", - "sharing of the models (detailed in Section 3.5).\n", - "3.2 Layout Data Structures\n", - "A critical feature of LayoutParser is the implementation of a series of data\n", - "structures and operations that can be used to efficiently process and manipulate\n", - "the layout elements. In document image analysis pipelines, various post-processing\n", - "on the layout analysis model outputs is usually required to obtain the final\n", - "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", - "the results into other pipelines. All model outputs from LayoutParser will be\n", - "stored in carefully engineered data types optimized for further processing, which\n", - "makes it possible to build an end-to-end document digitization pipeline within\n", - "LayoutParser. There are three key components in the data structure, namely\n", - "the Coordinate system, the TextBlock, and the Layout. They provide different\n", - "levels of abstraction for the layout data, and a set of APIs are supported for\n", - "transformations or operations on these classes.\n", - "\n", - "\n", - "\n", - "![**Image Summary**: Diagram showing a data structure for layout elements including coordinates (intervals, rectangles, quadrilaterals) and text blocks with extra features (block text, type, reading order). It illustrates a hierarchy from coordinates to text blocks and a list of layout elements.\n", - "\n", - "**Extracted Text**:\n", - "```\n", - "Coordinate \n", - "Coordinate\n", - "\n", - "x-interval\n", - "x1, y1\n", - "(x1, y1)\n", - "y-interval\n", - "(x2, y2)\n", - "(x2, y2)\n", - "\n", - "Rectangle\n", - "Rectangle\n", - "\n", - "Quadrilateral\n", - "\n", - "textblock\n", - "\n", - "Coordinate\n", - "Coordinate\n", - "\n", - "+ +\n", - "Extra features\n", - "Extra features\n", - "\n", - "Block Block Reading ...\n", - "Block Text\n", - "Block Type\n", - "Reading Order\n", - "...\n", - "\n", - "layout\n", - "\n", - "[coordinate1, textblock1, textblock2, layout1\\\\]\n", - "...\n", - "\n", - "[\\\\]\n", - "A list of the layout elements\n", - "A list of the layout elements\n", - "\n", - "The same transformation and operation APIs\n", - "The same transformation\n", - "and operation APIs\n", - "```](#)\n", - "\n" - ] - } - ], - "execution_count": 18 - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Working with Files\n", - "\n", - "Many document loaders involve parsing files. The difference between such loaders usually stems from how the file is parsed, rather than how the file is loaded. For example, you can use `open` to read the binary content of either a PDF or a markdown file, but you need different parsing logic to convert that binary data into text.\n", - "\n", - "As a result, it can be helpful to decouple the parsing logic from the loading logic, which makes it easier to re-use a given parser regardless of how the data was loaded.\n", - "You can use this strategy to analyze different files, with the same parsing parameters." - ] - }, - { - "cell_type": "code", - "metadata": { - "ExecuteTime": { - "end_time": "2025-02-06T07:07:08.394894Z", - "start_time": "2025-02-06T07:07:08.164047Z" - } - }, - "source": [ - "from langchain_community.document_loaders import FileSystemBlobLoader\n", - "from langchain_community.document_loaders.generic import GenericLoader\n", - "from langchain_community.document_loaders.parsers import PyPDFium2Parser\n", - "\n", - "loader = GenericLoader(\n", - " blob_loader=FileSystemBlobLoader(\n", - " path=\"./example_data/\",\n", - " glob=\"*.pdf\",\n", - " ),\n", - " blob_parser=PyPDFium2Parser(),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[0].page_content)\n", - "pprint.pp(docs[0].metadata)" - ], - "outputs": [ + "cell_type": "code", + "execution_count": 2, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:18.630037Z", + "start_time": "2025-02-06T07:04:15.634391Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": "%pip install -qU langchain_community pypdfium2" + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "LayoutParser: A Unified Toolkit for Deep\n", - "Learning Based Document Image Analysis\n", - "Zejiang Shen\n", - "1\n", - "(), Ruochen Zhang\n", - "2\n", - ", Melissa Dell\n", - "3\n", - ", Benjamin Charles Germain\n", - "Lee\n", - "4\n", - ", Jacob Carlson\n", - "3\n", - ", and Weining Li\n", - "5\n", - "1 Allen Institute for AI\n", - "shannons@allenai.org 2 Brown University\n", - "ruochen zhang@brown.edu 3 Harvard University\n", - "{melissadell,jacob carlson\n", - "}@fas.harvard.edu\n", - "4 University of Washington\n", - "bcgl@cs.washington.edu 5 University of Waterloo\n", - "w422li@uwaterloo.ca\n", - "Abstract. Recent advances in document image analysis (DIA) have been\n", - "primarily driven by the application of neural networks. Ideally, research\n", - "outcomes could be easily deployed in production and extended for further\n", - "investigation. However, various factors like loosely organized codebases\n", - "and sophisticated model configurations complicate the easy reuse of im\u0002portant innovations by a wide audience. Though there have been on-going\n", - "efforts to improve reusability and simplify deep learning (DL) model\n", - "development in disciplines like natural language processing and computer\n", - "vision, none of them are optimized for challenges in the domain of DIA.\n", - "This represents a major gap in the existing toolkit, as DIA is central to\n", - "academic research across a wide range of disciplines in the social sciences\n", - "and humanities. This paper introduces LayoutParser, an open-source\n", - "library for streamlining the usage of DL in DIA research and applica\u0002tions. The core LayoutParser library comes with a set of simple and\n", - "intuitive interfaces for applying and customizing DL models for layout de\u0002tection, character recognition, and many other document processing tasks.\n", - "To promote extensibility, LayoutParser also incorporates a community\n", - "platform for sharing both pre-trained models and full document digiti\u0002zation pipelines. We demonstrate that LayoutParser is helpful for both\n", - "lightweight and large-scale digitization pipelines in real-word use cases.\n", - "The library is publicly available at https://layout-parser.github.io.\n", - "Keywords: Document Image Analysis· Deep Learning· Layout Analysis\n", - "· Character Recognition· Open Source library· Toolkit.\n", - "1 Introduction\n", - "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", - "document image analysis (DIA) tasks including document image classification [11,\n", - "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", - "\n", - "{'title': '',\n", - " 'author': '',\n", - " 'subject': '',\n", - " 'keywords': '',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'producer': 'pdfTeX-1.40.21',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'source': 'example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 0}\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:19.594910Z", + "start_time": "2025-02-06T07:04:18.671508Z" + } + }, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import PyPDFium2Loader\n", + "\n", + "file_path = \"./example_data/layout-parser-paper.pdf\"\n", + "loader = PyPDFium2Loader(file_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:19.717964Z", + "start_time": "2025-02-06T07:04:19.607741Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'title': '', 'author': '', 'subject': '', 'keywords': '', 'creator': 'LaTeX with hyperref', 'producer': 'pdfTeX-1.40.21', 'creationdate': '2021-06-22T01:27:10+00:00', 'moddate': '2021-06-22T01:27:10+00:00', 'source': './example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen\\n1\\n(), Ruochen Zhang\\n2\\n, Melissa Dell\\n3\\n, Benjamin Charles Germain\\nLee\\n4\\n, Jacob Carlson\\n3\\n, and Weining Li\\n5\\n1 Allen Institute for AI\\nshannons@allenai.org 2 Brown University\\nruochen zhang@brown.edu 3 Harvard University\\n{melissadell,jacob carlson\\n}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu 5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recent advances in document image analysis (DIA) have been\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im\\x02portant innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica\\x02tions. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout de\\x02tection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti\\x02zation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: Document Image Analysis· Deep Learning· Layout Analysis\\n· Character Recognition· Open Source library· Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\narXiv:2103.15348v2 [cs.CV] 21 Jun 2021\\n')" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:19.784617Z", + "start_time": "2025-02-06T07:04:19.782020Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'producer': 'pdfTeX-1.40.21',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 0}\n" + ] + } + ], + "source": [ + "import pprint\n", + "\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:22.359295Z", + "start_time": "2025-02-06T07:04:22.143306Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "6" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pages = []\n", + "for doc in loader.lazy_load():\n", + " pages.append(doc)\n", + " if len(pages) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " pages = []\n", + "len(pages)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:23.200681Z", + "start_time": "2025-02-06T07:04:23.189169Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LayoutParser: A Unified Toolkit for DL-Based DIA 11\n", + "focuses on precision, efficiency, and robustness\n", + "{'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'producer': 'pdfTeX-1.40.21',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 10}\n" + ] + } + ], + "source": [ + "print(pages[0].page_content[:100])\n", + "pprint.pp(pages[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The metadata attribute contains at least the following keys:\n", + "- source\n", + "- page (if in mode *page*)\n", + "- total_page\n", + "- creationdate\n", + "- creator\n", + "- producer\n", + "\n", + "Additional metadata are specific to each parser.\n", + "These pieces of information can be helpful (to categorize your PDFs for example)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "## Splitting mode & custom pages delimiter" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When loading the PDF file you can split it in two different ways:\n", + "- By page\n", + "- As a single text flow\n", + "\n", + "By default PyPDFLoader will split the PDF as a single text flow." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract the PDF by page. Each page is extracted as a langchain Document object:" + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:27.102894Z", + "start_time": "2025-02-06T07:04:26.941787Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "16\n", + "{'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'producer': 'pdfTeX-1.40.21',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 0}\n" + ] + } + ], + "source": [ + "loader = PyPDFium2Loader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + ")\n", + "docs = loader.load()\n", + "print(len(docs))\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "In this mode the pdf is split by pages and the resulting Documents metadata contains the page number. But in some cases we could want to process the pdf as a single text flow (so we don't cut some paragraphs in half). In this case you can use the *single* mode :" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract the whole PDF as a single langchain Document object:" + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:29.714085Z", + "start_time": "2025-02-06T07:04:29.646263Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n", + "{'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'producer': 'pdfTeX-1.40.21',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16}\n" + ] + } + ], + "source": [ + "loader = PyPDFium2Loader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"single\",\n", + ")\n", + "docs = loader.load()\n", + "print(len(docs))\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "Logically, in this mode, the ‘page_number’ metadata disappears. Here's how to clearly identify where pages end in the text flow :" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Add a custom *pages_delimiter* to identify where are ends of pages in *single* mode:" + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:33.462591Z", + "start_time": "2025-02-06T07:04:33.299846Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LayoutParser: A Unified Toolkit for Deep\n", + "Learning Based Document Image Analysis\n", + "Zejiang Shen\n", + "1\n", + "(), Ruochen Zhang\n", + "2\n", + ", Melissa Dell\n", + "3\n", + ", Benjamin Charles Germain\n", + "Lee\n", + "4\n", + ", Jacob Carlson\n", + "3\n", + ", and Weining Li\n", + "5\n", + "1 Allen Institute for AI\n", + "shannons@allenai.org 2 Brown University\n", + "ruochen zhang@brown.edu 3 Harvard University\n", + "{melissadell,jacob carlson\n", + "}@fas.harvard.edu\n", + "4 University of Washington\n", + "bcgl@cs.washington.edu 5 University of Waterloo\n", + "w422li@uwaterloo.ca\n", + "Abstract. Recent advances in document image analysis (DIA) have been\n", + "primarily driven by the application of neural networks. Ideally, research\n", + "outcomes could be easily deployed in production and extended for further\n", + "investigation. However, various factors like loosely organized codebases\n", + "and sophisticated model configurations complicate the easy reuse of im\u0002portant innovations by a wide audience. Though there have been on-going\n", + "efforts to improve reusability and simplify deep learning (DL) model\n", + "development in disciplines like natural language processing and computer\n", + "vision, none of them are optimized for challenges in the domain of DIA.\n", + "This represents a major gap in the existing toolkit, as DIA is central to\n", + "academic research across a wide range of disciplines in the social sciences\n", + "and humanities. This paper introduces LayoutParser, an open-source\n", + "library for streamlining the usage of DL in DIA research and applica\u0002tions. The core LayoutParser library comes with a set of simple and\n", + "intuitive interfaces for applying and customizing DL models for layout de\u0002tection, character recognition, and many other document processing tasks.\n", + "To promote extensibility, LayoutParser also incorporates a community\n", + "platform for sharing both pre-trained models and full document digiti\u0002zation pipelines. We demonstrate that LayoutParser is helpful for both\n", + "lightweight and large-scale digitization pipelines in real-word use cases.\n", + "The library is publicly available at https://layout-parser.github.io.\n", + "Keywords: Document Image Analysis· Deep Learning· Layout Analysis\n", + "· Character Recognition· Open Source library· Toolkit.\n", + "1 Introduction\n", + "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", + "document image analysis (DIA) tasks including document image classification [11,\n", + "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", + "-------THIS IS A CUSTOM END OF PAGE-------\n", + "2 Z. Shen et al.\n", + "37], layout detection [38, 22], table detection [26], and scene text detection [4].\n", + "A generalized learning-based framework dramatically reduces the need for the\n", + "manual specification of complicated rules, which is the status quo with traditional\n", + "methods. DL has the potential to transform DIA pipelines and benefit a broad\n", + "spectrum of large-scale document digitization projects.\n", + "However, there are several practical difficulties for taking advantages of re\u0002cent advances in DL-based methods: 1) DL models are notoriously convoluted\n", + "for reuse and extension. Existing models are developed using distinct frame\u0002works like TensorFlow [1] or PyTorch [24], and the high-level parameters can\n", + "be obfuscated by implementation details [8]. It can be a time-consuming and\n", + "frustrating experience to debug, reproduce, and adapt existing models for DIA,\n", + "and many researchers who would benefit the most from using these methods lack\n", + "the technical background to implement them from scratch. 2) Document images\n", + "contain diverse and disparate patterns across domains, and customized training\n", + "is often required to achieve a desirable detection accuracy. Currently there is no\n", + "full-fledged infrastructure for easily curating the target document image datasets\n", + "and fine-tuning or re-training the models. 3) DIA usually requires a sequence of\n", + "models and other processing to obtain the final outputs. Often research teams use\n", + "DL models and then perform further document analyses in separate processes,\n", + "and these pipelines are not documented in any central location (and often not\n", + "documented at all). This makes it difficult for research teams to learn about how\n", + "full pipelines are implemented and leads them to invest significant resources in\n", + "reinventing the DIA wheel.\n", + "LayoutParser provides a unified toolkit to support DL-based document image\n", + "analysis and processing. To address the aforementioned challenges, LayoutParser\n", + "is built with the following components:\n", + "1. An off-the-shelf toolkit for applying DL models for layout detection, character\n", + "recognition, and other DIA tasks (Section 3)\n", + "2. A rich repository of pre-trained neural network models (Model Zoo) that\n", + "underlies the off-the-shelf usage\n", + "3. Comprehensive tools for efficient document image data annotation and model\n", + "tuning to support different levels of customization\n", + "4. A DL model hub and community platform for the easy sharing, distribu\u0002tion, and discussion of DIA models and pipelines, to promote reusability,\n", + "reproducibility, and extensibility (Section 4)\n", + "The library implements simple and intuitive Python APIs without sacrificing\n", + "generalizability and versatility, and can be easily installed via pip. Its convenient\n", + "functions for handling document image data can be seamlessly integrated with\n", + "existing DIA pipelines. With detailed documentations and carefully curated\n", + "tutorials, we hope this tool will benefit a variety of end-users, and will lead to\n", + "advances in applications in both industry and academic research.\n", + "LayoutParser is well aligned with recent efforts for improving DL model\n", + "reusability in other disciplines like natural language processing [8, 34] and com\u0002puter vision [35], but with a focus on unique challenges in DIA. We show\n", + "LayoutParser can be applied in sophisticated and large-scale digitization projects\n", + "-------THIS IS A CUSTOM END OF PAGE-------\n", + "LayoutParser: A Unified Toolkit for DL-Based DIA 3\n", + "that require precision, efficiency, and robustness, as well as \n" + ] + } + ], + "source": [ + "loader = PyPDFium2Loader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"single\",\n", + " pages_delimiter=\"\\n-------THIS IS A CUSTOM END OF PAGE-------\\n\",\n", + ")\n", + "docs = loader.load()\n", + "print(docs[0].page_content[:5780])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "This could simply be \\n, or \\f to clearly indicate a page change, or \\ for seamless injection in a Markdown viewer without a visual effect." + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "# Extract images from the PDF" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can extract images from your PDFs with a choice of three different solutions:\n", + "- rapidOCR (lightweight Optical Character Recognition tool)\n", + "- Tesseract (OCR tool with high precision)\n", + "- Multimodal language model\n", + "\n", + "You can tune these functions to choose the output format of the extracted images among *html*, *markdown* or *text*\n", + "\n", + "The result is inserted between the last and the second-to-last paragraphs of text of the page." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract images from the PDF with rapidOCR:" + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:04:39.419623Z", + "start_time": "2025-02-06T07:04:37.250297Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU rapidocr-onnxruntime" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:05:02.902374Z", + "start_time": "2025-02-06T07:04:39.500569Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "6 Z. Shen et al.\n", + "Fig. 2: The relationship between the three types of layout data structures.\n", + "Coordinate supports three kinds of variation; TextBlock consists of the co\u0002ordinate information and extra features like block text, types, and reading orders;\n", + "a Layout object is a list of all possible layout elements, including other Layout\n", + "objects. They all support the same set of transformation and operation APIs for\n", + "maximum flexibility.\n", + "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", + "on 5 different datasets. Description of the training dataset is provided alongside\n", + "with the trained models such that users can quickly identify the most suitable\n", + "models for their tasks. Additionally, when such a model is not readily available,\n", + "LayoutParser also supports training customized layout models and community\n", + "sharing of the models (detailed in Section 3.5).\n", + "3.2 Layout Data Structures\n", + "A critical feature of LayoutParser is the implementation of a series of data\n", + "structures and operations that can be used to efficiently process and manipulate\n", + "the layout elements. In document image analysis pipelines, various post-processing\n", + "on the layout analysis model outputs is usually required to obtain the final\n", + "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", + "the results into other pipelines. All model outputs from LayoutParser will be\n", + "stored in carefully engineered data types optimized for further processing, which\n", + "makes it possible to build an end-to-end document digitization pipeline within\n", + "LayoutParser. There are three key components in the data structure, namely\n", + "the Coordinate system, the TextBlock, and the Layout. They provide different\n", + "levels of abstraction for the layout data, and a set of APIs are supported for\n", + "transformations or operations on these classes.\n", + "\n", + "\n", + "\n", + "![Coordinate\n", + "(x1, y1)\n", + "(X1, y1)\n", + "(x2,y2)\n", + "APIS\n", + "x-interval\n", + "tart\n", + "end\n", + "Quadrilateral\n", + "operation\n", + "Rectangle\n", + "y-interval\n", + "ena\n", + "(x2, y2)\n", + "(x4, y4)\n", + "(x3, y3)\n", + "and\n", + "textblock\n", + "Coordinate\n", + "transformation\n", + "+\n", + "Block\n", + "Block\n", + "Reading\n", + "Extra features\n", + "Text\n", + "Type\n", + "Order\n", + "coordinatel\n", + "textblockl\n", + "layout\n", + " same\n", + "textblock2\n", + "layoutl\n", + "The\n", + "A list of the layout elements](#)\n", + "\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders.parsers import RapidOCRBlobParser\n", + "\n", + "loader = PyPDFium2Loader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " images_inner_format=\"markdown-img\",\n", + " images_parser=RapidOCRBlobParser(),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "print(docs[5].page_content)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "Be careful, RapidOCR is designed to work with Chinese and English, not other languages." + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract images from the PDF with Tesseract:" + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:05:07.656993Z", + "start_time": "2025-02-06T07:05:05.890565Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU pytesseract" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:05:16.677336Z", + "start_time": "2025-02-06T07:05:07.724790Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "6 Z. Shen et al.\n", + "Fig. 2: The relationship between the three types of layout data structures.\n", + "Coordinate supports three kinds of variation; TextBlock consists of the co\u0002ordinate information and extra features like block text, types, and reading orders;\n", + "a Layout object is a list of all possible layout elements, including other Layout\n", + "objects. They all support the same set of transformation and operation APIs for\n", + "maximum flexibility.\n", + "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", + "on 5 different datasets. Description of the training dataset is provided alongside\n", + "with the trained models such that users can quickly identify the most suitable\n", + "models for their tasks. Additionally, when such a model is not readily available,\n", + "LayoutParser also supports training customized layout models and community\n", + "sharing of the models (detailed in Section 3.5).\n", + "3.2 Layout Data Structures\n", + "A critical feature of LayoutParser is the implementation of a series of data\n", + "structures and operations that can be used to efficiently process and manipulate\n", + "the layout elements. In document image analysis pipelines, various post-processing\n", + "on the layout analysis model outputs is usually required to obtain the final\n", + "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", + "the results into other pipelines. All model outputs from LayoutParser will be\n", + "stored in carefully engineered data types optimized for further processing, which\n", + "makes it possible to build an end-to-end document digitization pipeline within\n", + "LayoutParser. There are three key components in the data structure, namely\n", + "the Coordinate system, the TextBlock, and the Layout. They provide different\n", + "levels of abstraction for the layout data, and a set of APIs are supported for\n", + "transformations or operations on these classes.\n", + "\n", + "\n", + "\n", + "\"Coordinate\n",\n", + "\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders.parsers import TesseractBlobParser\n", + "\n", + "loader = PyPDFium2Loader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " images_inner_format=\"html-img\",\n", + " images_parser=TesseractBlobParser(),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[5].page_content)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract images from the PDF with multimodal model:" + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:05:57.591688Z", + "start_time": "2025-02-06T07:05:54.591989Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain_openai" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:05:58.280055Z", + "start_time": "2025-02-06T07:05:58.180689Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "\n", + "from dotenv import load_dotenv\n", + "\n", + "load_dotenv()" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:05:59.170560Z", + "start_time": "2025-02-06T07:05:59.167117Z" + } + }, + "outputs": [], + "source": [ + "from getpass import getpass\n", + "\n", + "if not os.environ.get(\"OPENAI_API_KEY\"):\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass(\"OpenAI API key =\")" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:07:05.416416Z", + "start_time": "2025-02-06T07:06:00.694853Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "6 Z. Shen et al.\n", + "Fig. 2: The relationship between the three types of layout data structures.\n", + "Coordinate supports three kinds of variation; TextBlock consists of the co\u0002ordinate information and extra features like block text, types, and reading orders;\n", + "a Layout object is a list of all possible layout elements, including other Layout\n", + "objects. They all support the same set of transformation and operation APIs for\n", + "maximum flexibility.\n", + "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", + "on 5 different datasets. Description of the training dataset is provided alongside\n", + "with the trained models such that users can quickly identify the most suitable\n", + "models for their tasks. Additionally, when such a model is not readily available,\n", + "LayoutParser also supports training customized layout models and community\n", + "sharing of the models (detailed in Section 3.5).\n", + "3.2 Layout Data Structures\n", + "A critical feature of LayoutParser is the implementation of a series of data\n", + "structures and operations that can be used to efficiently process and manipulate\n", + "the layout elements. In document image analysis pipelines, various post-processing\n", + "on the layout analysis model outputs is usually required to obtain the final\n", + "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", + "the results into other pipelines. All model outputs from LayoutParser will be\n", + "stored in carefully engineered data types optimized for further processing, which\n", + "makes it possible to build an end-to-end document digitization pipeline within\n", + "LayoutParser. There are three key components in the data structure, namely\n", + "the Coordinate system, the TextBlock, and the Layout. They provide different\n", + "levels of abstraction for the layout data, and a set of APIs are supported for\n", + "transformations or operations on these classes.\n", + "\n", + "\n", + "\n", + "![**Image Summary**: Diagram showing a data structure for layout elements including coordinates (intervals, rectangles, quadrilaterals) and text blocks with extra features (block text, type, reading order). It illustrates a hierarchy from coordinates to text blocks and a list of layout elements.\n", + "\n", + "**Extracted Text**:\n", + "```\n", + "Coordinate \n", + "Coordinate\n", + "\n", + "x-interval\n", + "x1, y1\n", + "(x1, y1)\n", + "y-interval\n", + "(x2, y2)\n", + "(x2, y2)\n", + "\n", + "Rectangle\n", + "Rectangle\n", + "\n", + "Quadrilateral\n", + "\n", + "textblock\n", + "\n", + "Coordinate\n", + "Coordinate\n", + "\n", + "+ +\n", + "Extra features\n", + "Extra features\n", + "\n", + "Block Block Reading ...\n", + "Block Text\n", + "Block Type\n", + "Reading Order\n", + "...\n", + "\n", + "layout\n", + "\n", + "[coordinate1, textblock1, textblock2, layout1\\\\]\n", + "...\n", + "\n", + "[\\\\]\n", + "A list of the layout elements\n", + "A list of the layout elements\n", + "\n", + "The same transformation and operation APIs\n", + "The same transformation\n", + "and operation APIs\n", + "```](#)\n", + "\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders.parsers import LLMImageBlobParser\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "loader = PyPDFium2Loader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " images_inner_format=\"markdown-img\",\n", + " images_parser=LLMImageBlobParser(model=ChatOpenAI(model=\"gpt-4o\", max_tokens=1024)),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[5].page_content)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Working with Files\n", + "\n", + "Many document loaders involve parsing files. The difference between such loaders usually stems from how the file is parsed, rather than how the file is loaded. For example, you can use `open` to read the binary content of either a PDF or a markdown file, but you need different parsing logic to convert that binary data into text.\n", + "\n", + "As a result, it can be helpful to decouple the parsing logic from the loading logic, which makes it easier to re-use a given parser regardless of how the data was loaded.\n", + "You can use this strategy to analyze different files, with the same parsing parameters." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-06T07:07:08.394894Z", + "start_time": "2025-02-06T07:07:08.164047Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LayoutParser: A Unified Toolkit for Deep\n", + "Learning Based Document Image Analysis\n", + "Zejiang Shen\n", + "1\n", + "(), Ruochen Zhang\n", + "2\n", + ", Melissa Dell\n", + "3\n", + ", Benjamin Charles Germain\n", + "Lee\n", + "4\n", + ", Jacob Carlson\n", + "3\n", + ", and Weining Li\n", + "5\n", + "1 Allen Institute for AI\n", + "shannons@allenai.org 2 Brown University\n", + "ruochen zhang@brown.edu 3 Harvard University\n", + "{melissadell,jacob carlson\n", + "}@fas.harvard.edu\n", + "4 University of Washington\n", + "bcgl@cs.washington.edu 5 University of Waterloo\n", + "w422li@uwaterloo.ca\n", + "Abstract. Recent advances in document image analysis (DIA) have been\n", + "primarily driven by the application of neural networks. Ideally, research\n", + "outcomes could be easily deployed in production and extended for further\n", + "investigation. However, various factors like loosely organized codebases\n", + "and sophisticated model configurations complicate the easy reuse of im\u0002portant innovations by a wide audience. Though there have been on-going\n", + "efforts to improve reusability and simplify deep learning (DL) model\n", + "development in disciplines like natural language processing and computer\n", + "vision, none of them are optimized for challenges in the domain of DIA.\n", + "This represents a major gap in the existing toolkit, as DIA is central to\n", + "academic research across a wide range of disciplines in the social sciences\n", + "and humanities. This paper introduces LayoutParser, an open-source\n", + "library for streamlining the usage of DL in DIA research and applica\u0002tions. The core LayoutParser library comes with a set of simple and\n", + "intuitive interfaces for applying and customizing DL models for layout de\u0002tection, character recognition, and many other document processing tasks.\n", + "To promote extensibility, LayoutParser also incorporates a community\n", + "platform for sharing both pre-trained models and full document digiti\u0002zation pipelines. We demonstrate that LayoutParser is helpful for both\n", + "lightweight and large-scale digitization pipelines in real-word use cases.\n", + "The library is publicly available at https://layout-parser.github.io.\n", + "Keywords: Document Image Analysis· Deep Learning· Layout Analysis\n", + "· Character Recognition· Open Source library· Toolkit.\n", + "1 Introduction\n", + "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", + "document image analysis (DIA) tasks including document image classification [11,\n", + "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", + "\n", + "{'title': '',\n", + " 'author': '',\n", + " 'subject': '',\n", + " 'keywords': '',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'producer': 'pdfTeX-1.40.21',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'source': 'example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 0}\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders import FileSystemBlobLoader\n", + "from langchain_community.document_loaders.generic import GenericLoader\n", + "from langchain_community.document_loaders.parsers import PyPDFium2Parser\n", + "\n", + "loader = GenericLoader(\n", + " blob_loader=FileSystemBlobLoader(\n", + " path=\"./example_data/\",\n", + " glob=\"*.pdf\",\n", + " ),\n", + " blob_parser=PyPDFium2Parser(),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[0].page_content)\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "It is possible to work with files from cloud storage." + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import CloudBlobLoader\n", + "from langchain_community.document_loaders.generic import GenericLoader\n", + "\n", + "loader = GenericLoader(\n", + " blob_loader=CloudBlobLoader(\n", + " url=\"s3://mybucket\", # Supports s3://, az://, gs://, file:// schemes.\n", + " glob=\"*.pdf\",\n", + " ),\n", + " blob_parser=PyPDFium2Parser(),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[0].page_content)\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `PyPDFium2Loader` features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFium2Loader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.1" } - ], - "execution_count": 19 }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "It is possible to work with files from cloud storage." - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import CloudBlobLoader\n", - "from langchain_community.document_loaders.generic import GenericLoader\n", - "\n", - "loader = GenericLoader(\n", - " blob_loader=CloudBlobLoader(\n", - " url=\"s3://mybucket\", # Supports s3://, az://, gs://, file:// schemes.\n", - " glob=\"*.pdf\",\n", - " ),\n", - " blob_parser=PyPDFium2Parser(),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[0].page_content)\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `PyPDFium2Loader` features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFium2Loader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.1" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/document_loaders/pypdfloader.ipynb b/docs/docs/integrations/document_loaders/pypdfloader.ipynb index 70b4a52258f..9945df0200b 100644 --- a/docs/docs/integrations/document_loaders/pypdfloader.ipynb +++ b/docs/docs/integrations/document_loaders/pypdfloader.ipynb @@ -1,1121 +1,1121 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# PyPDFLoader\n", - "\n", - "This notebook provides a quick overview for getting started with `PyPDF` [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all DocumentLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html).\n", - "\n", - " \n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | JS support|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [PyPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", - " \n", - "--------- \n", - "\n", - "### Loader features\n", - "\n", - "| Source | Document Lazy Loading | Native Async Support | Extract Images | Extract Tables |\n", - "|:-----------:| :---: | :---: | :---: |:---: |\n", - "| PyPDFLoader | ✅ | ❌ | ✅ | ❌ |\n", - "\n", - " \n", - "\n", - "## Setup\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are required to use `PyPDFLoader`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:08.825630Z", - "start_time": "2025-01-21T08:01:08.823315Z" - } - }, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community** and **pypdf**." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:10.997240Z", - "start_time": "2025-01-21T08:01:09.529997Z" - } - }, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain_community pypdf" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:14.685958Z", - "start_time": "2025-01-21T08:01:13.653438Z" - } - }, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import PyPDFLoader\n", - "\n", - "file_path = \"./example_data/layout-parser-paper.pdf\"\n", - "loader = PyPDFLoader(file_path)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:17.433959Z", - "start_time": "2025-01-21T08:01:17.080724Z" - } - }, - "outputs": [ - { - "data": { - "text/plain": [ - "Document(metadata={'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'author': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'subject': '', 'title': '', 'trapped': '/False', 'source': './example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0, 'page_label': '1'}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen1 (\\x00 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\\nLee4, Jacob Carlson3, and Weining Li5\\n1 Allen Institute for AI\\nshannons@allenai.org\\n2 Brown University\\nruochen zhang@brown.edu\\n3 Harvard University\\n{melissadell,jacob carlson}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu\\n5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recent advances in document image analysis (DIA) have been\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im-\\nportant innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica-\\ntions. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout de-\\ntection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti-\\nzation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis\\n· Character Recognition · Open Source library · Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\narXiv:2103.15348v2 [cs.CV] 21 Jun 2021')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PyPDFLoader\n", + "\n", + "This notebook provides a quick overview for getting started with `PyPDF` [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all DocumentLoader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html).\n", + "\n", + " \n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | JS support|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [PyPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ❌ | \n", + " \n", + "--------- \n", + "\n", + "### Loader features\n", + "\n", + "| Source | Document Lazy Loading | Native Async Support | Extract Images | Extract Tables |\n", + "|:-----------:| :---: | :---: | :---: |:---: |\n", + "| PyPDFLoader | ✅ | ❌ | ✅ | ❌ |\n", + "\n", + " \n", + "\n", + "## Setup\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are required to use `PyPDFLoader`." ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:18.619845Z", - "start_time": "2025-01-21T08:01:18.615643Z" - } - }, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'author': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", - " '2020) kpathsea version 6.3.2',\n", - " 'subject': '',\n", - " 'title': '',\n", - " 'trapped': '/False',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 0,\n", - " 'page_label': '1'}\n" - ] - } - ], - "source": [ - "import pprint\n", - "\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load\n" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:20.450806Z", - "start_time": "2025-01-21T08:01:20.176333Z" - } - }, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, { - "data": { - "text/plain": [ - "6" + "cell_type": "code", + "execution_count": 1, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:08.825630Z", + "start_time": "2025-01-21T08:01:08.823315Z" + } + }, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "pages = []\n", - "for doc in loader.lazy_load():\n", - " pages.append(doc)\n", - " if len(pages) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " pages = []\n", - "len(pages)" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:21.267444Z", - "start_time": "2025-01-21T08:01:21.263726Z" - } - }, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "LayoutParser: A Unified Toolkit for DL-Based DIA 11\n", - "focuses on precision, efficiency, and robustness. T\n", - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'author': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", - " '2020) kpathsea version 6.3.2',\n", - " 'subject': '',\n", - " 'title': '',\n", - " 'trapped': '/False',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 10,\n", - " 'page_label': '11'}\n" - ] - } - ], - "source": [ - "print(pages[0].page_content[:100])\n", - "pprint.pp(pages[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The metadata attribute contains at least the following keys:\n", - "- source\n", - "- page (if in mode *page*)\n", - "- total_page\n", - "- creationdate\n", - "- creator\n", - "- producer\n", - "\n", - "Additional metadata are specific to each parser.\n", - "These pieces of information can be helpful (to categorize your PDFs for example)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "## Splitting mode & custom pages delimiter" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When loading the PDF file you can split it in two different ways:\n", - "- By page\n", - "- As a single text flow\n", - "\n", - "By default PyPDFLoader will split the PDF as a single text flow." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract the PDF by page. Each page is extracted as a langchain Document object:" - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:28.128153Z", - "start_time": "2025-01-21T08:01:27.823798Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "16\n", - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'author': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", - " '2020) kpathsea version 6.3.2',\n", - " 'subject': '',\n", - " 'title': '',\n", - " 'trapped': '/False',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 0,\n", - " 'page_label': '1'}\n" - ] - } - ], - "source": [ - "loader = PyPDFLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - ")\n", - "docs = loader.load()\n", - "print(len(docs))\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "In this mode the pdf is split by pages and the resulting Documents metadata contains the page number. But in some cases we could want to process the pdf as a single text flow (so we don't cut some paragraphs in half). In this case you can use the *single* mode :" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract the whole PDF as a single langchain Document object:" - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:31.794895Z", - "start_time": "2025-01-21T08:01:31.470806Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1\n", - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'author': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", - " '2020) kpathsea version 6.3.2',\n", - " 'subject': '',\n", - " 'title': '',\n", - " 'trapped': '/False',\n", - " 'source': './example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16}\n" - ] - } - ], - "source": [ - "loader = PyPDFLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"single\",\n", - ")\n", - "docs = loader.load()\n", - "print(len(docs))\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "Logically, in this mode, the ‘page_number’ metadata disappears. Here's how to clearly identify where pages end in the text flow :" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Add a custom *pages_delimiter* to identify where are ends of pages in *single* mode:" - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:35.848808Z", - "start_time": "2025-01-21T08:01:35.575903Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "LayoutParser: A Unified Toolkit for Deep\n", - "Learning Based Document Image Analysis\n", - "Zejiang Shen1 (\u0000 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\n", - "Lee4, Jacob Carlson3, and Weining Li5\n", - "1 Allen Institute for AI\n", - "shannons@allenai.org\n", - "2 Brown University\n", - "ruochen zhang@brown.edu\n", - "3 Harvard University\n", - "{melissadell,jacob carlson}@fas.harvard.edu\n", - "4 University of Washington\n", - "bcgl@cs.washington.edu\n", - "5 University of Waterloo\n", - "w422li@uwaterloo.ca\n", - "Abstract. Recent advances in document image analysis (DIA) have been\n", - "primarily driven by the application of neural networks. Ideally, research\n", - "outcomes could be easily deployed in production and extended for further\n", - "investigation. However, various factors like loosely organized codebases\n", - "and sophisticated model configurations complicate the easy reuse of im-\n", - "portant innovations by a wide audience. Though there have been on-going\n", - "efforts to improve reusability and simplify deep learning (DL) model\n", - "development in disciplines like natural language processing and computer\n", - "vision, none of them are optimized for challenges in the domain of DIA.\n", - "This represents a major gap in the existing toolkit, as DIA is central to\n", - "academic research across a wide range of disciplines in the social sciences\n", - "and humanities. This paper introduces LayoutParser, an open-source\n", - "library for streamlining the usage of DL in DIA research and applica-\n", - "tions. The core LayoutParser library comes with a set of simple and\n", - "intuitive interfaces for applying and customizing DL models for layout de-\n", - "tection, character recognition, and many other document processing tasks.\n", - "To promote extensibility, LayoutParser also incorporates a community\n", - "platform for sharing both pre-trained models and full document digiti-\n", - "zation pipelines. We demonstrate that LayoutParser is helpful for both\n", - "lightweight and large-scale digitization pipelines in real-word use cases.\n", - "The library is publicly available at https://layout-parser.github.io.\n", - "Keywords: Document Image Analysis · Deep Learning · Layout Analysis\n", - "· Character Recognition · Open Source library · Toolkit.\n", - "1 Introduction\n", - "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", - "document image analysis (DIA) tasks including document image classification [11,\n", - "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", - "-------THIS IS A CUSTOM END OF PAGE-------\n", - "2 Z. Shen et al.\n", - "37], layout detection [38, 22], table detection [ 26], and scene text detection [ 4].\n", - "A generalized learning-based framework dramatically reduces the need for the\n", - "manual specification of complicated rules, which is the status quo with traditional\n", - "methods. DL has the potential to transform DIA pipelines and benefit a broad\n", - "spectrum of large-scale document digitization projects.\n", - "However, there are several practical difficulties for taking advantages of re-\n", - "cent advances in DL-based methods: 1) DL models are notoriously convoluted\n", - "for reuse and extension. Existing models are developed using distinct frame-\n", - "works like TensorFlow [1] or PyTorch [ 24], and the high-level parameters can\n", - "be obfuscated by implementation details [ 8]. It can be a time-consuming and\n", - "frustrating experience to debug, reproduce, and adapt existing models for DIA,\n", - "and many researchers who would benefit the most from using these methods lack\n", - "the technical background to implement them from scratch. 2) Document images\n", - "contain diverse and disparate patterns across domains, and customized training\n", - "is often required to achieve a desirable detection accuracy. Currently there is no\n", - "full-fledged infrastructure for easily curating the target document image datasets\n", - "and fine-tuning or re-training the models. 3) DIA usually requires a sequence of\n", - "models and other processing to obtain the final outputs. Often research teams use\n", - "DL models and then perform further document analyses in separate processes,\n", - "and these pipelines are not documented in any central location (and often not\n", - "documented at all). This makes it difficult for research teams to learn about how\n", - "full pipelines are implemented and leads them to invest significant resources in\n", - "reinventing the DIA wheel .\n", - "LayoutParser provides a unified toolkit to support DL-based document image\n", - "analysis and processing. To address the aforementioned challenges,LayoutParser\n", - "is built with the following components:\n", - "1. An off-the-shelf toolkit for applying DL models for layout detection, character\n", - "recognition, and other DIA tasks (Section 3)\n", - "2. A rich repository of pre-trained neural network models (Model Zoo) that\n", - "underlies the off-the-shelf usage\n", - "3. Comprehensive tools for efficient document image data annotation and model\n", - "tuning to support different levels of customization\n", - "4. A DL model hub and community platform for the easy sharing, distribu-\n", - "tion, and discussion of DIA models and pipelines, to promote reusability,\n", - "reproducibility, and extensibility (Section 4)\n", - "The library implements simple and intuitive Python APIs without sacrificing\n", - "generalizability and versatility, and can be easily installed via pip. Its convenient\n", - "functions for handling document image data can be seamlessly integrated with\n", - "existing DIA pipelines. With detailed documentations and carefully curated\n", - "tutorials, we hope this tool will benefit a variety of end-users, and will lead to\n", - "advances in applications in both industry and academic research.\n", - "LayoutParser is well aligned with recent efforts for improving DL model\n", - "reusability in other disciplines like natural language processing [ 8, 34] and com-\n", - "puter vision [ 35], but with a focus on unique challenges in DIA. We show\n", - "LayoutParser can be applied in sophisticated and large-scale digitization projects\n", - "-------THIS IS A CUSTOM END OF PAGE-------\n", - "LayoutParser: A Unified Toolkit for DL-Based DIA 3\n", - "that require precision, efficiency, and robustness, as well as simple and light\n" - ] - } - ], - "source": [ - "loader = PyPDFLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"single\",\n", - " pages_delimiter=\"\\n-------THIS IS A CUSTOM END OF PAGE-------\\n\",\n", - ")\n", - "docs = loader.load()\n", - "print(docs[0].page_content[:5780])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "This could simply be \\n, or \\f to clearly indicate a page change, or \\ for seamless injection in a Markdown viewer without a visual effect." - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "# Extract images from the PDF" - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can extract images from your PDFs with a choice of three different solutions:\n", - "- rapidOCR (lightweight Optical Character Recognition tool)\n", - "- Tesseract (OCR tool with high precision)\n", - "- Multimodal language model\n", - "\n", - "You can tune these functions to choose the output format of the extracted images among *html*, *markdown* or *text*\n", - "\n", - "The result is inserted between the last and the second-to-last paragraphs of text of the page." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract images from the PDF with rapidOCR:" - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:01:40.692855Z", - "start_time": "2025-01-21T08:01:39.293791Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU rapidocr-onnxruntime" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:02:07.273962Z", - "start_time": "2025-01-21T08:01:42.848244Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "6 Z. Shen et al.\n", - "Fig. 2: The relationship between the three types of layout data structures.\n", - "Coordinate supports three kinds of variation; TextBlock consists of the co-\n", - "ordinate information and extra features like block text, types, and reading orders;\n", - "a Layout object is a list of all possible layout elements, including other Layout\n", - "objects. They all support the same set of transformation and operation APIs for\n", - "maximum flexibility.\n", - "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", - "on 5 different datasets. Description of the training dataset is provided alongside\n", - "with the trained models such that users can quickly identify the most suitable\n", - "models for their tasks. Additionally, when such a model is not readily available,\n", - "LayoutParser also supports training customized layout models and community\n", - "sharing of the models (detailed in Section 3.5).\n", - "3.2 Layout Data Structures\n", - "A critical feature of LayoutParser is the implementation of a series of data\n", - "structures and operations that can be used to efficiently process and manipulate\n", - "the layout elements. In document image analysis pipelines, various post-processing\n", - "on the layout analysis model outputs is usually required to obtain the final\n", - "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", - "the results into other pipelines. All model outputs from LayoutParser will be\n", - "stored in carefully engineered data types optimized for further processing, which\n", - "makes it possible to build an end-to-end document digitization pipeline within\n", - "LayoutParser. There are three key components in the data structure, namely\n", - "the Coordinate system, the TextBlock, and the Layout. They provide different\n", - "levels of abstraction for the layout data, and a set of APIs are supported for\n", - "transformations or operations on these classes.\n", - "\n", - "\n", - "\n", - "![Coordinate\n", - "(x1, y1)\n", - "(X1, y1)\n", - "(x2,y2)\n", - "APIS\n", - "x-interval\n", - "tart\n", - "end\n", - "Quadrilateral\n", - "operation\n", - "Rectangle\n", - "y-interval\n", - "ena\n", - "(x2, y2)\n", - "(x4, y4)\n", - "(x3, y3)\n", - "and\n", - "textblock\n", - "Coordinate\n", - "transformation\n", - "+\n", - "Block\n", - "Block\n", - "Reading\n", - "Extra features\n", - "Text\n", - "Type\n", - "Order\n", - "coordinatel\n", - "textblockl\n", - "layout\n", - " same\n", - "textblock2\n", - "layoutl\n", - "The\n", - "A list of the layout elements](#)\n" - ] - } - ], - "source": [ - "from langchain_community.document_loaders.parsers import RapidOCRBlobParser\n", - "\n", - "loader = PyPDFLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " images_inner_format=\"markdown-img\",\n", - " images_parser=RapidOCRBlobParser(),\n", - ")\n", - "docs = loader.load()\n", - "\n", - "print(docs[5].page_content)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "Be careful, RapidOCR is designed to work with Chinese and English, not other languages." - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract images from the PDF with Tesseract:" - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:02:12.070378Z", - "start_time": "2025-01-21T08:02:10.696635Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU pytesseract" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:02:21.712219Z", - "start_time": "2025-01-21T08:02:12.081700Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "6 Z. Shen et al.\n", - "Fig. 2: The relationship between the three types of layout data structures.\n", - "Coordinate supports three kinds of variation; TextBlock consists of the co-\n", - "ordinate information and extra features like block text, types, and reading orders;\n", - "a Layout object is a list of all possible layout elements, including other Layout\n", - "objects. They all support the same set of transformation and operation APIs for\n", - "maximum flexibility.\n", - "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", - "on 5 different datasets. Description of the training dataset is provided alongside\n", - "with the trained models such that users can quickly identify the most suitable\n", - "models for their tasks. Additionally, when such a model is not readily available,\n", - "LayoutParser also supports training customized layout models and community\n", - "sharing of the models (detailed in Section 3.5).\n", - "3.2 Layout Data Structures\n", - "A critical feature of LayoutParser is the implementation of a series of data\n", - "structures and operations that can be used to efficiently process and manipulate\n", - "the layout elements. In document image analysis pipelines, various post-processing\n", - "on the layout analysis model outputs is usually required to obtain the final\n", - "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", - "the results into other pipelines. All model outputs from LayoutParser will be\n", - "stored in carefully engineered data types optimized for further processing, which\n", - "makes it possible to build an end-to-end document digitization pipeline within\n", - "LayoutParser. There are three key components in the data structure, namely\n", - "the Coordinate system, the TextBlock, and the Layout. They provide different\n", - "levels of abstraction for the layout data, and a set of APIs are supported for\n", - "transformations or operations on these classes.\n", - "\n", - "\n", - "\n", - "\"Coordinate\n",\n" - ] - } - ], - "source": [ - "from langchain_community.document_loaders.parsers import TesseractBlobParser\n", - "\n", - "loader = PyPDFLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " images_inner_format=\"html-img\",\n", - " images_parser=TesseractBlobParser(),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[5].page_content)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "### Extract images from the PDF with multimodal model:" - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:02:25.912928Z", - "start_time": "2025-01-21T08:02:24.324014Z" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain_openai" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:02:28.494996Z", - "start_time": "2025-01-21T08:02:28.468181Z" - } - }, - "outputs": [ - { - "data": { - "text/plain": [ - "True" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community** and **pypdf**." ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "import os\n", - "\n", - "from dotenv import load_dotenv\n", - "\n", - "load_dotenv()" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:02:29.318093Z", - "start_time": "2025-01-21T08:02:29.314654Z" - } - }, - "outputs": [], - "source": [ - "from getpass import getpass\n", - "\n", - "if not os.environ.get(\"OPENAI_API_KEY\"):\n", - " os.environ[\"OPENAI_API_KEY\"] = getpass(\"OpenAI API key =\")" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:05:00.352337Z", - "start_time": "2025-01-21T08:02:30.723099Z" - } - }, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "6 Z. Shen et al.\n", - "Fig. 2: The relationship between the three types of layout data structures.\n", - "Coordinate supports three kinds of variation; TextBlock consists of the co-\n", - "ordinate information and extra features like block text, types, and reading orders;\n", - "a Layout object is a list of all possible layout elements, including other Layout\n", - "objects. They all support the same set of transformation and operation APIs for\n", - "maximum flexibility.\n", - "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", - "on 5 different datasets. Description of the training dataset is provided alongside\n", - "with the trained models such that users can quickly identify the most suitable\n", - "models for their tasks. Additionally, when such a model is not readily available,\n", - "LayoutParser also supports training customized layout models and community\n", - "sharing of the models (detailed in Section 3.5).\n", - "3.2 Layout Data Structures\n", - "A critical feature of LayoutParser is the implementation of a series of data\n", - "structures and operations that can be used to efficiently process and manipulate\n", - "the layout elements. In document image analysis pipelines, various post-processing\n", - "on the layout analysis model outputs is usually required to obtain the final\n", - "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", - "the results into other pipelines. All model outputs from LayoutParser will be\n", - "stored in carefully engineered data types optimized for further processing, which\n", - "makes it possible to build an end-to-end document digitization pipeline within\n", - "LayoutParser. There are three key components in the data structure, namely\n", - "the Coordinate system, the TextBlock, and the Layout. They provide different\n", - "levels of abstraction for the layout data, and a set of APIs are supported for\n", - "transformations or operations on these classes.\n", - "\n", - "\n", - "\n", - "![**Image Summary:** \n", - "Diagram explaining coordinate systems and layout elements with labels for coordinate intervals, rectangle, quadrilateral, textblock with extra features, and layout list. Includes transformation and operation APIs.\n", - "\n", - "**Extracted Text:** \n", - "Coordinate \n", - "coordinate \n", - "start \n", - "start \n", - "x-interval \n", - "end \n", - "y-interval \n", - "end \n", - "(x1, y1) \n", - "Rectangle \n", - "(x2, y2) \n", - "(x1, y1) \n", - "Quadrilateral \n", - "(x2, y2) \n", - "(x4, y4) \n", - "(x3, y3) \n", - "The same transformation and operation APIs \n", - "textblock \n", - "Coordinate \n", - "Extra features \n", - "Block Text \n", - "Block Type \n", - "Reading Order \n", - "... \n", - "layout \n", - "coordinate1, textblock1, ... \n", - "..., textblock2, layout1 \n", - "A list of the layout elements ](#)\n" - ] - } - ], - "source": [ - "from langchain_community.document_loaders.parsers import LLMImageBlobParser\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "loader = PyPDFLoader(\n", - " \"./example_data/layout-parser-paper.pdf\",\n", - " mode=\"page\",\n", - " images_inner_format=\"markdown-img\",\n", - " images_parser=LLMImageBlobParser(model=ChatOpenAI(model=\"gpt-4o\", max_tokens=1024)),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[5].page_content)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Working with Files\n", - "\n", - "Many document loaders involve parsing files. The difference between such loaders usually stems from how the file is parsed, rather than how the file is loaded. For example, you can use `open` to read the binary content of either a PDF or a markdown file, but you need different parsing logic to convert that binary data into text.\n", - "\n", - "As a result, it can be helpful to decouple the parsing logic from the loading logic, which makes it easier to re-use a given parser regardless of how the data was loaded.\n", - "You can use this strategy to analyze different files, with the same parsing parameters." - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": { - "ExecuteTime": { - "end_time": "2025-01-21T08:05:08.330141Z", - "start_time": "2025-01-21T08:05:07.997956Z" - } - }, - "outputs": [ + "cell_type": "code", + "execution_count": 2, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:10.997240Z", + "start_time": "2025-01-21T08:01:09.529997Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain_community pypdf" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "LayoutParser: A Unified Toolkit for Deep\n", - "Learning Based Document Image Analysis\n", - "Zejiang Shen1 (\u0000 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\n", - "Lee4, Jacob Carlson3, and Weining Li5\n", - "1 Allen Institute for AI\n", - "shannons@allenai.org\n", - "2 Brown University\n", - "ruochen zhang@brown.edu\n", - "3 Harvard University\n", - "{melissadell,jacob carlson}@fas.harvard.edu\n", - "4 University of Washington\n", - "bcgl@cs.washington.edu\n", - "5 University of Waterloo\n", - "w422li@uwaterloo.ca\n", - "Abstract. Recent advances in document image analysis (DIA) have been\n", - "primarily driven by the application of neural networks. Ideally, research\n", - "outcomes could be easily deployed in production and extended for further\n", - "investigation. However, various factors like loosely organized codebases\n", - "and sophisticated model configurations complicate the easy reuse of im-\n", - "portant innovations by a wide audience. Though there have been on-going\n", - "efforts to improve reusability and simplify deep learning (DL) model\n", - "development in disciplines like natural language processing and computer\n", - "vision, none of them are optimized for challenges in the domain of DIA.\n", - "This represents a major gap in the existing toolkit, as DIA is central to\n", - "academic research across a wide range of disciplines in the social sciences\n", - "and humanities. This paper introduces LayoutParser, an open-source\n", - "library for streamlining the usage of DL in DIA research and applica-\n", - "tions. The core LayoutParser library comes with a set of simple and\n", - "intuitive interfaces for applying and customizing DL models for layout de-\n", - "tection, character recognition, and many other document processing tasks.\n", - "To promote extensibility, LayoutParser also incorporates a community\n", - "platform for sharing both pre-trained models and full document digiti-\n", - "zation pipelines. We demonstrate that LayoutParser is helpful for both\n", - "lightweight and large-scale digitization pipelines in real-word use cases.\n", - "The library is publicly available at https://layout-parser.github.io.\n", - "Keywords: Document Image Analysis · Deep Learning · Layout Analysis\n", - "· Character Recognition · Open Source library · Toolkit.\n", - "1 Introduction\n", - "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", - "document image analysis (DIA) tasks including document image classification [11,\n", - "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", - "{'producer': 'pdfTeX-1.40.21',\n", - " 'creator': 'LaTeX with hyperref',\n", - " 'creationdate': '2021-06-22T01:27:10+00:00',\n", - " 'author': '',\n", - " 'keywords': '',\n", - " 'moddate': '2021-06-22T01:27:10+00:00',\n", - " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", - " '2020) kpathsea version 6.3.2',\n", - " 'subject': '',\n", - " 'title': '',\n", - " 'trapped': '/False',\n", - " 'source': 'example_data/layout-parser-paper.pdf',\n", - " 'total_pages': 16,\n", - " 'page': 0,\n", - " 'page_label': '1'}\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:14.685958Z", + "start_time": "2025-01-21T08:01:13.653438Z" + } + }, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import PyPDFLoader\n", + "\n", + "file_path = \"./example_data/layout-parser-paper.pdf\"\n", + "loader = PyPDFLoader(file_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:17.433959Z", + "start_time": "2025-01-21T08:01:17.080724Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'producer': 'pdfTeX-1.40.21', 'creator': 'LaTeX with hyperref', 'creationdate': '2021-06-22T01:27:10+00:00', 'author': '', 'keywords': '', 'moddate': '2021-06-22T01:27:10+00:00', 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) kpathsea version 6.3.2', 'subject': '', 'title': '', 'trapped': '/False', 'source': './example_data/layout-parser-paper.pdf', 'total_pages': 16, 'page': 0, 'page_label': '1'}, page_content='LayoutParser: A Unified Toolkit for Deep\\nLearning Based Document Image Analysis\\nZejiang Shen1 (\\x00 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\\nLee4, Jacob Carlson3, and Weining Li5\\n1 Allen Institute for AI\\nshannons@allenai.org\\n2 Brown University\\nruochen zhang@brown.edu\\n3 Harvard University\\n{melissadell,jacob carlson}@fas.harvard.edu\\n4 University of Washington\\nbcgl@cs.washington.edu\\n5 University of Waterloo\\nw422li@uwaterloo.ca\\nAbstract. Recent advances in document image analysis (DIA) have been\\nprimarily driven by the application of neural networks. Ideally, research\\noutcomes could be easily deployed in production and extended for further\\ninvestigation. However, various factors like loosely organized codebases\\nand sophisticated model configurations complicate the easy reuse of im-\\nportant innovations by a wide audience. Though there have been on-going\\nefforts to improve reusability and simplify deep learning (DL) model\\ndevelopment in disciplines like natural language processing and computer\\nvision, none of them are optimized for challenges in the domain of DIA.\\nThis represents a major gap in the existing toolkit, as DIA is central to\\nacademic research across a wide range of disciplines in the social sciences\\nand humanities. This paper introduces LayoutParser, an open-source\\nlibrary for streamlining the usage of DL in DIA research and applica-\\ntions. The core LayoutParser library comes with a set of simple and\\nintuitive interfaces for applying and customizing DL models for layout de-\\ntection, character recognition, and many other document processing tasks.\\nTo promote extensibility, LayoutParser also incorporates a community\\nplatform for sharing both pre-trained models and full document digiti-\\nzation pipelines. We demonstrate that LayoutParser is helpful for both\\nlightweight and large-scale digitization pipelines in real-word use cases.\\nThe library is publicly available at https://layout-parser.github.io.\\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis\\n· Character Recognition · Open Source library · Toolkit.\\n1 Introduction\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\\ndocument image analysis (DIA) tasks including document image classification [11,\\narXiv:2103.15348v2 [cs.CV] 21 Jun 2021')" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:18.619845Z", + "start_time": "2025-01-21T08:01:18.615643Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'author': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", + " '2020) kpathsea version 6.3.2',\n", + " 'subject': '',\n", + " 'title': '',\n", + " 'trapped': '/False',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 0,\n", + " 'page_label': '1'}\n" + ] + } + ], + "source": [ + "import pprint\n", + "\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:20.450806Z", + "start_time": "2025-01-21T08:01:20.176333Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "6" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pages = []\n", + "for doc in loader.lazy_load():\n", + " pages.append(doc)\n", + " if len(pages) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " pages = []\n", + "len(pages)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:21.267444Z", + "start_time": "2025-01-21T08:01:21.263726Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LayoutParser: A Unified Toolkit for DL-Based DIA 11\n", + "focuses on precision, efficiency, and robustness. T\n", + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'author': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", + " '2020) kpathsea version 6.3.2',\n", + " 'subject': '',\n", + " 'title': '',\n", + " 'trapped': '/False',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 10,\n", + " 'page_label': '11'}\n" + ] + } + ], + "source": [ + "print(pages[0].page_content[:100])\n", + "pprint.pp(pages[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The metadata attribute contains at least the following keys:\n", + "- source\n", + "- page (if in mode *page*)\n", + "- total_page\n", + "- creationdate\n", + "- creator\n", + "- producer\n", + "\n", + "Additional metadata are specific to each parser.\n", + "These pieces of information can be helpful (to categorize your PDFs for example)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "## Splitting mode & custom pages delimiter" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When loading the PDF file you can split it in two different ways:\n", + "- By page\n", + "- As a single text flow\n", + "\n", + "By default PyPDFLoader will split the PDF as a single text flow." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract the PDF by page. Each page is extracted as a langchain Document object:" + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:28.128153Z", + "start_time": "2025-01-21T08:01:27.823798Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "16\n", + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'author': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", + " '2020) kpathsea version 6.3.2',\n", + " 'subject': '',\n", + " 'title': '',\n", + " 'trapped': '/False',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 0,\n", + " 'page_label': '1'}\n" + ] + } + ], + "source": [ + "loader = PyPDFLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + ")\n", + "docs = loader.load()\n", + "print(len(docs))\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "In this mode the pdf is split by pages and the resulting Documents metadata contains the page number. But in some cases we could want to process the pdf as a single text flow (so we don't cut some paragraphs in half). In this case you can use the *single* mode :" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract the whole PDF as a single langchain Document object:" + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:31.794895Z", + "start_time": "2025-01-21T08:01:31.470806Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n", + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'author': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", + " '2020) kpathsea version 6.3.2',\n", + " 'subject': '',\n", + " 'title': '',\n", + " 'trapped': '/False',\n", + " 'source': './example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16}\n" + ] + } + ], + "source": [ + "loader = PyPDFLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"single\",\n", + ")\n", + "docs = loader.load()\n", + "print(len(docs))\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "Logically, in this mode, the ‘page_number’ metadata disappears. Here's how to clearly identify where pages end in the text flow :" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Add a custom *pages_delimiter* to identify where are ends of pages in *single* mode:" + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:35.848808Z", + "start_time": "2025-01-21T08:01:35.575903Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LayoutParser: A Unified Toolkit for Deep\n", + "Learning Based Document Image Analysis\n", + "Zejiang Shen1 (\u0000 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\n", + "Lee4, Jacob Carlson3, and Weining Li5\n", + "1 Allen Institute for AI\n", + "shannons@allenai.org\n", + "2 Brown University\n", + "ruochen zhang@brown.edu\n", + "3 Harvard University\n", + "{melissadell,jacob carlson}@fas.harvard.edu\n", + "4 University of Washington\n", + "bcgl@cs.washington.edu\n", + "5 University of Waterloo\n", + "w422li@uwaterloo.ca\n", + "Abstract. Recent advances in document image analysis (DIA) have been\n", + "primarily driven by the application of neural networks. Ideally, research\n", + "outcomes could be easily deployed in production and extended for further\n", + "investigation. However, various factors like loosely organized codebases\n", + "and sophisticated model configurations complicate the easy reuse of im-\n", + "portant innovations by a wide audience. Though there have been on-going\n", + "efforts to improve reusability and simplify deep learning (DL) model\n", + "development in disciplines like natural language processing and computer\n", + "vision, none of them are optimized for challenges in the domain of DIA.\n", + "This represents a major gap in the existing toolkit, as DIA is central to\n", + "academic research across a wide range of disciplines in the social sciences\n", + "and humanities. This paper introduces LayoutParser, an open-source\n", + "library for streamlining the usage of DL in DIA research and applica-\n", + "tions. The core LayoutParser library comes with a set of simple and\n", + "intuitive interfaces for applying and customizing DL models for layout de-\n", + "tection, character recognition, and many other document processing tasks.\n", + "To promote extensibility, LayoutParser also incorporates a community\n", + "platform for sharing both pre-trained models and full document digiti-\n", + "zation pipelines. We demonstrate that LayoutParser is helpful for both\n", + "lightweight and large-scale digitization pipelines in real-word use cases.\n", + "The library is publicly available at https://layout-parser.github.io.\n", + "Keywords: Document Image Analysis · Deep Learning · Layout Analysis\n", + "· Character Recognition · Open Source library · Toolkit.\n", + "1 Introduction\n", + "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", + "document image analysis (DIA) tasks including document image classification [11,\n", + "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", + "-------THIS IS A CUSTOM END OF PAGE-------\n", + "2 Z. Shen et al.\n", + "37], layout detection [38, 22], table detection [ 26], and scene text detection [ 4].\n", + "A generalized learning-based framework dramatically reduces the need for the\n", + "manual specification of complicated rules, which is the status quo with traditional\n", + "methods. DL has the potential to transform DIA pipelines and benefit a broad\n", + "spectrum of large-scale document digitization projects.\n", + "However, there are several practical difficulties for taking advantages of re-\n", + "cent advances in DL-based methods: 1) DL models are notoriously convoluted\n", + "for reuse and extension. Existing models are developed using distinct frame-\n", + "works like TensorFlow [1] or PyTorch [ 24], and the high-level parameters can\n", + "be obfuscated by implementation details [ 8]. It can be a time-consuming and\n", + "frustrating experience to debug, reproduce, and adapt existing models for DIA,\n", + "and many researchers who would benefit the most from using these methods lack\n", + "the technical background to implement them from scratch. 2) Document images\n", + "contain diverse and disparate patterns across domains, and customized training\n", + "is often required to achieve a desirable detection accuracy. Currently there is no\n", + "full-fledged infrastructure for easily curating the target document image datasets\n", + "and fine-tuning or re-training the models. 3) DIA usually requires a sequence of\n", + "models and other processing to obtain the final outputs. Often research teams use\n", + "DL models and then perform further document analyses in separate processes,\n", + "and these pipelines are not documented in any central location (and often not\n", + "documented at all). This makes it difficult for research teams to learn about how\n", + "full pipelines are implemented and leads them to invest significant resources in\n", + "reinventing the DIA wheel .\n", + "LayoutParser provides a unified toolkit to support DL-based document image\n", + "analysis and processing. To address the aforementioned challenges,LayoutParser\n", + "is built with the following components:\n", + "1. An off-the-shelf toolkit for applying DL models for layout detection, character\n", + "recognition, and other DIA tasks (Section 3)\n", + "2. A rich repository of pre-trained neural network models (Model Zoo) that\n", + "underlies the off-the-shelf usage\n", + "3. Comprehensive tools for efficient document image data annotation and model\n", + "tuning to support different levels of customization\n", + "4. A DL model hub and community platform for the easy sharing, distribu-\n", + "tion, and discussion of DIA models and pipelines, to promote reusability,\n", + "reproducibility, and extensibility (Section 4)\n", + "The library implements simple and intuitive Python APIs without sacrificing\n", + "generalizability and versatility, and can be easily installed via pip. Its convenient\n", + "functions for handling document image data can be seamlessly integrated with\n", + "existing DIA pipelines. With detailed documentations and carefully curated\n", + "tutorials, we hope this tool will benefit a variety of end-users, and will lead to\n", + "advances in applications in both industry and academic research.\n", + "LayoutParser is well aligned with recent efforts for improving DL model\n", + "reusability in other disciplines like natural language processing [ 8, 34] and com-\n", + "puter vision [ 35], but with a focus on unique challenges in DIA. We show\n", + "LayoutParser can be applied in sophisticated and large-scale digitization projects\n", + "-------THIS IS A CUSTOM END OF PAGE-------\n", + "LayoutParser: A Unified Toolkit for DL-Based DIA 3\n", + "that require precision, efficiency, and robustness, as well as simple and light\n" + ] + } + ], + "source": [ + "loader = PyPDFLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"single\",\n", + " pages_delimiter=\"\\n-------THIS IS A CUSTOM END OF PAGE-------\\n\",\n", + ")\n", + "docs = loader.load()\n", + "print(docs[0].page_content[:5780])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "This could simply be \\n, or \\f to clearly indicate a page change, or \\ for seamless injection in a Markdown viewer without a visual effect." + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "# Extract images from the PDF" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can extract images from your PDFs with a choice of three different solutions:\n", + "- rapidOCR (lightweight Optical Character Recognition tool)\n", + "- Tesseract (OCR tool with high precision)\n", + "- Multimodal language model\n", + "\n", + "You can tune these functions to choose the output format of the extracted images among *html*, *markdown* or *text*\n", + "\n", + "The result is inserted between the last and the second-to-last paragraphs of text of the page." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract images from the PDF with rapidOCR:" + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:01:40.692855Z", + "start_time": "2025-01-21T08:01:39.293791Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU rapidocr-onnxruntime" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:02:07.273962Z", + "start_time": "2025-01-21T08:01:42.848244Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "6 Z. Shen et al.\n", + "Fig. 2: The relationship between the three types of layout data structures.\n", + "Coordinate supports three kinds of variation; TextBlock consists of the co-\n", + "ordinate information and extra features like block text, types, and reading orders;\n", + "a Layout object is a list of all possible layout elements, including other Layout\n", + "objects. They all support the same set of transformation and operation APIs for\n", + "maximum flexibility.\n", + "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", + "on 5 different datasets. Description of the training dataset is provided alongside\n", + "with the trained models such that users can quickly identify the most suitable\n", + "models for their tasks. Additionally, when such a model is not readily available,\n", + "LayoutParser also supports training customized layout models and community\n", + "sharing of the models (detailed in Section 3.5).\n", + "3.2 Layout Data Structures\n", + "A critical feature of LayoutParser is the implementation of a series of data\n", + "structures and operations that can be used to efficiently process and manipulate\n", + "the layout elements. In document image analysis pipelines, various post-processing\n", + "on the layout analysis model outputs is usually required to obtain the final\n", + "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", + "the results into other pipelines. All model outputs from LayoutParser will be\n", + "stored in carefully engineered data types optimized for further processing, which\n", + "makes it possible to build an end-to-end document digitization pipeline within\n", + "LayoutParser. There are three key components in the data structure, namely\n", + "the Coordinate system, the TextBlock, and the Layout. They provide different\n", + "levels of abstraction for the layout data, and a set of APIs are supported for\n", + "transformations or operations on these classes.\n", + "\n", + "\n", + "\n", + "![Coordinate\n", + "(x1, y1)\n", + "(X1, y1)\n", + "(x2,y2)\n", + "APIS\n", + "x-interval\n", + "tart\n", + "end\n", + "Quadrilateral\n", + "operation\n", + "Rectangle\n", + "y-interval\n", + "ena\n", + "(x2, y2)\n", + "(x4, y4)\n", + "(x3, y3)\n", + "and\n", + "textblock\n", + "Coordinate\n", + "transformation\n", + "+\n", + "Block\n", + "Block\n", + "Reading\n", + "Extra features\n", + "Text\n", + "Type\n", + "Order\n", + "coordinatel\n", + "textblockl\n", + "layout\n", + " same\n", + "textblock2\n", + "layoutl\n", + "The\n", + "A list of the layout elements](#)\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders.parsers import RapidOCRBlobParser\n", + "\n", + "loader = PyPDFLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " images_inner_format=\"markdown-img\",\n", + " images_parser=RapidOCRBlobParser(),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "print(docs[5].page_content)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "Be careful, RapidOCR is designed to work with Chinese and English, not other languages." + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract images from the PDF with Tesseract:" + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:02:12.070378Z", + "start_time": "2025-01-21T08:02:10.696635Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU pytesseract" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:02:21.712219Z", + "start_time": "2025-01-21T08:02:12.081700Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "6 Z. Shen et al.\n", + "Fig. 2: The relationship between the three types of layout data structures.\n", + "Coordinate supports three kinds of variation; TextBlock consists of the co-\n", + "ordinate information and extra features like block text, types, and reading orders;\n", + "a Layout object is a list of all possible layout elements, including other Layout\n", + "objects. They all support the same set of transformation and operation APIs for\n", + "maximum flexibility.\n", + "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", + "on 5 different datasets. Description of the training dataset is provided alongside\n", + "with the trained models such that users can quickly identify the most suitable\n", + "models for their tasks. Additionally, when such a model is not readily available,\n", + "LayoutParser also supports training customized layout models and community\n", + "sharing of the models (detailed in Section 3.5).\n", + "3.2 Layout Data Structures\n", + "A critical feature of LayoutParser is the implementation of a series of data\n", + "structures and operations that can be used to efficiently process and manipulate\n", + "the layout elements. In document image analysis pipelines, various post-processing\n", + "on the layout analysis model outputs is usually required to obtain the final\n", + "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", + "the results into other pipelines. All model outputs from LayoutParser will be\n", + "stored in carefully engineered data types optimized for further processing, which\n", + "makes it possible to build an end-to-end document digitization pipeline within\n", + "LayoutParser. There are three key components in the data structure, namely\n", + "the Coordinate system, the TextBlock, and the Layout. They provide different\n", + "levels of abstraction for the layout data, and a set of APIs are supported for\n", + "transformations or operations on these classes.\n", + "\n", + "\n", + "\n", + "\"Coordinate\n",\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders.parsers import TesseractBlobParser\n", + "\n", + "loader = PyPDFLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " images_inner_format=\"html-img\",\n", + " images_parser=TesseractBlobParser(),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[5].page_content)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "### Extract images from the PDF with multimodal model:" + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:02:25.912928Z", + "start_time": "2025-01-21T08:02:24.324014Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain_openai" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:02:28.494996Z", + "start_time": "2025-01-21T08:02:28.468181Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "\n", + "from dotenv import load_dotenv\n", + "\n", + "load_dotenv()" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:02:29.318093Z", + "start_time": "2025-01-21T08:02:29.314654Z" + } + }, + "outputs": [], + "source": [ + "from getpass import getpass\n", + "\n", + "if not os.environ.get(\"OPENAI_API_KEY\"):\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass(\"OpenAI API key =\")" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:05:00.352337Z", + "start_time": "2025-01-21T08:02:30.723099Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "6 Z. Shen et al.\n", + "Fig. 2: The relationship between the three types of layout data structures.\n", + "Coordinate supports three kinds of variation; TextBlock consists of the co-\n", + "ordinate information and extra features like block text, types, and reading orders;\n", + "a Layout object is a list of all possible layout elements, including other Layout\n", + "objects. They all support the same set of transformation and operation APIs for\n", + "maximum flexibility.\n", + "Shown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\n", + "on 5 different datasets. Description of the training dataset is provided alongside\n", + "with the trained models such that users can quickly identify the most suitable\n", + "models for their tasks. Additionally, when such a model is not readily available,\n", + "LayoutParser also supports training customized layout models and community\n", + "sharing of the models (detailed in Section 3.5).\n", + "3.2 Layout Data Structures\n", + "A critical feature of LayoutParser is the implementation of a series of data\n", + "structures and operations that can be used to efficiently process and manipulate\n", + "the layout elements. In document image analysis pipelines, various post-processing\n", + "on the layout analysis model outputs is usually required to obtain the final\n", + "outputs. Traditionally, this requires exporting DL model outputs and then loading\n", + "the results into other pipelines. All model outputs from LayoutParser will be\n", + "stored in carefully engineered data types optimized for further processing, which\n", + "makes it possible to build an end-to-end document digitization pipeline within\n", + "LayoutParser. There are three key components in the data structure, namely\n", + "the Coordinate system, the TextBlock, and the Layout. They provide different\n", + "levels of abstraction for the layout data, and a set of APIs are supported for\n", + "transformations or operations on these classes.\n", + "\n", + "\n", + "\n", + "![**Image Summary:** \n", + "Diagram explaining coordinate systems and layout elements with labels for coordinate intervals, rectangle, quadrilateral, textblock with extra features, and layout list. Includes transformation and operation APIs.\n", + "\n", + "**Extracted Text:** \n", + "Coordinate \n", + "coordinate \n", + "start \n", + "start \n", + "x-interval \n", + "end \n", + "y-interval \n", + "end \n", + "(x1, y1) \n", + "Rectangle \n", + "(x2, y2) \n", + "(x1, y1) \n", + "Quadrilateral \n", + "(x2, y2) \n", + "(x4, y4) \n", + "(x3, y3) \n", + "The same transformation and operation APIs \n", + "textblock \n", + "Coordinate \n", + "Extra features \n", + "Block Text \n", + "Block Type \n", + "Reading Order \n", + "... \n", + "layout \n", + "coordinate1, textblock1, ... \n", + "..., textblock2, layout1 \n", + "A list of the layout elements ](#)\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders.parsers import LLMImageBlobParser\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "loader = PyPDFLoader(\n", + " \"./example_data/layout-parser-paper.pdf\",\n", + " mode=\"page\",\n", + " images_inner_format=\"markdown-img\",\n", + " images_parser=LLMImageBlobParser(model=ChatOpenAI(model=\"gpt-4o\", max_tokens=1024)),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[5].page_content)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Working with Files\n", + "\n", + "Many document loaders involve parsing files. The difference between such loaders usually stems from how the file is parsed, rather than how the file is loaded. For example, you can use `open` to read the binary content of either a PDF or a markdown file, but you need different parsing logic to convert that binary data into text.\n", + "\n", + "As a result, it can be helpful to decouple the parsing logic from the loading logic, which makes it easier to re-use a given parser regardless of how the data was loaded.\n", + "You can use this strategy to analyze different files, with the same parsing parameters." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "ExecuteTime": { + "end_time": "2025-01-21T08:05:08.330141Z", + "start_time": "2025-01-21T08:05:07.997956Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LayoutParser: A Unified Toolkit for Deep\n", + "Learning Based Document Image Analysis\n", + "Zejiang Shen1 (\u0000 ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\n", + "Lee4, Jacob Carlson3, and Weining Li5\n", + "1 Allen Institute for AI\n", + "shannons@allenai.org\n", + "2 Brown University\n", + "ruochen zhang@brown.edu\n", + "3 Harvard University\n", + "{melissadell,jacob carlson}@fas.harvard.edu\n", + "4 University of Washington\n", + "bcgl@cs.washington.edu\n", + "5 University of Waterloo\n", + "w422li@uwaterloo.ca\n", + "Abstract. Recent advances in document image analysis (DIA) have been\n", + "primarily driven by the application of neural networks. Ideally, research\n", + "outcomes could be easily deployed in production and extended for further\n", + "investigation. However, various factors like loosely organized codebases\n", + "and sophisticated model configurations complicate the easy reuse of im-\n", + "portant innovations by a wide audience. Though there have been on-going\n", + "efforts to improve reusability and simplify deep learning (DL) model\n", + "development in disciplines like natural language processing and computer\n", + "vision, none of them are optimized for challenges in the domain of DIA.\n", + "This represents a major gap in the existing toolkit, as DIA is central to\n", + "academic research across a wide range of disciplines in the social sciences\n", + "and humanities. This paper introduces LayoutParser, an open-source\n", + "library for streamlining the usage of DL in DIA research and applica-\n", + "tions. The core LayoutParser library comes with a set of simple and\n", + "intuitive interfaces for applying and customizing DL models for layout de-\n", + "tection, character recognition, and many other document processing tasks.\n", + "To promote extensibility, LayoutParser also incorporates a community\n", + "platform for sharing both pre-trained models and full document digiti-\n", + "zation pipelines. We demonstrate that LayoutParser is helpful for both\n", + "lightweight and large-scale digitization pipelines in real-word use cases.\n", + "The library is publicly available at https://layout-parser.github.io.\n", + "Keywords: Document Image Analysis · Deep Learning · Layout Analysis\n", + "· Character Recognition · Open Source library · Toolkit.\n", + "1 Introduction\n", + "Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n", + "document image analysis (DIA) tasks including document image classification [11,\n", + "arXiv:2103.15348v2 [cs.CV] 21 Jun 2021\n", + "{'producer': 'pdfTeX-1.40.21',\n", + " 'creator': 'LaTeX with hyperref',\n", + " 'creationdate': '2021-06-22T01:27:10+00:00',\n", + " 'author': '',\n", + " 'keywords': '',\n", + " 'moddate': '2021-06-22T01:27:10+00:00',\n", + " 'ptex.fullbanner': 'This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live '\n", + " '2020) kpathsea version 6.3.2',\n", + " 'subject': '',\n", + " 'title': '',\n", + " 'trapped': '/False',\n", + " 'source': 'example_data/layout-parser-paper.pdf',\n", + " 'total_pages': 16,\n", + " 'page': 0,\n", + " 'page_label': '1'}\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders import FileSystemBlobLoader\n", + "from langchain_community.document_loaders.generic import GenericLoader\n", + "from langchain_community.document_loaders.parsers import PyPDFParser\n", + "\n", + "loader = GenericLoader(\n", + " blob_loader=FileSystemBlobLoader(\n", + " path=\"./example_data/\",\n", + " glob=\"*.pdf\",\n", + " ),\n", + " blob_parser=PyPDFParser(),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[0].page_content)\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "It is possible to work with files from cloud storage." + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import CloudBlobLoader\n", + "from langchain_community.document_loaders.generic import GenericLoader\n", + "\n", + "loader = GenericLoader(\n", + " blob_loader=CloudBlobLoader(\n", + " url=\"s3://mybucket\", # Supports s3://, az://, gs://, file:// schemes.\n", + " glob=\"*.pdf\",\n", + " ),\n", + " blob_parser=PyPDFParser(),\n", + ")\n", + "docs = loader.load()\n", + "print(docs[0].page_content)\n", + "pprint.pp(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `PyPDFLoader` features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.1" } - ], - "source": [ - "from langchain_community.document_loaders import FileSystemBlobLoader\n", - "from langchain_community.document_loaders.generic import GenericLoader\n", - "from langchain_community.document_loaders.parsers import PyPDFParser\n", - "\n", - "loader = GenericLoader(\n", - " blob_loader=FileSystemBlobLoader(\n", - " path=\"./example_data/\",\n", - " glob=\"*.pdf\",\n", - " ),\n", - " blob_parser=PyPDFParser(),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[0].page_content)\n", - "pprint.pp(docs[0].metadata)" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "It is possible to work with files from cloud storage." - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import CloudBlobLoader\n", - "from langchain_community.document_loaders.generic import GenericLoader\n", - "\n", - "loader = GenericLoader(\n", - " blob_loader=CloudBlobLoader(\n", - " url=\"s3://mybucket\", # Supports s3://, az://, gs://, file:// schemes.\n", - " glob=\"*.pdf\",\n", - " ),\n", - " blob_parser=PyPDFParser(),\n", - ")\n", - "docs = loader.load()\n", - "print(docs[0].page_content)\n", - "pprint.pp(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `PyPDFLoader` features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.PyPDFLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.1" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/document_loaders/sitemap.ipynb b/docs/docs/integrations/document_loaders/sitemap.ipynb index ed6c74d406b..3fced297125 100644 --- a/docs/docs/integrations/document_loaders/sitemap.ipynb +++ b/docs/docs/integrations/document_loaders/sitemap.ipynb @@ -1,380 +1,378 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Sitemap\n", - "\n", - "Extends from the `WebBaseLoader`, `SitemapLoader` loads a sitemap from a given URL, and then scrapes and loads all pages in the sitemap, returning each page as a Document.\n", - "\n", - "The scraping is done concurrently. There are reasonable limits to concurrent requests, defaulting to 2 per second. If you aren't concerned about being a good citizen, or you control the scrapped server, or don't care about load you can increase this limit. Note, while this will speed up the scraping process, it may cause the server to block you. Be careful!\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/web_loaders/sitemap/)|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [SiteMapLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.sitemap.SitemapLoader.html#langchain_community.document_loaders.sitemap.SitemapLoader) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| SiteMapLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access SiteMap document loader you'll need to install the `langchain-community` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are needed to run this." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community**." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-community" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Fix notebook asyncio bug" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "import nest_asyncio\n", - "\n", - "nest_asyncio.apply()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders.sitemap import SitemapLoader" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sitemap_loader = SitemapLoader(web_path=\"https://api.python.langchain.com/sitemap.xml\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stderr", - "output_type": "stream", - "text": [ - "Fetching pages: 100%|##########| 28/28 [00:04<00:00, 6.83it/s]\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Sitemap\n", + "\n", + "Extends from the `WebBaseLoader`, `SitemapLoader` loads a sitemap from a given URL, and then scrapes and loads all pages in the sitemap, returning each page as a Document.\n", + "\n", + "The scraping is done concurrently. There are reasonable limits to concurrent requests, defaulting to 2 per second. If you aren't concerned about being a good citizen, or you control the scrapped server, or don't care about load you can increase this limit. Note, while this will speed up the scraping process, it may cause the server to block you. Be careful!\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/web_loaders/sitemap/)|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [SiteMapLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.sitemap.SitemapLoader.html#langchain_community.document_loaders.sitemap.SitemapLoader) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| SiteMapLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "To access SiteMap document loader you'll need to install the `langchain-community` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are needed to run this." + ] }, { - "data": { - "text/plain": [ - "Document(metadata={'source': 'https://api.python.langchain.com/en/stable/', 'loc': 'https://api.python.langchain.com/en/stable/', 'lastmod': '2024-05-15T00:29:42.163001+00:00', 'changefreq': 'weekly', 'priority': '1'}, page_content='\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nLangChain Python API Reference Documentation.\\n\\n\\nYou will be automatically redirected to the new location of this page.\\n\\n')" + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = sitemap_loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': 'https://api.python.langchain.com/en/stable/', 'loc': 'https://api.python.langchain.com/en/stable/', 'lastmod': '2024-05-15T00:29:42.163001+00:00', 'changefreq': 'weekly', 'priority': '1'}\n" - ] - } - ], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can change the `requests_per_second` parameter to increase the max concurrent requests. and use `requests_kwargs` to pass kwargs when send requests." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sitemap_loader.requests_per_second = 2\n", - "# Optional: avoid `[SSL: CERTIFICATE_VERIFY_FAILED]` issue\n", - "sitemap_loader.requests_kwargs = {\"verify\": False}" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load\n", - "\n", - "You can also load the pages lazily in order to minimize the memory load." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Fetching pages: 100%|##########| 28/28 [00:01<00:00, 19.06it/s]\n" - ] - } - ], - "source": [ - "page = []\n", - "for doc in sitemap_loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Filtering sitemap URLs\n", - "\n", - "Sitemaps can be massive files, with thousands of URLs. Often you don't need every single one of them. You can filter the URLs by passing a list of strings or regex patterns to the `filter_urls` parameter. Only URLs that match one of the patterns will be loaded." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "loader = SitemapLoader(\n", - " web_path=\"https://api.python.langchain.com/sitemap.xml\",\n", - " filter_urls=[\"https://api.python.langchain.com/en/latest\"],\n", - ")\n", - "documents = loader.load()" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "data": { - "text/plain": [ - "Document(page_content='\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nLangChain Python API Reference Documentation.\\n\\n\\nYou will be automatically redirected to the new location of this page.\\n\\n', metadata={'source': 'https://api.python.langchain.com/en/latest/', 'loc': 'https://api.python.langchain.com/en/latest/', 'lastmod': '2024-02-12T05:26:10.971077+00:00', 'changefreq': 'daily', 'priority': '0.9'})" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community**." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-community" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Fix notebook asyncio bug" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "import nest_asyncio\n", + "\n", + "nest_asyncio.apply()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders.sitemap import SitemapLoader" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "sitemap_loader = SitemapLoader(web_path=\"https://api.python.langchain.com/sitemap.xml\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Fetching pages: 100%|##########| 28/28 [00:04<00:00, 6.83it/s]\n" + ] + }, + { + "data": { + "text/plain": [ + "Document(metadata={'source': 'https://api.python.langchain.com/en/stable/', 'loc': 'https://api.python.langchain.com/en/stable/', 'lastmod': '2024-05-15T00:29:42.163001+00:00', 'changefreq': 'weekly', 'priority': '1'}, page_content='\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nLangChain Python API Reference Documentation.\\n\\n\\nYou will be automatically redirected to the new location of this page.\\n\\n')" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = sitemap_loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': 'https://api.python.langchain.com/en/stable/', 'loc': 'https://api.python.langchain.com/en/stable/', 'lastmod': '2024-05-15T00:29:42.163001+00:00', 'changefreq': 'weekly', 'priority': '1'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can change the `requests_per_second` parameter to increase the max concurrent requests. and use `requests_kwargs` to pass kwargs when send requests." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "sitemap_loader.requests_per_second = 2\n", + "# Optional: avoid `[SSL: CERTIFICATE_VERIFY_FAILED]` issue\n", + "sitemap_loader.requests_kwargs = {\"verify\": False}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load\n", + "\n", + "You can also load the pages lazily in order to minimize the memory load." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Fetching pages: 100%|##########| 28/28 [00:01<00:00, 19.06it/s]\n" + ] + } + ], + "source": [ + "page = []\n", + "for doc in sitemap_loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Filtering sitemap URLs\n", + "\n", + "Sitemaps can be massive files, with thousands of URLs. Often you don't need every single one of them. You can filter the URLs by passing a list of strings or regex patterns to the `filter_urls` parameter. Only URLs that match one of the patterns will be loaded." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "loader = SitemapLoader(\n", + " web_path=\"https://api.python.langchain.com/sitemap.xml\",\n", + " filter_urls=[\"https://api.python.langchain.com/en/latest\"],\n", + ")\n", + "documents = loader.load()" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(page_content='\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nLangChain Python API Reference Documentation.\\n\\n\\nYou will be automatically redirected to the new location of this page.\\n\\n', metadata={'source': 'https://api.python.langchain.com/en/latest/', 'loc': 'https://api.python.langchain.com/en/latest/', 'lastmod': '2024-02-12T05:26:10.971077+00:00', 'changefreq': 'daily', 'priority': '0.9'})" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "documents[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Add custom scraping rules\n", + "\n", + "The `SitemapLoader` uses `beautifulsoup4` for the scraping process, and it scrapes every element on the page by default. The `SitemapLoader` constructor accepts a custom scraping function. This feature can be helpful to tailor the scraping process to your specific needs; for example, you might want to avoid scraping headers or navigation elements.\n", + "\n", + " The following example shows how to develop and use a custom function to avoid navigation and header elements." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Import the `beautifulsoup4` library and define the custom function." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pip install beautifulsoup4" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "from bs4 import BeautifulSoup\n", + "\n", + "\n", + "def remove_nav_and_header_elements(content: BeautifulSoup) -> str:\n", + " # Find all 'nav' and 'header' elements in the BeautifulSoup object\n", + " nav_elements = content.find_all(\"nav\")\n", + " header_elements = content.find_all(\"header\")\n", + "\n", + " # Remove each 'nav' and 'header' element from the BeautifulSoup object\n", + " for element in nav_elements + header_elements:\n", + " element.decompose()\n", + "\n", + " return str(content.get_text())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Add your custom function to the `SitemapLoader` object." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "loader = SitemapLoader(\n", + " \"https://api.python.langchain.com/sitemap.xml\",\n", + " filter_urls=[\"https://api.python.langchain.com/en/latest/\"],\n", + " parsing_function=remove_nav_and_header_elements,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Local Sitemap\n", + "\n", + "The sitemap loader can also be used to load local files." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "sitemap_loader = SitemapLoader(web_path=\"example_data/sitemap.xml\", is_local=True)\n", + "\n", + "docs = sitemap_loader.load()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all SiteMapLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.sitemap.SitemapLoader.html#langchain_community.document_loaders.sitemap.SitemapLoader" ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "documents[0]" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Add custom scraping rules\n", - "\n", - "The `SitemapLoader` uses `beautifulsoup4` for the scraping process, and it scrapes every element on the page by default. The `SitemapLoader` constructor accepts a custom scraping function. This feature can be helpful to tailor the scraping process to your specific needs; for example, you might want to avoid scraping headers or navigation elements.\n", - "\n", - " The following example shows how to develop and use a custom function to avoid navigation and header elements." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Import the `beautifulsoup4` library and define the custom function." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "pip install beautifulsoup4" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - "from bs4 import BeautifulSoup\n", - "\n", - "\n", - "def remove_nav_and_header_elements(content: BeautifulSoup) -> str:\n", - " # Find all 'nav' and 'header' elements in the BeautifulSoup object\n", - " nav_elements = content.find_all(\"nav\")\n", - " header_elements = content.find_all(\"header\")\n", - "\n", - " # Remove each 'nav' and 'header' element from the BeautifulSoup object\n", - " for element in nav_elements + header_elements:\n", - " element.decompose()\n", - "\n", - " return str(content.get_text())" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Add your custom function to the `SitemapLoader` object." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [], - "source": [ - "loader = SitemapLoader(\n", - " \"https://api.python.langchain.com/sitemap.xml\",\n", - " filter_urls=[\"https://api.python.langchain.com/en/latest/\"],\n", - " parsing_function=remove_nav_and_header_elements,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Local Sitemap\n", - "\n", - "The sitemap loader can also be used to load local files." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "sitemap_loader = SitemapLoader(web_path=\"example_data/sitemap.xml\", is_local=True)\n", - "\n", - "docs = sitemap_loader.load()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all SiteMapLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.sitemap.SitemapLoader.html#langchain_community.document_loaders.sitemap.SitemapLoader" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/document_loaders/unstructured_markdown.ipynb b/docs/docs/integrations/document_loaders/unstructured_markdown.ipynb index 3bd98c0bed3..ca62516a949 100644 --- a/docs/docs/integrations/document_loaders/unstructured_markdown.ipynb +++ b/docs/docs/integrations/document_loaders/unstructured_markdown.ipynb @@ -1,269 +1,267 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# UnstructuredMarkdownLoader\n", - "\n", - "This notebook provides a quick overview for getting started with UnstructuredMarkdown [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all __ModuleName__Loader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.markdown.UnstructuredMarkdownLoader.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/unstructured/)|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [UnstructuredMarkdownLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.markdown.UnstructuredMarkdownLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ❌ | ❌ | ✅ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| UnstructuredMarkdownLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access UnstructuredMarkdownLoader document loader you'll need to install the `langchain-community` integration package and the `unstructured` python package.\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are needed to use this loader." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community** and **unstructured**" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community unstructured" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents. \n", - "\n", - "You can run the loader in one of two modes: \"single\" and \"elements\". If you use \"single\" mode, the document will be returned as a single `Document` object. If you use \"elements\" mode, the unstructured library will split the document into elements such as `Title` and `NarrativeText`. You can pass in additional `unstructured` kwargs after mode to apply different `unstructured` settings." - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import UnstructuredMarkdownLoader\n", - "\n", - "loader = UnstructuredMarkdownLoader(\n", - " \"./example_data/example.md\",\n", - " mode=\"single\",\n", - " strategy=\"fast\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/example.md'}, page_content='Sample Markdown Document\\n\\nIntroduction\\n\\nWelcome to this sample Markdown document. Markdown is a lightweight markup language used for formatting text. It\\'s widely used for documentation, readme files, and more.\\n\\nFeatures\\n\\nHeaders\\n\\nMarkdown supports multiple levels of headers:\\n\\nHeader 1: # Header 1\\n\\nHeader 2: ## Header 2\\n\\nHeader 3: ### Header 3\\n\\nLists\\n\\nUnordered List\\n\\nItem 1\\n\\nItem 2\\n\\nSubitem 2.1\\n\\nSubitem 2.2\\n\\nOrdered List\\n\\nFirst item\\n\\nSecond item\\n\\nThird item\\n\\nLinks\\n\\nOpenAI is an AI research organization.\\n\\nImages\\n\\nHere\\'s an example image:\\n\\nCode\\n\\nInline Code\\n\\nUse code for inline code snippets.\\n\\nCode Block\\n\\n```python def greet(name): return f\"Hello, {name}!\"\\n\\nprint(greet(\"World\")) ```')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# UnstructuredMarkdownLoader\n", + "\n", + "This notebook provides a quick overview for getting started with UnstructuredMarkdown [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all __ModuleName__Loader features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.markdown.UnstructuredMarkdownLoader.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/unstructured/)|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [UnstructuredMarkdownLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.markdown.UnstructuredMarkdownLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ❌ | ❌ | ✅ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| UnstructuredMarkdownLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "To access UnstructuredMarkdownLoader document loader you'll need to install the `langchain-community` integration package and the `unstructured` python package.\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are needed to use this loader." ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': './example_data/example.md'}\n" - ] - } - ], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/example.md', 'link_texts': ['OpenAI'], 'link_urls': ['https://www.openai.com'], 'last_modified': '2024-08-14T15:04:18', 'languages': ['eng'], 'parent_id': 'de1f74bf226224377ab4d8b54f215bb9', 'filetype': 'text/markdown', 'file_directory': './example_data', 'filename': 'example.md', 'category': 'NarrativeText', 'element_id': '898a542a261f7dc65e0072d1e847d535'}, page_content='OpenAI is an AI research organization.')" + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []\n", - "page[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load Elements\n", - "\n", - "In this example we will load in the `elements` mode, which will return a list of the different elements in the markdown document:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "29" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community** and **unstructured**" ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_community.document_loaders import UnstructuredMarkdownLoader\n", - "\n", - "loader = UnstructuredMarkdownLoader(\n", - " \"./example_data/example.md\",\n", - " mode=\"elements\",\n", - " strategy=\"fast\",\n", - ")\n", - "\n", - "docs = loader.load()\n", - "len(docs)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As you see there are 29 elements that were pulled from the `example.md` file. The first element is the title of the document as expected:" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "'Sample Markdown Document'" + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community unstructured" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents. \n", + "\n", + "You can run the loader in one of two modes: \"single\" and \"elements\". If you use \"single\" mode, the document will be returned as a single `Document` object. If you use \"elements\" mode, the unstructured library will split the document into elements such as `Title` and `NarrativeText`. You can pass in additional `unstructured` kwargs after mode to apply different `unstructured` settings." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import UnstructuredMarkdownLoader\n", + "\n", + "loader = UnstructuredMarkdownLoader(\n", + " \"./example_data/example.md\",\n", + " mode=\"single\",\n", + " strategy=\"fast\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/example.md'}, page_content='Sample Markdown Document\\n\\nIntroduction\\n\\nWelcome to this sample Markdown document. Markdown is a lightweight markup language used for formatting text. It\\'s widely used for documentation, readme files, and more.\\n\\nFeatures\\n\\nHeaders\\n\\nMarkdown supports multiple levels of headers:\\n\\nHeader 1: # Header 1\\n\\nHeader 2: ## Header 2\\n\\nHeader 3: ### Header 3\\n\\nLists\\n\\nUnordered List\\n\\nItem 1\\n\\nItem 2\\n\\nSubitem 2.1\\n\\nSubitem 2.2\\n\\nOrdered List\\n\\nFirst item\\n\\nSecond item\\n\\nThird item\\n\\nLinks\\n\\nOpenAI is an AI research organization.\\n\\nImages\\n\\nHere\\'s an example image:\\n\\nCode\\n\\nInline Code\\n\\nUse code for inline code snippets.\\n\\nCode Block\\n\\n```python def greet(name): return f\"Hello, {name}!\"\\n\\nprint(greet(\"World\")) ```')" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': './example_data/example.md'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/example.md', 'link_texts': ['OpenAI'], 'link_urls': ['https://www.openai.com'], 'last_modified': '2024-08-14T15:04:18', 'languages': ['eng'], 'parent_id': 'de1f74bf226224377ab4d8b54f215bb9', 'filetype': 'text/markdown', 'file_directory': './example_data', 'filename': 'example.md', 'category': 'NarrativeText', 'element_id': '898a542a261f7dc65e0072d1e847d535'}, page_content='OpenAI is an AI research organization.')" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []\n", + "page[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load Elements\n", + "\n", + "In this example we will load in the `elements` mode, which will return a list of the different elements in the markdown document:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "29" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_community.document_loaders import UnstructuredMarkdownLoader\n", + "\n", + "loader = UnstructuredMarkdownLoader(\n", + " \"./example_data/example.md\",\n", + " mode=\"elements\",\n", + " strategy=\"fast\",\n", + ")\n", + "\n", + "docs = loader.load()\n", + "len(docs)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you see there are 29 elements that were pulled from the `example.md` file. The first element is the title of the document as expected:" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Sample Markdown Document'" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs[0].page_content" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all UnstructuredMarkdownLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.markdown.UnstructuredMarkdownLoader.html" ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "docs[0].page_content" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all UnstructuredMarkdownLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.markdown.UnstructuredMarkdownLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/docs/docs/integrations/document_loaders/unstructured_pdfloader.ipynb b/docs/docs/integrations/document_loaders/unstructured_pdfloader.ipynb index 546e09674cf..60cb109496b 100644 --- a/docs/docs/integrations/document_loaders/unstructured_pdfloader.ipynb +++ b/docs/docs/integrations/document_loaders/unstructured_pdfloader.ipynb @@ -1,284 +1,282 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# UnstructuredPDFLoader\n", - "\n", - "## Overview\n", - "\n", - "[Unstructured](https://unstructured-io.github.io/unstructured/) supports a common interface for working with unstructured or semi-structured file formats, such as Markdown or PDF. LangChain's [UnstructuredPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.UnstructuredPDFLoader.html) integrates with Unstructured to parse PDF documents into LangChain [Document](https://python.langchain.com/api_reference/core/documents/langchain_core.documents.base.Document.html) objects.\n", - "\n", - "Please see [this page](/docs/integrations/providers/unstructured/) for more information on installing system requirements.\n", - "\n", - "\n", - "### Integration details\n", - "\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/unstructured/)|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [UnstructuredPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.UnstructuredPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| UnstructuredPDFLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are needed to use this loader." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community** and **unstructured**." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-community unstructured" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can initialize our loader:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import UnstructuredPDFLoader\n", - "\n", - "file_path = \"./example_data/layout-parser-paper.pdf\"\n", - "loader = UnstructuredPDFLoader(file_path)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/layout-parser-paper.pdf'}, page_content='1 2 0 2\\n\\nn u J\\n\\n1 2\\n\\n]\\n\\nV C . s c [\\n\\n2 v 8 4 3 5 1 . 3 0 1 2 : v i X r a\\n\\nLayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis\\n\\nZejiang Shen1 ((cid:0)), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain Lee4, Jacob Carlson3, and Weining Li5\\n\\n1 Allen Institute for AI shannons@allenai.org 2 Brown University ruochen zhang@brown.edu 3 Harvard University {melissadell,jacob carlson}@fas.harvard.edu 4 University of Washington bcgl@cs.washington.edu 5 University of Waterloo w422li@uwaterloo.ca\\n\\nAbstract. Recent advances in document image analysis (DIA) have been primarily driven by the application of neural networks. Ideally, research outcomes could be easily deployed in production and extended for further investigation. However, various factors like loosely organized codebases and sophisticated model configurations complicate the easy reuse of im- portant innovations by a wide audience. Though there have been on-going efforts to improve reusability and simplify deep learning (DL) model development in disciplines like natural language processing and computer vision, none of them are optimized for challenges in the domain of DIA. This represents a major gap in the existing toolkit, as DIA is central to academic research across a wide range of disciplines in the social sciences and humanities. This paper introduces LayoutParser, an open-source library for streamlining the usage of DL in DIA research and applica- tions. The core LayoutParser library comes with a set of simple and intuitive interfaces for applying and customizing DL models for layout de- tection, character recognition, and many other document processing tasks. To promote extensibility, LayoutParser also incorporates a community platform for sharing both pre-trained models and full document digiti- zation pipelines. We demonstrate that LayoutParser is helpful for both lightweight and large-scale digitization pipelines in real-word use cases. The library is publicly available at https://layout-parser.github.io.\\n\\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis · Character Recognition · Open Source library · Toolkit.\\n\\n1\\n\\nIntroduction\\n\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of document image analysis (DIA) tasks including document image classification [11,\\n\\n2\\n\\nZ. Shen et al.\\n\\n37], layout detection [38, 22], table detection [26], and scene text detection [4]. A generalized learning-based framework dramatically reduces the need for the manual specification of complicated rules, which is the status quo with traditional methods. DL has the potential to transform DIA pipelines and benefit a broad spectrum of large-scale document digitization projects.\\n\\nHowever, there are several practical difficulties for taking advantages of re- cent advances in DL-based methods: 1) DL models are notoriously convoluted for reuse and extension. Existing models are developed using distinct frame- works like TensorFlow [1] or PyTorch [24], and the high-level parameters can be obfuscated by implementation details [8]. It can be a time-consuming and frustrating experience to debug, reproduce, and adapt existing models for DIA, and many researchers who would benefit the most from using these methods lack the technical background to implement them from scratch. 2) Document images contain diverse and disparate patterns across domains, and customized training is often required to achieve a desirable detection accuracy. Currently there is no full-fledged infrastructure for easily curating the target document image datasets and fine-tuning or re-training the models. 3) DIA usually requires a sequence of models and other processing to obtain the final outputs. Often research teams use DL models and then perform further document analyses in separate processes, and these pipelines are not documented in any central location (and often not documented at all). This makes it difficult for research teams to learn about how full pipelines are implemented and leads them to invest significant resources in reinventing the DIA wheel.\\n\\nLayoutParser provides a unified toolkit to support DL-based document image analysis and processing. To address the aforementioned challenges, LayoutParser is built with the following components:\\n\\n1. An off-the-shelf toolkit for applying DL models for layout detection, character recognition, and other DIA tasks (Section 3)\\n\\n2. A rich repository of pre-trained neural network models (Model Zoo) that underlies the off-the-shelf usage\\n\\n3. Comprehensive tools for efficient document image data annotation and model tuning to support different levels of customization\\n\\n4. A DL model hub and community platform for the easy sharing, distribu- tion, and discussion of DIA models and pipelines, to promote reusability, reproducibility, and extensibility (Section 4)\\n\\nThe library implements simple and intuitive Python APIs without sacrificing generalizability and versatility, and can be easily installed via pip. Its convenient functions for handling document image data can be seamlessly integrated with existing DIA pipelines. With detailed documentations and carefully curated tutorials, we hope this tool will benefit a variety of end-users, and will lead to advances in applications in both industry and academic research.\\n\\nLayoutParser is well aligned with recent efforts for improving DL model reusability in other disciplines like natural language processing [8, 34] and com- puter vision [35], but with a focus on unique challenges in DIA. We show LayoutParser can be applied in sophisticated and large-scale digitization projects\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nthat require precision, efficiency, and robustness, as well as simple and light- weight document processing tasks focusing on efficacy and flexibility (Section 5). LayoutParser is being actively maintained, and support for more deep learning models and novel methods in text-based layout analysis methods [37, 34] is planned.\\n\\nThe rest of the paper is organized as follows. Section 2 provides an overview of related work. The core LayoutParser library, DL Model Zoo, and customized model training are described in Section 3, and the DL model hub and commu- nity platform are detailed in Section 4. Section 5 shows two examples of how LayoutParser can be used in practical DIA projects, and Section 6 concludes.\\n\\n2 Related Work\\n\\nRecently, various DL models and datasets have been developed for layout analysis tasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen- tation tasks on historical documents. Object detection-based methods like Faster R-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38] and detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also been used in table detection [27]. However, these models are usually implemented individually and there is no unified framework to load and use such models.\\n\\nThere has been a surge of interest in creating open-source tools for document image processing: a search of document image analysis in Github leads to 5M relevant code pieces 6; yet most of them rely on traditional rule-based methods or provide limited functionalities. The closest prior research to our work is the OCR-D project7, which also tries to build a complete toolkit for DIA. However, similar to the platform developed by Neudecker et al. [21], it is designed for analyzing historical documents, and provides no supports for recent DL models. The DocumentLayoutAnalysis project8 focuses on processing born-digital PDF documents via analyzing the stored PDF data. Repositories like DeepLayout9 and Detectron2-PubLayNet10 are individual deep learning models trained on layout analysis datasets without support for the full DIA pipeline. The Document Analysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2] aim to improve the reproducibility of DIA methods (or DL models), yet they are not actively maintained. OCR engines like Tesseract [14], easyOCR11 and paddleOCR12 usually do not come with comprehensive functionalities for other DIA tasks like layout analysis.\\n\\nRecent years have also seen numerous efforts to create libraries for promoting reproducibility and reusability in the field of DL. Libraries like Dectectron2 [35],\\n\\n6 The number shown is obtained by specifying the search type as ‘code’. 7 https://ocr-d.de/en/about 8 https://github.com/BobLd/DocumentLayoutAnalysis 9 https://github.com/leonlulu/DeepLayout 10 https://github.com/hpanwar08/detectron2 11 https://github.com/JaidedAI/EasyOCR 12 https://github.com/PaddlePaddle/PaddleOCR\\n\\n3\\n\\n4\\n\\nZ. Shen et al.\\n\\nDIA Model Hub\\n\\nStorage & Visualization\\n\\nLayout Detection Models\\n\\nOCR Module\\n\\nCustomized Model Training\\n\\nModel Customization\\n\\nCommunity Platform\\n\\nThe Core LayoutParser Library\\n\\nLayout Data Structure\\n\\nEfficient Data Annotation\\n\\nDocument Images\\n\\nDIA Pipeline Sharing\\n\\nFig. 1: The overall architecture of LayoutParser. For an input document image, the core LayoutParser library provides a set of off-the-shelf tools for layout detection, OCR, visualization, and storage, backed by a carefully designed layout data structure. LayoutParser also supports high level customization via efficient layout annotation and model training functions. These improve model accuracy on the target samples. The community platform enables the easy sharing of DIA models and whole digitization pipelines to promote reusability and reproducibility. A collection of detailed documentation, tutorials and exemplar projects make LayoutParser easy to learn and use.\\n\\nAllenNLP [8] and transformers [34] have provided the community with complete DL-based support for developing and deploying models for general computer vision and natural language processing problems. LayoutParser, on the other hand, specializes specifically in DIA tasks. LayoutParser is also equipped with a community platform inspired by established model hubs such as Torch Hub [23] and TensorFlow Hub [1]. It enables the sharing of pretrained models as well as full document processing pipelines that are unique to DIA tasks.\\n\\nThere have been a variety of document data collections to facilitate the development of DL models. Some examples include PRImA [3](magazine layouts), PubLayNet [38](academic paper layouts), Table Bank [18](tables in academic papers), Newspaper Navigator Dataset [16, 17](newspaper figure layouts) and HJDataset [31](historical Japanese document layouts). A spectrum of models trained on these datasets are currently available in the LayoutParser model zoo to support different use cases.\\n\\n3 The Core LayoutParser Library\\n\\nAt the core of LayoutParser is an off-the-shelf toolkit that streamlines DL- based document image analysis. Five components support a simple interface with comprehensive functionalities: 1) The layout detection models enable using pre-trained or self-trained DL models for layout detection with just four lines of code. 2) The detected layout information is stored in carefully engineered\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nTable 1: Current layout detection models in the LayoutParser model zoo\\n\\nDataset\\n\\nBase Model1 Large Model Notes\\n\\nPubLayNet [38] PRImA [3] Newspaper [17] TableBank [18] HJDataset [31]\\n\\nF / M M F F F / M\\n\\nM - - F -\\n\\nLayouts of modern scientific documents Layouts of scanned modern magazines and scientific reports Layouts of scanned US newspapers from the 20th century Table region on modern scientific and business document Layouts of history Japanese documents\\n\\n1 For each dataset, we train several models of different sizes for different needs (the trade-off between accuracy vs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101 backbones [13], respectively. One can train models of different architectures, like Faster R-CNN [28] (F) and Mask R-CNN [12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained using the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model zoo in coming months.\\n\\nlayout data structures, which are optimized for efficiency and versatility. 3) When necessary, users can employ existing or customized OCR models via the unified API provided in the OCR module. 4) LayoutParser comes with a set of utility functions for the visualization and storage of the layout data. 5) LayoutParser is also highly customizable, via its integration with functions for layout data annotation and model training. We now provide detailed descriptions for each component.\\n\\n3.1 Layout Detection Models\\n\\nIn LayoutParser, a layout model takes a document image as an input and generates a list of rectangular boxes for the target content regions. Different from traditional methods, it relies on deep convolutional neural networks rather than manually curated rules to identify content regions. It is formulated as an object detection problem and state-of-the-art models like Faster R-CNN [28] and Mask R-CNN [12] are used. This yields prediction results of high accuracy and makes it possible to build a concise, generalized interface for layout detection. LayoutParser, built upon Detectron2 [35], provides a minimal API that can perform layout detection with only four lines of code in Python:\\n\\n1 import layoutparser as lp 2 image = cv2 . imread ( \" image_file \" ) # load images 3 model = lp . De t e c tro n2 Lay outM odel (\\n\\n\" lp :// PubLayNet / f as t er _ r c nn _ R _ 50 _ F P N_ 3 x / config \" )\\n\\n4 5 layout = model . detect ( image )\\n\\nLayoutParser provides a wealth of pre-trained model weights using various datasets covering different languages, time periods, and document types. Due to domain shift [7], the prediction performance can notably drop when models are ap- plied to target samples that are significantly different from the training dataset. As document structures and layouts vary greatly in different domains, it is important to select models trained on a dataset similar to the test samples. A semantic syntax is used for initializing the model weights in LayoutParser, using both the dataset name and model name lp:///.\\n\\n5\\n\\n6\\n\\nZ. Shen et al.\\n\\nFig. 2: The relationship between the three types of layout data structures. Coordinate supports three kinds of variation; TextBlock consists of the co- ordinate information and extra features like block text, types, and reading orders; a Layout object is a list of all possible layout elements, including other Layout objects. They all support the same set of transformation and operation APIs for maximum flexibility.\\n\\nShown in Table 1, LayoutParser currently hosts 9 pre-trained models trained on 5 different datasets. Description of the training dataset is provided alongside with the trained models such that users can quickly identify the most suitable models for their tasks. Additionally, when such a model is not readily available, LayoutParser also supports training customized layout models and community sharing of the models (detailed in Section 3.5).\\n\\n3.2 Layout Data Structures\\n\\nA critical feature of LayoutParser is the implementation of a series of data structures and operations that can be used to efficiently process and manipulate the layout elements. In document image analysis pipelines, various post-processing on the layout analysis model outputs is usually required to obtain the final outputs. Traditionally, this requires exporting DL model outputs and then loading the results into other pipelines. All model outputs from LayoutParser will be stored in carefully engineered data types optimized for further processing, which makes it possible to build an end-to-end document digitization pipeline within LayoutParser. There are three key components in the data structure, namely the Coordinate system, the TextBlock, and the Layout. They provide different levels of abstraction for the layout data, and a set of APIs are supported for transformations or operations on these classes.\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nCoordinates are the cornerstones for storing layout information. Currently, three types of Coordinate data structures are provided in LayoutParser, shown in Figure 2. Interval and Rectangle are the most common data types and support specifying 1D or 2D regions within a document. They are parameterized with 2 and 4 parameters. A Quadrilateral class is also implemented to support a more generalized representation of rectangular regions when the document is skewed or distorted, where the 4 corner points can be specified and a total of 8 degrees of freedom are supported. A wide collection of transformations like shift, pad, and scale, and operations like intersect, union, and is_in, are supported for these classes. Notably, it is common to separate a segment of the image and analyze it individually. LayoutParser provides full support for this scenario via image cropping operations crop_image and coordinate transformations like relative_to and condition_on that transform coordinates to and from their relative representations. We refer readers to Table 2 for a more detailed description of these operations13.\\n\\nBased on Coordinates, we implement the TextBlock class that stores both the positional and extra features of individual layout elements. It also supports specifying the reading orders via setting the parent field to the index of the parent object. A Layout class is built that takes in a list of TextBlocks and supports processing the elements in batch. Layout can also be nested to support hierarchical layout structures. They support the same operations and transformations as the Coordinate classes, minimizing both learning and deployment effort.\\n\\n3.3 OCR\\n\\nLayoutParser provides a unified interface for existing OCR tools. Though there are many OCR tools available, they are usually configured differently with distinct APIs or protocols for using them. It can be inefficient to add new OCR tools into an existing pipeline, and difficult to make direct comparisons among the available tools to find the best option for a particular project. To this end, LayoutParser builds a series of wrappers among existing OCR engines, and provides nearly the same syntax for using them. It supports a plug-and-play style of using OCR engines, making it effortless to switch, evaluate, and compare different OCR modules:\\n\\n1 ocr_agent = lp . TesseractAgent () 2 # Can be easily switched to other OCR software 3 tokens = ocr_agent . detect ( image )\\n\\nThe OCR outputs will also be stored in the aforementioned layout data structures and can be seamlessly incorporated into the digitization pipeline. Currently LayoutParser supports the Tesseract and Google Cloud Vision OCR engines.\\n\\nLayoutParser also comes with a DL-based CNN-RNN OCR model [6] trained with the Connectionist Temporal Classification (CTC) loss [10]. It can be used like the other OCR modules, and can be easily trained on customized datasets.\\n\\n13 This is also available in the LayoutParser documentation pages.\\n\\n7\\n\\n8\\n\\nZ. Shen et al.\\n\\nTable 2: All operations supported by the layout elements. The same APIs are supported across different layout element classes including Coordinate types, TextBlock and Layout.\\n\\nOperation Name\\n\\nDescription\\n\\nblock.pad(top, bottom, right, left) Enlarge the current block according to the input\\n\\nblock.scale(fx, fy)\\n\\nScale the current block given the ratio in x and y direction\\n\\nblock.shift(dx, dy)\\n\\nMove the current block with the shift distances in x and y direction\\n\\nblock1.is in(block2)\\n\\nWhether block1 is inside of block2\\n\\nblock1.intersect(block2)\\n\\nReturn the intersection region of block1 and block2. Coordinate type to be determined based on the inputs.\\n\\nblock1.union(block2)\\n\\nReturn the union region of block1 and block2. Coordinate type to be determined based on the inputs.\\n\\nblock1.relative to(block2)\\n\\nConvert the absolute coordinates of block1 to relative coordinates to block2\\n\\nblock1.condition on(block2)\\n\\nCalculate the absolute coordinates of block1 given the canvas block2’s absolute coordinates\\n\\nblock.crop image(image)\\n\\nObtain the image segments in the block region\\n\\n3.4 Storage and visualization\\n\\nThe end goal of DIA is to transform the image-based document data into a structured database. LayoutParser supports exporting layout data into different formats like JSON, csv, and will add the support for the METS/ALTO XML format 14 . It can also load datasets from layout analysis-specific formats like COCO [38] and the Page Format [25] for training layout models (Section 3.5). Visualization of the layout detection results is critical for both presentation and debugging. LayoutParser is built with an integrated API for displaying the layout information along with the original document image. Shown in Figure 3, it enables presenting layout data with rich meta information and features in different modes. More detailed information can be found in the online LayoutParser documentation page.\\n\\n3.5 Customized Model Training\\n\\nBesides the off-the-shelf library, LayoutParser is also highly customizable with supports for highly unique and challenging document analysis tasks. Target document images can be vastly different from the existing datasets for train- ing layout models, which leads to low layout detection accuracy. Training data\\n\\n14 https://altoxml.github.io\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nFig. 3: Layout detection and OCR results visualization generated by the LayoutParser APIs. Mode I directly overlays the layout region bounding boxes and categories over the original image. Mode II recreates the original document via drawing the OCR’d texts at their corresponding positions on the image canvas. In this figure, tokens in textual regions are filtered using the API and then displayed.\\n\\ncan also be highly sensitive and not sharable publicly. To overcome these chal- lenges, LayoutParser is built with rich features for efficient data annotation and customized model training.\\n\\nLayoutParser incorporates a toolkit optimized for annotating document lay- outs using object-level active learning [32]. With the help from a layout detection model trained along with labeling, only the most important layout objects within each image, rather than the whole image, are required for labeling. The rest of the regions are automatically annotated with high confidence predictions from the layout detection model. This allows a layout dataset to be created more efficiently with only around 60% of the labeling budget.\\n\\nAfter the training dataset is curated, LayoutParser supports different modes for training the layout models. Fine-tuning can be used for training models on a small newly-labeled dataset by initializing the model with existing pre-trained weights. Training from scratch can be helpful when the source dataset and target are significantly different and a large training set is available. However, as suggested in Studer et al.’s work[33], loading pre-trained weights on large-scale datasets like ImageNet [5], even from totally different domains, can still boost model performance. Through the integrated API provided by LayoutParser, users can easily compare model performances on the benchmark datasets.\\n\\n9\\n\\n10\\n\\nZ. Shen et al.\\n\\nFig. 4: Illustration of (a) the original historical Japanese document with layout detection results and (b) a recreated version of the document image that achieves much better character recognition recall. The reorganization algorithm rearranges the tokens based on the their detected bounding boxes given a maximum allowed height.\\n\\n4 LayoutParser Community Platform\\n\\nAnother focus of LayoutParser is promoting the reusability of layout detection models and full digitization pipelines. Similar to many existing deep learning libraries, LayoutParser comes with a community model hub for distributing layout models. End-users can upload their self-trained models to the model hub, and these models can be loaded into a similar interface as the currently available LayoutParser pre-trained models. For example, the model trained on the News Navigator dataset [17] has been incorporated in the model hub.\\n\\nBeyond DL models, LayoutParser also promotes the sharing of entire doc- ument digitization pipelines. For example, sometimes the pipeline requires the combination of multiple DL models to achieve better accuracy. Currently, pipelines are mainly described in academic papers and implementations are often not pub- licly available. To this end, the LayoutParser community platform also enables the sharing of layout pipelines to promote the discussion and reuse of techniques. For each shared pipeline, it has a dedicated project page, with links to the source code, documentation, and an outline of the approaches. A discussion panel is provided for exchanging ideas. Combined with the core LayoutParser library, users can easily build reusable components based on the shared pipelines and apply them to solve their unique problems.\\n\\n5 Use Cases\\n\\nThe core objective of LayoutParser is to make it easier to create both large-scale and light-weight document digitization pipelines. Large-scale document processing\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nfocuses on precision, efficiency, and robustness. The target documents may have complicated structures, and may require training multiple layout detection models to achieve the optimal accuracy. Light-weight pipelines are built for relatively simple documents, with an emphasis on development ease, speed and flexibility. Ideally one only needs to use existing resources, and model training should be avoided. Through two exemplar projects, we show how practitioners in both academia and industry can easily build such pipelines using LayoutParser and extract high-quality structured document data for their downstream tasks. The source code for these projects will be publicly available in the LayoutParser community hub.\\n\\n5.1 A Comprehensive Historical Document Digitization Pipeline\\n\\nThe digitization of historical documents can unlock valuable data that can shed light on many important social, economic, and historical questions. Yet due to scan noises, page wearing, and the prevalence of complicated layout structures, ob- taining a structured representation of historical document scans is often extremely complicated. In this example, LayoutParser was used to develop a comprehensive pipeline, shown in Figure 5, to gener- ate high-quality structured data from historical Japanese firm financial ta- bles with complicated layouts. The pipeline applies two layout models to identify different levels of document structures and two customized OCR engines for optimized character recog- nition accuracy.\\n\\nAs shown in Figure 4 (a), the document contains columns of text written vertically 15, a common style in Japanese. Due to scanning noise and archaic printing technology, the columns can be skewed or have vari- able widths, and hence cannot be eas- ily identified via rule-based methods. Within each column, words are sepa- rated by white spaces of variable size, and the vertical positions of objects can be an indicator of their layout type.\\n\\nFig. 5: Illustration of how LayoutParser helps with the historical document digi- tization pipeline.\\n\\n15 A document page consists of eight rows like this. For simplicity we skip the row\\n\\nsegmentation discussion and refer readers to the source code when available.\\n\\n11\\n\\n12\\n\\nZ. Shen et al.\\n\\nTo decipher the complicated layout\\n\\nstructure, two object detection models have been trained to recognize individual columns and tokens, respectively. A small training set (400 images with approxi- mately 100 annotations each) is curated via the active learning based annotation tool [32] in LayoutParser. The models learn to identify both the categories and regions for each token or column via their distinct visual features. The layout data structure enables easy grouping of the tokens within each column, and rearranging columns to achieve the correct reading orders based on the horizontal position. Errors are identified and rectified via checking the consistency of the model predictions. Therefore, though trained on a small dataset, the pipeline achieves a high level of layout detection accuracy: it achieves a 96.97 AP [19] score across 5 categories for the column detection model, and a 89.23 AP across 4 categories for the token detection model.\\n\\nA combination of character recognition methods is developed to tackle the unique challenges in this document. In our experiments, we found that irregular spacing between the tokens led to a low character recognition recall rate, whereas existing OCR models tend to perform better on densely-arranged texts. To overcome this challenge, we create a document reorganization algorithm that rearranges the text based on the token bounding boxes detected in the layout analysis step. Figure 4 (b) illustrates the generated image of dense text, which is sent to the OCR APIs as a whole to reduce the transaction costs. The flexible coordinate system in LayoutParser is used to transform the OCR results relative to their original positions on the page.\\n\\nAdditionally, it is common for historical documents to use unique fonts with different glyphs, which significantly degrades the accuracy of OCR models trained on modern texts. In this document, a special flat font is used for printing numbers and could not be detected by off-the-shelf OCR engines. Using the highly flexible functionalities from LayoutParser, a pipeline approach is constructed that achieves a high recognition accuracy with minimal effort. As the characters have unique visual structures and are usually clustered together, we train the layout model to identify number regions with a dedicated category. Subsequently, LayoutParser crops images within these regions, and identifies characters within them using a self-trained OCR model based on a CNN-RNN [6]. The model detects a total of 15 possible categories, and achieves a 0.98 Jaccard score16 and a 0.17 average Levinstein distances17 for token prediction on the test set.\\n\\nOverall, it is possible to create an intricate and highly accurate digitization pipeline for large-scale digitization using LayoutParser. The pipeline avoids specifying the complicated rules used in traditional methods, is straightforward to develop, and is robust to outliers. The DL models also generate fine-grained results that enable creative approaches like page reorganization for OCR.\\n\\n16 This measures the overlap between the detected and ground-truth characters, and\\n\\nthe maximum is 1.\\n\\n17 This measures the number of edits from the ground-truth text to the predicted text,\\n\\nand lower is better.\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nFig. 6: This lightweight table detector can identify tables (outlined in red) and cells (shaded in blue) in different locations on a page. In very few cases (d), it might generate minor error predictions, e.g, failing to capture the top text line of a table.\\n\\n5.2 A light-weight Visual Table Extractor\\n\\nDetecting tables and parsing their structures (table extraction) are of central im- portance for many document digitization tasks. Many previous works [26, 30, 27] and tools 18 have been developed to identify and parse table structures. Yet they might require training complicated models from scratch, or are only applicable for born-digital PDF documents. In this section, we show how LayoutParser can help build a light-weight accurate visual table extractor for legal docket tables using the existing resources with minimal effort.\\n\\nThe extractor uses a pre-trained layout detection model for identifying the table regions and some simple rules for pairing the rows and the columns in the PDF image. Mask R-CNN [12] trained on the PubLayNet dataset [38] from the LayoutParser Model Zoo can be used for detecting table regions. By filtering out model predictions of low confidence and removing overlapping predictions, LayoutParser can identify the tabular regions on each page, which significantly simplifies the subsequent steps. By applying the line detection functions within the tabular segments, provided in the utility module from LayoutParser, the pipeline can identify the three distinct columns in the tables. A row clustering method is then applied via analyzing the y coordinates of token bounding boxes in the left-most column, which are obtained from the OCR engines. A non-maximal suppression algorithm is used to remove duplicated rows with extremely small gaps. Shown in Figure 6, the built pipeline can detect tables at different positions on a page accurately. Continued tables from different pages are concatenated, and a structured table representation has been easily created.\\n\\n18 https://github.com/atlanhq/camelot, https://github.com/tabulapdf/tabula\\n\\n13\\n\\n14\\n\\nZ. Shen et al.\\n\\n6 Conclusion\\n\\nLayoutParser provides a comprehensive toolkit for deep learning-based document image analysis. The off-the-shelf library is easy to install, and can be used to build flexible and accurate pipelines for processing documents with complicated structures. It also supports high-level customization and enables easy labeling and training of DL models on unique document image datasets. The LayoutParser community platform facilitates sharing DL models and DIA pipelines, inviting discussion and promoting code reproducibility and reusability. The LayoutParser team is committed to keeping the library updated continuously and bringing the most recent advances in DL-based DIA, such as multi-modal document modeling [37, 36, 9] (an upcoming priority), to a diverse audience of end-users.\\n\\nAcknowledgements We thank the anonymous reviewers for their comments and suggestions. This project is supported in part by NSF Grant OIA-2033558 and funding from the Harvard Data Science Initiative and Harvard Catalyst. Zejiang Shen thanks Doug Downey for suggestions.\\n\\nReferences\\n\\n[1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Man´e, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Vi´egas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015), https://www.tensorflow.org/, software available from tensorflow.org\\n\\n[2] Alberti, M., Pondenkandath, V., W¨ursch, M., Ingold, R., Liwicki, M.: Deepdiva: a highly-functional python framework for reproducible experiments. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 423–428. IEEE (2018)\\n\\n[3] Antonacopoulos, A., Bridson, D., Papadopoulos, C., Pletschacher, S.: A realistic dataset for performance evaluation of document layout analysis. In: 2009 10th International Conference on Document Analysis and Recognition. pp. 296–300. IEEE (2009)\\n\\n[4] Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9365–9374 (2019)\\n\\n[5] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale\\n\\nHierarchical Image Database. In: CVPR09 (2009)\\n\\n[6] Deng, Y., Kanervisto, A., Ling, J., Rush, A.M.: Image-to-markup generation with coarse-to-fine attention. In: International Conference on Machine Learning. pp. 980–989. PMLR (2017)\\n\\n[7] Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International conference on machine learning. pp. 1180–1189. PMLR (2015)\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\n[8] Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters, M., Schmitz, M., Zettlemoyer, L.: Allennlp: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640 (2018) (cid:32)Lukasz Garncarek, Powalski, R., Stanis(cid:32)lawek, T., Topolski, B., Halama, P., Grali´nski, F.: Lambert: Layout-aware (language) modeling using bert for in- formation extraction (2020)\\n\\n[9]\\n\\n[10] Graves, A., Fern´andez, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on Machine learning. pp. 369–376 (2006)\\n\\n[11] Harley, A.W., Ufkes, A., Derpanis, K.G.: Evaluation of deep convolutional nets for document image classification and retrieval. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR). pp. 991–995. IEEE (2015) [12] He, K., Gkioxari, G., Doll´ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the\\n\\nIEEE international conference on computer vision. pp. 2961–2969 (2017)\\n\\n[13] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)\\n\\n[14] Kay, A.: Tesseract: An open-source optical character recognition engine. Linux J.\\n\\n2007(159), 2 (Jul 2007)\\n\\n[15] Lamiroy, B., Lopresti, D.: An open architecture for end-to-end document analysis benchmarking. In: 2011 International Conference on Document Analysis and Recognition. pp. 42–47. IEEE (2011)\\n\\n[16] Lee, B.C., Weld, D.S.: Newspaper navigator: Open faceted search for 1.5 million images. In: Adjunct Publication of the 33rd Annual ACM Sym- posium on User Interface Software and Technology. p. 120–122. UIST ’20 Adjunct, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3379350.3416143, https://doi-org.offcampus. lib.washington.edu/10.1145/3379350.3416143\\n\\n[17] Lee, B.C.G., Mears, J., Jakeway, E., Ferriter, M., Adams, C., Yarasavage, N., Thomas, D., Zwaard, K., Weld, D.S.: The Newspaper Navigator Dataset: Extracting Headlines and Visual Content from 16 Million Historic Newspaper Pages in Chronicling America, p. 3055–3062. Association for Computing Machinery, New York, NY, USA (2020), https://doi.org/10.1145/3340531.3412767\\n\\n[18] Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: Tablebank: Table benchmark for image-based table detection and recognition. arXiv preprint arXiv:1903.01949 (2019)\\n\\n[19] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ar, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)\\n\\n[20] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3431–3440 (2015)\\n\\n[21] Neudecker, C., Schlarb, S., Dogan, Z.M., Missier, P., Sufi, S., Williams, A., Wolsten- croft, K.: An experimental workflow development platform for historical document digitisation and analysis. In: Proceedings of the 2011 workshop on historical document imaging and processing. pp. 161–168 (2011)\\n\\n[22] Oliveira, S.A., Seguin, B., Kaplan, F.: dhsegment: A generic deep-learning approach for document segmentation. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 7–12. IEEE (2018)\\n\\n15\\n\\n16\\n\\nZ. Shen et al.\\n\\n[23] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017) [24] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019) [25] Pletschacher, S., Antonacopoulos, A.: The page (page analysis and ground-truth elements) format framework. In: 2010 20th International Conference on Pattern Recognition. pp. 257–260. IEEE (2010)\\n\\n[26] Prasad, D., Gadpal, A., Kapadni, K., Visave, M., Sultanpure, K.: Cascadetabnet: An approach for end to end table detection and structure recognition from image- based documents. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 572–573 (2020)\\n\\n[27] Qasim, S.R., Mahmood, H., Shafait, F.: Rethinking table recognition using graph neural networks. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 142–147. IEEE (2019)\\n\\n[28] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. pp. 91–99 (2015)\\n\\n[29] Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE transactions on neural networks 20(1), 61–80 (2008) [30] Schreiber, S., Agne, S., Wolf, I., Dengel, A., Ahmed, S.: Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR). vol. 1, pp. 1162–1167. IEEE (2017)\\n\\n[31] Shen, Z., Zhang, K., Dell, M.: A large dataset of historical japanese documents with complex layouts. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 548–549 (2020)\\n\\n[32] Shen, Z., Zhao, J., Dell, M., Yu, Y., Li, W.: Olala: Object-level active learning\\n\\nbased layout annotation. arXiv preprint arXiv:2010.01762 (2020)\\n\\n[33] Studer, L., Alberti, M., Pondenkandath, V., Goktepe, P., Kolonko, T., Fischer, A., Liwicki, M., Ingold, R.: A comprehensive study of imagenet pre-training for historical document image analysis. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 720–725. IEEE (2019)\\n\\n[34] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of- the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019) [35] Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. https://\\n\\ngithub.com/facebookresearch/detectron2 (2019)\\n\\n[36] Xu, Y., Xu, Y., Lv, T., Cui, L., Wei, F., Wang, G., Lu, Y., Florencio, D., Zhang, C., Che, W., et al.: Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740 (2020)\\n\\n[37] Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: Pre-training of\\n\\ntext and layout for document image understanding (2019)\\n\\n[38] Zhong, X., Tang, J., Yepes, A.J.: Publaynet:\\n\\nlargest dataset ever for doc- In: 2019 International Conference on Document IEEE (Sep 2019).\\n\\nument Analysis and Recognition (ICDAR). pp. 1015–1022. https://doi.org/10.1109/ICDAR.2019.00166\\n\\nlayout analysis.')" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# UnstructuredPDFLoader\n", + "\n", + "## Overview\n", + "\n", + "[Unstructured](https://unstructured-io.github.io/unstructured/) supports a common interface for working with unstructured or semi-structured file formats, such as Markdown or PDF. LangChain's [UnstructuredPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.UnstructuredPDFLoader.html) integrates with Unstructured to parse PDF documents into LangChain [Document](https://python.langchain.com/api_reference/core/documents/langchain_core.documents.base.Document.html) objects.\n", + "\n", + "Please see [this page](/docs/integrations/providers/unstructured/) for more information on installing system requirements.\n", + "\n", + "\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/unstructured/)|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [UnstructuredPDFLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.UnstructuredPDFLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| UnstructuredPDFLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are needed to use this loader." ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': './example_data/layout-parser-paper.pdf'}\n" - ] - } - ], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Retain Elements\n", - "\n", - "Under the hood, Unstructured creates different \"elements\" for different chunks of text. By default we combine those together, but you can easily keep that separation by specifying `mode=\"elements\"`." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/layout-parser-paper.pdf', 'coordinates': {'points': ((16.34, 213.36), (16.34, 253.36), (36.34, 253.36), (36.34, 213.36)), 'system': 'PixelSpace', 'layout_width': 612, 'layout_height': 792}, 'file_directory': './example_data', 'filename': 'layout-parser-paper.pdf', 'languages': ['eng'], 'last_modified': '2024-07-25T21:28:58', 'page_number': 1, 'filetype': 'application/pdf', 'category': 'UncategorizedText', 'element_id': 'd3ce55f220dfb75891b4394a18bcb973'}, page_content='1 2 0 2')" + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "file_path = \"./example_data/layout-parser-paper.pdf\"\n", - "loader = UnstructuredPDFLoader(file_path, mode=\"elements\")\n", - "\n", - "data = loader.load()\n", - "data[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "See the full set of element types for this particular document:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "{'ListItem', 'NarrativeText', 'Title', 'UncategorizedText'}" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community** and **unstructured**." ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "set(doc.metadata[\"category\"] for doc in data)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Fetching remote PDFs using Unstructured\n", - "\n", - "This covers how to load online PDFs into a document format that we can use downstream. This can be used for various online PDF sites such as https://open.umn.edu/opentextbooks/textbooks/ and https://arxiv.org/archive/\n", - "\n", - "Note: all other PDF loaders can also be used to fetch remote PDFs, but `OnlinePDFLoader` is a legacy function, and works specifically with `UnstructuredPDFLoader`." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "Document(metadata={'source': '/var/folders/td/vzm913rx77x21csd90g63_7c0000gn/T/tmp3qdyy7e8/tmp.pdf'}, page_content='3 2 0 2\\n\\nb e F 7\\n\\n]\\n\\nG A . h t a m\\n\\n[\\n\\n1 v 3 0 8 3 0 . 2 0 3 2 : v i X r a\\n\\nA WEAK (k, k)-LEFSCHETZ THEOREM FOR PROJECTIVE TORIC ORBIFOLDS\\n\\nWilliam D. Montoya\\n\\nInstituto de Matem´atica, Estat´ıstica e Computa¸c˜ao Cient´ıfica, Universidade Estadual de Campinas (UNICAMP),\\n\\nRua S´ergio Buarque de Holanda 651, 13083-859, Campinas, SP, Brazil\\n\\nFebruary 9, 2023\\n\\nAbstract\\n\\nFirstly we show a generalization of the (1, 1)-Lefschetz theorem for projective toric orbifolds and secondly we prove that on 2k-dimensional quasi-smooth hyper- surfaces coming from quasi-smooth intersection surfaces, under the Cayley trick, every rational (k, k)-cohomology class is algebraic, i.e., the Hodge conjecture holds on them.\\n\\n1\\n\\nIntroduction\\n\\nIn [3] we proved that, under suitable conditions, on a very general codimension s quasi- smooth intersection subvariety X in a projective toric orbifold Pd Σ with d + s = 2(k + 1) the Hodge conjecture holds, that is, every (p, p)-cohomology class, under the Poincar´e duality is a rational linear combination of fundamental classes of algebraic subvarieties of X. The proof of the above-mentioned result relies, for p ≠ d + 1 − s, on a Lefschetz\\n\\nDate: February 9, 2023 2020 Mathematics Subject Classification: 14C30, 14M10, 14J70, 14M25 Keywords: (1,1)- Lefschetz theorem, Hodge conjecture, toric varieties, complete intersection Email: wmontoya@ime.unicamp.br\\n\\n1\\n\\ntheorem ([7]) and the Hard Lefschetz theorem for projective orbifolds ([11]). When p = d + 1 − s the proof relies on the Cayley trick, a trick which associates to X a quasi-smooth hypersurface Y in a projective vector bundle, and the Cayley Proposition (4.3) which gives an isomorphism of some primitive cohomologies (4.2) of X and Y . The Cayley trick, following the philosophy of Mavlyutov in [7], reduces results known for quasi-smooth hypersurfaces to quasi-smooth intersection subvarieties. The idea in this paper goes the other way around, we translate some results for quasi-smooth intersection subvarieties to quasi-smooth hypersurfaces, mainly the (1, 1)-Lefschetz theorem.\\n\\nAcknowledgement. I thank Prof. Ugo Bruzzo and Tiago Fonseca for useful discus-\\n\\nsions. I also acknowledge support from FAPESP postdoctoral grant No. 2019/23499-7.\\n\\n2 Preliminaries and Notation\\n\\n2.1 Toric varieties\\n\\nLet M be a free abelian group of rank d, let N = Hom(M, Z), and NR = N ⊗Z R.\\n\\nA convex subset σ ⊂ NR is a rational k-dimensional simplicial cone if there exist k linearly independent primitive elements e1, . . . , ek ∈ N such that σ = {µ1e1 + ⋯ + µkek}.\\n\\nDefinition 2.1.\\n\\nThe generators ei are integral if for every i and any nonnegative rational number µ the product µei is in N only if µ is an integer.\\n\\nGiven two rational simplicial cones σ, σ′ one says that σ′ is a face of σ (σ′ < σ) if the set of integral generators of σ′ is a subset of the set of integral generators of σ.\\n\\nA finite set Σ = {σ1, . . . , σt} of rational simplicial cones is called a rational simplicial complete d-dimensional fan if:\\n\\n1. all faces of cones in Σ are in Σ;\\n\\n2. if σ, σ′ ∈ Σ then σ ∩ σ′ < σ and σ ∩ σ′ < σ′;\\n\\n3. NR = σ1 ∪ ⋅ ⋅ ⋅ ∪ σt.\\n\\nA rational simplicial complete d-dimensional fan Σ defines a d-dimensional toric variety Σ having only orbifold singularities which we assume to be projective. Moreover, T ∶= Pd N ⊗Z C∗ ≃ (C∗)d is the torus action on Pd Σ. We denote by Σ(i) the i-dimensional cones\\n\\n2\\n\\nof Σ and each ρ ∈ Σ corresponds to an irreducible T -invariant Weil divisor Dρ on Pd Cl(Σ) be the group of Weil divisors on Pd\\n\\nΣ module rational equivalences.\\n\\nΣ. Let\\n\\nThe total coordinate ring of Pd\\n\\nΣ is the polynomial ring S = C[xρ ∣ ρ ∈ Σ(1)], S has the ρ ∈\\n\\nCl(Σ)-grading, a Weil divisor D = ∑ρ∈Σ(1) uρDρ determines the monomial xu ∶= ∏ρ∈Σ(1) xuρ S and conversely deg(xu) = [D] ∈ Cl(Σ).\\n\\nFor a cone σ ∈ Σ, ˆσ is the set of 1-dimensional cone in Σ that are not contained in σ\\n\\nand xˆσ ∶= ∏ρ∈ˆσ xρ is the associated monomial in S.\\n\\nΣ is the monomial ideal BΣ ∶=< xˆσ ∣ σ ∈ Σ > and\\n\\nDefinition 2.2. The irrelevant ideal of Pd the zero locus Z(Σ) ∶= V(BΣ) in the affine space Ad ∶= Spec(S) is the irrelevant locus.\\n\\nProposition 2.3 (Theorem 5.1.11 [5]). The toric variety Pd Σ is a categorical quotient Ad ∖ Z(Σ) by the group Hom(Cl(Σ), C∗) and the group action is induced by the Cl(Σ)- grading of S.\\n\\n2.2 Orbifolds\\n\\nNow we give a brief introduction to complex orbifolds and we mention the needed theorems for the next section. Namely: de Rham theorem and Dolbeault theorem for complex orbifolds.\\n\\nDefinition 2.4. A complex orbifold of complex dimension d is a singular complex space whose singularities are locally isomorphic to quotient singularities Cd/G, for finite sub- groups G ⊂ Gl(d, C).\\n\\nDefinition 2.5. A differential form on a complex orbifold Z is defined locally at z ∈ Z as a G-invariant differential form on Cd where G ⊂ Gl(d, C) and Z is locally isomorphic to Cd/G around z.\\n\\nRoughly speaking the local geometry of orbifolds reduces to local G-invariant geometry. We have a complex of differential forms (A●(Z), d) and a double complex (A●,●(Z), ∂, ¯∂) of bigraded differential forms which define the de Rham and the Dolbeault cohomology groups (for a fixed p ∈ N) respectively:\\n\\ndR(Z, C) ∶=\\n\\nH ●\\n\\nker d im d\\n\\nand H p,●(Z, ¯∂) ∶=\\n\\nker ¯∂ im ¯∂\\n\\nTheorem 2.6 (Theorem 3.4.4 in [4] and Theorem 1.2 in [1] ). Let Z be a compact complex orbifold. There are natural isomorphisms:\\n\\n3\\n\\nH ●\\n\\ndR(Z, C) ≃ H ●(Z, C)\\n\\nH p,●(Z, ¯∂) ≃ H ●(X, Ωp Z )\\n\\n3\\n\\n(1,1)-Lefschetz theorem for projective toric orbifolds\\n\\nDefinition 3.1. A subvariety X ⊂ Pd Z(Σ).\\n\\nΣ is quasi-smooth if V(IX ) ⊂ A#Σ(1) is smooth outside\\n\\nExample 3.2. Quasi-smooth hypersurfaces or more generally quasi-smooth intersection sub- varieties are quasi-smooth subvarieties (see [2] or [7] for more details).\\n\\nRemark 3.3. Quasi-smooth subvarieties are suborbifolds of Pd Σ in the sense of Satake in [8]. Intuitively speaking they are subvarieties whose only singularities come from the ambient space.\\n\\nTheorem 3.4. Let X ⊂ Pd class λ ∈ H 1,1(X) ∩ H 2(X, Z) is algebraic\\n\\nΣ be a quasi-smooth subvariety. Then every (1, 1)-cohomology\\n\\nProof. From the exponential short exact sequence\\n\\n0 → Z → OX → O∗ X\\n\\n→ 0\\n\\nwe have a long exact sequence in cohomology\\n\\nX ) → H 2(X, Z) → H 2(OX ) ≃ H 0,2(X)\\n\\nH 1(O∗\\n\\nwhere the last isomorphisms is due to Steenbrink in [9]. Now, it is enough to prove the commutativity of the next diagram\\n\\nH 2(X, Z)\\n\\nH 2(X, OX )\\n\\nH 2(X, C)\\n\\n≃ Dolbeault\\n\\nde Rham ≃\\n\\n(cid:15)\\n\\n(cid:15)\\n\\nH 2\\n\\ndR(X, C)\\n\\n/\\n\\n/ H 0,2\\n\\n¯∂ (X)\\n\\n4\\n\\n△\\n\\n△\\n\\nThe key points are the de Rham and Dolbeault’s isomorphisms for orbifolds. The rest\\n\\nof the proof follows as the (1, 1)-Lefschetz theorem in [6].\\n\\nRemark 3.5. For k = 1 and Pd Lefschetz theorem.\\n\\nΣ as the projective space, we recover the classical (1, 1)-\\n\\nBy the Hard Lefschetz Theorem for projective orbifolds (see [11] for details) we get an\\n\\nisomorphism of cohomologies :\\n\\nH ●(X, Q) ≃ H 2 dim X−●(X, Q)\\n\\ngiven by the Lefschetz morphism and since it is a morphism of Hodge structures, we have:\\n\\nH 1,1(X, Q) ≃ H dim X−1,dim X−1(X, Q)\\n\\nFor X as before:\\n\\nCorollary 3.6. If the dimension of X is 1, 2 or 3. The Hodge conjecture holds on X.\\n\\nProof. If the dimCX = 1 the result is clear by the Hard Lefschetz theorem for projective orbifolds. The dimension 2 and 3 cases are covered by Theorem 3.5 and the Hard Lefschetz. theorem.\\n\\n4 Cayley trick and Cayley proposition\\n\\nThe Cayley trick is a way to associate to a quasi-smooth intersection subvariety a quasi- smooth hypersurface. Let L1, . . . , Ls be line bundles on Pd Σ be the projective space bundle associated to the vector bundle E = L1 ⊕ ⋯ ⊕ Ls. It is known that P(E) is a (d + s − 1)-dimensional simplicial toric variety whose fan depends on the degrees of the line bundles and the fan Σ. Furthermore, if the Cox ring, without considering the grading, of Pd\\n\\nΣ and let π ∶ P(E) → Pd\\n\\nΣ is C[x1, . . . , xm] then the Cox ring of P(E) is\\n\\nC[x1, . . . , xm, y1, . . . , ys]\\n\\nMoreover for X a quasi-smooth intersection subvariety cut off by f1, . . . , fs with deg(fi) = [Li] we relate the hypersurface Y cut off by F = y1f1 + ⋅ ⋅ ⋅ + ysfs which turns out to be quasi-smooth. For more details see Section 2 in [7].\\n\\n5\\n\\n△\\n\\nWe will denote P(E) as Pd+s−1\\n\\nΣ,X to keep track of its relation with X and Pd Σ.\\n\\nThe following is a key remark.\\n\\nRemark 4.1. There is a morphism ι ∶ X → Y ⊂ Pd+s−1 with y ≠ 0 has a preimage. Hence for any subvariety W = V(IW ) ⊂ X ⊂ Pd W ′ ⊂ Y ⊂ Pd+s−1 Σ,X such that π(W ′) = W , i.e., W ′ = {z = (x, y) ∣ x ∈ W }.\\n\\nΣ,X . Moreover every point z ∶= (x, y) ∈ Y Σ there exists\\n\\n△\\n\\nFor X ⊂ Pd\\n\\nΣ a quasi-smooth intersection variety the morphism in cohomology induced\\n\\nby the inclusion i∗ ∶ H d−s(Pd\\n\\nΣ, C) → H d−s(X, C) is injective by Proposition 1.4 in [7].\\n\\nDefinition 4.2. The primitive cohomology of H d−s and H d−s prim(X, Q) with rational coefficients.\\n\\nprim(X) is the quotient H d−s(X, C)/i∗(H d−s(Pd\\n\\nH d−s(Pd\\n\\nΣ, C) and H d−s(X, C) have pure Hodge structures, and the morphism i∗ is com-\\n\\npatible with them, so that H d−s\\n\\nprim(X) gets a pure Hodge structure.\\n\\nThe next Proposition is the Cayley proposition.\\n\\nProposition 4.3. [Proposition 2.3 in [3] ] Let X = X1 ∩⋅ ⋅ ⋅∩Xs be a quasi-smooth intersec- , d+s−3 tion subvariety in Pd 2\\n\\nΣ cut off by homogeneous polynomials f1 . . . fs. Then for p ≠ d+s−1\\n\\n2\\n\\nH p−1,d+s−1−p\\n\\nprim\\n\\n(Y ) ≃ H p−s,d−p\\n\\nprim (X).\\n\\nCorollary 4.4. If d + s = 2(k + 1),\\n\\nH k+1−s,k+1−s\\n\\nprim\\n\\n(X) ≃ H k,k\\n\\nprim(Y )\\n\\nRemark 4.5. The above isomorphisms are also true with rational coefficients since H ●(X, C) = H ●(X, Q) ⊗Q C. See the beginning of Section 7.1 in [10] for more details.\\n\\n△\\n\\n5 Main result\\n\\nTheorem 5.1. Let Y = {F = y1f1 + ⋯ + ykfk = 0} ⊂ P2k+1 associated to the quasi-smooth intersection surface X = Xf1 ∩ ⋅ ⋅ ⋅ ∩ Xfk ⊂ Pk+2 the Hodge conjecture holds.\\n\\nΣ,X be the quasi-smooth hypersurface Σ . Then on Y\\n\\nProof. If H k,k proposition H k,k\\n\\nprim(X, Q) = 0 we are done. So let us assume H k,k\\n\\nprim(X, Q) ≠ 0. By the Cayley prim(X, Q) and by the (1, 1)-Lefschetz theorem for projective\\n\\nprim(Y, Q) ≃ H 1,1\\n\\n6\\n\\nΣ, C))\\n\\ntoric orbifolds there is a non-zero algebraic basis λC1, . . . , λCn with rational coefficients of H 1,1 prim(X, Q) algebraic curves C1, . . . , Cn in X such that under the Poincar´e duality the class in homology [Ci] goes to λCi, [Ci] ↦ λCi. Recall that the Cox ring of Pk+2 is contained in the Cox ring of P2k+1 Σ,X without considering the Σ ) then (α, 0) ∈ Cl(P2k+1 grading. Considering the grading we have that if α ∈ Cl(Pk+2 Σ,X ). So the polynomials defining Ci ⊂ Pk+2 X,Σ but with different degree. Moreover, by Remark 4.1 each Ci is contained in Y = {F = y1f1 + ⋯ + ykfk = 0} and furthermore it has codimension k.\\n\\nprim(X, Q), that is, there are n ∶= h1,1\\n\\ncan be interpreted in P2k+1\\n\\nΣ\\n\\ni=1 is a basis of H k,k It is enough to prove that λCi is different from zero in H k,k prim(Y, Q) or equivalently that the cohomology classes {λCi}n i=1 do not come from the ambient space. By contradiction, let us assume that there exists a j and C ⊂ P2k+1 Σ,X , Q) with i∗(λC) = λCj or in terms of homology there exists a (k + 2)-dimensional algebraic subvariety V ⊂ P2k+1 Σ,X such that V ∩ Y = Cj so they are equal as a homology class of P2k+1 Σ,X ,i.e., [V ∩ Y ] = [Cj] . Σ where π ∶ (x, y) ↦ x. Hence It is easy to check that π(V ) ∩ X = Cj as a subvariety of Pk+2 [π(V ) ∩ X] = [Cj] which is equivalent to say that λCj comes from Pk+2 Σ which contradicts the choice of [Cj].\\n\\nClaim: {λCi}n\\n\\nprim(Y, Q).\\n\\nΣ,X such that λC ∈ H k,k(P2k+1\\n\\nRemark 5.2. Into the proof of the previous theorem, the key fact was that on X the Hodge conjecture holds and we translate it to Y by contradiction. So, using an analogous argument we have:\\n\\nProposition 5.3. Let Y = {F = y1fs+⋯+ysfs = 0} ⊂ P2k+1 associated to a quasi-smooth intersection subvariety X = Xf1 ∩ ⋅ ⋅ ⋅ ∩ Xfs ⊂ Pd d + s = 2(k + 1). If the Hodge conjecture holds on X then it holds as well on Y .\\n\\nΣ,X be the quasi-smooth hypersurface Σ such that\\n\\nCorollary 5.4. If the dimension of Y is 2s − 1, 2s or 2s + 1 then the Hodge conjecture holds on Y .\\n\\nProof. By Proposition 5.3 and Corollary 3.6.\\n\\n7\\n\\n△\\n\\nReferences\\n\\n[1] Angella, D. Cohomologies of certain orbifolds. Journal of Geometry and Physics\\n\\n71 (2013), 117–126.\\n\\n[2] Batyrev, V. V., and Cox, D. A. On the Hodge structure of projective hypersur-\\n\\nfaces in toric varieties. Duke Mathematical Journal 75, 2 (Aug 1994).\\n\\n[3] Bruzzo, U., and Montoya, W. On the Hodge conjecture for quasi-smooth in- tersections in toric varieties. S˜ao Paulo J. Math. Sci. Special Section: Geometry in Algebra and Algebra in Geometry (2021).\\n\\n[4] Caramello Jr, F. C. Introduction to orbifolds. arXiv:1909.08699v6 (2019).\\n\\n[5] Cox, D., Little, J., and Schenck, H. Toric varieties, vol. 124. American Math-\\n\\nematical Soc., 2011.\\n\\n[6] Griffiths, P., and Harris, J. Principles of Algebraic Geometry. John Wiley &\\n\\nSons, Ltd, 1978.\\n\\n[7] Mavlyutov, A. R. Cohomology of complete intersections in toric varieties. Pub-\\n\\nlished in Pacific J. of Math. 191 No. 1 (1999), 133–144.\\n\\n[8] Satake, I. On a Generalization of the Notion of Manifold. Proceedings of the National Academy of Sciences of the United States of America 42, 6 (1956), 359–363.\\n\\n[9] Steenbrink, J. H. M. Intersection form for quasi-homogeneous singularities. Com-\\n\\npositio Mathematica 34, 2 (1977), 211–223.\\n\\n[10] Voisin, C. Hodge Theory and Complex Algebraic Geometry I, vol. 1 of Cambridge\\n\\nStudies in Advanced Mathematics. Cambridge University Press, 2002.\\n\\n[11] Wang, Z. Z., and Zaffran, D. A remark on the Hard Lefschetz theorem for K¨ahler orbifolds. Proceedings of the American Mathematical Society 137, 08 (Aug 2009).\\n\\n8')" + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-community unstructured" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can initialize our loader:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import UnstructuredPDFLoader\n", + "\n", + "file_path = \"./example_data/layout-parser-paper.pdf\"\n", + "loader = UnstructuredPDFLoader(file_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/layout-parser-paper.pdf'}, page_content='1 2 0 2\\n\\nn u J\\n\\n1 2\\n\\n]\\n\\nV C . s c [\\n\\n2 v 8 4 3 5 1 . 3 0 1 2 : v i X r a\\n\\nLayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis\\n\\nZejiang Shen1 ((cid:0)), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain Lee4, Jacob Carlson3, and Weining Li5\\n\\n1 Allen Institute for AI shannons@allenai.org 2 Brown University ruochen zhang@brown.edu 3 Harvard University {melissadell,jacob carlson}@fas.harvard.edu 4 University of Washington bcgl@cs.washington.edu 5 University of Waterloo w422li@uwaterloo.ca\\n\\nAbstract. Recent advances in document image analysis (DIA) have been primarily driven by the application of neural networks. Ideally, research outcomes could be easily deployed in production and extended for further investigation. However, various factors like loosely organized codebases and sophisticated model configurations complicate the easy reuse of im- portant innovations by a wide audience. Though there have been on-going efforts to improve reusability and simplify deep learning (DL) model development in disciplines like natural language processing and computer vision, none of them are optimized for challenges in the domain of DIA. This represents a major gap in the existing toolkit, as DIA is central to academic research across a wide range of disciplines in the social sciences and humanities. This paper introduces LayoutParser, an open-source library for streamlining the usage of DL in DIA research and applica- tions. The core LayoutParser library comes with a set of simple and intuitive interfaces for applying and customizing DL models for layout de- tection, character recognition, and many other document processing tasks. To promote extensibility, LayoutParser also incorporates a community platform for sharing both pre-trained models and full document digiti- zation pipelines. We demonstrate that LayoutParser is helpful for both lightweight and large-scale digitization pipelines in real-word use cases. The library is publicly available at https://layout-parser.github.io.\\n\\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis · Character Recognition · Open Source library · Toolkit.\\n\\n1\\n\\nIntroduction\\n\\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of document image analysis (DIA) tasks including document image classification [11,\\n\\n2\\n\\nZ. Shen et al.\\n\\n37], layout detection [38, 22], table detection [26], and scene text detection [4]. A generalized learning-based framework dramatically reduces the need for the manual specification of complicated rules, which is the status quo with traditional methods. DL has the potential to transform DIA pipelines and benefit a broad spectrum of large-scale document digitization projects.\\n\\nHowever, there are several practical difficulties for taking advantages of re- cent advances in DL-based methods: 1) DL models are notoriously convoluted for reuse and extension. Existing models are developed using distinct frame- works like TensorFlow [1] or PyTorch [24], and the high-level parameters can be obfuscated by implementation details [8]. It can be a time-consuming and frustrating experience to debug, reproduce, and adapt existing models for DIA, and many researchers who would benefit the most from using these methods lack the technical background to implement them from scratch. 2) Document images contain diverse and disparate patterns across domains, and customized training is often required to achieve a desirable detection accuracy. Currently there is no full-fledged infrastructure for easily curating the target document image datasets and fine-tuning or re-training the models. 3) DIA usually requires a sequence of models and other processing to obtain the final outputs. Often research teams use DL models and then perform further document analyses in separate processes, and these pipelines are not documented in any central location (and often not documented at all). This makes it difficult for research teams to learn about how full pipelines are implemented and leads them to invest significant resources in reinventing the DIA wheel.\\n\\nLayoutParser provides a unified toolkit to support DL-based document image analysis and processing. To address the aforementioned challenges, LayoutParser is built with the following components:\\n\\n1. An off-the-shelf toolkit for applying DL models for layout detection, character recognition, and other DIA tasks (Section 3)\\n\\n2. A rich repository of pre-trained neural network models (Model Zoo) that underlies the off-the-shelf usage\\n\\n3. Comprehensive tools for efficient document image data annotation and model tuning to support different levels of customization\\n\\n4. A DL model hub and community platform for the easy sharing, distribu- tion, and discussion of DIA models and pipelines, to promote reusability, reproducibility, and extensibility (Section 4)\\n\\nThe library implements simple and intuitive Python APIs without sacrificing generalizability and versatility, and can be easily installed via pip. Its convenient functions for handling document image data can be seamlessly integrated with existing DIA pipelines. With detailed documentations and carefully curated tutorials, we hope this tool will benefit a variety of end-users, and will lead to advances in applications in both industry and academic research.\\n\\nLayoutParser is well aligned with recent efforts for improving DL model reusability in other disciplines like natural language processing [8, 34] and com- puter vision [35], but with a focus on unique challenges in DIA. We show LayoutParser can be applied in sophisticated and large-scale digitization projects\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nthat require precision, efficiency, and robustness, as well as simple and light- weight document processing tasks focusing on efficacy and flexibility (Section 5). LayoutParser is being actively maintained, and support for more deep learning models and novel methods in text-based layout analysis methods [37, 34] is planned.\\n\\nThe rest of the paper is organized as follows. Section 2 provides an overview of related work. The core LayoutParser library, DL Model Zoo, and customized model training are described in Section 3, and the DL model hub and commu- nity platform are detailed in Section 4. Section 5 shows two examples of how LayoutParser can be used in practical DIA projects, and Section 6 concludes.\\n\\n2 Related Work\\n\\nRecently, various DL models and datasets have been developed for layout analysis tasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen- tation tasks on historical documents. Object detection-based methods like Faster R-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38] and detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also been used in table detection [27]. However, these models are usually implemented individually and there is no unified framework to load and use such models.\\n\\nThere has been a surge of interest in creating open-source tools for document image processing: a search of document image analysis in Github leads to 5M relevant code pieces 6; yet most of them rely on traditional rule-based methods or provide limited functionalities. The closest prior research to our work is the OCR-D project7, which also tries to build a complete toolkit for DIA. However, similar to the platform developed by Neudecker et al. [21], it is designed for analyzing historical documents, and provides no supports for recent DL models. The DocumentLayoutAnalysis project8 focuses on processing born-digital PDF documents via analyzing the stored PDF data. Repositories like DeepLayout9 and Detectron2-PubLayNet10 are individual deep learning models trained on layout analysis datasets without support for the full DIA pipeline. The Document Analysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2] aim to improve the reproducibility of DIA methods (or DL models), yet they are not actively maintained. OCR engines like Tesseract [14], easyOCR11 and paddleOCR12 usually do not come with comprehensive functionalities for other DIA tasks like layout analysis.\\n\\nRecent years have also seen numerous efforts to create libraries for promoting reproducibility and reusability in the field of DL. Libraries like Dectectron2 [35],\\n\\n6 The number shown is obtained by specifying the search type as ‘code’. 7 https://ocr-d.de/en/about 8 https://github.com/BobLd/DocumentLayoutAnalysis 9 https://github.com/leonlulu/DeepLayout 10 https://github.com/hpanwar08/detectron2 11 https://github.com/JaidedAI/EasyOCR 12 https://github.com/PaddlePaddle/PaddleOCR\\n\\n3\\n\\n4\\n\\nZ. Shen et al.\\n\\nDIA Model Hub\\n\\nStorage & Visualization\\n\\nLayout Detection Models\\n\\nOCR Module\\n\\nCustomized Model Training\\n\\nModel Customization\\n\\nCommunity Platform\\n\\nThe Core LayoutParser Library\\n\\nLayout Data Structure\\n\\nEfficient Data Annotation\\n\\nDocument Images\\n\\nDIA Pipeline Sharing\\n\\nFig. 1: The overall architecture of LayoutParser. For an input document image, the core LayoutParser library provides a set of off-the-shelf tools for layout detection, OCR, visualization, and storage, backed by a carefully designed layout data structure. LayoutParser also supports high level customization via efficient layout annotation and model training functions. These improve model accuracy on the target samples. The community platform enables the easy sharing of DIA models and whole digitization pipelines to promote reusability and reproducibility. A collection of detailed documentation, tutorials and exemplar projects make LayoutParser easy to learn and use.\\n\\nAllenNLP [8] and transformers [34] have provided the community with complete DL-based support for developing and deploying models for general computer vision and natural language processing problems. LayoutParser, on the other hand, specializes specifically in DIA tasks. LayoutParser is also equipped with a community platform inspired by established model hubs such as Torch Hub [23] and TensorFlow Hub [1]. It enables the sharing of pretrained models as well as full document processing pipelines that are unique to DIA tasks.\\n\\nThere have been a variety of document data collections to facilitate the development of DL models. Some examples include PRImA [3](magazine layouts), PubLayNet [38](academic paper layouts), Table Bank [18](tables in academic papers), Newspaper Navigator Dataset [16, 17](newspaper figure layouts) and HJDataset [31](historical Japanese document layouts). A spectrum of models trained on these datasets are currently available in the LayoutParser model zoo to support different use cases.\\n\\n3 The Core LayoutParser Library\\n\\nAt the core of LayoutParser is an off-the-shelf toolkit that streamlines DL- based document image analysis. Five components support a simple interface with comprehensive functionalities: 1) The layout detection models enable using pre-trained or self-trained DL models for layout detection with just four lines of code. 2) The detected layout information is stored in carefully engineered\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nTable 1: Current layout detection models in the LayoutParser model zoo\\n\\nDataset\\n\\nBase Model1 Large Model Notes\\n\\nPubLayNet [38] PRImA [3] Newspaper [17] TableBank [18] HJDataset [31]\\n\\nF / M M F F F / M\\n\\nM - - F -\\n\\nLayouts of modern scientific documents Layouts of scanned modern magazines and scientific reports Layouts of scanned US newspapers from the 20th century Table region on modern scientific and business document Layouts of history Japanese documents\\n\\n1 For each dataset, we train several models of different sizes for different needs (the trade-off between accuracy vs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101 backbones [13], respectively. One can train models of different architectures, like Faster R-CNN [28] (F) and Mask R-CNN [12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained using the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model zoo in coming months.\\n\\nlayout data structures, which are optimized for efficiency and versatility. 3) When necessary, users can employ existing or customized OCR models via the unified API provided in the OCR module. 4) LayoutParser comes with a set of utility functions for the visualization and storage of the layout data. 5) LayoutParser is also highly customizable, via its integration with functions for layout data annotation and model training. We now provide detailed descriptions for each component.\\n\\n3.1 Layout Detection Models\\n\\nIn LayoutParser, a layout model takes a document image as an input and generates a list of rectangular boxes for the target content regions. Different from traditional methods, it relies on deep convolutional neural networks rather than manually curated rules to identify content regions. It is formulated as an object detection problem and state-of-the-art models like Faster R-CNN [28] and Mask R-CNN [12] are used. This yields prediction results of high accuracy and makes it possible to build a concise, generalized interface for layout detection. LayoutParser, built upon Detectron2 [35], provides a minimal API that can perform layout detection with only four lines of code in Python:\\n\\n1 import layoutparser as lp 2 image = cv2 . imread ( \" image_file \" ) # load images 3 model = lp . De t e c tro n2 Lay outM odel (\\n\\n\" lp :// PubLayNet / f as t er _ r c nn _ R _ 50 _ F P N_ 3 x / config \" )\\n\\n4 5 layout = model . detect ( image )\\n\\nLayoutParser provides a wealth of pre-trained model weights using various datasets covering different languages, time periods, and document types. Due to domain shift [7], the prediction performance can notably drop when models are ap- plied to target samples that are significantly different from the training dataset. As document structures and layouts vary greatly in different domains, it is important to select models trained on a dataset similar to the test samples. A semantic syntax is used for initializing the model weights in LayoutParser, using both the dataset name and model name lp:///.\\n\\n5\\n\\n6\\n\\nZ. Shen et al.\\n\\nFig. 2: The relationship between the three types of layout data structures. Coordinate supports three kinds of variation; TextBlock consists of the co- ordinate information and extra features like block text, types, and reading orders; a Layout object is a list of all possible layout elements, including other Layout objects. They all support the same set of transformation and operation APIs for maximum flexibility.\\n\\nShown in Table 1, LayoutParser currently hosts 9 pre-trained models trained on 5 different datasets. Description of the training dataset is provided alongside with the trained models such that users can quickly identify the most suitable models for their tasks. Additionally, when such a model is not readily available, LayoutParser also supports training customized layout models and community sharing of the models (detailed in Section 3.5).\\n\\n3.2 Layout Data Structures\\n\\nA critical feature of LayoutParser is the implementation of a series of data structures and operations that can be used to efficiently process and manipulate the layout elements. In document image analysis pipelines, various post-processing on the layout analysis model outputs is usually required to obtain the final outputs. Traditionally, this requires exporting DL model outputs and then loading the results into other pipelines. All model outputs from LayoutParser will be stored in carefully engineered data types optimized for further processing, which makes it possible to build an end-to-end document digitization pipeline within LayoutParser. There are three key components in the data structure, namely the Coordinate system, the TextBlock, and the Layout. They provide different levels of abstraction for the layout data, and a set of APIs are supported for transformations or operations on these classes.\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nCoordinates are the cornerstones for storing layout information. Currently, three types of Coordinate data structures are provided in LayoutParser, shown in Figure 2. Interval and Rectangle are the most common data types and support specifying 1D or 2D regions within a document. They are parameterized with 2 and 4 parameters. A Quadrilateral class is also implemented to support a more generalized representation of rectangular regions when the document is skewed or distorted, where the 4 corner points can be specified and a total of 8 degrees of freedom are supported. A wide collection of transformations like shift, pad, and scale, and operations like intersect, union, and is_in, are supported for these classes. Notably, it is common to separate a segment of the image and analyze it individually. LayoutParser provides full support for this scenario via image cropping operations crop_image and coordinate transformations like relative_to and condition_on that transform coordinates to and from their relative representations. We refer readers to Table 2 for a more detailed description of these operations13.\\n\\nBased on Coordinates, we implement the TextBlock class that stores both the positional and extra features of individual layout elements. It also supports specifying the reading orders via setting the parent field to the index of the parent object. A Layout class is built that takes in a list of TextBlocks and supports processing the elements in batch. Layout can also be nested to support hierarchical layout structures. They support the same operations and transformations as the Coordinate classes, minimizing both learning and deployment effort.\\n\\n3.3 OCR\\n\\nLayoutParser provides a unified interface for existing OCR tools. Though there are many OCR tools available, they are usually configured differently with distinct APIs or protocols for using them. It can be inefficient to add new OCR tools into an existing pipeline, and difficult to make direct comparisons among the available tools to find the best option for a particular project. To this end, LayoutParser builds a series of wrappers among existing OCR engines, and provides nearly the same syntax for using them. It supports a plug-and-play style of using OCR engines, making it effortless to switch, evaluate, and compare different OCR modules:\\n\\n1 ocr_agent = lp . TesseractAgent () 2 # Can be easily switched to other OCR software 3 tokens = ocr_agent . detect ( image )\\n\\nThe OCR outputs will also be stored in the aforementioned layout data structures and can be seamlessly incorporated into the digitization pipeline. Currently LayoutParser supports the Tesseract and Google Cloud Vision OCR engines.\\n\\nLayoutParser also comes with a DL-based CNN-RNN OCR model [6] trained with the Connectionist Temporal Classification (CTC) loss [10]. It can be used like the other OCR modules, and can be easily trained on customized datasets.\\n\\n13 This is also available in the LayoutParser documentation pages.\\n\\n7\\n\\n8\\n\\nZ. Shen et al.\\n\\nTable 2: All operations supported by the layout elements. The same APIs are supported across different layout element classes including Coordinate types, TextBlock and Layout.\\n\\nOperation Name\\n\\nDescription\\n\\nblock.pad(top, bottom, right, left) Enlarge the current block according to the input\\n\\nblock.scale(fx, fy)\\n\\nScale the current block given the ratio in x and y direction\\n\\nblock.shift(dx, dy)\\n\\nMove the current block with the shift distances in x and y direction\\n\\nblock1.is in(block2)\\n\\nWhether block1 is inside of block2\\n\\nblock1.intersect(block2)\\n\\nReturn the intersection region of block1 and block2. Coordinate type to be determined based on the inputs.\\n\\nblock1.union(block2)\\n\\nReturn the union region of block1 and block2. Coordinate type to be determined based on the inputs.\\n\\nblock1.relative to(block2)\\n\\nConvert the absolute coordinates of block1 to relative coordinates to block2\\n\\nblock1.condition on(block2)\\n\\nCalculate the absolute coordinates of block1 given the canvas block2’s absolute coordinates\\n\\nblock.crop image(image)\\n\\nObtain the image segments in the block region\\n\\n3.4 Storage and visualization\\n\\nThe end goal of DIA is to transform the image-based document data into a structured database. LayoutParser supports exporting layout data into different formats like JSON, csv, and will add the support for the METS/ALTO XML format 14 . It can also load datasets from layout analysis-specific formats like COCO [38] and the Page Format [25] for training layout models (Section 3.5). Visualization of the layout detection results is critical for both presentation and debugging. LayoutParser is built with an integrated API for displaying the layout information along with the original document image. Shown in Figure 3, it enables presenting layout data with rich meta information and features in different modes. More detailed information can be found in the online LayoutParser documentation page.\\n\\n3.5 Customized Model Training\\n\\nBesides the off-the-shelf library, LayoutParser is also highly customizable with supports for highly unique and challenging document analysis tasks. Target document images can be vastly different from the existing datasets for train- ing layout models, which leads to low layout detection accuracy. Training data\\n\\n14 https://altoxml.github.io\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nFig. 3: Layout detection and OCR results visualization generated by the LayoutParser APIs. Mode I directly overlays the layout region bounding boxes and categories over the original image. Mode II recreates the original document via drawing the OCR’d texts at their corresponding positions on the image canvas. In this figure, tokens in textual regions are filtered using the API and then displayed.\\n\\ncan also be highly sensitive and not sharable publicly. To overcome these chal- lenges, LayoutParser is built with rich features for efficient data annotation and customized model training.\\n\\nLayoutParser incorporates a toolkit optimized for annotating document lay- outs using object-level active learning [32]. With the help from a layout detection model trained along with labeling, only the most important layout objects within each image, rather than the whole image, are required for labeling. The rest of the regions are automatically annotated with high confidence predictions from the layout detection model. This allows a layout dataset to be created more efficiently with only around 60% of the labeling budget.\\n\\nAfter the training dataset is curated, LayoutParser supports different modes for training the layout models. Fine-tuning can be used for training models on a small newly-labeled dataset by initializing the model with existing pre-trained weights. Training from scratch can be helpful when the source dataset and target are significantly different and a large training set is available. However, as suggested in Studer et al.’s work[33], loading pre-trained weights on large-scale datasets like ImageNet [5], even from totally different domains, can still boost model performance. Through the integrated API provided by LayoutParser, users can easily compare model performances on the benchmark datasets.\\n\\n9\\n\\n10\\n\\nZ. Shen et al.\\n\\nFig. 4: Illustration of (a) the original historical Japanese document with layout detection results and (b) a recreated version of the document image that achieves much better character recognition recall. The reorganization algorithm rearranges the tokens based on the their detected bounding boxes given a maximum allowed height.\\n\\n4 LayoutParser Community Platform\\n\\nAnother focus of LayoutParser is promoting the reusability of layout detection models and full digitization pipelines. Similar to many existing deep learning libraries, LayoutParser comes with a community model hub for distributing layout models. End-users can upload their self-trained models to the model hub, and these models can be loaded into a similar interface as the currently available LayoutParser pre-trained models. For example, the model trained on the News Navigator dataset [17] has been incorporated in the model hub.\\n\\nBeyond DL models, LayoutParser also promotes the sharing of entire doc- ument digitization pipelines. For example, sometimes the pipeline requires the combination of multiple DL models to achieve better accuracy. Currently, pipelines are mainly described in academic papers and implementations are often not pub- licly available. To this end, the LayoutParser community platform also enables the sharing of layout pipelines to promote the discussion and reuse of techniques. For each shared pipeline, it has a dedicated project page, with links to the source code, documentation, and an outline of the approaches. A discussion panel is provided for exchanging ideas. Combined with the core LayoutParser library, users can easily build reusable components based on the shared pipelines and apply them to solve their unique problems.\\n\\n5 Use Cases\\n\\nThe core objective of LayoutParser is to make it easier to create both large-scale and light-weight document digitization pipelines. Large-scale document processing\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nfocuses on precision, efficiency, and robustness. The target documents may have complicated structures, and may require training multiple layout detection models to achieve the optimal accuracy. Light-weight pipelines are built for relatively simple documents, with an emphasis on development ease, speed and flexibility. Ideally one only needs to use existing resources, and model training should be avoided. Through two exemplar projects, we show how practitioners in both academia and industry can easily build such pipelines using LayoutParser and extract high-quality structured document data for their downstream tasks. The source code for these projects will be publicly available in the LayoutParser community hub.\\n\\n5.1 A Comprehensive Historical Document Digitization Pipeline\\n\\nThe digitization of historical documents can unlock valuable data that can shed light on many important social, economic, and historical questions. Yet due to scan noises, page wearing, and the prevalence of complicated layout structures, ob- taining a structured representation of historical document scans is often extremely complicated. In this example, LayoutParser was used to develop a comprehensive pipeline, shown in Figure 5, to gener- ate high-quality structured data from historical Japanese firm financial ta- bles with complicated layouts. The pipeline applies two layout models to identify different levels of document structures and two customized OCR engines for optimized character recog- nition accuracy.\\n\\nAs shown in Figure 4 (a), the document contains columns of text written vertically 15, a common style in Japanese. Due to scanning noise and archaic printing technology, the columns can be skewed or have vari- able widths, and hence cannot be eas- ily identified via rule-based methods. Within each column, words are sepa- rated by white spaces of variable size, and the vertical positions of objects can be an indicator of their layout type.\\n\\nFig. 5: Illustration of how LayoutParser helps with the historical document digi- tization pipeline.\\n\\n15 A document page consists of eight rows like this. For simplicity we skip the row\\n\\nsegmentation discussion and refer readers to the source code when available.\\n\\n11\\n\\n12\\n\\nZ. Shen et al.\\n\\nTo decipher the complicated layout\\n\\nstructure, two object detection models have been trained to recognize individual columns and tokens, respectively. A small training set (400 images with approxi- mately 100 annotations each) is curated via the active learning based annotation tool [32] in LayoutParser. The models learn to identify both the categories and regions for each token or column via their distinct visual features. The layout data structure enables easy grouping of the tokens within each column, and rearranging columns to achieve the correct reading orders based on the horizontal position. Errors are identified and rectified via checking the consistency of the model predictions. Therefore, though trained on a small dataset, the pipeline achieves a high level of layout detection accuracy: it achieves a 96.97 AP [19] score across 5 categories for the column detection model, and a 89.23 AP across 4 categories for the token detection model.\\n\\nA combination of character recognition methods is developed to tackle the unique challenges in this document. In our experiments, we found that irregular spacing between the tokens led to a low character recognition recall rate, whereas existing OCR models tend to perform better on densely-arranged texts. To overcome this challenge, we create a document reorganization algorithm that rearranges the text based on the token bounding boxes detected in the layout analysis step. Figure 4 (b) illustrates the generated image of dense text, which is sent to the OCR APIs as a whole to reduce the transaction costs. The flexible coordinate system in LayoutParser is used to transform the OCR results relative to their original positions on the page.\\n\\nAdditionally, it is common for historical documents to use unique fonts with different glyphs, which significantly degrades the accuracy of OCR models trained on modern texts. In this document, a special flat font is used for printing numbers and could not be detected by off-the-shelf OCR engines. Using the highly flexible functionalities from LayoutParser, a pipeline approach is constructed that achieves a high recognition accuracy with minimal effort. As the characters have unique visual structures and are usually clustered together, we train the layout model to identify number regions with a dedicated category. Subsequently, LayoutParser crops images within these regions, and identifies characters within them using a self-trained OCR model based on a CNN-RNN [6]. The model detects a total of 15 possible categories, and achieves a 0.98 Jaccard score16 and a 0.17 average Levinstein distances17 for token prediction on the test set.\\n\\nOverall, it is possible to create an intricate and highly accurate digitization pipeline for large-scale digitization using LayoutParser. The pipeline avoids specifying the complicated rules used in traditional methods, is straightforward to develop, and is robust to outliers. The DL models also generate fine-grained results that enable creative approaches like page reorganization for OCR.\\n\\n16 This measures the overlap between the detected and ground-truth characters, and\\n\\nthe maximum is 1.\\n\\n17 This measures the number of edits from the ground-truth text to the predicted text,\\n\\nand lower is better.\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\nFig. 6: This lightweight table detector can identify tables (outlined in red) and cells (shaded in blue) in different locations on a page. In very few cases (d), it might generate minor error predictions, e.g, failing to capture the top text line of a table.\\n\\n5.2 A light-weight Visual Table Extractor\\n\\nDetecting tables and parsing their structures (table extraction) are of central im- portance for many document digitization tasks. Many previous works [26, 30, 27] and tools 18 have been developed to identify and parse table structures. Yet they might require training complicated models from scratch, or are only applicable for born-digital PDF documents. In this section, we show how LayoutParser can help build a light-weight accurate visual table extractor for legal docket tables using the existing resources with minimal effort.\\n\\nThe extractor uses a pre-trained layout detection model for identifying the table regions and some simple rules for pairing the rows and the columns in the PDF image. Mask R-CNN [12] trained on the PubLayNet dataset [38] from the LayoutParser Model Zoo can be used for detecting table regions. By filtering out model predictions of low confidence and removing overlapping predictions, LayoutParser can identify the tabular regions on each page, which significantly simplifies the subsequent steps. By applying the line detection functions within the tabular segments, provided in the utility module from LayoutParser, the pipeline can identify the three distinct columns in the tables. A row clustering method is then applied via analyzing the y coordinates of token bounding boxes in the left-most column, which are obtained from the OCR engines. A non-maximal suppression algorithm is used to remove duplicated rows with extremely small gaps. Shown in Figure 6, the built pipeline can detect tables at different positions on a page accurately. Continued tables from different pages are concatenated, and a structured table representation has been easily created.\\n\\n18 https://github.com/atlanhq/camelot, https://github.com/tabulapdf/tabula\\n\\n13\\n\\n14\\n\\nZ. Shen et al.\\n\\n6 Conclusion\\n\\nLayoutParser provides a comprehensive toolkit for deep learning-based document image analysis. The off-the-shelf library is easy to install, and can be used to build flexible and accurate pipelines for processing documents with complicated structures. It also supports high-level customization and enables easy labeling and training of DL models on unique document image datasets. The LayoutParser community platform facilitates sharing DL models and DIA pipelines, inviting discussion and promoting code reproducibility and reusability. The LayoutParser team is committed to keeping the library updated continuously and bringing the most recent advances in DL-based DIA, such as multi-modal document modeling [37, 36, 9] (an upcoming priority), to a diverse audience of end-users.\\n\\nAcknowledgements We thank the anonymous reviewers for their comments and suggestions. This project is supported in part by NSF Grant OIA-2033558 and funding from the Harvard Data Science Initiative and Harvard Catalyst. Zejiang Shen thanks Doug Downey for suggestions.\\n\\nReferences\\n\\n[1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Man´e, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Vi´egas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015), https://www.tensorflow.org/, software available from tensorflow.org\\n\\n[2] Alberti, M., Pondenkandath, V., W¨ursch, M., Ingold, R., Liwicki, M.: Deepdiva: a highly-functional python framework for reproducible experiments. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 423–428. IEEE (2018)\\n\\n[3] Antonacopoulos, A., Bridson, D., Papadopoulos, C., Pletschacher, S.: A realistic dataset for performance evaluation of document layout analysis. In: 2009 10th International Conference on Document Analysis and Recognition. pp. 296–300. IEEE (2009)\\n\\n[4] Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9365–9374 (2019)\\n\\n[5] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale\\n\\nHierarchical Image Database. In: CVPR09 (2009)\\n\\n[6] Deng, Y., Kanervisto, A., Ling, J., Rush, A.M.: Image-to-markup generation with coarse-to-fine attention. In: International Conference on Machine Learning. pp. 980–989. PMLR (2017)\\n\\n[7] Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International conference on machine learning. pp. 1180–1189. PMLR (2015)\\n\\nLayoutParser: A Unified Toolkit for DL-Based DIA\\n\\n[8] Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters, M., Schmitz, M., Zettlemoyer, L.: Allennlp: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640 (2018) (cid:32)Lukasz Garncarek, Powalski, R., Stanis(cid:32)lawek, T., Topolski, B., Halama, P., Grali´nski, F.: Lambert: Layout-aware (language) modeling using bert for in- formation extraction (2020)\\n\\n[9]\\n\\n[10] Graves, A., Fern´andez, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on Machine learning. pp. 369–376 (2006)\\n\\n[11] Harley, A.W., Ufkes, A., Derpanis, K.G.: Evaluation of deep convolutional nets for document image classification and retrieval. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR). pp. 991–995. IEEE (2015) [12] He, K., Gkioxari, G., Doll´ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the\\n\\nIEEE international conference on computer vision. pp. 2961–2969 (2017)\\n\\n[13] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)\\n\\n[14] Kay, A.: Tesseract: An open-source optical character recognition engine. Linux J.\\n\\n2007(159), 2 (Jul 2007)\\n\\n[15] Lamiroy, B., Lopresti, D.: An open architecture for end-to-end document analysis benchmarking. In: 2011 International Conference on Document Analysis and Recognition. pp. 42–47. IEEE (2011)\\n\\n[16] Lee, B.C., Weld, D.S.: Newspaper navigator: Open faceted search for 1.5 million images. In: Adjunct Publication of the 33rd Annual ACM Sym- posium on User Interface Software and Technology. p. 120–122. UIST ’20 Adjunct, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3379350.3416143, https://doi-org.offcampus. lib.washington.edu/10.1145/3379350.3416143\\n\\n[17] Lee, B.C.G., Mears, J., Jakeway, E., Ferriter, M., Adams, C., Yarasavage, N., Thomas, D., Zwaard, K., Weld, D.S.: The Newspaper Navigator Dataset: Extracting Headlines and Visual Content from 16 Million Historic Newspaper Pages in Chronicling America, p. 3055–3062. Association for Computing Machinery, New York, NY, USA (2020), https://doi.org/10.1145/3340531.3412767\\n\\n[18] Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: Tablebank: Table benchmark for image-based table detection and recognition. arXiv preprint arXiv:1903.01949 (2019)\\n\\n[19] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ar, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)\\n\\n[20] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3431–3440 (2015)\\n\\n[21] Neudecker, C., Schlarb, S., Dogan, Z.M., Missier, P., Sufi, S., Williams, A., Wolsten- croft, K.: An experimental workflow development platform for historical document digitisation and analysis. In: Proceedings of the 2011 workshop on historical document imaging and processing. pp. 161–168 (2011)\\n\\n[22] Oliveira, S.A., Seguin, B., Kaplan, F.: dhsegment: A generic deep-learning approach for document segmentation. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 7–12. IEEE (2018)\\n\\n15\\n\\n16\\n\\nZ. Shen et al.\\n\\n[23] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017) [24] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019) [25] Pletschacher, S., Antonacopoulos, A.: The page (page analysis and ground-truth elements) format framework. In: 2010 20th International Conference on Pattern Recognition. pp. 257–260. IEEE (2010)\\n\\n[26] Prasad, D., Gadpal, A., Kapadni, K., Visave, M., Sultanpure, K.: Cascadetabnet: An approach for end to end table detection and structure recognition from image- based documents. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 572–573 (2020)\\n\\n[27] Qasim, S.R., Mahmood, H., Shafait, F.: Rethinking table recognition using graph neural networks. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 142–147. IEEE (2019)\\n\\n[28] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. pp. 91–99 (2015)\\n\\n[29] Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE transactions on neural networks 20(1), 61–80 (2008) [30] Schreiber, S., Agne, S., Wolf, I., Dengel, A., Ahmed, S.: Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR). vol. 1, pp. 1162–1167. IEEE (2017)\\n\\n[31] Shen, Z., Zhang, K., Dell, M.: A large dataset of historical japanese documents with complex layouts. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 548–549 (2020)\\n\\n[32] Shen, Z., Zhao, J., Dell, M., Yu, Y., Li, W.: Olala: Object-level active learning\\n\\nbased layout annotation. arXiv preprint arXiv:2010.01762 (2020)\\n\\n[33] Studer, L., Alberti, M., Pondenkandath, V., Goktepe, P., Kolonko, T., Fischer, A., Liwicki, M., Ingold, R.: A comprehensive study of imagenet pre-training for historical document image analysis. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 720–725. IEEE (2019)\\n\\n[34] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of- the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019) [35] Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. https://\\n\\ngithub.com/facebookresearch/detectron2 (2019)\\n\\n[36] Xu, Y., Xu, Y., Lv, T., Cui, L., Wei, F., Wang, G., Lu, Y., Florencio, D., Zhang, C., Che, W., et al.: Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740 (2020)\\n\\n[37] Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: Pre-training of\\n\\ntext and layout for document image understanding (2019)\\n\\n[38] Zhong, X., Tang, J., Yepes, A.J.: Publaynet:\\n\\nlargest dataset ever for doc- In: 2019 International Conference on Document IEEE (Sep 2019).\\n\\nument Analysis and Recognition (ICDAR). pp. 1015–1022. https://doi.org/10.1109/ICDAR.2019.00166\\n\\nlayout analysis.')" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': './example_data/layout-parser-paper.pdf'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Retain Elements\n", + "\n", + "Under the hood, Unstructured creates different \"elements\" for different chunks of text. By default we combine those together, but you can easily keep that separation by specifying `mode=\"elements\"`." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/layout-parser-paper.pdf', 'coordinates': {'points': ((16.34, 213.36), (16.34, 253.36), (36.34, 253.36), (36.34, 213.36)), 'system': 'PixelSpace', 'layout_width': 612, 'layout_height': 792}, 'file_directory': './example_data', 'filename': 'layout-parser-paper.pdf', 'languages': ['eng'], 'last_modified': '2024-07-25T21:28:58', 'page_number': 1, 'filetype': 'application/pdf', 'category': 'UncategorizedText', 'element_id': 'd3ce55f220dfb75891b4394a18bcb973'}, page_content='1 2 0 2')" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "file_path = \"./example_data/layout-parser-paper.pdf\"\n", + "loader = UnstructuredPDFLoader(file_path, mode=\"elements\")\n", + "\n", + "data = loader.load()\n", + "data[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "See the full set of element types for this particular document:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'ListItem', 'NarrativeText', 'Title', 'UncategorizedText'}" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "set(doc.metadata[\"category\"] for doc in data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Fetching remote PDFs using Unstructured\n", + "\n", + "This covers how to load online PDFs into a document format that we can use downstream. This can be used for various online PDF sites such as https://open.umn.edu/opentextbooks/textbooks/ and https://arxiv.org/archive/\n", + "\n", + "Note: all other PDF loaders can also be used to fetch remote PDFs, but `OnlinePDFLoader` is a legacy function, and works specifically with `UnstructuredPDFLoader`." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': '/var/folders/td/vzm913rx77x21csd90g63_7c0000gn/T/tmp3qdyy7e8/tmp.pdf'}, page_content='3 2 0 2\\n\\nb e F 7\\n\\n]\\n\\nG A . h t a m\\n\\n[\\n\\n1 v 3 0 8 3 0 . 2 0 3 2 : v i X r a\\n\\nA WEAK (k, k)-LEFSCHETZ THEOREM FOR PROJECTIVE TORIC ORBIFOLDS\\n\\nWilliam D. Montoya\\n\\nInstituto de Matem´atica, Estat´ıstica e Computa¸c˜ao Cient´ıfica, Universidade Estadual de Campinas (UNICAMP),\\n\\nRua S´ergio Buarque de Holanda 651, 13083-859, Campinas, SP, Brazil\\n\\nFebruary 9, 2023\\n\\nAbstract\\n\\nFirstly we show a generalization of the (1, 1)-Lefschetz theorem for projective toric orbifolds and secondly we prove that on 2k-dimensional quasi-smooth hyper- surfaces coming from quasi-smooth intersection surfaces, under the Cayley trick, every rational (k, k)-cohomology class is algebraic, i.e., the Hodge conjecture holds on them.\\n\\n1\\n\\nIntroduction\\n\\nIn [3] we proved that, under suitable conditions, on a very general codimension s quasi- smooth intersection subvariety X in a projective toric orbifold Pd Σ with d + s = 2(k + 1) the Hodge conjecture holds, that is, every (p, p)-cohomology class, under the Poincar´e duality is a rational linear combination of fundamental classes of algebraic subvarieties of X. The proof of the above-mentioned result relies, for p ≠ d + 1 − s, on a Lefschetz\\n\\nDate: February 9, 2023 2020 Mathematics Subject Classification: 14C30, 14M10, 14J70, 14M25 Keywords: (1,1)- Lefschetz theorem, Hodge conjecture, toric varieties, complete intersection Email: wmontoya@ime.unicamp.br\\n\\n1\\n\\ntheorem ([7]) and the Hard Lefschetz theorem for projective orbifolds ([11]). When p = d + 1 − s the proof relies on the Cayley trick, a trick which associates to X a quasi-smooth hypersurface Y in a projective vector bundle, and the Cayley Proposition (4.3) which gives an isomorphism of some primitive cohomologies (4.2) of X and Y . The Cayley trick, following the philosophy of Mavlyutov in [7], reduces results known for quasi-smooth hypersurfaces to quasi-smooth intersection subvarieties. The idea in this paper goes the other way around, we translate some results for quasi-smooth intersection subvarieties to quasi-smooth hypersurfaces, mainly the (1, 1)-Lefschetz theorem.\\n\\nAcknowledgement. I thank Prof. Ugo Bruzzo and Tiago Fonseca for useful discus-\\n\\nsions. I also acknowledge support from FAPESP postdoctoral grant No. 2019/23499-7.\\n\\n2 Preliminaries and Notation\\n\\n2.1 Toric varieties\\n\\nLet M be a free abelian group of rank d, let N = Hom(M, Z), and NR = N ⊗Z R.\\n\\nA convex subset σ ⊂ NR is a rational k-dimensional simplicial cone if there exist k linearly independent primitive elements e1, . . . , ek ∈ N such that σ = {µ1e1 + ⋯ + µkek}.\\n\\nDefinition 2.1.\\n\\nThe generators ei are integral if for every i and any nonnegative rational number µ the product µei is in N only if µ is an integer.\\n\\nGiven two rational simplicial cones σ, σ′ one says that σ′ is a face of σ (σ′ < σ) if the set of integral generators of σ′ is a subset of the set of integral generators of σ.\\n\\nA finite set Σ = {σ1, . . . , σt} of rational simplicial cones is called a rational simplicial complete d-dimensional fan if:\\n\\n1. all faces of cones in Σ are in Σ;\\n\\n2. if σ, σ′ ∈ Σ then σ ∩ σ′ < σ and σ ∩ σ′ < σ′;\\n\\n3. NR = σ1 ∪ ⋅ ⋅ ⋅ ∪ σt.\\n\\nA rational simplicial complete d-dimensional fan Σ defines a d-dimensional toric variety Σ having only orbifold singularities which we assume to be projective. Moreover, T ∶= Pd N ⊗Z C∗ ≃ (C∗)d is the torus action on Pd Σ. We denote by Σ(i) the i-dimensional cones\\n\\n2\\n\\nof Σ and each ρ ∈ Σ corresponds to an irreducible T -invariant Weil divisor Dρ on Pd Cl(Σ) be the group of Weil divisors on Pd\\n\\nΣ module rational equivalences.\\n\\nΣ. Let\\n\\nThe total coordinate ring of Pd\\n\\nΣ is the polynomial ring S = C[xρ ∣ ρ ∈ Σ(1)], S has the ρ ∈\\n\\nCl(Σ)-grading, a Weil divisor D = ∑ρ∈Σ(1) uρDρ determines the monomial xu ∶= ∏ρ∈Σ(1) xuρ S and conversely deg(xu) = [D] ∈ Cl(Σ).\\n\\nFor a cone σ ∈ Σ, ˆσ is the set of 1-dimensional cone in Σ that are not contained in σ\\n\\nand xˆσ ∶= ∏ρ∈ˆσ xρ is the associated monomial in S.\\n\\nΣ is the monomial ideal BΣ ∶=< xˆσ ∣ σ ∈ Σ > and\\n\\nDefinition 2.2. The irrelevant ideal of Pd the zero locus Z(Σ) ∶= V(BΣ) in the affine space Ad ∶= Spec(S) is the irrelevant locus.\\n\\nProposition 2.3 (Theorem 5.1.11 [5]). The toric variety Pd Σ is a categorical quotient Ad ∖ Z(Σ) by the group Hom(Cl(Σ), C∗) and the group action is induced by the Cl(Σ)- grading of S.\\n\\n2.2 Orbifolds\\n\\nNow we give a brief introduction to complex orbifolds and we mention the needed theorems for the next section. Namely: de Rham theorem and Dolbeault theorem for complex orbifolds.\\n\\nDefinition 2.4. A complex orbifold of complex dimension d is a singular complex space whose singularities are locally isomorphic to quotient singularities Cd/G, for finite sub- groups G ⊂ Gl(d, C).\\n\\nDefinition 2.5. A differential form on a complex orbifold Z is defined locally at z ∈ Z as a G-invariant differential form on Cd where G ⊂ Gl(d, C) and Z is locally isomorphic to Cd/G around z.\\n\\nRoughly speaking the local geometry of orbifolds reduces to local G-invariant geometry. We have a complex of differential forms (A●(Z), d) and a double complex (A●,●(Z), ∂, ¯∂) of bigraded differential forms which define the de Rham and the Dolbeault cohomology groups (for a fixed p ∈ N) respectively:\\n\\ndR(Z, C) ∶=\\n\\nH ●\\n\\nker d im d\\n\\nand H p,●(Z, ¯∂) ∶=\\n\\nker ¯∂ im ¯∂\\n\\nTheorem 2.6 (Theorem 3.4.4 in [4] and Theorem 1.2 in [1] ). Let Z be a compact complex orbifold. There are natural isomorphisms:\\n\\n3\\n\\nH ●\\n\\ndR(Z, C) ≃ H ●(Z, C)\\n\\nH p,●(Z, ¯∂) ≃ H ●(X, Ωp Z )\\n\\n3\\n\\n(1,1)-Lefschetz theorem for projective toric orbifolds\\n\\nDefinition 3.1. A subvariety X ⊂ Pd Z(Σ).\\n\\nΣ is quasi-smooth if V(IX ) ⊂ A#Σ(1) is smooth outside\\n\\nExample 3.2. Quasi-smooth hypersurfaces or more generally quasi-smooth intersection sub- varieties are quasi-smooth subvarieties (see [2] or [7] for more details).\\n\\nRemark 3.3. Quasi-smooth subvarieties are suborbifolds of Pd Σ in the sense of Satake in [8]. Intuitively speaking they are subvarieties whose only singularities come from the ambient space.\\n\\nTheorem 3.4. Let X ⊂ Pd class λ ∈ H 1,1(X) ∩ H 2(X, Z) is algebraic\\n\\nΣ be a quasi-smooth subvariety. Then every (1, 1)-cohomology\\n\\nProof. From the exponential short exact sequence\\n\\n0 → Z → OX → O∗ X\\n\\n→ 0\\n\\nwe have a long exact sequence in cohomology\\n\\nX ) → H 2(X, Z) → H 2(OX ) ≃ H 0,2(X)\\n\\nH 1(O∗\\n\\nwhere the last isomorphisms is due to Steenbrink in [9]. Now, it is enough to prove the commutativity of the next diagram\\n\\nH 2(X, Z)\\n\\nH 2(X, OX )\\n\\nH 2(X, C)\\n\\n≃ Dolbeault\\n\\nde Rham ≃\\n\\n(cid:15)\\n\\n(cid:15)\\n\\nH 2\\n\\ndR(X, C)\\n\\n/\\n\\n/ H 0,2\\n\\n¯∂ (X)\\n\\n4\\n\\n△\\n\\n△\\n\\nThe key points are the de Rham and Dolbeault’s isomorphisms for orbifolds. The rest\\n\\nof the proof follows as the (1, 1)-Lefschetz theorem in [6].\\n\\nRemark 3.5. For k = 1 and Pd Lefschetz theorem.\\n\\nΣ as the projective space, we recover the classical (1, 1)-\\n\\nBy the Hard Lefschetz Theorem for projective orbifolds (see [11] for details) we get an\\n\\nisomorphism of cohomologies :\\n\\nH ●(X, Q) ≃ H 2 dim X−●(X, Q)\\n\\ngiven by the Lefschetz morphism and since it is a morphism of Hodge structures, we have:\\n\\nH 1,1(X, Q) ≃ H dim X−1,dim X−1(X, Q)\\n\\nFor X as before:\\n\\nCorollary 3.6. If the dimension of X is 1, 2 or 3. The Hodge conjecture holds on X.\\n\\nProof. If the dimCX = 1 the result is clear by the Hard Lefschetz theorem for projective orbifolds. The dimension 2 and 3 cases are covered by Theorem 3.5 and the Hard Lefschetz. theorem.\\n\\n4 Cayley trick and Cayley proposition\\n\\nThe Cayley trick is a way to associate to a quasi-smooth intersection subvariety a quasi- smooth hypersurface. Let L1, . . . , Ls be line bundles on Pd Σ be the projective space bundle associated to the vector bundle E = L1 ⊕ ⋯ ⊕ Ls. It is known that P(E) is a (d + s − 1)-dimensional simplicial toric variety whose fan depends on the degrees of the line bundles and the fan Σ. Furthermore, if the Cox ring, without considering the grading, of Pd\\n\\nΣ and let π ∶ P(E) → Pd\\n\\nΣ is C[x1, . . . , xm] then the Cox ring of P(E) is\\n\\nC[x1, . . . , xm, y1, . . . , ys]\\n\\nMoreover for X a quasi-smooth intersection subvariety cut off by f1, . . . , fs with deg(fi) = [Li] we relate the hypersurface Y cut off by F = y1f1 + ⋅ ⋅ ⋅ + ysfs which turns out to be quasi-smooth. For more details see Section 2 in [7].\\n\\n5\\n\\n△\\n\\nWe will denote P(E) as Pd+s−1\\n\\nΣ,X to keep track of its relation with X and Pd Σ.\\n\\nThe following is a key remark.\\n\\nRemark 4.1. There is a morphism ι ∶ X → Y ⊂ Pd+s−1 with y ≠ 0 has a preimage. Hence for any subvariety W = V(IW ) ⊂ X ⊂ Pd W ′ ⊂ Y ⊂ Pd+s−1 Σ,X such that π(W ′) = W , i.e., W ′ = {z = (x, y) ∣ x ∈ W }.\\n\\nΣ,X . Moreover every point z ∶= (x, y) ∈ Y Σ there exists\\n\\n△\\n\\nFor X ⊂ Pd\\n\\nΣ a quasi-smooth intersection variety the morphism in cohomology induced\\n\\nby the inclusion i∗ ∶ H d−s(Pd\\n\\nΣ, C) → H d−s(X, C) is injective by Proposition 1.4 in [7].\\n\\nDefinition 4.2. The primitive cohomology of H d−s and H d−s prim(X, Q) with rational coefficients.\\n\\nprim(X) is the quotient H d−s(X, C)/i∗(H d−s(Pd\\n\\nH d−s(Pd\\n\\nΣ, C) and H d−s(X, C) have pure Hodge structures, and the morphism i∗ is com-\\n\\npatible with them, so that H d−s\\n\\nprim(X) gets a pure Hodge structure.\\n\\nThe next Proposition is the Cayley proposition.\\n\\nProposition 4.3. [Proposition 2.3 in [3] ] Let X = X1 ∩⋅ ⋅ ⋅∩Xs be a quasi-smooth intersec- , d+s−3 tion subvariety in Pd 2\\n\\nΣ cut off by homogeneous polynomials f1 . . . fs. Then for p ≠ d+s−1\\n\\n2\\n\\nH p−1,d+s−1−p\\n\\nprim\\n\\n(Y ) ≃ H p−s,d−p\\n\\nprim (X).\\n\\nCorollary 4.4. If d + s = 2(k + 1),\\n\\nH k+1−s,k+1−s\\n\\nprim\\n\\n(X) ≃ H k,k\\n\\nprim(Y )\\n\\nRemark 4.5. The above isomorphisms are also true with rational coefficients since H ●(X, C) = H ●(X, Q) ⊗Q C. See the beginning of Section 7.1 in [10] for more details.\\n\\n△\\n\\n5 Main result\\n\\nTheorem 5.1. Let Y = {F = y1f1 + ⋯ + ykfk = 0} ⊂ P2k+1 associated to the quasi-smooth intersection surface X = Xf1 ∩ ⋅ ⋅ ⋅ ∩ Xfk ⊂ Pk+2 the Hodge conjecture holds.\\n\\nΣ,X be the quasi-smooth hypersurface Σ . Then on Y\\n\\nProof. If H k,k proposition H k,k\\n\\nprim(X, Q) = 0 we are done. So let us assume H k,k\\n\\nprim(X, Q) ≠ 0. By the Cayley prim(X, Q) and by the (1, 1)-Lefschetz theorem for projective\\n\\nprim(Y, Q) ≃ H 1,1\\n\\n6\\n\\nΣ, C))\\n\\ntoric orbifolds there is a non-zero algebraic basis λC1, . . . , λCn with rational coefficients of H 1,1 prim(X, Q) algebraic curves C1, . . . , Cn in X such that under the Poincar´e duality the class in homology [Ci] goes to λCi, [Ci] ↦ λCi. Recall that the Cox ring of Pk+2 is contained in the Cox ring of P2k+1 Σ,X without considering the Σ ) then (α, 0) ∈ Cl(P2k+1 grading. Considering the grading we have that if α ∈ Cl(Pk+2 Σ,X ). So the polynomials defining Ci ⊂ Pk+2 X,Σ but with different degree. Moreover, by Remark 4.1 each Ci is contained in Y = {F = y1f1 + ⋯ + ykfk = 0} and furthermore it has codimension k.\\n\\nprim(X, Q), that is, there are n ∶= h1,1\\n\\ncan be interpreted in P2k+1\\n\\nΣ\\n\\ni=1 is a basis of H k,k It is enough to prove that λCi is different from zero in H k,k prim(Y, Q) or equivalently that the cohomology classes {λCi}n i=1 do not come from the ambient space. By contradiction, let us assume that there exists a j and C ⊂ P2k+1 Σ,X , Q) with i∗(λC) = λCj or in terms of homology there exists a (k + 2)-dimensional algebraic subvariety V ⊂ P2k+1 Σ,X such that V ∩ Y = Cj so they are equal as a homology class of P2k+1 Σ,X ,i.e., [V ∩ Y ] = [Cj] . Σ where π ∶ (x, y) ↦ x. Hence It is easy to check that π(V ) ∩ X = Cj as a subvariety of Pk+2 [π(V ) ∩ X] = [Cj] which is equivalent to say that λCj comes from Pk+2 Σ which contradicts the choice of [Cj].\\n\\nClaim: {λCi}n\\n\\nprim(Y, Q).\\n\\nΣ,X such that λC ∈ H k,k(P2k+1\\n\\nRemark 5.2. Into the proof of the previous theorem, the key fact was that on X the Hodge conjecture holds and we translate it to Y by contradiction. So, using an analogous argument we have:\\n\\nProposition 5.3. Let Y = {F = y1fs+⋯+ysfs = 0} ⊂ P2k+1 associated to a quasi-smooth intersection subvariety X = Xf1 ∩ ⋅ ⋅ ⋅ ∩ Xfs ⊂ Pd d + s = 2(k + 1). If the Hodge conjecture holds on X then it holds as well on Y .\\n\\nΣ,X be the quasi-smooth hypersurface Σ such that\\n\\nCorollary 5.4. If the dimension of Y is 2s − 1, 2s or 2s + 1 then the Hodge conjecture holds on Y .\\n\\nProof. By Proposition 5.3 and Corollary 3.6.\\n\\n7\\n\\n△\\n\\nReferences\\n\\n[1] Angella, D. Cohomologies of certain orbifolds. Journal of Geometry and Physics\\n\\n71 (2013), 117–126.\\n\\n[2] Batyrev, V. V., and Cox, D. A. On the Hodge structure of projective hypersur-\\n\\nfaces in toric varieties. Duke Mathematical Journal 75, 2 (Aug 1994).\\n\\n[3] Bruzzo, U., and Montoya, W. On the Hodge conjecture for quasi-smooth in- tersections in toric varieties. S˜ao Paulo J. Math. Sci. Special Section: Geometry in Algebra and Algebra in Geometry (2021).\\n\\n[4] Caramello Jr, F. C. Introduction to orbifolds. arXiv:1909.08699v6 (2019).\\n\\n[5] Cox, D., Little, J., and Schenck, H. Toric varieties, vol. 124. American Math-\\n\\nematical Soc., 2011.\\n\\n[6] Griffiths, P., and Harris, J. Principles of Algebraic Geometry. John Wiley &\\n\\nSons, Ltd, 1978.\\n\\n[7] Mavlyutov, A. R. Cohomology of complete intersections in toric varieties. Pub-\\n\\nlished in Pacific J. of Math. 191 No. 1 (1999), 133–144.\\n\\n[8] Satake, I. On a Generalization of the Notion of Manifold. Proceedings of the National Academy of Sciences of the United States of America 42, 6 (1956), 359–363.\\n\\n[9] Steenbrink, J. H. M. Intersection form for quasi-homogeneous singularities. Com-\\n\\npositio Mathematica 34, 2 (1977), 211–223.\\n\\n[10] Voisin, C. Hodge Theory and Complex Algebraic Geometry I, vol. 1 of Cambridge\\n\\nStudies in Advanced Mathematics. Cambridge University Press, 2002.\\n\\n[11] Wang, Z. Z., and Zaffran, D. A remark on the Hard Lefschetz theorem for K¨ahler orbifolds. Proceedings of the American Mathematical Society 137, 08 (Aug 2009).\\n\\n8')" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_community.document_loaders import OnlinePDFLoader\n", + "\n", + "loader = OnlinePDFLoader(\"https://arxiv.org/pdf/2302.03803.pdf\")\n", + "data = loader.load()\n", + "data[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all UnstructuredPDFLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.UnstructuredPDFLoader.html" ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_community.document_loaders import OnlinePDFLoader\n", - "\n", - "loader = OnlinePDFLoader(\"https://arxiv.org/pdf/2302.03803.pdf\")\n", - "data = loader.load()\n", - "data[0]" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all UnstructuredPDFLoader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.pdf.UnstructuredPDFLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/docs/docs/integrations/document_loaders/xml.ipynb b/docs/docs/integrations/document_loaders/xml.ipynb index 39fb2fb507c..15976a6bf68 100644 --- a/docs/docs/integrations/document_loaders/xml.ipynb +++ b/docs/docs/integrations/document_loaders/xml.ipynb @@ -1,202 +1,200 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "72ccbe2b", - "metadata": {}, - "source": [ - "# UnstructuredXMLLoader\n", - "\n", - "This notebook provides a quick overview for getting started with UnstructuredXMLLoader [document loader](https://python.langchain.com/docs/concepts/document_loaders). The `UnstructuredXMLLoader` is used to load `XML` files. The loader works with `.xml` files. The page content will be the text extracted from the XML tags.\n", - "\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/unstructured/)|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [UnstructuredXMLLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.xml.UnstructuredXMLLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| UnstructuredXMLLoader | ✅ | ❌ | \n", - "\n", - "## Setup\n", - "\n", - "To access UnstructuredXMLLoader document loader you'll need to install the `langchain-community` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "No credentials are needed to use the UnstructuredXMLLoader" - ] - }, - { - "cell_type": "markdown", - "id": "fc4ba987", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9fa4d5e5", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "38e53f22", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community**." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "fcd320ec", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community" - ] - }, - { - "cell_type": "markdown", - "id": "a102f199", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "2d198582", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import UnstructuredXMLLoader\n", - "\n", - "loader = UnstructuredXMLLoader(\n", - " \"./example_data/factbook.xml\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "9bbb463c", - "metadata": {}, - "source": [ - "## Load" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "cd875e75", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "Document(metadata={'source': './example_data/factbook.xml'}, page_content='United States\\n\\nWashington, DC\\n\\nJoe Biden\\n\\nBaseball\\n\\nCanada\\n\\nOttawa\\n\\nJustin Trudeau\\n\\nHockey\\n\\nFrance\\n\\nParis\\n\\nEmmanuel Macron\\n\\nSoccer\\n\\nTrinidad & Tobado\\n\\nPort of Spain\\n\\nKeith Rowley\\n\\nTrack & Field')" + "cell_type": "markdown", + "id": "72ccbe2b", + "metadata": {}, + "source": [ + "# UnstructuredXMLLoader\n", + "\n", + "This notebook provides a quick overview for getting started with UnstructuredXMLLoader [document loader](https://python.langchain.com/docs/concepts/document_loaders). The `UnstructuredXMLLoader` is used to load `XML` files. The loader works with `.xml` files. The page content will be the text extracted from the XML tags.\n", + "\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/file_loaders/unstructured/)|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [UnstructuredXMLLoader](https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.xml.UnstructuredXMLLoader.html) | [langchain_community](https://python.langchain.com/api_reference/community/index.html) | ✅ | ❌ | ✅ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| UnstructuredXMLLoader | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "To access UnstructuredXMLLoader document loader you'll need to install the `langchain-community` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "No credentials are needed to use the UnstructuredXMLLoader" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "79b52cc0", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'source': './example_data/factbook.xml'}\n" - ] + "cell_type": "markdown", + "id": "fc4ba987", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9fa4d5e5", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "38e53f22", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community**." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fcd320ec", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community" + ] + }, + { + "cell_type": "markdown", + "id": "a102f199", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "2d198582", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import UnstructuredXMLLoader\n", + "\n", + "loader = UnstructuredXMLLoader(\n", + " \"./example_data/factbook.xml\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "9bbb463c", + "metadata": {}, + "source": [ + "## Load" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cd875e75", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(metadata={'source': './example_data/factbook.xml'}, page_content='United States\\n\\nWashington, DC\\n\\nJoe Biden\\n\\nBaseball\\n\\nCanada\\n\\nOttawa\\n\\nJustin Trudeau\\n\\nHockey\\n\\nFrance\\n\\nParis\\n\\nEmmanuel Macron\\n\\nSoccer\\n\\nTrinidad & Tobado\\n\\nPort of Spain\\n\\nKeith Rowley\\n\\nTrack & Field')" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "79b52cc0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'source': './example_data/factbook.xml'}\n" + ] + } + ], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "id": "557608e5", + "metadata": {}, + "source": [ + "## Lazy Load" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "e3b9e75c", + "metadata": {}, + "outputs": [], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "id": "712aa98f", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all __ModuleName__Loader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.xml.UnstructuredXMLLoader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" } - ], - "source": [ - "print(docs[0].metadata)" - ] }, - { - "cell_type": "markdown", - "id": "557608e5", - "metadata": {}, - "source": [ - "## Lazy Load" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "e3b9e75c", - "metadata": {}, - "outputs": [], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ] - }, - { - "cell_type": "markdown", - "id": "712aa98f", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all __ModuleName__Loader features and configurations head to the API reference: https://python.langchain.com/api_reference/community/document_loaders/langchain_community.document_loaders.xml.UnstructuredXMLLoader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/llms/openai.ipynb b/docs/docs/integrations/llms/openai.ipynb index e24d5a5fa83..2accff6e953 100644 --- a/docs/docs/integrations/llms/openai.ipynb +++ b/docs/docs/integrations/llms/openai.ipynb @@ -1,263 +1,261 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "9597802c", - "metadata": {}, - "source": [ - "# OpenAI\n", - "\n", - ":::caution\n", - "You are currently on a page documenting the use of OpenAI [text completion models](/docs/concepts/text_llms). The latest and most popular OpenAI models are [chat completion models](/docs/concepts/chat_models).\n", - "\n", - "Unless you are specifically using `gpt-3.5-turbo-instruct`, you are probably looking for [this page instead](/docs/integrations/chat/openai/).\n", - ":::\n", - "\n", - "[OpenAI](https://platform.openai.com/docs/introduction) offers a spectrum of models with different levels of power suitable for different tasks.\n", - "\n", - "This example goes over how to use LangChain to interact with `OpenAI` [models](https://platform.openai.com/docs/models)" - ] - }, - { - "cell_type": "markdown", - "id": "74312161", - "metadata": {}, - "source": [ - "## Overview\n", - "\n", - "### Integration details\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/openai) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [ChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html) | [langchain-openai](https://python.langchain.com/api_reference/openai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-openai?style=flat-square&label=%20) |\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access OpenAI models you'll need to create an OpenAI account, get an API key, and install the `langchain-openai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to https://platform.openai.com to sign up to OpenAI and generate an API key. Once you've done this set the OPENAI_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "efcdb2b6", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if \"OPENAI_API_KEY\" not in os.environ:\n", - " os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "f5d528fa", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "52fa46e8", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0fad78d8", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain OpenAI integration lives in the `langchain-openai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2e300149", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-openai" - ] - }, - { - "cell_type": "markdown", - "id": "129a3275", - "metadata": {}, - "source": [ - "Should you need to specify your organization ID, you can use the following cell. However, it is not required if you are only part of a single organization or intend to use your default organization. You can check your default organization [here](https://platform.openai.com/account/api-keys).\n", - "\n", - "To specify your organization, you can use this:\n", - "```python\n", - "OPENAI_ORGANIZATION = getpass()\n", - "\n", - "os.environ[\"OPENAI_ORGANIZATION\"] = OPENAI_ORGANIZATION\n", - "```\n", - "\n", - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "6fb585dd", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain_openai import OpenAI\n", - "\n", - "llm = OpenAI()" - ] - }, - { - "cell_type": "markdown", - "id": "464003c1", - "metadata": {}, - "source": [ - "## Invocation" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "85b49da0", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'\\n\\nI am an AI and do not have emotions like humans do, so I am always functioning at my optimal level. Thank you for asking! How can I assist you today?'" + "cell_type": "markdown", + "id": "9597802c", + "metadata": {}, + "source": [ + "# OpenAI\n", + "\n", + ":::caution\n", + "You are currently on a page documenting the use of OpenAI [text completion models](/docs/concepts/text_llms). The latest and most popular OpenAI models are [chat completion models](/docs/concepts/chat_models).\n", + "\n", + "Unless you are specifically using `gpt-3.5-turbo-instruct`, you are probably looking for [this page instead](/docs/integrations/chat/openai/).\n", + ":::\n", + "\n", + "[OpenAI](https://platform.openai.com/docs/introduction) offers a spectrum of models with different levels of power suitable for different tasks.\n", + "\n", + "This example goes over how to use LangChain to interact with `OpenAI` [models](https://platform.openai.com/docs/models)" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "llm.invoke(\"Hello how are you?\")" - ] - }, - { - "cell_type": "markdown", - "id": "2b7e0dfc", - "metadata": {}, - "source": [ - "## Chaining" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "a641dbd9", - "metadata": { - "tags": [] - }, - "outputs": [ + }, { - "data": { - "text/plain": [ - "'\\nIch liebe Programmieren.'" + "cell_type": "markdown", + "id": "74312161", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "### Integration details\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/openai) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [ChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html) | [langchain-openai](https://python.langchain.com/api_reference/openai/index.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-openai?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-openai?style=flat-square&label=%20) |\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access OpenAI models you'll need to create an OpenAI account, get an API key, and install the `langchain-openai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to https://platform.openai.com to sign up to OpenAI and generate an API key. Once you've done this set the OPENAI_API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "efcdb2b6", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if \"OPENAI_API_KEY\" not in os.environ:\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "f5d528fa", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "52fa46e8", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0fad78d8", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain OpenAI integration lives in the `langchain-openai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e300149", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-openai" + ] + }, + { + "cell_type": "markdown", + "id": "129a3275", + "metadata": {}, + "source": [ + "Should you need to specify your organization ID, you can use the following cell. However, it is not required if you are only part of a single organization or intend to use your default organization. You can check your default organization [here](https://platform.openai.com/account/api-keys).\n", + "\n", + "To specify your organization, you can use this:\n", + "```python\n", + "OPENAI_ORGANIZATION = getpass()\n", + "\n", + "os.environ[\"OPENAI_ORGANIZATION\"] = OPENAI_ORGANIZATION\n", + "```\n", + "\n", + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "6fb585dd", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain_openai import OpenAI\n", + "\n", + "llm = OpenAI()" + ] + }, + { + "cell_type": "markdown", + "id": "464003c1", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "85b49da0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'\\n\\nI am an AI and do not have emotions like humans do, so I am always functioning at my optimal level. Thank you for asking! How can I assist you today?'" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "llm.invoke(\"Hello how are you?\")" + ] + }, + { + "cell_type": "markdown", + "id": "2b7e0dfc", + "metadata": {}, + "source": [ + "## Chaining" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "a641dbd9", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'\\nIch liebe Programmieren.'" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import PromptTemplate\n", + "\n", + "prompt = PromptTemplate.from_template(\"How to say {input} in {output_language}:\\n\")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "58a9ddb1", + "metadata": {}, + "source": [ + "## Using a proxy\n", + "\n", + "If you are behind an explicit proxy, you can specify the http_client to pass through" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "55142cec", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install httpx\n", + "\n", + "import httpx\n", + "\n", + "openai = OpenAI(\n", + " model_name=\"gpt-3.5-turbo-instruct\",\n", + " http_client=httpx.Client(proxies=\"http://proxy.yourcompany.com:8080\"),\n", + ")" + ] + }, + { + "cell_type": "raw", + "id": "2fd99e97-013f-4c28-bb47-426faa42a2cf", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `OpenAI` llm features and configurations head to the API reference: https://python.langchain.com/api_reference/openai/llms/langchain_openai.llms.base.OpenAI.html" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "from langchain_core.prompts import PromptTemplate\n", - "\n", - "prompt = PromptTemplate.from_template(\"How to say {input} in {output_language}:\\n\")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" + }, + "vscode": { + "interpreter": { + "hash": "e971737741ff4ec9aff7dc6155a1060a59a8a6d52c757dbbe66bf8ee389494b1" + } + } }, - { - "cell_type": "markdown", - "id": "58a9ddb1", - "metadata": {}, - "source": [ - "## Using a proxy\n", - "\n", - "If you are behind an explicit proxy, you can specify the http_client to pass through" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "55142cec", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install httpx\n", - "\n", - "import httpx\n", - "\n", - "openai = OpenAI(\n", - " model_name=\"gpt-3.5-turbo-instruct\",\n", - " http_client=httpx.Client(proxies=\"http://proxy.yourcompany.com:8080\"),\n", - ")" - ] - }, - { - "cell_type": "raw", - "id": "2fd99e97-013f-4c28-bb47-426faa42a2cf", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `OpenAI` llm features and configurations head to the API reference: https://python.langchain.com/api_reference/openai/llms/langchain_openai.llms.base.OpenAI.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.4" - }, - "vscode": { - "interpreter": { - "hash": "e971737741ff4ec9aff7dc6155a1060a59a8a6d52c757dbbe66bf8ee389494b1" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/retrievers/wikipedia.ipynb b/docs/docs/integrations/retrievers/wikipedia.ipynb index 53c393e503b..bc36b7d947e 100644 --- a/docs/docs/integrations/retrievers/wikipedia.ipynb +++ b/docs/docs/integrations/retrievers/wikipedia.ipynb @@ -1,262 +1,262 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "62727aaa-bcff-4087-891c-e539f824ee1f", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Wikipedia\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "d62a16c1-10de-4f99-b392-c4ad2e6123a1", - "metadata": {}, - "source": [ - "# WikipediaRetriever\n", - "\n", - "## Overview\n", - ">[Wikipedia](https://wikipedia.org/) is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system called MediaWiki. `Wikipedia` is the largest and most-read reference work in history.\n", - "\n", - "This notebook shows how to retrieve wiki pages from `wikipedia.org` into the [Document](https://python.langchain.com/api_reference/core/documents/langchain_core.documents.base.Document.html) format that is used downstream.\n", - "\n", - "### Integration details\n", - "\n", - "import {ItemTable} from \"@theme/FeatureTables\";\n", - "\n", - "" - ] - }, - { - "cell_type": "markdown", - "id": "eb7d377c-168b-40e8-bd61-af6a4fb1b44f", - "metadata": {}, - "source": [ - "## Setup\n", - "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1bbc6013-2617-4f7e-9d8b-7453d09315c0", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "51489529-5dcd-4b86-bda6-de0a39d8ffd1", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The integration lives in the `langchain-community` package. We also need to install the `wikipedia` python package itself." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1a737220", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "%pip install -qU langchain_community wikipedia" - ] - }, - { - "cell_type": "markdown", - "id": "ae622ac6-d18a-4754-a4bd-d30a078c19b5", - "metadata": {}, - "source": [ - "## Instantiation" - ] - }, - { - "cell_type": "markdown", - "id": "6c15470b-a16b-4e0d-bc6a-6998bafbb5a4", - "metadata": {}, - "source": [ - "Now we can instantiate our retriever:\n", - "\n", - "`WikipediaRetriever` parameters include:\n", - "- optional `lang`: default=\"en\". Use it to search in a specific language part of Wikipedia\n", - "- optional `load_max_docs`: default=100. Use it to limit number of downloaded documents. It takes time to download all 100 documents, so use a small number for experiments. There is a hard limit of 300 for now.\n", - "- optional `load_all_available_meta`: default=False. By default only the most important fields downloaded: `Published` (date when document was published/last updated), `title`, `Summary`. If True, other fields also downloaded.\n", - "\n", - "`get_relevant_documents()` has one argument, `query`: free text which used to find documents in Wikipedia" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "b78f0cd0-ffea-4fe3-9d1d-54639c4ef1ff", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.retrievers import WikipediaRetriever\n", - "\n", - "retriever = WikipediaRetriever()" - ] - }, - { - "cell_type": "markdown", - "id": "12aead36-7b97-4d9c-82e7-ec644a3127f9", - "metadata": {}, - "source": [ - "## Usage" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "54a76605-6b1e-44bf-b8a2-7d48119290c4", - "metadata": {}, - "outputs": [], - "source": [ - "docs = retriever.invoke(\"TOKYO GHOUL\")" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "65ada2b7-3507-4dcb-9982-5f8f4e97a2e1", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Tokyo Ghoul (Japanese: 東京喰種（トーキョーグール）, Hepburn: Tōkyō Gūru) is a Japanese dark fantasy manga series written and illustrated by Sui Ishida. It was serialized in Shueisha's seinen manga magazine Weekly Young Jump from September 2011 to September 2014, with its chapters collected in 14 tankōbon volumes. The story is set in an alternate version of Tokyo where humans coexist with ghouls, beings who loo\n" - ] - } - ], - "source": [ - "print(docs[0].page_content[:400])" - ] - }, - { - "cell_type": "markdown", - "id": "ae3c3d16", - "metadata": {}, - "source": [ - "## Use within a chain\n", - "Like other retrievers, `WikipediaRetriever` can be incorporated into LLM applications via [chains](/docs/how_to/sequence/).\n", - "\n", - "We will need a LLM or chat model:\n", - "\n", - "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "4bd3d268-eb8c-46e9-930a-18f5e2a50008", - "metadata": {}, - "outputs": [], - "source": [ - "# | output: false\n", - "# | echo: false\n", - "\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "9b52bc65-1b2e-4c30-ab43-41eaa5bf79c3", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.output_parsers import StrOutputParser\n", - "from langchain_core.prompts import ChatPromptTemplate\n", - "from langchain_core.runnables import RunnablePassthrough\n", - "\n", - "prompt = ChatPromptTemplate.from_template(\n", - " \"\"\"\n", - " Answer the question based only on the context provided.\n", - " Context: {context}\n", - " Question: {question}\n", - " \"\"\"\n", - ")\n", - "\n", - "\n", - "def format_docs(docs):\n", - " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", - "\n", - "\n", - "chain = (\n", - " {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n", - " | prompt\n", - " | llm\n", - " | StrOutputParser()\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "0d268905-3b19-4338-ac10-223c0fe4d5e4", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'The main character in Tokyo Ghoul is Ken Kaneki, who transforms into a ghoul after receiving an organ transplant from a ghoul named Rize.'" + "cell_type": "raw", + "id": "62727aaa-bcff-4087-891c-e539f824ee1f", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Wikipedia\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "d62a16c1-10de-4f99-b392-c4ad2e6123a1", + "metadata": {}, + "source": [ + "# WikipediaRetriever\n", + "\n", + "## Overview\n", + ">[Wikipedia](https://wikipedia.org/) is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system called MediaWiki. `Wikipedia` is the largest and most-read reference work in history.\n", + "\n", + "This notebook shows how to retrieve wiki pages from `wikipedia.org` into the [Document](https://python.langchain.com/api_reference/core/documents/langchain_core.documents.base.Document.html) format that is used downstream.\n", + "\n", + "### Integration details\n", + "\n", + "import {ItemTable} from \"@theme/FeatureTables\";\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "eb7d377c-168b-40e8-bd61-af6a4fb1b44f", + "metadata": {}, + "source": [ + "## Setup\n", + "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1bbc6013-2617-4f7e-9d8b-7453d09315c0", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "51489529-5dcd-4b86-bda6-de0a39d8ffd1", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The integration lives in the `langchain-community` package. We also need to install the `wikipedia` python package itself." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1a737220", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "%pip install -qU langchain_community wikipedia" + ] + }, + { + "cell_type": "markdown", + "id": "ae622ac6-d18a-4754-a4bd-d30a078c19b5", + "metadata": {}, + "source": [ + "## Instantiation" + ] + }, + { + "cell_type": "markdown", + "id": "6c15470b-a16b-4e0d-bc6a-6998bafbb5a4", + "metadata": {}, + "source": [ + "Now we can instantiate our retriever:\n", + "\n", + "`WikipediaRetriever` parameters include:\n", + "- optional `lang`: default=\"en\". Use it to search in a specific language part of Wikipedia\n", + "- optional `load_max_docs`: default=100. Use it to limit number of downloaded documents. It takes time to download all 100 documents, so use a small number for experiments. There is a hard limit of 300 for now.\n", + "- optional `load_all_available_meta`: default=False. By default only the most important fields downloaded: `Published` (date when document was published/last updated), `title`, `Summary`. If True, other fields also downloaded.\n", + "\n", + "`get_relevant_documents()` has one argument, `query`: free text which used to find documents in Wikipedia" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "b78f0cd0-ffea-4fe3-9d1d-54639c4ef1ff", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.retrievers import WikipediaRetriever\n", + "\n", + "retriever = WikipediaRetriever()" + ] + }, + { + "cell_type": "markdown", + "id": "12aead36-7b97-4d9c-82e7-ec644a3127f9", + "metadata": {}, + "source": [ + "## Usage" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "54a76605-6b1e-44bf-b8a2-7d48119290c4", + "metadata": {}, + "outputs": [], + "source": [ + "docs = retriever.invoke(\"TOKYO GHOUL\")" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "65ada2b7-3507-4dcb-9982-5f8f4e97a2e1", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tokyo Ghoul (Japanese: 東京喰種（トーキョーグール）, Hepburn: Tōkyō Gūru) is a Japanese dark fantasy manga series written and illustrated by Sui Ishida. It was serialized in Shueisha's seinen manga magazine Weekly Young Jump from September 2011 to September 2014, with its chapters collected in 14 tankōbon volumes. The story is set in an alternate version of Tokyo where humans coexist with ghouls, beings who loo\n" + ] + } + ], + "source": [ + "print(docs[0].page_content[:400])" + ] + }, + { + "cell_type": "markdown", + "id": "ae3c3d16", + "metadata": {}, + "source": [ + "## Use within a chain\n", + "Like other retrievers, `WikipediaRetriever` can be incorporated into LLM applications via [chains](/docs/how_to/sequence/).\n", + "\n", + "We will need a LLM or chat model:\n", + "\n", + "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "4bd3d268-eb8c-46e9-930a-18f5e2a50008", + "metadata": {}, + "outputs": [], + "source": [ + "# | output: false\n", + "# | echo: false\n", + "\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "9b52bc65-1b2e-4c30-ab43-41eaa5bf79c3", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "from langchain_core.runnables import RunnablePassthrough\n", + "\n", + "prompt = ChatPromptTemplate.from_template(\n", + " \"\"\"\n", + " Answer the question based only on the context provided.\n", + " Context: {context}\n", + " Question: {question}\n", + " \"\"\"\n", + ")\n", + "\n", + "\n", + "def format_docs(docs):\n", + " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", + "\n", + "\n", + "chain = (\n", + " {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0d268905-3b19-4338-ac10-223c0fe4d5e4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The main character in Tokyo Ghoul is Ken Kaneki, who transforms into a ghoul after receiving an organ transplant from a ghoul named Rize.'" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chain.invoke(\n", + " \"Who is the main character in `Tokyo Ghoul` and does he transform into a ghoul?\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "236bbafb-ebd4-4165-9b8f-d47605f6eef3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `WikipediaRetriever` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/retrievers/langchain_community.retrievers.wikipedia.WikipediaRetriever.html#langchain-community-retrievers-wikipedia-wikipediaretriever)." ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "chain.invoke(\n", - " \"Who is the main character in `Tokyo Ghoul` and does he transform into a ghoul?\"\n", - ")" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "236bbafb-ebd4-4165-9b8f-d47605f6eef3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `WikipediaRetriever` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/retrievers/langchain_community.retrievers.wikipedia.WikipediaRetriever.html#langchain-community-retrievers-wikipedia-wikipediaretriever)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/ai21.ipynb b/docs/docs/integrations/text_embedding/ai21.ipynb index 813e8ccf6b8..beeb4ec48b8 100644 --- a/docs/docs/integrations/text_embedding/ai21.ipynb +++ b/docs/docs/integrations/text_embedding/ai21.ipynb @@ -1,272 +1,270 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: AI21\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# AI21Embeddings\n", - "\n", - ":::caution This service is deprecated. :::\n", - "\n", - "This will help you get started with AI21 embedding models using LangChain. For detailed documentation on `AI21Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ai21/embeddings/langchain_ai21.embeddings.AI21Embeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access AI21 embedding models you'll need to create an AI21 account, get an API key, and install the `langchain-ai21` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [https://docs.ai21.com/](https://docs.ai21.com/) to sign up to AI21 and generate an API key. Once you've done this set the `AI21_API_KEY` environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"AI21_API_KEY\"):\n", - " os.environ[\"AI21_API_KEY\"] = getpass.getpass(\"Enter your AI21 API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain AI21 integration lives in the `langchain-ai21` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-ai21" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_ai21 import AI21Embeddings\n", - "\n", - "embeddings = AI21Embeddings(\n", - " # Can optionally increase or decrease the batch_size\n", - " # to improve latency.\n", - " # Use larger batch sizes with smaller documents, and\n", - " # smaller batch sizes with larger documents.\n", - " # batch_size=256,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "d817716b", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: AI21\n", + "---" ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.01913362182676792, 0.004960147198289633, -0.01582135073840618, -0.042474791407585144, 0.040200788\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# AI21Embeddings\n", + "\n", + ":::caution This service is deprecated. :::\n", + "\n", + "This will help you get started with AI21 embedding models using LangChain. For detailed documentation on `AI21Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ai21/embeddings/langchain_ai21.embeddings.AI21Embeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access AI21 embedding models you'll need to create an AI21 account, get an API key, and install the `langchain-ai21` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [https://docs.ai21.com/](https://docs.ai21.com/) to sign up to AI21 and generate an API key. Once you've done this set the `AI21_API_KEY` environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.03029559925198555, 0.002908500377088785, -0.02700909972190857, -0.04616579785943031, 0.0382771529\n", - "[0.018214847892522812, 0.011460083536803722, -0.03329407051205635, -0.04951060563325882, 0.032756105\n" - ] + "cell_type": "code", + "execution_count": 2, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"AI21_API_KEY\"):\n", + " os.environ[\"AI21_API_KEY\"] = getpass.getpass(\"Enter your AI21 API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain AI21 integration lives in the `langchain-ai21` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-ai21" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_ai21 import AI21Embeddings\n", + "\n", + "embeddings = AI21Embeddings(\n", + " # Can optionally increase or decrease the batch_size\n", + " # to improve latency.\n", + " # Use larger batch sizes with smaller documents, and\n", + " # smaller batch sizes with larger documents.\n", + " # batch_size=256,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.01913362182676792, 0.004960147198289633, -0.01582135073840618, -0.042474791407585144, 0.040200788\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.03029559925198555, 0.002908500377088785, -0.02700909972190857, -0.04616579785943031, 0.0382771529\n", + "[0.018214847892522812, 0.011460083536803722, -0.03329407051205635, -0.04951060563325882, 0.032756105\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `AI21Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ai21/embeddings/langchain_ai21.embeddings.AI21Embeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `AI21Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ai21/embeddings/langchain_ai21.embeddings.AI21Embeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.6" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/azureopenai.ipynb b/docs/docs/integrations/text_embedding/azureopenai.ipynb index d94846038aa..d484b2883a8 100644 --- a/docs/docs/integrations/text_embedding/azureopenai.ipynb +++ b/docs/docs/integrations/text_embedding/azureopenai.ipynb @@ -1,280 +1,278 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: AzureOpenAI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# AzureOpenAIEmbeddings\n", - "\n", - "This will help you get started with AzureOpenAI embedding models using LangChain. For detailed documentation on `AzureOpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.azure.AzureOpenAIEmbeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access AzureOpenAI embedding models you'll need to create an Azure account, get an API key, and install the `langchain-openai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "You’ll need to have an Azure OpenAI instance deployed. You can deploy a version on Azure Portal following this [guide](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal).\n", - "\n", - "Once you have your instance running, make sure you have the name of your instance and key. You can find the key in the Azure Portal, under the “Keys and Endpoint” section of your instance.\n", - "\n", - "```bash\n", - "AZURE_OPENAI_ENDPOINT=\n", - "AZURE_OPENAI_API_KEY=\n", - "AZURE_OPENAI_API_VERSION=\"2024-02-01\"\n", - "```" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"AZURE_OPENAI_API_KEY\"):\n", - " os.environ[\"AZURE_OPENAI_API_KEY\"] = getpass.getpass(\n", - " \"Enter your AzureOpenAI API key: \"\n", - " )" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain AzureOpenAI integration lives in the `langchain-openai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-openai" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_openai import AzureOpenAIEmbeddings\n", - "\n", - "embeddings = AzureOpenAIEmbeddings(\n", - " model=\"text-embedding-3-large\",\n", - " # dimensions: Optional[int] = None, # Can specify dimensions with new text-embedding-3 models\n", - " # azure_endpoint=\"https://.openai.azure.com/\", If not provided, will read env variable AZURE_OPENAI_ENDPOINT\n", - " # api_key=... # Can provide an API key directly. If missing read env variable AZURE_OPENAI_API_KEY\n", - " # openai_api_version=..., # If not provided, will read env variable AZURE_OPENAI_API_VERSION\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d817716b", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: AzureOpenAI\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.0011676070280373096, 0.007125577889382839, -0.014674457721412182, -0.034061674028635025, 0.01128\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# AzureOpenAIEmbeddings\n", + "\n", + "This will help you get started with AzureOpenAI embedding models using LangChain. For detailed documentation on `AzureOpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.azure.AzureOpenAIEmbeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access AzureOpenAI embedding models you'll need to create an Azure account, get an API key, and install the `langchain-openai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "You’ll need to have an Azure OpenAI instance deployed. You can deploy a version on Azure Portal following this [guide](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal).\n", + "\n", + "Once you have your instance running, make sure you have the name of your instance and key. You can find the key in the Azure Portal, under the “Keys and Endpoint” section of your instance.\n", + "\n", + "```bash\n", + "AZURE_OPENAI_ENDPOINT=\n", + "AZURE_OPENAI_API_KEY=\n", + "AZURE_OPENAI_API_VERSION=\"2024-02-01\"\n", + "```" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.0011966148158535361, 0.007160289213061333, -0.014659193344414234, -0.03403077274560928, 0.011280\n", - "[-0.005595256108790636, 0.016757294535636902, -0.011055258102715015, -0.031094247475266457, -0.00363\n" - ] + "cell_type": "code", + "execution_count": null, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"AZURE_OPENAI_API_KEY\"):\n", + " os.environ[\"AZURE_OPENAI_API_KEY\"] = getpass.getpass(\n", + " \"Enter your AzureOpenAI API key: \"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain AzureOpenAI integration lives in the `langchain-openai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-openai" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_openai import AzureOpenAIEmbeddings\n", + "\n", + "embeddings = AzureOpenAIEmbeddings(\n", + " model=\"text-embedding-3-large\",\n", + " # dimensions: Optional[int] = None, # Can specify dimensions with new text-embedding-3 models\n", + " # azure_endpoint=\"https://.openai.azure.com/\", If not provided, will read env variable AZURE_OPENAI_ENDPOINT\n", + " # api_key=... # Can provide an API key directly. If missing read env variable AZURE_OPENAI_API_KEY\n", + " # openai_api_version=..., # If not provided, will read env variable AZURE_OPENAI_API_VERSION\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.0011676070280373096, 0.007125577889382839, -0.014674457721412182, -0.034061674028635025, 0.01128\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.0011966148158535361, 0.007160289213061333, -0.014659193344414234, -0.03403077274560928, 0.011280\n", + "[-0.005595256108790636, 0.016757294535636902, -0.011055258102715015, -0.031094247475266457, -0.00363\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `AzureOpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.azure.AzureOpenAIEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `AzureOpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.azure.AzureOpenAIEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.6" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/cohere.ipynb b/docs/docs/integrations/text_embedding/cohere.ipynb index b1334f0ec40..7d72399976e 100644 --- a/docs/docs/integrations/text_embedding/cohere.ipynb +++ b/docs/docs/integrations/text_embedding/cohere.ipynb @@ -1,267 +1,265 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Cohere\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# CohereEmbeddings\n", - "\n", - "This will help you get started with Cohere embedding models using LangChain. For detailed documentation on `CohereEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/cohere/embeddings/langchain_cohere.embeddings.CohereEmbeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access Cohere embedding models you'll need to create a/an Cohere account, get an API key, and install the `langchain-cohere` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "\n", - "Head to [cohere.com](https://cohere.com) to sign up to Cohere and generate an API key. Once you’ve done this set the COHERE_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"COHERE_API_KEY\"):\n", - " os.environ[\"COHERE_API_KEY\"] = getpass.getpass(\"Enter your Cohere API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Cohere integration lives in the `langchain-cohere` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-cohere" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_cohere import CohereEmbeddings\n", - "\n", - "embeddings = CohereEmbeddings(\n", - " model=\"embed-english-v3.0\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "d817716b", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Cohere\n", + "---" ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.022979736, -0.030212402, -0.08886719, -0.08569336, 0.007030487, -0.0010671616, -0.033813477, 0.0\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# CohereEmbeddings\n", + "\n", + "This will help you get started with Cohere embedding models using LangChain. For detailed documentation on `CohereEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/cohere/embeddings/langchain_cohere.embeddings.CohereEmbeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access Cohere embedding models you'll need to create a/an Cohere account, get an API key, and install the `langchain-cohere` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "\n", + "Head to [cohere.com](https://cohere.com) to sign up to Cohere and generate an API key. Once you’ve done this set the COHERE_API_KEY environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.028869629, -0.030410767, -0.099121094, -0.07116699, -0.012748718, -0.0059432983, -0.04360962, 0.\n", - "[-0.047332764, -0.049957275, -0.07458496, -0.034332275, -0.057922363, -0.0112838745, -0.06994629, 0.\n" - ] + "cell_type": "code", + "execution_count": 8, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"COHERE_API_KEY\"):\n", + " os.environ[\"COHERE_API_KEY\"] = getpass.getpass(\"Enter your Cohere API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Cohere integration lives in the `langchain-cohere` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-cohere" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_cohere import CohereEmbeddings\n", + "\n", + "embeddings = CohereEmbeddings(\n", + " model=\"embed-english-v3.0\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.022979736, -0.030212402, -0.08886719, -0.08569336, 0.007030487, -0.0010671616, -0.033813477, 0.0\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.028869629, -0.030410767, -0.099121094, -0.07116699, -0.012748718, -0.0059432983, -0.04360962, 0.\n", + "[-0.047332764, -0.049957275, -0.07458496, -0.034332275, -0.057922363, -0.0112838745, -0.06994629, 0.\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `CohereEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/cohere/embeddings/langchain_cohere.embeddings.CohereEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `CohereEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/cohere/embeddings/langchain_cohere.embeddings.CohereEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/fireworks.ipynb b/docs/docs/integrations/text_embedding/fireworks.ipynb index 5a593ebac52..815bc9fa57d 100644 --- a/docs/docs/integrations/text_embedding/fireworks.ipynb +++ b/docs/docs/integrations/text_embedding/fireworks.ipynb @@ -1,267 +1,265 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Fireworks\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# FireworksEmbeddings\n", - "\n", - "This will help you get started with Fireworks embedding models using LangChain. For detailed documentation on `FireworksEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/fireworks/embeddings/langchain_fireworks.embeddings.FireworksEmbeddings.html).\n", - "\n", - "## Overview\n", - "\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access Fireworks embedding models you'll need to create a Fireworks account, get an API key, and install the `langchain-fireworks` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [fireworks.ai](https://fireworks.ai/) to sign up to Fireworks and generate an API key. Once you’ve done this set the FIREWORKS_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"FIREWORKS_API_KEY\"):\n", - " os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Enter your Fireworks API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Fireworks integration lives in the `langchain-fireworks` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-fireworks" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_fireworks import FireworksEmbeddings\n", - "\n", - "embeddings = FireworksEmbeddings(\n", - " model=\"nomic-ai/nomic-embed-text-v1.5\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d817716b", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Fireworks\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.01666259765625, 0.011688232421875, -0.1181640625, -0.10205078125, 0.05438232421875, -0.0890502929\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# FireworksEmbeddings\n", + "\n", + "This will help you get started with Fireworks embedding models using LangChain. For detailed documentation on `FireworksEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/fireworks/embeddings/langchain_fireworks.embeddings.FireworksEmbeddings.html).\n", + "\n", + "## Overview\n", + "\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access Fireworks embedding models you'll need to create a Fireworks account, get an API key, and install the `langchain-fireworks` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [fireworks.ai](https://fireworks.ai/) to sign up to Fireworks and generate an API key. Once you’ve done this set the FIREWORKS_API_KEY environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.016632080078125, 0.01165008544921875, -0.1181640625, -0.10186767578125, 0.05438232421875, -0.0890\n", - "[-0.02667236328125, 0.036651611328125, -0.1630859375, -0.0904541015625, -0.022430419921875, -0.09545\n" - ] + "cell_type": "code", + "execution_count": 1, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"FIREWORKS_API_KEY\"):\n", + " os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Enter your Fireworks API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Fireworks integration lives in the `langchain-fireworks` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-fireworks" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_fireworks import FireworksEmbeddings\n", + "\n", + "embeddings = FireworksEmbeddings(\n", + " model=\"nomic-ai/nomic-embed-text-v1.5\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.01666259765625, 0.011688232421875, -0.1181640625, -0.10205078125, 0.05438232421875, -0.0890502929\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.016632080078125, 0.01165008544921875, -0.1181640625, -0.10186767578125, 0.05438232421875, -0.0890\n", + "[-0.02667236328125, 0.036651611328125, -0.1630859375, -0.0904541015625, -0.022430419921875, -0.09545\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "3fba556a-b53d-431c-b0c6-ffb1e2fa5a6e", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation of all `FireworksEmbeddings` features and configurations head to the [API reference](https://python.langchain.com/api_reference/fireworks/embeddings/langchain_fireworks.embeddings.FireworksEmbeddings.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "3fba556a-b53d-431c-b0c6-ffb1e2fa5a6e", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation of all `FireworksEmbeddings` features and configurations head to the [API reference](https://python.langchain.com/api_reference/fireworks/embeddings/langchain_fireworks.embeddings.FireworksEmbeddings.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/google_vertex_ai_palm.ipynb b/docs/docs/integrations/text_embedding/google_vertex_ai_palm.ipynb index 6cabee7a1d3..ac90af8488c 100644 --- a/docs/docs/integrations/text_embedding/google_vertex_ai_palm.ipynb +++ b/docs/docs/integrations/text_embedding/google_vertex_ai_palm.ipynb @@ -1,323 +1,321 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Google Vertex AI \n", - "keywords: [Vertex AI, vertexai , Google Cloud, embeddings]\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# Google Vertex AI Embeddings \n", - "\n", - "This will help you get started with Google Vertex AI Embeddings models using LangChain. For detailed documentation on `Google Vertex AI Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/google_vertexai/embeddings/langchain_google_vertexai.embeddings.VertexAIEmbeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Provider | Package |\n", - "|:--------:|:-------:|\n", - "| [Google](https://python.langchain.com/docs/integrations/providers/google/) | [langchain-google-vertexai](https://python.langchain.com/api_reference/google_vertexai/embeddings/langchain_google_vertexai.embeddings.VertexAIEmbeddings.html) |\n", - "\n", - "## Setup\n", - "\n", - "To access Google Vertex AI Embeddings models you'll need to \n", - "- Create a Google Cloud account \n", - "- Install the `langchain-google-vertexai` integration package.\n", - "\n", - "\n", - "\n", - "\n", - "### Credentials\n", - "\n", - "\n", - "Head to [Google Cloud](https://cloud.google.com/free/) to sign up to create an account. Once you've done this set the GOOGLE_APPLICATION_CREDENTIALS environment variable:\n", - "\n", - "For more information, see:\n", - "\n", - "https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n", - "https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth" - ] - }, - { - "cell_type": "markdown", - "id": "caaba519-3476-423b-a5e4-d99a10929506", - "metadata": {}, - "source": [ - "**OPTIONAL : Authenticate your notebook environment (Colab only)**\n", - "\n", - "If you're running this notebook on Google Colab, run the cell below to authenticate your environment." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b0770000-3667-439b-8c46-acc5af7c8e40", - "metadata": {}, - "outputs": [], - "source": [ - "import sys\n", - "\n", - "if \"google.colab\" in sys.modules:\n", - " from google.colab import auth\n", - "\n", - " auth.authenticate_user()" - ] - }, - { - "cell_type": "markdown", - "id": "9fbd4a33-2480-4ad1-8d56-aec730b3662b", - "metadata": {}, - "source": [ - "**Set Google Cloud project information and initialize Vertex AI SDK**\n", - "\n", - "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", - "\n", - "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n", - "LOCATION = \"us-central1\" # @param {type:\"string\"}\n", - "\n", - "import vertexai\n", - "\n", - "vertexai.init(project=PROJECT_ID, location=LOCATION)" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Google Vertex AI Embeddings integration lives in the `langchain-google-vertexai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-google-vertexai" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate embeddings:\n", - ">Check the list of [Supported Models](https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings#supported-models)" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_google_vertexai import VertexAIEmbeddings\n", - "\n", - "# Initialize the a specific Embeddings Model version\n", - "embeddings = VertexAIEmbeddings(model_name=\"text-embedding-004\")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "d817716b", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Google Vertex AI\n", + "keywords: [Vertex AI, vertexai , Google Cloud, embeddings]\n", + "---" ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.02831101417541504, 0.022063178941607475, -0.07454229146242142, 0.006448323838412762, 0.001955120\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# Google Vertex AI Embeddings\n", + "\n", + "This will help you get started with Google Vertex AI Embeddings models using LangChain. For detailed documentation on `Google Vertex AI Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/google_vertexai/embeddings/langchain_google_vertexai.embeddings.VertexAIEmbeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Provider | Package |\n", + "|:--------:|:-------:|\n", + "| [Google](https://python.langchain.com/docs/integrations/providers/google/) | [langchain-google-vertexai](https://python.langchain.com/api_reference/google_vertexai/embeddings/langchain_google_vertexai.embeddings.VertexAIEmbeddings.html) |\n", + "\n", + "## Setup\n", + "\n", + "To access Google Vertex AI Embeddings models you'll need to\n", + "- Create a Google Cloud account\n", + "- Install the `langchain-google-vertexai` integration package.\n", + "\n", + "\n", + "\n", + "\n", + "### Credentials\n", + "\n", + "\n", + "Head to [Google Cloud](https://cloud.google.com/free/) to sign up to create an account. Once you've done this set the GOOGLE_APPLICATION_CREDENTIALS environment variable:\n", + "\n", + "For more information, see:\n", + "\n", + "https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n", + "https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.01092718355357647, 0.01213780976831913, -0.05650627985596657, 0.006737854331731796, 0.0085973171\n", - "[0.010135706514120102, 0.01234869472682476, -0.07284046709537506, 0.00027134662377648056, 0.01546290\n" - ] + "cell_type": "markdown", + "id": "caaba519-3476-423b-a5e4-d99a10929506", + "metadata": {}, + "source": [ + "**OPTIONAL : Authenticate your notebook environment (Colab only)**\n", + "\n", + "If you're running this notebook on Google Colab, run the cell below to authenticate your environment." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b0770000-3667-439b-8c46-acc5af7c8e40", + "metadata": {}, + "outputs": [], + "source": [ + "import sys\n", + "\n", + "if \"google.colab\" in sys.modules:\n", + " from google.colab import auth\n", + "\n", + " auth.authenticate_user()" + ] + }, + { + "cell_type": "markdown", + "id": "9fbd4a33-2480-4ad1-8d56-aec730b3662b", + "metadata": {}, + "source": [ + "**Set Google Cloud project information and initialize Vertex AI SDK**\n", + "\n", + "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", + "\n", + "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n", + "LOCATION = \"us-central1\" # @param {type:\"string\"}\n", + "\n", + "import vertexai\n", + "\n", + "vertexai.init(project=PROJECT_ID, location=LOCATION)" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Google Vertex AI Embeddings integration lives in the `langchain-google-vertexai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-google-vertexai" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate embeddings:\n", + ">Check the list of [Supported Models](https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings#supported-models)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_google_vertexai import VertexAIEmbeddings\n", + "\n", + "# Initialize the a specific Embeddings Model version\n", + "embeddings = VertexAIEmbeddings(model_name=\"text-embedding-004\")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.02831101417541504, 0.022063178941607475, -0.07454229146242142, 0.006448323838412762, 0.001955120\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.01092718355357647, 0.01213780976831913, -0.05650627985596657, 0.006737854331731796, 0.0085973171\n", + "[0.010135706514120102, 0.01234869472682476, -0.07284046709537506, 0.00027134662377648056, 0.01546290\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `Google Vertex AI Embeddings\n", + "` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/google_vertexai/embeddings/langchain_google_vertexai.embeddings.VertexAIEmbeddings.html).\n" + ] + } + ], + "metadata": { + "environment": { + "kernel": "python310", + "name": "tf2-gpu.2-6.m104", + "type": "gcloud", + "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-6:m104" + }, + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.6" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `Google Vertex AI Embeddings\n", - "` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/google_vertexai/embeddings/langchain_google_vertexai.embeddings.VertexAIEmbeddings.html).\n" - ] - } - ], - "metadata": { - "environment": { - "kernel": "python310", - "name": "tf2-gpu.2-6.m104", - "type": "gcloud", - "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-6:m104" - }, - "kernelspec": { - "display_name": ".venv", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.6" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/mistralai.ipynb b/docs/docs/integrations/text_embedding/mistralai.ipynb index af008b75d14..56efd00bc19 100644 --- a/docs/docs/integrations/text_embedding/mistralai.ipynb +++ b/docs/docs/integrations/text_embedding/mistralai.ipynb @@ -1,266 +1,264 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: MistralAI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# MistralAIEmbeddings\n", - "\n", - "This will help you get started with MistralAI embedding models using LangChain. For detailed documentation on `MistralAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/mistralai/embeddings/langchain_mistralai.embeddings.MistralAIEmbeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access MistralAI embedding models you'll need to create a/an MistralAI account, get an API key, and install the `langchain-mistralai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [https://console.mistral.ai/](https://console.mistral.ai/) to sign up to MistralAI and generate an API key. Once you've done this set the MISTRALAI_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"MISTRALAI_API_KEY\"):\n", - " os.environ[\"MISTRALAI_API_KEY\"] = getpass.getpass(\"Enter your MistralAI API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain MistralAI integration lives in the `langchain-mistralai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-mistralai" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_mistralai import MistralAIEmbeddings\n", - "\n", - "embeddings = MistralAIEmbeddings(\n", - " model=\"mistral-embed\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d817716b", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: MistralAI\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.04443359375, 0.01885986328125, 0.018035888671875, -0.00864410400390625, 0.049652099609375, -0.00\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# MistralAIEmbeddings\n", + "\n", + "This will help you get started with MistralAI embedding models using LangChain. For detailed documentation on `MistralAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/mistralai/embeddings/langchain_mistralai.embeddings.MistralAIEmbeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access MistralAI embedding models you'll need to create a/an MistralAI account, get an API key, and install the `langchain-mistralai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [https://console.mistral.ai/](https://console.mistral.ai/) to sign up to MistralAI and generate an API key. Once you've done this set the MISTRALAI_API_KEY environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.04443359375, 0.01885986328125, 0.0180511474609375, -0.0086517333984375, 0.049652099609375, -0.00\n", - "[-0.02032470703125, 0.02606201171875, 0.051605224609375, -0.0281982421875, 0.055755615234375, 0.0019\n" - ] + "cell_type": "code", + "execution_count": 1, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"MISTRALAI_API_KEY\"):\n", + " os.environ[\"MISTRALAI_API_KEY\"] = getpass.getpass(\"Enter your MistralAI API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain MistralAI integration lives in the `langchain-mistralai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-mistralai" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_mistralai import MistralAIEmbeddings\n", + "\n", + "embeddings = MistralAIEmbeddings(\n", + " model=\"mistral-embed\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.04443359375, 0.01885986328125, 0.018035888671875, -0.00864410400390625, 0.049652099609375, -0.00\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.04443359375, 0.01885986328125, 0.0180511474609375, -0.0086517333984375, 0.049652099609375, -0.00\n", + "[-0.02032470703125, 0.02606201171875, 0.051605224609375, -0.0281982421875, 0.055755615234375, 0.0019\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `MistralAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/mistralai/embeddings/langchain_mistralai.embeddings.MistralAIEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `MistralAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/mistralai/embeddings/langchain_mistralai.embeddings.MistralAIEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/nomic.ipynb b/docs/docs/integrations/text_embedding/nomic.ipynb index c06a8d7d428..60142aa0a76 100644 --- a/docs/docs/integrations/text_embedding/nomic.ipynb +++ b/docs/docs/integrations/text_embedding/nomic.ipynb @@ -1,287 +1,285 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Nomic\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# NomicEmbeddings\n", - "\n", - "This will help you get started with Nomic embedding models using LangChain. For detailed documentation on `NomicEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/nomic/embeddings/langchain_nomic.embeddings.NomicEmbeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access Nomic embedding models you'll need to create a/an Nomic account, get an API key, and install the `langchain-nomic` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [https://atlas.nomic.ai/](https://atlas.nomic.ai/) to sign up to Nomic and generate an API key. Once you've done this set the `NOMIC_API_KEY` environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"NOMIC_API_KEY\"):\n", - " os.environ[\"NOMIC_API_KEY\"] = getpass.getpass(\"Enter your Nomic API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Nomic integration lives in the `langchain-nomic` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "64853226", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain-nomic" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_nomic import NomicEmbeddings\n", - "\n", - "embeddings = NomicEmbeddings(\n", - " model=\"nomic-embed-text-v1.5\",\n", - " # dimensionality=256,\n", - " # Nomic's `nomic-embed-text-v1.5` model was [trained with Matryoshka learning](https://blog.nomic.ai/posts/nomic-embed-matryoshka)\n", - " # to enable variable-length embeddings with a single model.\n", - " # This means that you can specify the dimensionality of the embeddings at inference time.\n", - " # The model supports dimensionality from 64 to 768.\n", - " # inference_mode=\"remote\",\n", - " # One of `remote`, `local` (Embed4All), or `dynamic` (automatic). Defaults to `remote`.\n", - " # api_key=... , # if using remote inference,\n", - " # device=\"cpu\",\n", - " # The device to use for local embeddings. Choices include\n", - " # `cpu`, `gpu`, `nvidia`, `amd`, or a specific device name. See\n", - " # the docstring for `GPT4All.__init__` for more info. Typically\n", - " # defaults to CPU. Do not use on macOS.\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d817716b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Nomic\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.024642944, 0.029083252, -0.14013672, -0.09082031, 0.058898926, -0.07489014, -0.0138168335, 0.0037\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# NomicEmbeddings\n", + "\n", + "This will help you get started with Nomic embedding models using LangChain. For detailed documentation on `NomicEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/nomic/embeddings/langchain_nomic.embeddings.NomicEmbeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access Nomic embedding models you'll need to create a/an Nomic account, get an API key, and install the `langchain-nomic` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [https://atlas.nomic.ai/](https://atlas.nomic.ai/) to sign up to Nomic and generate an API key. Once you've done this set the `NOMIC_API_KEY` environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.012771606, 0.023727417, -0.12365723, -0.083740234, 0.06530762, -0.07110596, -0.021896362, -0.0068\n", - "[-0.019058228, 0.04058838, -0.15222168, -0.06842041, -0.012130737, -0.07128906, -0.04534912, 0.00522\n" - ] + "cell_type": "code", + "execution_count": 2, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"NOMIC_API_KEY\"):\n", + " os.environ[\"NOMIC_API_KEY\"] = getpass.getpass(\"Enter your Nomic API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Nomic integration lives in the `langchain-nomic` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "64853226", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain-nomic" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_nomic import NomicEmbeddings\n", + "\n", + "embeddings = NomicEmbeddings(\n", + " model=\"nomic-embed-text-v1.5\",\n", + " # dimensionality=256,\n", + " # Nomic's `nomic-embed-text-v1.5` model was [trained with Matryoshka learning](https://blog.nomic.ai/posts/nomic-embed-matryoshka)\n", + " # to enable variable-length embeddings with a single model.\n", + " # This means that you can specify the dimensionality of the embeddings at inference time.\n", + " # The model supports dimensionality from 64 to 768.\n", + " # inference_mode=\"remote\",\n", + " # One of `remote`, `local` (Embed4All), or `dynamic` (automatic). Defaults to `remote`.\n", + " # api_key=... , # if using remote inference,\n", + " # device=\"cpu\",\n", + " # The device to use for local embeddings. Choices include\n", + " # `cpu`, `gpu`, `nvidia`, `amd`, or a specific device name. See\n", + " # the docstring for `GPT4All.__init__` for more info. Typically\n", + " # defaults to CPU. Do not use on macOS.\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.024642944, 0.029083252, -0.14013672, -0.09082031, 0.058898926, -0.07489014, -0.0138168335, 0.0037\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.012771606, 0.023727417, -0.12365723, -0.083740234, 0.06530762, -0.07110596, -0.021896362, -0.0068\n", + "[-0.019058228, 0.04058838, -0.15222168, -0.06842041, -0.012130737, -0.07128906, -0.04534912, 0.00522\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `NomicEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/nomic/embeddings/langchain_nomic.embeddings.NomicEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `NomicEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/nomic/embeddings/langchain_nomic.embeddings.NomicEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.6" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/ollama.ipynb b/docs/docs/integrations/text_embedding/ollama.ipynb index ddf5c154636..5d93c266ca1 100644 --- a/docs/docs/integrations/text_embedding/ollama.ipynb +++ b/docs/docs/integrations/text_embedding/ollama.ipynb @@ -1,276 +1,274 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Ollama\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# OllamaEmbeddings\n", - "\n", - "This will help you get started with Ollama embedding models using LangChain. For detailed documentation on `OllamaEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ollama/embeddings/langchain_ollama.embeddings.OllamaEmbeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n", - "\n", - "* [Download](https://ollama.ai/download) and install Ollama onto the available supported platforms (including Windows Subsystem for Linux)\n", - "* Fetch available LLM model via `ollama pull `\n", - " * View a list of available models via the [model library](https://ollama.ai/library)\n", - " * e.g., `ollama pull llama3`\n", - "* This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.\n", - "\n", - "> On Mac, the models will be download to `~/.ollama/models`\n", - "> \n", - "> On Linux (or WSL), the models will be stored at `/usr/share/ollama/.ollama/models`\n", - "\n", - "* Specify the exact version of the model of interest as such `ollama pull vicuna:13b-v1.5-16k-q4_0` (View the [various tags for the `Vicuna`](https://ollama.ai/library/vicuna/tags) model in this instance)\n", - "* To view all pulled models, use `ollama list`\n", - "* To chat directly with a model from the command line, use `ollama run `\n", - "* View the [Ollama documentation](https://github.com/jmorganca/ollama) for more commands. Run `ollama help` in the terminal to see available commands too.\n", - "\n", - "\n", - "### Credentials\n", - "\n", - "There is no built-in auth mechanism for Ollama." - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Ollama integration lives in the `langchain-ollama` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "64853226", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain-ollama" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate embeddings:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_ollama import OllamaEmbeddings\n", - "\n", - "embeddings = OllamaEmbeddings(\n", - " model=\"llama3\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "d817716b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Ollama\n", + "---" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.001288981, 0.006547121, 0.018376578, 0.025603496, 0.009599175, -0.0042578303, -0.023250086, -0.0\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# OllamaEmbeddings\n", + "\n", + "This will help you get started with Ollama embedding models using LangChain. For detailed documentation on `OllamaEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ollama/embeddings/langchain_ollama.embeddings.OllamaEmbeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n", + "\n", + "* [Download](https://ollama.ai/download) and install Ollama onto the available supported platforms (including Windows Subsystem for Linux)\n", + "* Fetch available LLM model via `ollama pull `\n", + " * View a list of available models via the [model library](https://ollama.ai/library)\n", + " * e.g., `ollama pull llama3`\n", + "* This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.\n", + "\n", + "> On Mac, the models will be download to `~/.ollama/models`\n", + ">\n", + "> On Linux (or WSL), the models will be stored at `/usr/share/ollama/.ollama/models`\n", + "\n", + "* Specify the exact version of the model of interest as such `ollama pull vicuna:13b-v1.5-16k-q4_0` (View the [various tags for the `Vicuna`](https://ollama.ai/library/vicuna/tags) model in this instance)\n", + "* To view all pulled models, use `ollama list`\n", + "* To chat directly with a model from the command line, use `ollama run `\n", + "* View the [Ollama documentation](https://github.com/jmorganca/ollama) for more commands. Run `ollama help` in the terminal to see available commands too.\n", + "\n", + "\n", + "### Credentials\n", + "\n", + "There is no built-in auth mechanism for Ollama." + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.0013138362, 0.006438795, 0.018304596, 0.025530428, 0.009717592, -0.004225636, -0.023363983, -0.0\n", - "[-0.010317663, 0.01632489, 0.0070348927, 0.017076202, 0.008924255, 0.007399284, -0.023064945, -0.003\n" - ] + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Ollama integration lives in the `langchain-ollama` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "64853226", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain-ollama" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate embeddings:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_ollama import OllamaEmbeddings\n", + "\n", + "embeddings = OllamaEmbeddings(\n", + " model=\"llama3\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.001288981, 0.006547121, 0.018376578, 0.025603496, 0.009599175, -0.0042578303, -0.023250086, -0.0\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.0013138362, 0.006438795, 0.018304596, 0.025530428, 0.009717592, -0.004225636, -0.023363983, -0.0\n", + "[-0.010317663, 0.01632489, 0.0070348927, 0.017076202, 0.008924255, 0.007399284, -0.023064945, -0.003\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `OllamaEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ollama/embeddings/langchain_ollama.embeddings.OllamaEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `OllamaEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/ollama/embeddings/langchain_ollama.embeddings.OllamaEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.6" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/openai.ipynb b/docs/docs/integrations/text_embedding/openai.ipynb index b61440cd344..61b964ce308 100644 --- a/docs/docs/integrations/text_embedding/openai.ipynb +++ b/docs/docs/integrations/text_embedding/openai.ipynb @@ -1,272 +1,270 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: OpenAI\n", - "keywords: [openaiembeddings]\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# OpenAIEmbeddings\n", - "\n", - "This will help you get started with OpenAI embedding models using LangChain. For detailed documentation on `OpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.base.OpenAIEmbeddings.html).\n", - "\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access OpenAI embedding models you'll need to create a/an OpenAI account, get an API key, and install the `langchain-openai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [platform.openai.com](https://platform.openai.com) to sign up to OpenAI and generate an API key. Once you’ve done this set the OPENAI_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"OPENAI_API_KEY\"):\n", - " os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain OpenAI integration lives in the `langchain-openai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-openai" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_openai import OpenAIEmbeddings\n", - "\n", - "embeddings = OpenAIEmbeddings(\n", - " model=\"text-embedding-3-large\",\n", - " # With the `text-embedding-3` class\n", - " # of models, you can specify the size\n", - " # of the embeddings you want returned.\n", - " # dimensions=1024\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "d817716b", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: OpenAI\n", + "keywords: [openaiembeddings]\n", + "---" ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.019276829436421394, 0.0037708976306021214, -0.03294256329536438, 0.0037671267054975033, 0.008175\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# OpenAIEmbeddings\n", + "\n", + "This will help you get started with OpenAI embedding models using LangChain. For detailed documentation on `OpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.base.OpenAIEmbeddings.html).\n", + "\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access OpenAI embedding models you'll need to create a/an OpenAI account, get an API key, and install the `langchain-openai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [platform.openai.com](https://platform.openai.com) to sign up to OpenAI and generate an API key. Once you’ve done this set the OPENAI_API_KEY environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.019260549917817116, 0.0037612367887049913, -0.03291035071015358, 0.003757466096431017, 0.0082049\n", - "[-0.010181212797760963, 0.023419594392180443, -0.04215526953339577, -0.001532090245746076, -0.023573\n" - ] + "cell_type": "code", + "execution_count": 6, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"OPENAI_API_KEY\"):\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain OpenAI integration lives in the `langchain-openai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-openai" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_openai import OpenAIEmbeddings\n", + "\n", + "embeddings = OpenAIEmbeddings(\n", + " model=\"text-embedding-3-large\",\n", + " # With the `text-embedding-3` class\n", + " # of models, you can specify the size\n", + " # of the embeddings you want returned.\n", + " # dimensions=1024\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.019276829436421394, 0.0037708976306021214, -0.03294256329536438, 0.0037671267054975033, 0.008175\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.019260549917817116, 0.0037612367887049913, -0.03291035071015358, 0.003757466096431017, 0.0082049\n", + "[-0.010181212797760963, 0.023419594392180443, -0.04215526953339577, -0.001532090245746076, -0.023573\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `OpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.base.OpenAIEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `OpenAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/openai/embeddings/langchain_openai.embeddings.base.OpenAIEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/together.ipynb b/docs/docs/integrations/text_embedding/together.ipynb index 5cb07b6f852..5d2867c69cd 100644 --- a/docs/docs/integrations/text_embedding/together.ipynb +++ b/docs/docs/integrations/text_embedding/together.ipynb @@ -1,277 +1,275 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Together AI\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# TogetherEmbeddings\n", - "\n", - "This will help you get started with Together embedding models using LangChain. For detailed documentation on `TogetherEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/together/embeddings/langchain_together.embeddings.TogetherEmbeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "import { ItemTable } from \"@theme/FeatureTables\";\n", - "\n", - "\n", - "\n", - "## Setup\n", - "\n", - "To access Together embedding models you'll need to create a/an Together account, get an API key, and install the `langchain-together` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [https://api.together.xyz/](https://api.together.xyz/) to sign up to Together and generate an API key. Once you've done this set the TOGETHER_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"TOGETHER_API_KEY\"):\n", - " os.environ[\"TOGETHER_API_KEY\"] = getpass.getpass(\"Enter your Together API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain Together integration lives in the `langchain-together` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "64853226", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n", - "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython -m pip install --upgrade pip\u001b[0m\n", - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain-together" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_together import TogetherEmbeddings\n", - "\n", - "embeddings = TogetherEmbeddings(\n", - " model=\"togethercomputer/m2-bert-80M-8k-retrieval\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "d817716b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Together AI\n", + "---" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.3812227, -0.052848946, -0.10564975, 0.03480297, 0.2878488, 0.0084609175, 0.11605915, 0.05303011, \n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# TogetherEmbeddings\n", + "\n", + "This will help you get started with Together embedding models using LangChain. For detailed documentation on `TogetherEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/together/embeddings/langchain_together.embeddings.TogetherEmbeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "import { ItemTable } from \"@theme/FeatureTables\";\n", + "\n", + "\n", + "\n", + "## Setup\n", + "\n", + "To access Together embedding models you'll need to create a/an Together account, get an API key, and install the `langchain-together` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [https://api.together.xyz/](https://api.together.xyz/) to sign up to Together and generate an API key. Once you've done this set the TOGETHER_API_KEY environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[0.3812227, -0.052848946, -0.10564975, 0.03480297, 0.2878488, 0.0084609175, 0.11605915, 0.05303011, \n", - "[0.066308185, -0.032866564, 0.115751594, 0.19082588, 0.14017, -0.26976448, -0.056340694, -0.26923394\n" - ] + "cell_type": "code", + "execution_count": 1, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"TOGETHER_API_KEY\"):\n", + " os.environ[\"TOGETHER_API_KEY\"] = getpass.getpass(\"Enter your Together API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain Together integration lives in the `langchain-together` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "64853226", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython -m pip install --upgrade pip\u001b[0m\n", + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain-together" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_together import TogetherEmbeddings\n", + "\n", + "embeddings = TogetherEmbeddings(\n", + " model=\"togethercomputer/m2-bert-80M-8k-retrieval\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.3812227, -0.052848946, -0.10564975, 0.03480297, 0.2878488, 0.0084609175, 0.11605915, 0.05303011, \n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.3812227, -0.052848946, -0.10564975, 0.03480297, 0.2878488, 0.0084609175, 0.11605915, 0.05303011, \n", + "[0.066308185, -0.032866564, 0.115751594, 0.19082588, 0.14017, -0.26976448, -0.056340694, -0.26923394\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `TogetherEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/together/embeddings/langchain_together.embeddings.TogetherEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `TogetherEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/together/embeddings/langchain_together.embeddings.TogetherEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/text_embedding/zhipuai.ipynb b/docs/docs/integrations/text_embedding/zhipuai.ipynb index 5c4d9235bb3..39235d4ae8f 100644 --- a/docs/docs/integrations/text_embedding/zhipuai.ipynb +++ b/docs/docs/integrations/text_embedding/zhipuai.ipynb @@ -1,279 +1,277 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: ZhipuAI\n", - "keywords: [zhipuaiembeddings]\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# ZhipuAIEmbeddings\n", - "\n", - "This will help you get started with ZhipuAI embedding models using LangChain. For detailed documentation on `ZhipuAIEmbeddings` features and configuration options, please refer to the [API reference](https://bigmodel.cn/dev/api#vector).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Provider | Package |\n", - "|:--------:|:-------:|\n", - "| [ZhipuAI](/docs/integrations/providers/zhipuai/) | [langchain-community](https://python.langchain.com/api_reference/community/embeddings/langchain_community.embeddings.zhipuai.ZhipuAIEmbeddings.html) |\n", - "\n", - "## Setup\n", - "\n", - "To access ZhipuAI embedding models you'll need to create a/an ZhipuAI account, get an API key, and install the `zhipuai` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "Head to [https://bigmodel.cn/](https://bigmodel.cn/usercenter/apikeys) to sign up to ZhipuAI and generate an API key. Once you've done this set the ZHIPUAI_API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"ZHIPUAI_API_KEY\"):\n", - " os.environ[\"ZHIPUAI_API_KEY\"] = getpass.getpass(\"Enter your ZhipuAI API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain ZhipuAI integration lives in the `zhipuai` package:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "64853226", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU zhipuai" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.embeddings import ZhipuAIEmbeddings\n", - "\n", - "embeddings = ZhipuAIEmbeddings(\n", - " model=\"embedding-3\",\n", - " # With the `embedding-3` class\n", - " # of models, you can specify the size\n", - " # of the embeddings you want returned.\n", - " # dimensions=1024\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "d817716b", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'LangChain is the framework for building context-aware reasoning applications'" + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: ZhipuAI\n", + "keywords: [zhipuaiembeddings]\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "0d2befcd", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.022979736, 0.007785797, 0.04598999, 0.012741089, -0.01689148, 0.008277893, 0.016464233, 0.009246\n" - ] - } - ], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# ZhipuAIEmbeddings\n", + "\n", + "This will help you get started with ZhipuAI embedding models using LangChain. For detailed documentation on `ZhipuAIEmbeddings` features and configuration options, please refer to the [API reference](https://bigmodel.cn/dev/api#vector).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Provider | Package |\n", + "|:--------:|:-------:|\n", + "| [ZhipuAI](/docs/integrations/providers/zhipuai/) | [langchain-community](https://python.langchain.com/api_reference/community/embeddings/langchain_community.embeddings.zhipuai.ZhipuAIEmbeddings.html) |\n", + "\n", + "## Setup\n", + "\n", + "To access ZhipuAI embedding models you'll need to create a/an ZhipuAI account, get an API key, and install the `zhipuai` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "Head to [https://bigmodel.cn/](https://bigmodel.cn/usercenter/apikeys) to sign up to ZhipuAI and generate an API key. Once you've done this set the ZHIPUAI_API_KEY environment variable:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "[-0.022979736, 0.007785797, 0.04598999, 0.012741089, -0.01689148, 0.008277893, 0.016464233, 0.009246\n", - "[-0.02330017, -0.013916016, 0.00022411346, 0.017196655, -0.034240723, 0.011131287, 0.011497498, -0.0\n" - ] + "cell_type": "code", + "execution_count": 1, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"ZHIPUAI_API_KEY\"):\n", + " os.environ[\"ZHIPUAI_API_KEY\"] = getpass.getpass(\"Enter your ZhipuAI API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain ZhipuAI integration lives in the `zhipuai` package:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "64853226", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU zhipuai" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.embeddings import ZhipuAIEmbeddings\n", + "\n", + "embeddings = ZhipuAIEmbeddings(\n", + " model=\"embedding-3\",\n", + " # With the `embedding-3` class\n", + " # of models, you can specify the size\n", + " # of the embeddings you want returned.\n", + " # dimensions=1024\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'LangChain is the framework for building context-aware reasoning applications'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.022979736, 0.007785797, 0.04598999, 0.012741089, -0.01689148, 0.008277893, 0.016464233, 0.009246\n" + ] + } + ], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[-0.022979736, 0.007785797, 0.04598999, 0.012741089, -0.01689148, 0.008277893, 0.016464233, 0.009246\n", + "[-0.02330017, -0.013916016, 0.00022411346, 0.017196655, -0.034240723, 0.011131287, 0.011497498, -0.0\n" + ] + } + ], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `ZhipuAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/community/embeddings/langchain_community.embeddings.zhipuai.ZhipuAIEmbeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.3" } - ], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `ZhipuAIEmbeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/api_reference/community/embeddings/langchain_community.embeddings.zhipuai.ZhipuAIEmbeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.12.3" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/tools/cdp_agentkit.ipynb b/docs/docs/integrations/tools/cdp_agentkit.ipynb index 738074116a3..9b07988c0e4 100644 --- a/docs/docs/integrations/tools/cdp_agentkit.ipynb +++ b/docs/docs/integrations/tools/cdp_agentkit.ipynb @@ -1,332 +1,330 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: CDP\n", - "---" - ] + "cells": [ + { + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: CDP\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# CDP Agentkit Toolkit\n", + "\n", + "The `CDP Agentkit` toolkit contains tools that enable an LLM agent to interact with the [Coinbase Developer Platform](https://docs.cdp.coinbase.com/). The toolkit provides a wrapper around the CDP SDK, allowing agents to perform onchain operations like transfers, trades, and smart contract interactions.\n", + "\n", + "## Overview\n", + "\n", + "### Integration details\n", + "\n", + "| Class | Package | Serializable | JS support | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| CdpToolkit | `cdp-langchain` | ❌ | ❌ | ![PyPI - Version](https://img.shields.io/pypi/v/cdp-langchain?style=flat-square&label=%20) |\n", + "\n", + "### Tool features\n", + "\n", + "The toolkit provides the following tools:\n", + "\n", + "1. **get_wallet_details** - Get details about the MPC Wallet\n", + "2. **get_balance** - Get balance for specific assets\n", + "3. **request_faucet_funds** - Request test tokens from faucet\n", + "4. **transfer** - Transfer assets between addresses\n", + "5. **trade** - Trade assets (Mainnet only)\n", + "6. **deploy_token** - Deploy ERC-20 token contracts\n", + "7. **mint_nft** - Mint NFTs from existing contracts\n", + "8. **deploy_nft** - Deploy new NFT contracts\n", + "9. **register_basename** - Register a basename for the wallet\n", + "\n", + "We encourage you to add your own tools, both using CDP and web2 APIs, to create an agent that is tailored to your needs.\n", + "\n", + "## Setup\n", + "\n", + "At a high-level, we will:\n", + "\n", + "1. Install the langchain package\n", + "2. Set up your CDP API credentials\n", + "3. Initialize the CDP wrapper and toolkit\n", + "4. Pass the tools to your agent with `toolkit.get_tools()`" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b", + "metadata": {}, + "source": "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "This toolkit lives in the `cdp-langchain` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU cdp-langchain" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65", + "metadata": {}, + "source": [ + "#### Set Environment Variables\n", + "\n", + "To use this toolkit, you must first set the following environment variables to access the [CDP APIs](https://docs.cdp.coinbase.com/mpc-wallet/docs/quickstart) to create wallets and interact onchain. You can sign up for an API key for free on the [CDP Portal](https://cdp.coinbase.com/):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb09c344", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "for env_var in [\n", + " \"CDP_API_KEY_NAME\",\n", + " \"CDP_API_KEY_PRIVATE_KEY\",\n", + "]:\n", + " if not os.getenv(env_var):\n", + " os.environ[env_var] = getpass.getpass(f\"Enter your {env_var}: \")\n", + "\n", + "# Optional: Set network (defaults to base-sepolia)\n", + "os.environ[\"NETWORK_ID\"] = \"base-sepolia\" # or \"base-mainnet\"" + ] + }, + { + "cell_type": "markdown", + "id": "5c5f2839", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our toolkit:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "51a60dbe", + "metadata": {}, + "outputs": [], + "source": [ + "from cdp_langchain.agent_toolkits import CdpToolkit\n", + "from cdp_langchain.utils import CdpAgentkitWrapper\n", + "\n", + "# Initialize CDP wrapper\n", + "cdp = CdpAgentkitWrapper()\n", + "\n", + "# Create toolkit from wrapper\n", + "toolkit = CdpToolkit.from_cdp_agentkit_wrapper(cdp)" + ] + }, + { + "cell_type": "markdown", + "id": "d11245ad", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "View [available tools](#tool-features):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "310bf18e", + "metadata": {}, + "outputs": [], + "source": [ + "tools = toolkit.get_tools()\n", + "for tool in tools:\n", + " print(tool.name)" + ] + }, + { + "cell_type": "markdown", + "id": "23e11cc9", + "metadata": {}, + "source": [ + "## Use within an agent\n", + "\n", + "We will need a LLM or chat model:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d1ee55bc", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca", + "metadata": {}, + "source": [ + "Initialize the agent with the tools:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f8a2c4b1", + "metadata": {}, + "outputs": [], + "source": [ + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "tools = toolkit.get_tools()\n", + "agent_executor = create_react_agent(llm, tools)" + ] + }, + { + "cell_type": "markdown", + "id": "b4a7c9d2", + "metadata": {}, + "source": [ + "Example usage:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c9a8e4f3", + "metadata": {}, + "outputs": [], + "source": [ + "example_query = \"Send 0.005 ETH to john2879.base.eth\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "id": "e5a7c9d4", + "metadata": {}, + "source": [ + "Expected output:\n", + "```\n", + "Transferred 0.005 of eth to john2879.base.eth.\n", + "Transaction hash for the transfer: 0x78c7c2878659a0de216d0764fc87eff0d38b47f3315fa02ba493a83d8e782d1e\n", + "Transaction link for the transfer: https://sepolia.basescan.org/tx/0x78c7c2878659a0de216d0764fc87eff0d38b47f3315fa02ba493a83d8e782d1\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "f5a7c9d5", + "metadata": {}, + "source": [ + "## CDP Toolkit Specific Features\n", + "\n", + "### Wallet Management\n", + "\n", + "The toolkit maintains an MPC wallet. The wallet data can be exported and imported to persist between sessions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "g5a7c9d6", + "metadata": {}, + "outputs": [], + "source": [ + "# Export wallet data\n", + "wallet_data = cdp.export_wallet()\n", + "\n", + "# Import wallet data\n", + "values = {\"cdp_wallet_data\": wallet_data}\n", + "cdp = CdpAgentkitWrapper(**values)" + ] + }, + { + "cell_type": "markdown", + "id": "h5a7c9d7", + "metadata": {}, + "source": [ + "### Network Support\n", + "\n", + "The toolkit supports [multiple networks](https://docs.cdp.coinbase.com/cdp-sdk/docs/networks)\n", + "\n", + "### Gasless Transactions\n", + "\n", + "Some operations support gasless transactions on Base Mainnet:\n", + "- USDC transfers\n", + "- EURC transfers\n", + "- cbBTC transfers" + ] + }, + { + "cell_type": "markdown", + "id": "i5a7c9d8", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all CDP features and configurations head to the [CDP docs](https://docs.cdp.coinbase.com/mpc-wallet/docs/welcome)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# CDP Agentkit Toolkit\n", - "\n", - "The `CDP Agentkit` toolkit contains tools that enable an LLM agent to interact with the [Coinbase Developer Platform](https://docs.cdp.coinbase.com/). The toolkit provides a wrapper around the CDP SDK, allowing agents to perform onchain operations like transfers, trades, and smart contract interactions.\n", - "\n", - "## Overview\n", - "\n", - "### Integration details\n", - "\n", - "| Class | Package | Serializable | JS support | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| CdpToolkit | `cdp-langchain` | ❌ | ❌ | ![PyPI - Version](https://img.shields.io/pypi/v/cdp-langchain?style=flat-square&label=%20) |\n", - "\n", - "### Tool features\n", - "\n", - "The toolkit provides the following tools:\n", - "\n", - "1. **get_wallet_details** - Get details about the MPC Wallet\n", - "2. **get_balance** - Get balance for specific assets\n", - "3. **request_faucet_funds** - Request test tokens from faucet\n", - "4. **transfer** - Transfer assets between addresses\n", - "5. **trade** - Trade assets (Mainnet only)\n", - "6. **deploy_token** - Deploy ERC-20 token contracts\n", - "7. **mint_nft** - Mint NFTs from existing contracts\n", - "8. **deploy_nft** - Deploy new NFT contracts\n", - "9. **register_basename** - Register a basename for the wallet\n", - "\n", - "We encourage you to add your own tools, both using CDP and web2 APIs, to create an agent that is tailored to your needs.\n", - "\n", - "## Setup\n", - "\n", - "At a high-level, we will:\n", - "\n", - "1. Install the langchain package\n", - "2. Set up your CDP API credentials\n", - "3. Initialize the CDP wrapper and toolkit\n", - "4. Pass the tools to your agent with `toolkit.get_tools()`" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b", - "metadata": {}, - "source": [ - "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "This toolkit lives in the `cdp-langchain` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU cdp-langchain" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65", - "metadata": {}, - "source": [ - "#### Set Environment Variables\n", - "\n", - "To use this toolkit, you must first set the following environment variables to access the [CDP APIs](https://docs.cdp.coinbase.com/mpc-wallet/docs/quickstart) to create wallets and interact onchain. You can sign up for an API key for free on the [CDP Portal](https://cdp.coinbase.com/):" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "cb09c344", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "for env_var in [\n", - " \"CDP_API_KEY_NAME\",\n", - " \"CDP_API_KEY_PRIVATE_KEY\",\n", - "]:\n", - " if not os.getenv(env_var):\n", - " os.environ[env_var] = getpass.getpass(f\"Enter your {env_var}: \")\n", - "\n", - "# Optional: Set network (defaults to base-sepolia)\n", - "os.environ[\"NETWORK_ID\"] = \"base-sepolia\" # or \"base-mainnet\"" - ] - }, - { - "cell_type": "markdown", - "id": "5c5f2839", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our toolkit:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "51a60dbe", - "metadata": {}, - "outputs": [], - "source": [ - "from cdp_langchain.agent_toolkits import CdpToolkit\n", - "from cdp_langchain.utils import CdpAgentkitWrapper\n", - "\n", - "# Initialize CDP wrapper\n", - "cdp = CdpAgentkitWrapper()\n", - "\n", - "# Create toolkit from wrapper\n", - "toolkit = CdpToolkit.from_cdp_agentkit_wrapper(cdp)" - ] - }, - { - "cell_type": "markdown", - "id": "d11245ad", - "metadata": {}, - "source": [ - "## Tools\n", - "\n", - "View [available tools](#tool-features):" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "310bf18e", - "metadata": {}, - "outputs": [], - "source": [ - "tools = toolkit.get_tools()\n", - "for tool in tools:\n", - " print(tool.name)" - ] - }, - { - "cell_type": "markdown", - "id": "23e11cc9", - "metadata": {}, - "source": [ - "## Use within an agent\n", - "\n", - "We will need a LLM or chat model:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "d1ee55bc", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_openai import ChatOpenAI\n", - "\n", - "llm = ChatOpenAI(model=\"gpt-4o-mini\")" - ] - }, - { - "cell_type": "markdown", - "id": "3a5bb5ca", - "metadata": {}, - "source": [ - "Initialize the agent with the tools:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f8a2c4b1", - "metadata": {}, - "outputs": [], - "source": [ - "from langgraph.prebuilt import create_react_agent\n", - "\n", - "tools = toolkit.get_tools()\n", - "agent_executor = create_react_agent(llm, tools)" - ] - }, - { - "cell_type": "markdown", - "id": "b4a7c9d2", - "metadata": {}, - "source": [ - "Example usage:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c9a8e4f3", - "metadata": {}, - "outputs": [], - "source": [ - "example_query = \"Send 0.005 ETH to john2879.base.eth\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " event[\"messages\"][-1].pretty_print()" - ] - }, - { - "cell_type": "markdown", - "id": "e5a7c9d4", - "metadata": {}, - "source": [ - "Expected output:\n", - "```\n", - "Transferred 0.005 of eth to john2879.base.eth.\n", - "Transaction hash for the transfer: 0x78c7c2878659a0de216d0764fc87eff0d38b47f3315fa02ba493a83d8e782d1e\n", - "Transaction link for the transfer: https://sepolia.basescan.org/tx/0x78c7c2878659a0de216d0764fc87eff0d38b47f3315fa02ba493a83d8e782d1\n", - "```" - ] - }, - { - "cell_type": "markdown", - "id": "f5a7c9d5", - "metadata": {}, - "source": [ - "## CDP Toolkit Specific Features\n", - "\n", - "### Wallet Management\n", - "\n", - "The toolkit maintains an MPC wallet. The wallet data can be exported and imported to persist between sessions:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "g5a7c9d6", - "metadata": {}, - "outputs": [], - "source": [ - "# Export wallet data\n", - "wallet_data = cdp.export_wallet()\n", - "\n", - "# Import wallet data\n", - "values = {\"cdp_wallet_data\": wallet_data}\n", - "cdp = CdpAgentkitWrapper(**values)" - ] - }, - { - "cell_type": "markdown", - "id": "h5a7c9d7", - "metadata": {}, - "source": [ - "### Network Support\n", - "\n", - "The toolkit supports [multiple networks](https://docs.cdp.coinbase.com/cdp-sdk/docs/networks)\n", - "\n", - "### Gasless Transactions\n", - "\n", - "Some operations support gasless transactions on Base Mainnet:\n", - "- USDC transfers\n", - "- EURC transfers\n", - "- cbBTC transfers" - ] - }, - { - "cell_type": "markdown", - "id": "i5a7c9d8", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all CDP features and configurations head to the [CDP docs](https://docs.cdp.coinbase.com/mpc-wallet/docs/welcome)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/tools/github.ipynb b/docs/docs/integrations/tools/github.ipynb index 3c5f8fa01eb..f12564bc357 100644 --- a/docs/docs/integrations/tools/github.ipynb +++ b/docs/docs/integrations/tools/github.ipynb @@ -1,360 +1,358 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Github Toolkit\n", - "\n", - "The `Github` toolkit contains tools that enable an LLM agent to interact with a github repository. \n", - "The tool is a wrapper for the [PyGitHub](https://github.com/PyGithub/PyGithub) library. \n", - "\n", - "For detailed documentation of all GithubToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.github.toolkit.GitHubToolkit.html).\n", - "\n", - "## Setup\n", - "\n", - "At a high-level, we will:\n", - "\n", - "1. Install the pygithub library\n", - "2. Create a Github app\n", - "3. Set your environmental variables\n", - "4. Pass the tools to your agent with `toolkit.get_tools()`" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "#### 1. Install dependencies\n", - "\n", - "This integration is implemented in `langchain-community`. We will also need the `pygithub` dependency:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install --upgrade --quiet pygithub langchain-community" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### 2. Create a Github App\n", - "\n", - "[Follow the instructions here](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app) to create and register a Github app. Make sure your app has the following [repository permissions:](https://docs.github.com/en/rest/overview/permissions-required-for-github-apps?apiVersion=2022-11-28)\n", - "\n", - "* Commit statuses (read only)\n", - "* Contents (read and write)\n", - "* Issues (read and write)\n", - "* Metadata (read only)\n", - "* Pull requests (read and write)\n", - "\n", - "Once the app has been registered, you must give your app permission to access each of the repositories you whish it to act upon. Use the App settings on [github.com here](https://github.com/settings/installations).\n", - "\n", - "\n", - "#### 3. Set Environment Variables\n", - "\n", - "Before initializing your agent, the following environment variables need to be set:\n", - "\n", - "* **GITHUB_APP_ID**- A six digit number found in your app's general settings\n", - "* **GITHUB_APP_PRIVATE_KEY**- The location of your app's private key .pem file, or the full text of that file as a string.\n", - "* **GITHUB_REPOSITORY**- The name of the Github repository you want your bot to act upon. Must follow the format \\{username\\}/\\{repo-name\\}. *Make sure the app has been added to this repository first!*\n", - "* Optional: **GITHUB_BRANCH**- The branch where the bot will make its commits. Defaults to `repo.default_branch`.\n", - "* Optional: **GITHUB_BASE_BRANCH**- The base branch of your repo upon which PRs will based from. Defaults to `repo.default_branch`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "for env_var in [\n", - " \"GITHUB_APP_ID\",\n", - " \"GITHUB_APP_PRIVATE_KEY\",\n", - " \"GITHUB_REPOSITORY\",\n", - "]:\n", - " if not os.getenv(env_var):\n", - " os.environ[env_var] = getpass.getpass()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our toolkit:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.agent_toolkits.github.toolkit import GitHubToolkit\n", - "from langchain_community.utilities.github import GitHubAPIWrapper\n", - "\n", - "github = GitHubAPIWrapper()\n", - "toolkit = GitHubToolkit.from_github_api_wrapper(github)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Tools\n", - "\n", - "View available tools:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Get Issues\n", - "Get Issue\n", - "Comment on Issue\n", - "List open pull requests (PRs)\n", - "Get Pull Request\n", - "Overview of files included in PR\n", - "Create Pull Request\n", - "List Pull Requests' Files\n", - "Create File\n", - "Read File\n", - "Update File\n", - "Delete File\n", - "Overview of existing files in Main branch\n", - "Overview of files in current working branch\n", - "List branches in this repository\n", - "Set active branch\n", - "Create a new branch\n", - "Get files from a directory\n", - "Search issues and pull requests\n", - "Search code\n", - "Create review request\n" - ] - } - ], - "source": [ - "tools = toolkit.get_tools()\n", - "\n", - "for tool in tools:\n", - " print(tool.name)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The purpose of these tools is as follows:\n", - "\n", - "Each of these steps will be explained in great detail below.\n", - "\n", - "1. **Get Issues**- fetches issues from the repository.\n", - "\n", - "2. **Get Issue**- fetches details about a specific issue.\n", - "\n", - "3. **Comment on Issue**- posts a comment on a specific issue.\n", - "\n", - "4. **Create Pull Request**- creates a pull request from the bot's working branch to the base branch.\n", - "\n", - "5. **Create File**- creates a new file in the repository.\n", - "\n", - "6. **Read File**- reads a file from the repository.\n", - "\n", - "7. **Update File**- updates a file in the repository.\n", - "\n", - "8. **Delete File**- deletes a file from the repository." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Include release tools\n", - "\n", - "By default, the toolkit does not include release-related tools. You can include them by setting `include_release_tools=True` when initializing the toolkit:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "toolkit = GitHubToolkit.from_github_api_wrapper(github, include_release_tools=True)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Settings `include_release_tools=True` will include the following tools:\n", - "\n", - "* **Get Latest Release**- fetches the latest release from the repository.\n", - "\n", - "* **Get Releases**- fetches the latest 5 releases from the repository.\n", - "\n", - "* **Get Release**- fetches a specific release from the repository by tag name, e.g. `v1.0.0`.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Use within an agent\n", - "\n", - "We will need a LLM or chat model:\n", - "\n", - "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "# | output: false\n", - "# | echo: false\n", - "\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Initialize the agent with a subset of tools:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [], - "source": [ - "from langgraph.prebuilt import create_react_agent\n", - "\n", - "tools = [tool for tool in toolkit.get_tools() if tool.name == \"Get Issue\"]\n", - "assert len(tools) == 1\n", - "tools[0].name = \"get_issue\"\n", - "\n", - "agent_executor = create_react_agent(llm, tools)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And issue it a query:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Github Toolkit\n", + "\n", + "The `Github` toolkit contains tools that enable an LLM agent to interact with a github repository. \n", + "The tool is a wrapper for the [PyGitHub](https://github.com/PyGithub/PyGithub) library. \n", + "\n", + "For detailed documentation of all GithubToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.github.toolkit.GitHubToolkit.html).\n", + "\n", + "## Setup\n", + "\n", + "At a high-level, we will:\n", + "\n", + "1. Install the pygithub library\n", + "2. Create a Github app\n", + "3. Set your environmental variables\n", + "4. Pass the tools to your agent with `toolkit.get_tools()`" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "================================\u001b[1m Human Message \u001b[0m=================================\n", - "\n", - "What is the title of issue 24888?\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " get_issue (call_iSYJVaM7uchfNHOMJoVPQsOi)\n", - " Call ID: call_iSYJVaM7uchfNHOMJoVPQsOi\n", - " Args:\n", - " issue_number: 24888\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: get_issue\n", - "\n", - "{\"number\": 24888, \"title\": \"Standardize KV-Store Docs\", \"body\": \"To make our KV-store integrations as easy to use as possible we need to make sure the docs for them are thorough and standardized. There are two parts to this: updating the KV-store docstrings and updating the actual integration docs.\\r\\n\\r\\nThis needs to be done for each KV-store integration, ideally with one PR per KV-store.\\r\\n\\r\\nRelated to broader issues #21983 and #22005.\\r\\n\\r\\n## Docstrings\\r\\nEach KV-store class docstring should have the sections shown in the [Appendix](#appendix) below. The sections should have input and output code blocks when relevant.\\r\\n\\r\\nTo build a preview of the API docs for the package you're working on run (from root of repo):\\r\\n\\r\\n```shell\\r\\nmake api_docs_clean; make api_docs_quick_preview API_PKG=openai\\r\\n```\\r\\n\\r\\nwhere `API_PKG=` should be the parent directory that houses the edited package (e.g. community, openai, anthropic, huggingface, together, mistralai, groq, fireworks, etc.). This should be quite fast for all the partner packages.\\r\\n\\r\\n## Doc pages\\r\\nEach KV-store [docs page](https://python.langchain.com/docs/integrations/stores/) should follow [this template](https://github.com/langchain-ai/langchain/blob/master/libs/cli/langchain_cli/integration_template/docs/kv_store.ipynb).\\r\\n\\r\\nHere is an example: https://python.langchain.com/docs/integrations/stores/in_memory/\\r\\n\\r\\nYou can use the `langchain-cli` to quickly get started with a new chat model integration docs page (run from root of repo):\\r\\n\\r\\n```shell\\r\\npoetry run pip install -e libs/cli\\r\\npoetry run langchain-cli integration create-doc --name \\\"foo-bar\\\" --name-class FooBar --component-type kv_store --destination-dir ./docs/docs/integrations/stores/\\r\\n```\\r\\n\\r\\nwhere `--name` is the integration package name without the \\\"langchain-\\\" prefix and `--name-class` is the class name without the \\\"ByteStore\\\" suffix. This will create a template doc with some autopopulated fields at docs/docs/integrations/stores/foo_bar.ipynb.\\r\\n\\r\\nTo build a preview of the docs you can run (from root):\\r\\n\\r\\n```shell\\r\\nmake docs_clean\\r\\nmake docs_build\\r\\ncd docs/build/output-new\\r\\nyarn\\r\\nyarn start\\r\\n```\\r\\n\\r\\n## Appendix\\r\\nExpected sections for the KV-store class docstring.\\r\\n\\r\\n```python\\r\\n \\\"\\\"\\\"__ModuleName__ completion KV-store integration.\\r\\n\\r\\n # TODO: Replace with relevant packages, env vars.\\r\\n Setup:\\r\\n Install ``__package_name__`` and set environment variable ``__MODULE_NAME___API_KEY``.\\r\\n\\r\\n .. code-block:: bash\\r\\n\\r\\n pip install -U __package_name__\\r\\n export __MODULE_NAME___API_KEY=\\\"your-api-key\\\"\\r\\n\\r\\n # TODO: Populate with relevant params.\\r\\n Key init args \\u2014 client params:\\r\\n api_key: Optional[str]\\r\\n __ModuleName__ API key. If not passed in will be read from env var __MODULE_NAME___API_KEY.\\r\\n\\r\\n See full list of supported init args and their descriptions in the params section.\\r\\n\\r\\n # TODO: Replace with relevant init params.\\r\\n Instantiate:\\r\\n .. code-block:: python\\r\\n\\r\\n from __module_name__ import __ModuleName__ByteStore\\r\\n\\r\\n kv_store = __ModuleName__ByteStore(\\r\\n # api_key=\\\"...\\\",\\r\\n # other params...\\r\\n )\\r\\n\\r\\n Set keys:\\r\\n .. code-block:: python\\r\\n\\r\\n kv_pairs = [\\r\\n [\\\"key1\\\", \\\"value1\\\"],\\r\\n [\\\"key2\\\", \\\"value2\\\"],\\r\\n ]\\r\\n\\r\\n kv_store.mset(kv_pairs)\\r\\n\\r\\n .. code-block:: python\\r\\n\\r\\n Get keys:\\r\\n .. code-block:: python\\r\\n\\r\\n kv_store.mget([\\\"key1\\\", \\\"key2\\\"])\\r\\n\\r\\n .. code-block:: python\\r\\n\\r\\n # TODO: Example output.\\r\\n\\r\\n Delete keys:\\r\\n ..code-block:: python\\r\\n\\r\\n kv_store.mdelete([\\\"key1\\\", \\\"key2\\\"])\\r\\n\\r\\n ..code-block:: python\\r\\n \\\"\\\"\\\" # noqa: E501\\r\\n```\", \"comments\": \"[]\", \"opened_by\": \"jacoblee93\"}\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "The title of issue 24888 is \"Standardize KV-Store Docs\".\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "#### 1. Install dependencies\n", + "\n", + "This integration is implemented in `langchain-community`. We will also need the `pygithub` dependency:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet pygithub langchain-community" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 2. Create a Github App\n", + "\n", + "[Follow the instructions here](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app) to create and register a Github app. Make sure your app has the following [repository permissions:](https://docs.github.com/en/rest/overview/permissions-required-for-github-apps?apiVersion=2022-11-28)\n", + "\n", + "* Commit statuses (read only)\n", + "* Contents (read and write)\n", + "* Issues (read and write)\n", + "* Metadata (read only)\n", + "* Pull requests (read and write)\n", + "\n", + "Once the app has been registered, you must give your app permission to access each of the repositories you whish it to act upon. Use the App settings on [github.com here](https://github.com/settings/installations).\n", + "\n", + "\n", + "#### 3. Set Environment Variables\n", + "\n", + "Before initializing your agent, the following environment variables need to be set:\n", + "\n", + "* **GITHUB_APP_ID**- A six digit number found in your app's general settings\n", + "* **GITHUB_APP_PRIVATE_KEY**- The location of your app's private key .pem file, or the full text of that file as a string.\n", + "* **GITHUB_REPOSITORY**- The name of the Github repository you want your bot to act upon. Must follow the format \\{username\\}/\\{repo-name\\}. *Make sure the app has been added to this repository first!*\n", + "* Optional: **GITHUB_BRANCH**- The branch where the bot will make its commits. Defaults to `repo.default_branch`.\n", + "* Optional: **GITHUB_BASE_BRANCH**- The base branch of your repo upon which PRs will based from. Defaults to `repo.default_branch`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "for env_var in [\n", + " \"GITHUB_APP_ID\",\n", + " \"GITHUB_APP_PRIVATE_KEY\",\n", + " \"GITHUB_REPOSITORY\",\n", + "]:\n", + " if not os.getenv(env_var):\n", + " os.environ[env_var] = getpass.getpass()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our toolkit:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.agent_toolkits.github.toolkit import GitHubToolkit\n", + "from langchain_community.utilities.github import GitHubAPIWrapper\n", + "\n", + "github = GitHubAPIWrapper()\n", + "toolkit = GitHubToolkit.from_github_api_wrapper(github)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "View available tools:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Get Issues\n", + "Get Issue\n", + "Comment on Issue\n", + "List open pull requests (PRs)\n", + "Get Pull Request\n", + "Overview of files included in PR\n", + "Create Pull Request\n", + "List Pull Requests' Files\n", + "Create File\n", + "Read File\n", + "Update File\n", + "Delete File\n", + "Overview of existing files in Main branch\n", + "Overview of files in current working branch\n", + "List branches in this repository\n", + "Set active branch\n", + "Create a new branch\n", + "Get files from a directory\n", + "Search issues and pull requests\n", + "Search code\n", + "Create review request\n" + ] + } + ], + "source": [ + "tools = toolkit.get_tools()\n", + "\n", + "for tool in tools:\n", + " print(tool.name)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The purpose of these tools is as follows:\n", + "\n", + "Each of these steps will be explained in great detail below.\n", + "\n", + "1. **Get Issues**- fetches issues from the repository.\n", + "\n", + "2. **Get Issue**- fetches details about a specific issue.\n", + "\n", + "3. **Comment on Issue**- posts a comment on a specific issue.\n", + "\n", + "4. **Create Pull Request**- creates a pull request from the bot's working branch to the base branch.\n", + "\n", + "5. **Create File**- creates a new file in the repository.\n", + "\n", + "6. **Read File**- reads a file from the repository.\n", + "\n", + "7. **Update File**- updates a file in the repository.\n", + "\n", + "8. **Delete File**- deletes a file from the repository." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Include release tools\n", + "\n", + "By default, the toolkit does not include release-related tools. You can include them by setting `include_release_tools=True` when initializing the toolkit:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "toolkit = GitHubToolkit.from_github_api_wrapper(github, include_release_tools=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Settings `include_release_tools=True` will include the following tools:\n", + "\n", + "* **Get Latest Release**- fetches the latest release from the repository.\n", + "\n", + "* **Get Releases**- fetches the latest 5 releases from the repository.\n", + "\n", + "* **Get Release**- fetches a specific release from the repository by tag name, e.g. `v1.0.0`.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Use within an agent\n", + "\n", + "We will need a LLM or chat model:\n", + "\n", + "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "# | output: false\n", + "# | echo: false\n", + "\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Initialize the agent with a subset of tools:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "tools = [tool for tool in toolkit.get_tools() if tool.name == \"Get Issue\"]\n", + "assert len(tools) == 1\n", + "tools[0].name = \"get_issue\"\n", + "\n", + "agent_executor = create_react_agent(llm, tools)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And issue it a query:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "What is the title of issue 24888?\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " get_issue (call_iSYJVaM7uchfNHOMJoVPQsOi)\n", + " Call ID: call_iSYJVaM7uchfNHOMJoVPQsOi\n", + " Args:\n", + " issue_number: 24888\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: get_issue\n", + "\n", + "{\"number\": 24888, \"title\": \"Standardize KV-Store Docs\", \"body\": \"To make our KV-store integrations as easy to use as possible we need to make sure the docs for them are thorough and standardized. There are two parts to this: updating the KV-store docstrings and updating the actual integration docs.\\r\\n\\r\\nThis needs to be done for each KV-store integration, ideally with one PR per KV-store.\\r\\n\\r\\nRelated to broader issues #21983 and #22005.\\r\\n\\r\\n## Docstrings\\r\\nEach KV-store class docstring should have the sections shown in the [Appendix](#appendix) below. The sections should have input and output code blocks when relevant.\\r\\n\\r\\nTo build a preview of the API docs for the package you're working on run (from root of repo):\\r\\n\\r\\n```shell\\r\\nmake api_docs_clean; make api_docs_quick_preview API_PKG=openai\\r\\n```\\r\\n\\r\\nwhere `API_PKG=` should be the parent directory that houses the edited package (e.g. community, openai, anthropic, huggingface, together, mistralai, groq, fireworks, etc.). This should be quite fast for all the partner packages.\\r\\n\\r\\n## Doc pages\\r\\nEach KV-store [docs page](https://python.langchain.com/docs/integrations/stores/) should follow [this template](https://github.com/langchain-ai/langchain/blob/master/libs/cli/langchain_cli/integration_template/docs/kv_store.ipynb).\\r\\n\\r\\nHere is an example: https://python.langchain.com/docs/integrations/stores/in_memory/\\r\\n\\r\\nYou can use the `langchain-cli` to quickly get started with a new chat model integration docs page (run from root of repo):\\r\\n\\r\\n```shell\\r\\npoetry run pip install -e libs/cli\\r\\npoetry run langchain-cli integration create-doc --name \\\"foo-bar\\\" --name-class FooBar --component-type kv_store --destination-dir ./docs/docs/integrations/stores/\\r\\n```\\r\\n\\r\\nwhere `--name` is the integration package name without the \\\"langchain-\\\" prefix and `--name-class` is the class name without the \\\"ByteStore\\\" suffix. This will create a template doc with some autopopulated fields at docs/docs/integrations/stores/foo_bar.ipynb.\\r\\n\\r\\nTo build a preview of the docs you can run (from root):\\r\\n\\r\\n```shell\\r\\nmake docs_clean\\r\\nmake docs_build\\r\\ncd docs/build/output-new\\r\\nyarn\\r\\nyarn start\\r\\n```\\r\\n\\r\\n## Appendix\\r\\nExpected sections for the KV-store class docstring.\\r\\n\\r\\n```python\\r\\n \\\"\\\"\\\"__ModuleName__ completion KV-store integration.\\r\\n\\r\\n # TODO: Replace with relevant packages, env vars.\\r\\n Setup:\\r\\n Install ``__package_name__`` and set environment variable ``__MODULE_NAME___API_KEY``.\\r\\n\\r\\n .. code-block:: bash\\r\\n\\r\\n pip install -U __package_name__\\r\\n export __MODULE_NAME___API_KEY=\\\"your-api-key\\\"\\r\\n\\r\\n # TODO: Populate with relevant params.\\r\\n Key init args \\u2014 client params:\\r\\n api_key: Optional[str]\\r\\n __ModuleName__ API key. If not passed in will be read from env var __MODULE_NAME___API_KEY.\\r\\n\\r\\n See full list of supported init args and their descriptions in the params section.\\r\\n\\r\\n # TODO: Replace with relevant init params.\\r\\n Instantiate:\\r\\n .. code-block:: python\\r\\n\\r\\n from __module_name__ import __ModuleName__ByteStore\\r\\n\\r\\n kv_store = __ModuleName__ByteStore(\\r\\n # api_key=\\\"...\\\",\\r\\n # other params...\\r\\n )\\r\\n\\r\\n Set keys:\\r\\n .. code-block:: python\\r\\n\\r\\n kv_pairs = [\\r\\n [\\\"key1\\\", \\\"value1\\\"],\\r\\n [\\\"key2\\\", \\\"value2\\\"],\\r\\n ]\\r\\n\\r\\n kv_store.mset(kv_pairs)\\r\\n\\r\\n .. code-block:: python\\r\\n\\r\\n Get keys:\\r\\n .. code-block:: python\\r\\n\\r\\n kv_store.mget([\\\"key1\\\", \\\"key2\\\"])\\r\\n\\r\\n .. code-block:: python\\r\\n\\r\\n # TODO: Example output.\\r\\n\\r\\n Delete keys:\\r\\n ..code-block:: python\\r\\n\\r\\n kv_store.mdelete([\\\"key1\\\", \\\"key2\\\"])\\r\\n\\r\\n ..code-block:: python\\r\\n \\\"\\\"\\\" # noqa: E501\\r\\n```\", \"comments\": \"[]\", \"opened_by\": \"jacoblee93\"}\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "The title of issue 24888 is \"Standardize KV-Store Docs\".\n" + ] + } + ], + "source": [ + "example_query = \"What is the title of issue 24888?\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `GithubToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.github.toolkit.GitHubToolkit.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.1" } - ], - "source": [ - "example_query = \"What is the title of issue 24888?\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " event[\"messages\"][-1].pretty_print()" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `GithubToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.github.toolkit.GitHubToolkit.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.13.1" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/tools/gmail.ipynb b/docs/docs/integrations/tools/gmail.ipynb index 0e3816c8bbd..d4781d6dc2f 100644 --- a/docs/docs/integrations/tools/gmail.ipynb +++ b/docs/docs/integrations/tools/gmail.ipynb @@ -1,269 +1,267 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Gmail Toolkit\n", - "\n", - "This will help you getting started with the GMail [toolkit](/docs/concepts/tools/#toolkits). This toolkit interacts with the GMail API to read messages, draft and send messages, and more. For detailed documentation of all GmailToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.toolkit.GmailToolkit.html).\n", - "\n", - "## Setup\n", - "\n", - "To use this toolkit, you will need to set up your credentials explained in the [Gmail API docs](https://developers.google.com/gmail/api/quickstart/python#authorize_credentials_for_a_desktop_application). Once you've downloaded the `credentials.json` file, you can start using the Gmail API." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "This toolkit lives in the `langchain-google-community` package. We'll need the `gmail` extra:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-google-community\\[gmail\\]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "By default the toolkit reads the local `credentials.json` file. You can also manually provide a `Credentials` object." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_google_community import GmailToolkit\n", - "\n", - "toolkit = GmailToolkit()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Customizing Authentication\n", - "\n", - "Behind the scenes, a `googleapi` resource is created using the following methods. \n", - "you can manually build a `googleapi` resource for more auth control. " - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain_google_community.gmail.utils import (\n", - " build_resource_service,\n", - " get_gmail_credentials,\n", - ")\n", - "\n", - "# Can review scopes here https://developers.google.com/gmail/api/auth/scopes\n", - "# For instance, readonly scope is 'https://www.googleapis.com/auth/gmail.readonly'\n", - "credentials = get_gmail_credentials(\n", - " token_file=\"token.json\",\n", - " scopes=[\"https://mail.google.com/\"],\n", - " client_secrets_file=\"credentials.json\",\n", - ")\n", - "api_resource = build_resource_service(credentials=credentials)\n", - "toolkit = GmailToolkit(api_resource=api_resource)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Tools\n", - "\n", - "View available tools:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "tags": [] - }, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "[GmailCreateDraft(api_resource=),\n", - " GmailSendMessage(api_resource=),\n", - " GmailSearch(api_resource=),\n", - " GmailGetMessage(api_resource=),\n", - " GmailGetThread(api_resource=)]" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Gmail Toolkit\n", + "\n", + "This will help you getting started with the GMail [toolkit](/docs/concepts/tools/#toolkits). This toolkit interacts with the GMail API to read messages, draft and send messages, and more. For detailed documentation of all GmailToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.toolkit.GmailToolkit.html).\n", + "\n", + "## Setup\n", + "\n", + "To use this toolkit, you will need to set up your credentials explained in the [Gmail API docs](https://developers.google.com/gmail/api/quickstart/python#authorize_credentials_for_a_desktop_application). Once you've downloaded the `credentials.json` file, you can start using the Gmail API." ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "tools = toolkit.get_tools()\n", - "tools" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "- [GmailCreateDraft](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.create_draft.GmailCreateDraft.html)\n", - "- [GmailSendMessage](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.send_message.GmailSendMessage.html)\n", - "- [GmailSearch](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.search.GmailSearch.html)\n", - "- [GmailGetMessage](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.get_message.GmailGetMessage.html)\n", - "- [GmailGetThread](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.get_thread.GmailGetThread.html)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Use within an agent\n", - "\n", - "Below we show how to incorporate the toolkit into an [agent](/docs/tutorials/agents).\n", - "\n", - "We will need a LLM or chat model:\n", - "\n", - "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], - "source": [ - "# | output: false\n", - "# | echo: false\n", - "\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "from langgraph.prebuilt import create_react_agent\n", - "\n", - "agent_executor = create_react_agent(llm, tools)" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "================================\u001b[1m Human Message \u001b[0m=================================\n", - "\n", - "Draft an email to fake@fake.com thanking them for coffee.\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " create_gmail_draft (call_slGkYKZKA6h3Mf1CraUBzs6M)\n", - " Call ID: call_slGkYKZKA6h3Mf1CraUBzs6M\n", - " Args:\n", - " message: Dear Fake,\n", - "\n", - "I wanted to take a moment to thank you for the coffee yesterday. It was a pleasure catching up with you. Let's do it again soon!\n", - "\n", - "Best regards,\n", - "[Your Name]\n", - " to: ['fake@fake.com']\n", - " subject: Thank You for the Coffee\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: create_gmail_draft\n", - "\n", - "Draft created. Draft Id: r-7233782721440261513\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "I have drafted an email to fake@fake.com thanking them for the coffee. You can review and send it from your email draft with the subject \"Thank You for the Coffee\".\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "This toolkit lives in the `langchain-google-community` package. We'll need the `gmail` extra:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-google-community\\[gmail\\]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "By default the toolkit reads the local `credentials.json` file. You can also manually provide a `Credentials` object." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_google_community import GmailToolkit\n", + "\n", + "toolkit = GmailToolkit()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Customizing Authentication\n", + "\n", + "Behind the scenes, a `googleapi` resource is created using the following methods. \n", + "you can manually build a `googleapi` resource for more auth control. " + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain_google_community.gmail.utils import (\n", + " build_resource_service,\n", + " get_gmail_credentials,\n", + ")\n", + "\n", + "# Can review scopes here https://developers.google.com/gmail/api/auth/scopes\n", + "# For instance, readonly scope is 'https://www.googleapis.com/auth/gmail.readonly'\n", + "credentials = get_gmail_credentials(\n", + " token_file=\"token.json\",\n", + " scopes=[\"https://mail.google.com/\"],\n", + " client_secrets_file=\"credentials.json\",\n", + ")\n", + "api_resource = build_resource_service(credentials=credentials)\n", + "toolkit = GmailToolkit(api_resource=api_resource)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "View available tools:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[GmailCreateDraft(api_resource=),\n", + " GmailSendMessage(api_resource=),\n", + " GmailSearch(api_resource=),\n", + " GmailGetMessage(api_resource=),\n", + " GmailGetThread(api_resource=)]" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tools = toolkit.get_tools()\n", + "tools" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- [GmailCreateDraft](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.create_draft.GmailCreateDraft.html)\n", + "- [GmailSendMessage](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.send_message.GmailSendMessage.html)\n", + "- [GmailSearch](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.search.GmailSearch.html)\n", + "- [GmailGetMessage](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.get_message.GmailGetMessage.html)\n", + "- [GmailGetThread](https://python.langchain.com/api_reference/google_community/gmail/langchain_google_community.gmail.get_thread.GmailGetThread.html)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Use within an agent\n", + "\n", + "Below we show how to incorporate the toolkit into an [agent](/docs/tutorials/agents).\n", + "\n", + "We will need a LLM or chat model:\n", + "\n", + "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "# | output: false\n", + "# | echo: false\n", + "\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "agent_executor = create_react_agent(llm, tools)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "Draft an email to fake@fake.com thanking them for coffee.\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " create_gmail_draft (call_slGkYKZKA6h3Mf1CraUBzs6M)\n", + " Call ID: call_slGkYKZKA6h3Mf1CraUBzs6M\n", + " Args:\n", + " message: Dear Fake,\n", + "\n", + "I wanted to take a moment to thank you for the coffee yesterday. It was a pleasure catching up with you. Let's do it again soon!\n", + "\n", + "Best regards,\n", + "[Your Name]\n", + " to: ['fake@fake.com']\n", + " subject: Thank You for the Coffee\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: create_gmail_draft\n", + "\n", + "Draft created. Draft Id: r-7233782721440261513\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "I have drafted an email to fake@fake.com thanking them for the coffee. You can review and send it from your email draft with the subject \"Thank You for the Coffee\".\n" + ] + } + ], + "source": [ + "example_query = \"Draft an email to fake@fake.com thanking them for coffee.\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `GmailToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.gmail.toolkit.GmailToolkit.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" } - ], - "source": [ - "example_query = \"Draft an email to fake@fake.com thanking them for coffee.\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " event[\"messages\"][-1].pretty_print()" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `GmailToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.gmail.toolkit.GmailToolkit.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/tools/requests.ipynb b/docs/docs/integrations/tools/requests.ipynb index 2fa15459016..54732db6181 100644 --- a/docs/docs/integrations/tools/requests.ipynb +++ b/docs/docs/integrations/tools/requests.ipynb @@ -1,351 +1,349 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "cfe4185a-34dc-4cdc-b831-001954f2d6e8", - "metadata": {}, - "source": [ - "# Requests Toolkit\n", - "\n", - "We can use the Requests [toolkit](/docs/concepts/tools/#toolkits) to construct agents that generate HTTP requests.\n", - "\n", - "For detailed documentation of all API toolkit features and configurations head to the API reference for [RequestsToolkit](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.openapi.toolkit.RequestsToolkit.html).\n", - "\n", - "## ⚠️ Security note ⚠️\n", - "There are inherent risks in giving models discretion to execute real-world actions. Take precautions to mitigate these risks:\n", - "\n", - "- Make sure that permissions associated with the tools are narrowly-scoped (e.g., for database operations or API requests);\n", - "- When desired, make use of human-in-the-loop workflows." - ] - }, - { - "cell_type": "markdown", - "id": "d968e982-f370-4614-8469-c1bc71ee3e32", - "metadata": {}, - "source": [ - "## Setup\n", - "\n", - "### Installation\n", - "\n", - "This toolkit lives in the `langchain-community` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f74f05fb-3f24-4c0b-a17f-cf4edeedbb9a", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-community" - ] - }, - { - "cell_type": "markdown", - "id": "36a178eb-1f2c-411e-bf25-0240ead4c62a", - "metadata": {}, - "source": [ - "Note that if you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8e68d0cd-6233-481c-b048-e8d95cba4c35", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "a7e2f64a-a72e-4fef-be52-eaf7c5072d24", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "First we will demonstrate a minimal example.\n", - "\n", - "**NOTE**: There are inherent risks in giving models discretion to execute real-world actions. We must \"opt-in\" to these risks by setting `allow_dangerous_request=True` to use these tools.\n", - "**This can be dangerous for calling unwanted requests**. Please make sure your custom OpenAPI spec (yaml) is safe and that permissions associated with the tools are narrowly-scoped." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "018bd070-9fc8-459b-8d28-b4a3e283e640", - "metadata": {}, - "outputs": [], - "source": [ - "ALLOW_DANGEROUS_REQUEST = True" - ] - }, - { - "cell_type": "markdown", - "id": "a024f7b3-5437-4878-bd16-c4783bff394c", - "metadata": {}, - "source": [ - "We can use the [JSONPlaceholder](https://jsonplaceholder.typicode.com) API as a testing ground.\n", - "\n", - "Let's create (a subset of) its API spec:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "2dcbcf92-2ad5-49c3-94ac-91047ccc8c5b", - "metadata": {}, - "outputs": [], - "source": [ - "from typing import Any, Dict, Union\n", - "\n", - "import requests\n", - "import yaml\n", - "\n", - "\n", - "def _get_schema(response_json: Union[dict, list]) -> dict:\n", - " if isinstance(response_json, list):\n", - " response_json = response_json[0] if response_json else {}\n", - " return {key: type(value).__name__ for key, value in response_json.items()}\n", - "\n", - "\n", - "def _get_api_spec() -> str:\n", - " base_url = \"https://jsonplaceholder.typicode.com\"\n", - " endpoints = [\n", - " \"/posts\",\n", - " \"/comments\",\n", - " ]\n", - " common_query_parameters = [\n", - " {\n", - " \"name\": \"_limit\",\n", - " \"in\": \"query\",\n", - " \"required\": False,\n", - " \"schema\": {\"type\": \"integer\", \"example\": 2},\n", - " \"description\": \"Limit the number of results\",\n", - " }\n", - " ]\n", - " openapi_spec: Dict[str, Any] = {\n", - " \"openapi\": \"3.0.0\",\n", - " \"info\": {\"title\": \"JSONPlaceholder API\", \"version\": \"1.0.0\"},\n", - " \"servers\": [{\"url\": base_url}],\n", - " \"paths\": {},\n", - " }\n", - " # Iterate over the endpoints to construct the paths\n", - " for endpoint in endpoints:\n", - " response = requests.get(base_url + endpoint)\n", - " if response.status_code == 200:\n", - " schema = _get_schema(response.json())\n", - " openapi_spec[\"paths\"][endpoint] = {\n", - " \"get\": {\n", - " \"summary\": f\"Get {endpoint[1:]}\",\n", - " \"parameters\": common_query_parameters,\n", - " \"responses\": {\n", - " \"200\": {\n", - " \"description\": \"Successful response\",\n", - " \"content\": {\n", - " \"application/json\": {\n", - " \"schema\": {\"type\": \"object\", \"properties\": schema}\n", - " }\n", - " },\n", - " }\n", - " },\n", - " }\n", - " }\n", - " return yaml.dump(openapi_spec, sort_keys=False)\n", - "\n", - "\n", - "api_spec = _get_api_spec()" - ] - }, - { - "cell_type": "markdown", - "id": "db3d6148-ae65-4a1d-91a6-59ee3e4e6efa", - "metadata": {}, - "source": [ - "Next we can instantiate the toolkit. We require no authorization or other headers for this API:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "63a630b3-45bb-4525-865b-083f322b944b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.agent_toolkits.openapi.toolkit import RequestsToolkit\n", - "from langchain_community.utilities.requests import TextRequestsWrapper\n", - "\n", - "toolkit = RequestsToolkit(\n", - " requests_wrapper=TextRequestsWrapper(headers={}),\n", - " allow_dangerous_requests=ALLOW_DANGEROUS_REQUEST,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "f4224a64-843a-479d-8a7b-84719e4b9d0c", - "metadata": {}, - "source": [ - "## Tools\n", - "\n", - "View available tools:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "70ea0f4e-9f10-4906-894b-08df832fd515", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "[RequestsGetTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", - " RequestsPostTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", - " RequestsPatchTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", - " RequestsPutTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", - " RequestsDeleteTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True)]" + "cell_type": "markdown", + "id": "cfe4185a-34dc-4cdc-b831-001954f2d6e8", + "metadata": {}, + "source": [ + "# Requests Toolkit\n", + "\n", + "We can use the Requests [toolkit](/docs/concepts/tools/#toolkits) to construct agents that generate HTTP requests.\n", + "\n", + "For detailed documentation of all API toolkit features and configurations head to the API reference for [RequestsToolkit](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.openapi.toolkit.RequestsToolkit.html).\n", + "\n", + "## ⚠️ Security note ⚠️\n", + "There are inherent risks in giving models discretion to execute real-world actions. Take precautions to mitigate these risks:\n", + "\n", + "- Make sure that permissions associated with the tools are narrowly-scoped (e.g., for database operations or API requests);\n", + "- When desired, make use of human-in-the-loop workflows." ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "tools = toolkit.get_tools()\n", - "\n", - "tools" - ] - }, - { - "cell_type": "markdown", - "id": "a21a6ca4-d650-4b7d-a944-1a8771b5293a", - "metadata": {}, - "source": [ - "- [RequestsGetTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsGetTool.html)\n", - "- [RequestsPostTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsPostTool.html)\n", - "- [RequestsPatchTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsPatchTool.html)\n", - "- [RequestsPutTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsPutTool.html)\n", - "- [RequestsDeleteTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsDeleteTool.html)" - ] - }, - { - "cell_type": "markdown", - "id": "e2dbb304-abf2-472a-9130-f03150a40549", - "metadata": {}, - "source": [ - "## Use within an agent" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "db062da7-f22c-4f36-9df8-1da96c9f7538", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_openai import ChatOpenAI\n", - "from langgraph.prebuilt import create_react_agent\n", - "\n", - "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n", - "\n", - "system_message = \"\"\"\n", - "You have access to an API to help answer user queries.\n", - "Here is documentation on the API:\n", - "{api_spec}\n", - "\"\"\".format(api_spec=api_spec)\n", - "\n", - "agent_executor = create_react_agent(llm, tools, prompt=system_message)" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "c1e47be9-374a-457c-928a-48f02b5530e3", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "================================\u001b[1m Human Message \u001b[0m=================================\n", - "\n", - "Fetch the top two posts. What are their titles?\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " requests_get (call_RV2SOyzCnV5h2sm4WPgG8fND)\n", - " Call ID: call_RV2SOyzCnV5h2sm4WPgG8fND\n", - " Args:\n", - " url: https://jsonplaceholder.typicode.com/posts?_limit=2\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: requests_get\n", - "\n", - "[\n", - " {\n", - " \"userId\": 1,\n", - " \"id\": 1,\n", - " \"title\": \"sunt aut facere repellat provident occaecati excepturi optio reprehenderit\",\n", - " \"body\": \"quia et suscipit\\nsuscipit recusandae consequuntur expedita et cum\\nreprehenderit molestiae ut ut quas totam\\nnostrum rerum est autem sunt rem eveniet architecto\"\n", - " },\n", - " {\n", - " \"userId\": 1,\n", - " \"id\": 2,\n", - " \"title\": \"qui est esse\",\n", - " \"body\": \"est rerum tempore vitae\\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\\nqui aperiam non debitis possimus qui neque nisi nulla\"\n", - " }\n", - "]\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "The titles of the top two posts are:\n", - "1. \"sunt aut facere repellat provident occaecati excepturi optio reprehenderit\"\n", - "2. \"qui est esse\"\n" - ] + "cell_type": "markdown", + "id": "d968e982-f370-4614-8469-c1bc71ee3e32", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "### Installation\n", + "\n", + "This toolkit lives in the `langchain-community` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f74f05fb-3f24-4c0b-a17f-cf4edeedbb9a", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-community" + ] + }, + { + "cell_type": "markdown", + "id": "36a178eb-1f2c-411e-bf25-0240ead4c62a", + "metadata": {}, + "source": "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8e68d0cd-6233-481c-b048-e8d95cba4c35", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "a7e2f64a-a72e-4fef-be52-eaf7c5072d24", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "First we will demonstrate a minimal example.\n", + "\n", + "**NOTE**: There are inherent risks in giving models discretion to execute real-world actions. We must \"opt-in\" to these risks by setting `allow_dangerous_request=True` to use these tools.\n", + "**This can be dangerous for calling unwanted requests**. Please make sure your custom OpenAPI spec (yaml) is safe and that permissions associated with the tools are narrowly-scoped." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "018bd070-9fc8-459b-8d28-b4a3e283e640", + "metadata": {}, + "outputs": [], + "source": [ + "ALLOW_DANGEROUS_REQUEST = True" + ] + }, + { + "cell_type": "markdown", + "id": "a024f7b3-5437-4878-bd16-c4783bff394c", + "metadata": {}, + "source": [ + "We can use the [JSONPlaceholder](https://jsonplaceholder.typicode.com) API as a testing ground.\n", + "\n", + "Let's create (a subset of) its API spec:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "2dcbcf92-2ad5-49c3-94ac-91047ccc8c5b", + "metadata": {}, + "outputs": [], + "source": [ + "from typing import Any, Dict, Union\n", + "\n", + "import requests\n", + "import yaml\n", + "\n", + "\n", + "def _get_schema(response_json: Union[dict, list]) -> dict:\n", + " if isinstance(response_json, list):\n", + " response_json = response_json[0] if response_json else {}\n", + " return {key: type(value).__name__ for key, value in response_json.items()}\n", + "\n", + "\n", + "def _get_api_spec() -> str:\n", + " base_url = \"https://jsonplaceholder.typicode.com\"\n", + " endpoints = [\n", + " \"/posts\",\n", + " \"/comments\",\n", + " ]\n", + " common_query_parameters = [\n", + " {\n", + " \"name\": \"_limit\",\n", + " \"in\": \"query\",\n", + " \"required\": False,\n", + " \"schema\": {\"type\": \"integer\", \"example\": 2},\n", + " \"description\": \"Limit the number of results\",\n", + " }\n", + " ]\n", + " openapi_spec: Dict[str, Any] = {\n", + " \"openapi\": \"3.0.0\",\n", + " \"info\": {\"title\": \"JSONPlaceholder API\", \"version\": \"1.0.0\"},\n", + " \"servers\": [{\"url\": base_url}],\n", + " \"paths\": {},\n", + " }\n", + " # Iterate over the endpoints to construct the paths\n", + " for endpoint in endpoints:\n", + " response = requests.get(base_url + endpoint)\n", + " if response.status_code == 200:\n", + " schema = _get_schema(response.json())\n", + " openapi_spec[\"paths\"][endpoint] = {\n", + " \"get\": {\n", + " \"summary\": f\"Get {endpoint[1:]}\",\n", + " \"parameters\": common_query_parameters,\n", + " \"responses\": {\n", + " \"200\": {\n", + " \"description\": \"Successful response\",\n", + " \"content\": {\n", + " \"application/json\": {\n", + " \"schema\": {\"type\": \"object\", \"properties\": schema}\n", + " }\n", + " },\n", + " }\n", + " },\n", + " }\n", + " }\n", + " return yaml.dump(openapi_spec, sort_keys=False)\n", + "\n", + "\n", + "api_spec = _get_api_spec()" + ] + }, + { + "cell_type": "markdown", + "id": "db3d6148-ae65-4a1d-91a6-59ee3e4e6efa", + "metadata": {}, + "source": [ + "Next we can instantiate the toolkit. We require no authorization or other headers for this API:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "63a630b3-45bb-4525-865b-083f322b944b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.agent_toolkits.openapi.toolkit import RequestsToolkit\n", + "from langchain_community.utilities.requests import TextRequestsWrapper\n", + "\n", + "toolkit = RequestsToolkit(\n", + " requests_wrapper=TextRequestsWrapper(headers={}),\n", + " allow_dangerous_requests=ALLOW_DANGEROUS_REQUEST,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f4224a64-843a-479d-8a7b-84719e4b9d0c", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "View available tools:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "70ea0f4e-9f10-4906-894b-08df832fd515", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[RequestsGetTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", + " RequestsPostTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", + " RequestsPatchTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", + " RequestsPutTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True),\n", + " RequestsDeleteTool(requests_wrapper=TextRequestsWrapper(headers={}, aiosession=None, auth=None, response_content_type='text', verify=True), allow_dangerous_requests=True)]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tools = toolkit.get_tools()\n", + "\n", + "tools" + ] + }, + { + "cell_type": "markdown", + "id": "a21a6ca4-d650-4b7d-a944-1a8771b5293a", + "metadata": {}, + "source": [ + "- [RequestsGetTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsGetTool.html)\n", + "- [RequestsPostTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsPostTool.html)\n", + "- [RequestsPatchTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsPatchTool.html)\n", + "- [RequestsPutTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsPutTool.html)\n", + "- [RequestsDeleteTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.requests.tool.RequestsDeleteTool.html)" + ] + }, + { + "cell_type": "markdown", + "id": "e2dbb304-abf2-472a-9130-f03150a40549", + "metadata": {}, + "source": [ + "## Use within an agent" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "db062da7-f22c-4f36-9df8-1da96c9f7538", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n", + "\n", + "system_message = \"\"\"\n", + "You have access to an API to help answer user queries.\n", + "Here is documentation on the API:\n", + "{api_spec}\n", + "\"\"\".format(api_spec=api_spec)\n", + "\n", + "agent_executor = create_react_agent(llm, tools, prompt=system_message)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "c1e47be9-374a-457c-928a-48f02b5530e3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "Fetch the top two posts. What are their titles?\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " requests_get (call_RV2SOyzCnV5h2sm4WPgG8fND)\n", + " Call ID: call_RV2SOyzCnV5h2sm4WPgG8fND\n", + " Args:\n", + " url: https://jsonplaceholder.typicode.com/posts?_limit=2\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: requests_get\n", + "\n", + "[\n", + " {\n", + " \"userId\": 1,\n", + " \"id\": 1,\n", + " \"title\": \"sunt aut facere repellat provident occaecati excepturi optio reprehenderit\",\n", + " \"body\": \"quia et suscipit\\nsuscipit recusandae consequuntur expedita et cum\\nreprehenderit molestiae ut ut quas totam\\nnostrum rerum est autem sunt rem eveniet architecto\"\n", + " },\n", + " {\n", + " \"userId\": 1,\n", + " \"id\": 2,\n", + " \"title\": \"qui est esse\",\n", + " \"body\": \"est rerum tempore vitae\\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\\nqui aperiam non debitis possimus qui neque nisi nulla\"\n", + " }\n", + "]\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "The titles of the top two posts are:\n", + "1. \"sunt aut facere repellat provident occaecati excepturi optio reprehenderit\"\n", + "2. \"qui est esse\"\n" + ] + } + ], + "source": [ + "example_query = \"Fetch the top two posts. What are their titles?\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "id": "01ec4886-de3d-4fda-bd05-e3f254810969", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all API toolkit features and configurations head to the API reference for [RequestsToolkit](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.openapi.toolkit.RequestsToolkit.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" } - ], - "source": [ - "example_query = \"Fetch the top two posts. What are their titles?\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " event[\"messages\"][-1].pretty_print()" - ] }, - { - "cell_type": "markdown", - "id": "01ec4886-de3d-4fda-bd05-e3f254810969", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all API toolkit features and configurations head to the API reference for [RequestsToolkit](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.openapi.toolkit.RequestsToolkit.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/tools/slack.ipynb b/docs/docs/integrations/tools/slack.ipynb index 84201881682..5c25c72c02c 100644 --- a/docs/docs/integrations/tools/slack.ipynb +++ b/docs/docs/integrations/tools/slack.ipynb @@ -1,274 +1,272 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Slack Toolkit\n", - "\n", - "This will help you getting started with the Slack [toolkit](/docs/concepts/tools/#toolkits). For detailed documentation of all SlackToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.slack.toolkit.SlackToolkit.html).\n", - "\n", - "## Setup\n", - "\n", - "To use this toolkit, you will need to get a token as explained in the [Slack API docs](https://api.slack.com/tutorials/tracks/getting-a-token). Once you've received a SLACK_USER_TOKEN, you can input it as an environment variable below." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"SLACK_USER_TOKEN\"):\n", - " os.environ[\"SLACK_USER_TOKEN\"] = getpass.getpass(\"Enter your Slack user token: \")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "This toolkit lives in the `langchain-community` package. We will also need the Slack SDK:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain-community slack_sdk" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Optionally, we can install beautifulsoup4 to assist in parsing HTML messages:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU beautifulsoup4 # This is optional but is useful for parsing HTML messages" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our toolkit:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.agent_toolkits import SlackToolkit\n", - "\n", - "toolkit = SlackToolkit()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Tools\n", - "\n", - "View available tools:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "[SlackGetChannel(client=),\n", - " SlackGetMessage(client=),\n", - " SlackScheduleMessage(client=),\n", - " SlackSendMessage(client=)]" + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Slack Toolkit\n", + "\n", + "This will help you getting started with the Slack [toolkit](/docs/concepts/tools/#toolkits). For detailed documentation of all SlackToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.slack.toolkit.SlackToolkit.html).\n", + "\n", + "## Setup\n", + "\n", + "To use this toolkit, you will need to get a token as explained in the [Slack API docs](https://api.slack.com/tutorials/tracks/getting-a-token). Once you've received a SLACK_USER_TOKEN, you can input it as an environment variable below." ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "tools = toolkit.get_tools()\n", - "\n", - "tools" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This toolkit loads:\n", - "\n", - "- [SlackGetChannel](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.get_channel.SlackGetChannel.html)\n", - "- [SlackGetMessage](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.get_message.SlackGetMessage.html)\n", - "- [SlackScheduleMessage](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.schedule_message.SlackScheduleMessage.html)\n", - "- [SlackSendMessage](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.send_message.SlackSendMessage.html)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Use within an agent\n", - "\n", - "Let's equip an agent with the Slack toolkit and query for information about a channel." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_openai import ChatOpenAI\n", - "from langgraph.prebuilt import create_react_agent\n", - "\n", - "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n", - "\n", - "agent_executor = create_react_agent(llm, tools)" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "================================\u001b[1m Human Message \u001b[0m=================================\n", - "\n", - "When was the #general channel created?\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " get_channelid_name_dict (call_NXDkALjoOx97uF1v0CoZTqtJ)\n", - " Call ID: call_NXDkALjoOx97uF1v0CoZTqtJ\n", - " Args:\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "The #general channel was created on timestamp 1671043305.\n" - ] - } - ], - "source": [ - "example_query = \"When was the #general channel created?\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " message = event[\"messages\"][-1]\n", - " if message.type != \"tool\": # mask sensitive information\n", - " event[\"messages\"][-1].pretty_print()" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"SLACK_USER_TOKEN\"):\n", + " os.environ[\"SLACK_USER_TOKEN\"] = getpass.getpass(\"Enter your Slack user token: \")" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "================================\u001b[1m Human Message \u001b[0m=================================\n", - "\n", - "Send a friendly greeting to channel C072Q1LP4QM.\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " send_message (call_xQxpv4wFeAZNZgSBJRIuaizi)\n", - " Call ID: call_xQxpv4wFeAZNZgSBJRIuaizi\n", - " Args:\n", - " message: Hello! Have a great day!\n", - " channel: C072Q1LP4QM\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "I have sent a friendly greeting to the channel C072Q1LP4QM.\n" - ] + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "This toolkit lives in the `langchain-community` package. We will also need the Slack SDK:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain-community slack_sdk" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Optionally, we can install beautifulsoup4 to assist in parsing HTML messages:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU beautifulsoup4 # This is optional but is useful for parsing HTML messages" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our toolkit:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.agent_toolkits import SlackToolkit\n", + "\n", + "toolkit = SlackToolkit()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "View available tools:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[SlackGetChannel(client=),\n", + " SlackGetMessage(client=),\n", + " SlackScheduleMessage(client=),\n", + " SlackSendMessage(client=)]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tools = toolkit.get_tools()\n", + "\n", + "tools" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This toolkit loads:\n", + "\n", + "- [SlackGetChannel](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.get_channel.SlackGetChannel.html)\n", + "- [SlackGetMessage](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.get_message.SlackGetMessage.html)\n", + "- [SlackScheduleMessage](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.schedule_message.SlackScheduleMessage.html)\n", + "- [SlackSendMessage](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.slack.send_message.SlackSendMessage.html)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Use within an agent\n", + "\n", + "Let's equip an agent with the Slack toolkit and query for information about a channel." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n", + "\n", + "agent_executor = create_react_agent(llm, tools)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "When was the #general channel created?\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " get_channelid_name_dict (call_NXDkALjoOx97uF1v0CoZTqtJ)\n", + " Call ID: call_NXDkALjoOx97uF1v0CoZTqtJ\n", + " Args:\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "The #general channel was created on timestamp 1671043305.\n" + ] + } + ], + "source": [ + "example_query = \"When was the #general channel created?\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " message = event[\"messages\"][-1]\n", + " if message.type != \"tool\": # mask sensitive information\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "Send a friendly greeting to channel C072Q1LP4QM.\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " send_message (call_xQxpv4wFeAZNZgSBJRIuaizi)\n", + " Call ID: call_xQxpv4wFeAZNZgSBJRIuaizi\n", + " Args:\n", + " message: Hello! Have a great day!\n", + " channel: C072Q1LP4QM\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "I have sent a friendly greeting to the channel C072Q1LP4QM.\n" + ] + } + ], + "source": [ + "example_query = \"Send a friendly greeting to channel C072Q1LP4QM.\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " message = event[\"messages\"][-1]\n", + " if message.type != \"tool\": # mask sensitive information\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `SlackToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.slack.toolkit.SlackToolkit.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" } - ], - "source": [ - "example_query = \"Send a friendly greeting to channel C072Q1LP4QM.\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " message = event[\"messages\"][-1]\n", - " if message.type != \"tool\": # mask sensitive information\n", - " event[\"messages\"][-1].pretty_print()" - ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `SlackToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.slack.toolkit.SlackToolkit.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/docs/docs/integrations/tools/sql_database.ipynb b/docs/docs/integrations/tools/sql_database.ipynb index 589c37b23d8..82f4b1999ba 100644 --- a/docs/docs/integrations/tools/sql_database.ipynb +++ b/docs/docs/integrations/tools/sql_database.ipynb @@ -1,620 +1,620 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "0e499e90-7a6d-4fab-8aab-31a4df417601", - "metadata": {}, - "source": [ - "# SQLDatabase Toolkit\n", - "\n", - "This will help you getting started with the SQL Database [toolkit](/docs/concepts/tools/#toolkits). For detailed documentation of all `SQLDatabaseToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html).\n", - "\n", - "Tools within the `SQLDatabaseToolkit` are designed to interact with a `SQL` database. \n", - "\n", - "A common application is to enable agents to answer questions using data in a relational database, potentially in an iterative fashion (e.g., recovering from errors).\n", - "\n", - "**⚠️ Security note ⚠️**\n", - "\n", - "Building Q&A systems of SQL databases requires executing model-generated SQL queries. There are inherent risks in doing this. Make sure that your database connection permissions are always scoped as narrowly as possible for your chain/agent's needs. This will mitigate though not eliminate the risks of building a model-driven system. For more on general security best practices, [see here](/docs/security).\n", - "\n", - "## Setup\n", - "\n", - "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3de6e3be-1fd9-42a3-8564-8ca7dca11e1c", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "31896b61-68d2-4b4d-be9d-b829eda327d1", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "This toolkit lives in the `langchain-community` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c4933e04-9120-4ccc-9ef7-369987823b0e", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install --upgrade --quiet langchain-community" - ] - }, - { - "cell_type": "markdown", - "id": "6ad08dbe-1642-448c-b58d-153810024375", - "metadata": {}, - "source": [ - "For demonstration purposes, we will access a prompt in the LangChain [Hub](https://smith.langchain.com/hub). We will also require `langgraph` to demonstrate the use of the toolkit with an agent. This is not required to use the toolkit." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f3dead45-9908-497d-a5a3-bce30642e88f", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install --upgrade --quiet langchainhub langgraph" - ] - }, - { - "cell_type": "markdown", - "id": "804533b1-2f16-497b-821b-c82d67fcf7b6", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "The `SQLDatabaseToolkit` toolkit requires:\n", - "\n", - "- a [SQLDatabase](https://python.langchain.com/api_reference/community/utilities/langchain_community.utilities.sql_database.SQLDatabase.html) object;\n", - "- a LLM or chat model (for instantiating the [QuerySQLCheckerTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.sql_database.tool.QuerySQLCheckerTool.html) tool).\n", - "\n", - "Below, we instantiate the toolkit with these objects. Let's first create a database object.\n", - "\n", - "This guide uses the example `Chinook` database based on [these instructions](https://database.guide/2-sample-databases-sqlite/).\n", - "\n", - "Below we will use the `requests` library to pull the `.sql` file and create an in-memory SQLite database. Note that this approach is lightweight, but ephemeral and not thread-safe. If you'd prefer, you can follow the instructions to save the file locally as `Chinook.db` and instantiate the database via `db = SQLDatabase.from_uri(\"sqlite:///Chinook.db\")`." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "40d05f9b-5a8f-4307-8f8b-4153db0fdfa9", - "metadata": {}, - "outputs": [], - "source": [ - "import sqlite3\n", - "\n", - "import requests\n", - "from langchain_community.utilities.sql_database import SQLDatabase\n", - "from sqlalchemy import create_engine\n", - "from sqlalchemy.pool import StaticPool\n", - "\n", - "\n", - "def get_engine_for_chinook_db():\n", - " \"\"\"Pull sql file, populate in-memory database, and create engine.\"\"\"\n", - " url = \"https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sql\"\n", - " response = requests.get(url)\n", - " sql_script = response.text\n", - "\n", - " connection = sqlite3.connect(\":memory:\", check_same_thread=False)\n", - " connection.executescript(sql_script)\n", - " return create_engine(\n", - " \"sqlite://\",\n", - " creator=lambda: connection,\n", - " poolclass=StaticPool,\n", - " connect_args={\"check_same_thread\": False},\n", - " )\n", - "\n", - "\n", - "engine = get_engine_for_chinook_db()\n", - "\n", - "db = SQLDatabase(engine)" - ] - }, - { - "cell_type": "markdown", - "id": "2b9a6326-78fd-4c42-a1cb-4316619ac449", - "metadata": {}, - "source": [ - "We will also need a LLM or chat model:\n", - "\n", - "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "cc6e6108-83d9-404f-8f31-474c2fbf5f6c", - "metadata": {}, - "outputs": [], - "source": [ - "# | output: false\n", - "# | echo: false\n", - "\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "llm = ChatOpenAI(temperature=0)" - ] - }, - { - "cell_type": "markdown", - "id": "77925e72-4730-43c3-8726-d68cedf635f4", - "metadata": {}, - "source": [ - "We can now instantiate the toolkit:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "42bd5a41-672a-4a53-b70a-2f0c0555758c", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit\n", - "\n", - "toolkit = SQLDatabaseToolkit(db=db, llm=llm)" - ] - }, - { - "cell_type": "markdown", - "id": "b2f882cf-4156-4a9f-a714-db97ec8ccc37", - "metadata": {}, - "source": [ - "## Tools\n", - "\n", - "View available tools:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "a18c3e69-bee0-4f5d-813e-eeb540f41b98", - "metadata": {}, - "outputs": [ + "cells": [ { - "data": { - "text/plain": [ - "[QuerySQLDatabaseTool(description=\"Input to this tool is a detailed and correct SQL query, output is a result from the database. If the query is not correct, an error message will be returned. If an error is returned, rewrite the query, check the query, and try again. If you encounter an issue with Unknown column 'xxxx' in 'field list', use sql_db_schema to query the correct table fields.\", db=),\n", - " InfoSQLDatabaseTool(description='Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables. Be sure that the tables actually exist by calling sql_db_list_tables first! Example Input: table1, table2, table3', db=),\n", - " ListSQLDatabaseTool(db=),\n", - " QuerySQLCheckerTool(description='Use this tool to double check if your query is correct before executing it. Always use this tool before executing a query with sql_db_query!', db=, llm=ChatOpenAI(client=, async_client=, root_client=, root_async_client=, temperature=0.0, model_kwargs={}, openai_api_key=SecretStr('**********')), llm_chain=LLMChain(verbose=False, prompt=PromptTemplate(input_variables=['dialect', 'query'], input_types={}, partial_variables={}, template='\\n{query}\\nDouble check the {dialect} query above for common mistakes, including:\\n- Using NOT IN with NULL values\\n- Using UNION when UNION ALL should have been used\\n- Using BETWEEN for exclusive ranges\\n- Data type mismatch in predicates\\n- Properly quoting identifiers\\n- Using the correct number of arguments for functions\\n- Casting to the correct data type\\n- Using the proper columns for joins\\n\\nIf there are any of the above mistakes, rewrite the query. If there are no mistakes, just reproduce the original query.\\n\\nOutput the final SQL query only.\\n\\nSQL Query: '), llm=ChatOpenAI(client=, async_client=, root_client=, root_async_client=, temperature=0.0, model_kwargs={}, openai_api_key=SecretStr('**********')), output_parser=StrOutputParser(), llm_kwargs={}))]" + "cell_type": "markdown", + "id": "0e499e90-7a6d-4fab-8aab-31a4df417601", + "metadata": {}, + "source": [ + "# SQLDatabase Toolkit\n", + "\n", + "This will help you getting started with the SQL Database [toolkit](/docs/concepts/tools/#toolkits). For detailed documentation of all `SQLDatabaseToolkit` features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html).\n", + "\n", + "Tools within the `SQLDatabaseToolkit` are designed to interact with a `SQL` database. \n", + "\n", + "A common application is to enable agents to answer questions using data in a relational database, potentially in an iterative fashion (e.g., recovering from errors).\n", + "\n", + "**⚠️ Security note ⚠️**\n", + "\n", + "Building Q&A systems of SQL databases requires executing model-generated SQL queries. There are inherent risks in doing this. Make sure that your database connection permissions are always scoped as narrowly as possible for your chain/agent's needs. This will mitigate though not eliminate the risks of building a model-driven system. For more on general security best practices, [see here](/docs/security).\n", + "\n", + "## Setup\n", + "\n", + "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "toolkit.get_tools()" - ] - }, - { - "cell_type": "markdown", - "id": "4e3fd064-aa86-448d-8db3-3c55eaa5bc15", - "metadata": {}, - "source": [ - "You can use the individual tools directly:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7fa8d00c-750c-4803-9b66-057d12b26b06", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.tools.sql_database.tool import (\n", - " InfoSQLDatabaseTool,\n", - " ListSQLDatabaseTool,\n", - " QuerySQLCheckerTool,\n", - " QuerySQLDatabaseTool,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "c067e0ed-dcca-4dcc-81b2-a0eeb4fc2a9f", - "metadata": {}, - "source": [ - "## Use within an agent\n", - "\n", - "Following the [SQL Q&A Tutorial](/docs/tutorials/sql_qa/#agents), below we equip a simple question-answering agent with the tools in our toolkit. First we pull a relevant prompt and populate it with its required parameters:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "eda12f8b-be90-4697-ac84-2ece9e2d1708", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "['dialect', 'top_k']\n" - ] - } - ], - "source": [ - "from langchain import hub\n", - "\n", - "prompt_template = hub.pull(\"langchain-ai/sql-agent-system-prompt\")\n", - "\n", - "assert len(prompt_template.messages) == 1\n", - "print(prompt_template.input_variables)" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "3470ae96-e5e5-4717-a6d6-d7d28c7b7347", - "metadata": {}, - "outputs": [], - "source": [ - "system_message = prompt_template.format(dialect=\"SQLite\", top_k=5)" - ] - }, - { - "cell_type": "markdown", - "id": "97930c07-36d1-4137-94ae-fe5ac83ecc44", - "metadata": {}, - "source": [ - "We then instantiate the agent:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "48bca92c-9b4b-4d5c-bcce-1b239c9e901c", - "metadata": {}, - "outputs": [], - "source": [ - "from langgraph.prebuilt import create_react_agent\n", - "\n", - "agent_executor = create_react_agent(llm, toolkit.get_tools(), prompt=system_message)" - ] - }, - { - "cell_type": "markdown", - "id": "09fb1845-1105-4f41-98b4-24756452a3e3", - "metadata": {}, - "source": [ - "And issue it a query:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "39e6d2bf-3194-4aba-854b-63faf919157b", - "metadata": {}, - "outputs": [ + "cell_type": "code", + "execution_count": null, + "id": "3de6e3be-1fd9-42a3-8564-8ca7dca11e1c", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "================================\u001b[1m Human Message \u001b[0m=================================\n", - "\n", - "Which country's customers spent the most?\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " sql_db_list_tables (call_EBPjyfzqXzFutDn8BklYACLj)\n", - " Call ID: call_EBPjyfzqXzFutDn8BklYACLj\n", - " Args:\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: sql_db_list_tables\n", - "\n", - "Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " sql_db_schema (call_kGcnKpxRVFIY8dPjYIJbRoVU)\n", - " Call ID: call_kGcnKpxRVFIY8dPjYIJbRoVU\n", - " Args:\n", - " table_names: Customer, Invoice, InvoiceLine\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: sql_db_schema\n", - "\n", - "\n", - "CREATE TABLE \"Customer\" (\n", - "\t\"CustomerId\" INTEGER NOT NULL, \n", - "\t\"FirstName\" NVARCHAR(40) NOT NULL, \n", - "\t\"LastName\" NVARCHAR(20) NOT NULL, \n", - "\t\"Company\" NVARCHAR(80), \n", - "\t\"Address\" NVARCHAR(70), \n", - "\t\"City\" NVARCHAR(40), \n", - "\t\"State\" NVARCHAR(40), \n", - "\t\"Country\" NVARCHAR(40), \n", - "\t\"PostalCode\" NVARCHAR(10), \n", - "\t\"Phone\" NVARCHAR(24), \n", - "\t\"Fax\" NVARCHAR(24), \n", - "\t\"Email\" NVARCHAR(60) NOT NULL, \n", - "\t\"SupportRepId\" INTEGER, \n", - "\tPRIMARY KEY (\"CustomerId\"), \n", - "\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\n", - ")\n", - "\n", - "/*\n", - "3 rows from Customer table:\n", - "CustomerId\tFirstName\tLastName\tCompany\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\tSupportRepId\n", - "1\tLuís\tGonçalves\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\tAv. Brigadeiro Faria Lima, 2170\tSão José dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\tluisg@embraer.com.br\t3\n", - "2\tLeonie\tKöhler\tNone\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n", - "3\tFrançois\tTremblay\tNone\t1498 rue Bélanger\tMontréal\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\tNone\tftremblay@gmail.com\t3\n", - "*/\n", - "\n", - "\n", - "CREATE TABLE \"Invoice\" (\n", - "\t\"InvoiceId\" INTEGER NOT NULL, \n", - "\t\"CustomerId\" INTEGER NOT NULL, \n", - "\t\"InvoiceDate\" DATETIME NOT NULL, \n", - "\t\"BillingAddress\" NVARCHAR(70), \n", - "\t\"BillingCity\" NVARCHAR(40), \n", - "\t\"BillingState\" NVARCHAR(40), \n", - "\t\"BillingCountry\" NVARCHAR(40), \n", - "\t\"BillingPostalCode\" NVARCHAR(10), \n", - "\t\"Total\" NUMERIC(10, 2) NOT NULL, \n", - "\tPRIMARY KEY (\"InvoiceId\"), \n", - "\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\n", - ")\n", - "\n", - "/*\n", - "3 rows from Invoice table:\n", - "InvoiceId\tCustomerId\tInvoiceDate\tBillingAddress\tBillingCity\tBillingState\tBillingCountry\tBillingPostalCode\tTotal\n", - "1\t2\t2021-01-01 00:00:00\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t1.98\n", - "2\t4\t2021-01-02 00:00:00\tUllevålsveien 14\tOslo\tNone\tNorway\t0171\t3.96\n", - "3\t8\t2021-01-03 00:00:00\tGrétrystraat 63\tBrussels\tNone\tBelgium\t1000\t5.94\n", - "*/\n", - "\n", - "\n", - "CREATE TABLE \"InvoiceLine\" (\n", - "\t\"InvoiceLineId\" INTEGER NOT NULL, \n", - "\t\"InvoiceId\" INTEGER NOT NULL, \n", - "\t\"TrackId\" INTEGER NOT NULL, \n", - "\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n", - "\t\"Quantity\" INTEGER NOT NULL, \n", - "\tPRIMARY KEY (\"InvoiceLineId\"), \n", - "\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n", - "\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\n", - ")\n", - "\n", - "/*\n", - "3 rows from InvoiceLine table:\n", - "InvoiceLineId\tInvoiceId\tTrackId\tUnitPrice\tQuantity\n", - "1\t1\t2\t0.99\t1\n", - "2\t1\t4\t0.99\t1\n", - "3\t2\t6\t0.99\t1\n", - "*/\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " sql_db_query (call_cTfI7OrY64FzJaDd49ILFWw7)\n", - " Call ID: call_cTfI7OrY64FzJaDd49ILFWw7\n", - " Args:\n", - " query: SELECT c.Country, SUM(i.Total) AS TotalSpent FROM Customer c JOIN Invoice i ON c.CustomerId = i.CustomerId GROUP BY c.Country ORDER BY TotalSpent DESC LIMIT 1\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: sql_db_query\n", - "\n", - "[('USA', 523.06)]\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "Customers from the USA spent the most, with a total amount spent of $523.06.\n" - ] - } - ], - "source": [ - "example_query = \"Which country's customers spent the most?\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " event[\"messages\"][-1].pretty_print()" - ] - }, - { - "cell_type": "markdown", - "id": "adbf3d8d-7570-45a5-950f-ce84db5145ab", - "metadata": {}, - "source": [ - "We can also observe the agent recover from an error:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "23c1235c-6d18-43e4-98ab-85b426b53d94", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "31896b61-68d2-4b4d-be9d-b829eda327d1", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "This toolkit lives in the `langchain-community` package:" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "================================\u001b[1m Human Message \u001b[0m=================================\n", - "\n", - "Who are the top 3 best selling artists?\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " sql_db_query (call_xAkvYiRFM7nCMKXsDNvk1OMx)\n", - " Call ID: call_xAkvYiRFM7nCMKXsDNvk1OMx\n", - " Args:\n", - " query: SELECT artist_name, SUM(quantity) AS total_sold FROM sales GROUP BY artist_name ORDER BY total_sold DESC LIMIT 3\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: sql_db_query\n", - "\n", - "Error: (sqlite3.OperationalError) no such table: sales\n", - "[SQL: SELECT artist_name, SUM(quantity) AS total_sold FROM sales GROUP BY artist_name ORDER BY total_sold DESC LIMIT 3]\n", - "(Background on this error at: https://sqlalche.me/e/20/e3q8)\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " sql_db_list_tables (call_K4Zvbowsq7XPgGFepbvc5G7i)\n", - " Call ID: call_K4Zvbowsq7XPgGFepbvc5G7i\n", - " Args:\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: sql_db_list_tables\n", - "\n", - "Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " sql_db_schema (call_tUztueSK7VO2klZ99xT4ZVhM)\n", - " Call ID: call_tUztueSK7VO2klZ99xT4ZVhM\n", - " Args:\n", - " table_names: Artist, Album, InvoiceLine\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: sql_db_schema\n", - "\n", - "\n", - "CREATE TABLE \"Album\" (\n", - "\t\"AlbumId\" INTEGER NOT NULL, \n", - "\t\"Title\" NVARCHAR(160) NOT NULL, \n", - "\t\"ArtistId\" INTEGER NOT NULL, \n", - "\tPRIMARY KEY (\"AlbumId\"), \n", - "\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\n", - ")\n", - "\n", - "/*\n", - "3 rows from Album table:\n", - "AlbumId\tTitle\tArtistId\n", - "1\tFor Those About To Rock We Salute You\t1\n", - "2\tBalls to the Wall\t2\n", - "3\tRestless and Wild\t2\n", - "*/\n", - "\n", - "\n", - "CREATE TABLE \"Artist\" (\n", - "\t\"ArtistId\" INTEGER NOT NULL, \n", - "\t\"Name\" NVARCHAR(120), \n", - "\tPRIMARY KEY (\"ArtistId\")\n", - ")\n", - "\n", - "/*\n", - "3 rows from Artist table:\n", - "ArtistId\tName\n", - "1\tAC/DC\n", - "2\tAccept\n", - "3\tAerosmith\n", - "*/\n", - "\n", - "\n", - "CREATE TABLE \"InvoiceLine\" (\n", - "\t\"InvoiceLineId\" INTEGER NOT NULL, \n", - "\t\"InvoiceId\" INTEGER NOT NULL, \n", - "\t\"TrackId\" INTEGER NOT NULL, \n", - "\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n", - "\t\"Quantity\" INTEGER NOT NULL, \n", - "\tPRIMARY KEY (\"InvoiceLineId\"), \n", - "\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n", - "\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\n", - ")\n", - "\n", - "/*\n", - "3 rows from InvoiceLine table:\n", - "InvoiceLineId\tInvoiceId\tTrackId\tUnitPrice\tQuantity\n", - "1\t1\t2\t0.99\t1\n", - "2\t1\t4\t0.99\t1\n", - "3\t2\t6\t0.99\t1\n", - "*/\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "Tool Calls:\n", - " sql_db_query (call_tVtLQIRPmCM6pukgpHFfq86A)\n", - " Call ID: call_tVtLQIRPmCM6pukgpHFfq86A\n", - " Args:\n", - " query: SELECT Artist.Name AS artist_name, SUM(InvoiceLine.Quantity) AS total_sold FROM Artist JOIN Album ON Artist.ArtistId = Album.ArtistId JOIN Track ON Album.AlbumId = Track.AlbumId JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY total_sold DESC LIMIT 3\n", - "=================================\u001b[1m Tool Message \u001b[0m=================================\n", - "Name: sql_db_query\n", - "\n", - "[('Iron Maiden', 140), ('U2', 107), ('Metallica', 91)]\n", - "==================================\u001b[1m Ai Message \u001b[0m==================================\n", - "\n", - "The top 3 best selling artists are:\n", - "1. Iron Maiden - 140 units sold\n", - "2. U2 - 107 units sold\n", - "3. Metallica - 91 units sold\n" - ] + "cell_type": "code", + "execution_count": null, + "id": "c4933e04-9120-4ccc-9ef7-369987823b0e", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchain-community" + ] + }, + { + "cell_type": "markdown", + "id": "6ad08dbe-1642-448c-b58d-153810024375", + "metadata": {}, + "source": [ + "For demonstration purposes, we will access a prompt in the LangChain [Hub](https://smith.langchain.com/hub). We will also require `langgraph` to demonstrate the use of the toolkit with an agent. This is not required to use the toolkit." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f3dead45-9908-497d-a5a3-bce30642e88f", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install --upgrade --quiet langchainhub langgraph" + ] + }, + { + "cell_type": "markdown", + "id": "804533b1-2f16-497b-821b-c82d67fcf7b6", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "The `SQLDatabaseToolkit` toolkit requires:\n", + "\n", + "- a [SQLDatabase](https://python.langchain.com/api_reference/community/utilities/langchain_community.utilities.sql_database.SQLDatabase.html) object;\n", + "- a LLM or chat model (for instantiating the [QuerySQLCheckerTool](https://python.langchain.com/api_reference/community/tools/langchain_community.tools.sql_database.tool.QuerySQLCheckerTool.html) tool).\n", + "\n", + "Below, we instantiate the toolkit with these objects. Let's first create a database object.\n", + "\n", + "This guide uses the example `Chinook` database based on [these instructions](https://database.guide/2-sample-databases-sqlite/).\n", + "\n", + "Below we will use the `requests` library to pull the `.sql` file and create an in-memory SQLite database. Note that this approach is lightweight, but ephemeral and not thread-safe. If you'd prefer, you can follow the instructions to save the file locally as `Chinook.db` and instantiate the database via `db = SQLDatabase.from_uri(\"sqlite:///Chinook.db\")`." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "40d05f9b-5a8f-4307-8f8b-4153db0fdfa9", + "metadata": {}, + "outputs": [], + "source": [ + "import sqlite3\n", + "\n", + "import requests\n", + "from langchain_community.utilities.sql_database import SQLDatabase\n", + "from sqlalchemy import create_engine\n", + "from sqlalchemy.pool import StaticPool\n", + "\n", + "\n", + "def get_engine_for_chinook_db():\n", + " \"\"\"Pull sql file, populate in-memory database, and create engine.\"\"\"\n", + " url = \"https://raw.githubusercontent.com/lerocha/chinook-database/master/ChinookDatabase/DataSources/Chinook_Sqlite.sql\"\n", + " response = requests.get(url)\n", + " sql_script = response.text\n", + "\n", + " connection = sqlite3.connect(\":memory:\", check_same_thread=False)\n", + " connection.executescript(sql_script)\n", + " return create_engine(\n", + " \"sqlite://\",\n", + " creator=lambda: connection,\n", + " poolclass=StaticPool,\n", + " connect_args={\"check_same_thread\": False},\n", + " )\n", + "\n", + "\n", + "engine = get_engine_for_chinook_db()\n", + "\n", + "db = SQLDatabase(engine)" + ] + }, + { + "cell_type": "markdown", + "id": "2b9a6326-78fd-4c42-a1cb-4316619ac449", + "metadata": {}, + "source": [ + "We will also need a LLM or chat model:\n", + "\n", + "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "cc6e6108-83d9-404f-8f31-474c2fbf5f6c", + "metadata": {}, + "outputs": [], + "source": [ + "# | output: false\n", + "# | echo: false\n", + "\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(temperature=0)" + ] + }, + { + "cell_type": "markdown", + "id": "77925e72-4730-43c3-8726-d68cedf635f4", + "metadata": {}, + "source": [ + "We can now instantiate the toolkit:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "42bd5a41-672a-4a53-b70a-2f0c0555758c", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit\n", + "\n", + "toolkit = SQLDatabaseToolkit(db=db, llm=llm)" + ] + }, + { + "cell_type": "markdown", + "id": "b2f882cf-4156-4a9f-a714-db97ec8ccc37", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "View available tools:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "a18c3e69-bee0-4f5d-813e-eeb540f41b98", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[QuerySQLDatabaseTool(description=\"Input to this tool is a detailed and correct SQL query, output is a result from the database. If the query is not correct, an error message will be returned. If an error is returned, rewrite the query, check the query, and try again. If you encounter an issue with Unknown column 'xxxx' in 'field list', use sql_db_schema to query the correct table fields.\", db=),\n", + " InfoSQLDatabaseTool(description='Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables. Be sure that the tables actually exist by calling sql_db_list_tables first! Example Input: table1, table2, table3', db=),\n", + " ListSQLDatabaseTool(db=),\n", + " QuerySQLCheckerTool(description='Use this tool to double check if your query is correct before executing it. Always use this tool before executing a query with sql_db_query!', db=, llm=ChatOpenAI(client=, async_client=, root_client=, root_async_client=, temperature=0.0, model_kwargs={}, openai_api_key=SecretStr('**********')), llm_chain=LLMChain(verbose=False, prompt=PromptTemplate(input_variables=['dialect', 'query'], input_types={}, partial_variables={}, template='\\n{query}\\nDouble check the {dialect} query above for common mistakes, including:\\n- Using NOT IN with NULL values\\n- Using UNION when UNION ALL should have been used\\n- Using BETWEEN for exclusive ranges\\n- Data type mismatch in predicates\\n- Properly quoting identifiers\\n- Using the correct number of arguments for functions\\n- Casting to the correct data type\\n- Using the proper columns for joins\\n\\nIf there are any of the above mistakes, rewrite the query. If there are no mistakes, just reproduce the original query.\\n\\nOutput the final SQL query only.\\n\\nSQL Query: '), llm=ChatOpenAI(client=, async_client=, root_client=, root_async_client=, temperature=0.0, model_kwargs={}, openai_api_key=SecretStr('**********')), output_parser=StrOutputParser(), llm_kwargs={}))]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "toolkit.get_tools()" + ] + }, + { + "cell_type": "markdown", + "id": "4e3fd064-aa86-448d-8db3-3c55eaa5bc15", + "metadata": {}, + "source": [ + "You can use the individual tools directly:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7fa8d00c-750c-4803-9b66-057d12b26b06", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.tools.sql_database.tool import (\n", + " InfoSQLDatabaseTool,\n", + " ListSQLDatabaseTool,\n", + " QuerySQLCheckerTool,\n", + " QuerySQLDatabaseTool,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "c067e0ed-dcca-4dcc-81b2-a0eeb4fc2a9f", + "metadata": {}, + "source": [ + "## Use within an agent\n", + "\n", + "Following the [SQL Q&A Tutorial](/docs/tutorials/sql_qa/#agents), below we equip a simple question-answering agent with the tools in our toolkit. First we pull a relevant prompt and populate it with its required parameters:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "eda12f8b-be90-4697-ac84-2ece9e2d1708", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['dialect', 'top_k']\n" + ] + } + ], + "source": [ + "from langchain import hub\n", + "\n", + "prompt_template = hub.pull(\"langchain-ai/sql-agent-system-prompt\")\n", + "\n", + "assert len(prompt_template.messages) == 1\n", + "print(prompt_template.input_variables)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "3470ae96-e5e5-4717-a6d6-d7d28c7b7347", + "metadata": {}, + "outputs": [], + "source": [ + "system_message = prompt_template.format(dialect=\"SQLite\", top_k=5)" + ] + }, + { + "cell_type": "markdown", + "id": "97930c07-36d1-4137-94ae-fe5ac83ecc44", + "metadata": {}, + "source": [ + "We then instantiate the agent:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "48bca92c-9b4b-4d5c-bcce-1b239c9e901c", + "metadata": {}, + "outputs": [], + "source": [ + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "agent_executor = create_react_agent(llm, toolkit.get_tools(), prompt=system_message)" + ] + }, + { + "cell_type": "markdown", + "id": "09fb1845-1105-4f41-98b4-24756452a3e3", + "metadata": {}, + "source": [ + "And issue it a query:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "39e6d2bf-3194-4aba-854b-63faf919157b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "Which country's customers spent the most?\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " sql_db_list_tables (call_EBPjyfzqXzFutDn8BklYACLj)\n", + " Call ID: call_EBPjyfzqXzFutDn8BklYACLj\n", + " Args:\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: sql_db_list_tables\n", + "\n", + "Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " sql_db_schema (call_kGcnKpxRVFIY8dPjYIJbRoVU)\n", + " Call ID: call_kGcnKpxRVFIY8dPjYIJbRoVU\n", + " Args:\n", + " table_names: Customer, Invoice, InvoiceLine\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: sql_db_schema\n", + "\n", + "\n", + "CREATE TABLE \"Customer\" (\n", + "\t\"CustomerId\" INTEGER NOT NULL, \n", + "\t\"FirstName\" NVARCHAR(40) NOT NULL, \n", + "\t\"LastName\" NVARCHAR(20) NOT NULL, \n", + "\t\"Company\" NVARCHAR(80), \n", + "\t\"Address\" NVARCHAR(70), \n", + "\t\"City\" NVARCHAR(40), \n", + "\t\"State\" NVARCHAR(40), \n", + "\t\"Country\" NVARCHAR(40), \n", + "\t\"PostalCode\" NVARCHAR(10), \n", + "\t\"Phone\" NVARCHAR(24), \n", + "\t\"Fax\" NVARCHAR(24), \n", + "\t\"Email\" NVARCHAR(60) NOT NULL, \n", + "\t\"SupportRepId\" INTEGER, \n", + "\tPRIMARY KEY (\"CustomerId\"), \n", + "\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\n", + ")\n", + "\n", + "/*\n", + "3 rows from Customer table:\n", + "CustomerId\tFirstName\tLastName\tCompany\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\tSupportRepId\n", + "1\tLuís\tGonçalves\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\tAv. Brigadeiro Faria Lima, 2170\tSão José dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\tluisg@embraer.com.br\t3\n", + "2\tLeonie\tKöhler\tNone\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n", + "3\tFrançois\tTremblay\tNone\t1498 rue Bélanger\tMontréal\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\tNone\tftremblay@gmail.com\t3\n", + "*/\n", + "\n", + "\n", + "CREATE TABLE \"Invoice\" (\n", + "\t\"InvoiceId\" INTEGER NOT NULL, \n", + "\t\"CustomerId\" INTEGER NOT NULL, \n", + "\t\"InvoiceDate\" DATETIME NOT NULL, \n", + "\t\"BillingAddress\" NVARCHAR(70), \n", + "\t\"BillingCity\" NVARCHAR(40), \n", + "\t\"BillingState\" NVARCHAR(40), \n", + "\t\"BillingCountry\" NVARCHAR(40), \n", + "\t\"BillingPostalCode\" NVARCHAR(10), \n", + "\t\"Total\" NUMERIC(10, 2) NOT NULL, \n", + "\tPRIMARY KEY (\"InvoiceId\"), \n", + "\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\n", + ")\n", + "\n", + "/*\n", + "3 rows from Invoice table:\n", + "InvoiceId\tCustomerId\tInvoiceDate\tBillingAddress\tBillingCity\tBillingState\tBillingCountry\tBillingPostalCode\tTotal\n", + "1\t2\t2021-01-01 00:00:00\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t1.98\n", + "2\t4\t2021-01-02 00:00:00\tUllevålsveien 14\tOslo\tNone\tNorway\t0171\t3.96\n", + "3\t8\t2021-01-03 00:00:00\tGrétrystraat 63\tBrussels\tNone\tBelgium\t1000\t5.94\n", + "*/\n", + "\n", + "\n", + "CREATE TABLE \"InvoiceLine\" (\n", + "\t\"InvoiceLineId\" INTEGER NOT NULL, \n", + "\t\"InvoiceId\" INTEGER NOT NULL, \n", + "\t\"TrackId\" INTEGER NOT NULL, \n", + "\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n", + "\t\"Quantity\" INTEGER NOT NULL, \n", + "\tPRIMARY KEY (\"InvoiceLineId\"), \n", + "\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n", + "\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\n", + ")\n", + "\n", + "/*\n", + "3 rows from InvoiceLine table:\n", + "InvoiceLineId\tInvoiceId\tTrackId\tUnitPrice\tQuantity\n", + "1\t1\t2\t0.99\t1\n", + "2\t1\t4\t0.99\t1\n", + "3\t2\t6\t0.99\t1\n", + "*/\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " sql_db_query (call_cTfI7OrY64FzJaDd49ILFWw7)\n", + " Call ID: call_cTfI7OrY64FzJaDd49ILFWw7\n", + " Args:\n", + " query: SELECT c.Country, SUM(i.Total) AS TotalSpent FROM Customer c JOIN Invoice i ON c.CustomerId = i.CustomerId GROUP BY c.Country ORDER BY TotalSpent DESC LIMIT 1\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: sql_db_query\n", + "\n", + "[('USA', 523.06)]\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "Customers from the USA spent the most, with a total amount spent of $523.06.\n" + ] + } + ], + "source": [ + "example_query = \"Which country's customers spent the most?\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "id": "adbf3d8d-7570-45a5-950f-ce84db5145ab", + "metadata": {}, + "source": [ + "We can also observe the agent recover from an error:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "23c1235c-6d18-43e4-98ab-85b426b53d94", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "================================\u001b[1m Human Message \u001b[0m=================================\n", + "\n", + "Who are the top 3 best selling artists?\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " sql_db_query (call_xAkvYiRFM7nCMKXsDNvk1OMx)\n", + " Call ID: call_xAkvYiRFM7nCMKXsDNvk1OMx\n", + " Args:\n", + " query: SELECT artist_name, SUM(quantity) AS total_sold FROM sales GROUP BY artist_name ORDER BY total_sold DESC LIMIT 3\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: sql_db_query\n", + "\n", + "Error: (sqlite3.OperationalError) no such table: sales\n", + "[SQL: SELECT artist_name, SUM(quantity) AS total_sold FROM sales GROUP BY artist_name ORDER BY total_sold DESC LIMIT 3]\n", + "(Background on this error at: https://sqlalche.me/e/20/e3q8)\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " sql_db_list_tables (call_K4Zvbowsq7XPgGFepbvc5G7i)\n", + " Call ID: call_K4Zvbowsq7XPgGFepbvc5G7i\n", + " Args:\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: sql_db_list_tables\n", + "\n", + "Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " sql_db_schema (call_tUztueSK7VO2klZ99xT4ZVhM)\n", + " Call ID: call_tUztueSK7VO2klZ99xT4ZVhM\n", + " Args:\n", + " table_names: Artist, Album, InvoiceLine\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: sql_db_schema\n", + "\n", + "\n", + "CREATE TABLE \"Album\" (\n", + "\t\"AlbumId\" INTEGER NOT NULL, \n", + "\t\"Title\" NVARCHAR(160) NOT NULL, \n", + "\t\"ArtistId\" INTEGER NOT NULL, \n", + "\tPRIMARY KEY (\"AlbumId\"), \n", + "\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\n", + ")\n", + "\n", + "/*\n", + "3 rows from Album table:\n", + "AlbumId\tTitle\tArtistId\n", + "1\tFor Those About To Rock We Salute You\t1\n", + "2\tBalls to the Wall\t2\n", + "3\tRestless and Wild\t2\n", + "*/\n", + "\n", + "\n", + "CREATE TABLE \"Artist\" (\n", + "\t\"ArtistId\" INTEGER NOT NULL, \n", + "\t\"Name\" NVARCHAR(120), \n", + "\tPRIMARY KEY (\"ArtistId\")\n", + ")\n", + "\n", + "/*\n", + "3 rows from Artist table:\n", + "ArtistId\tName\n", + "1\tAC/DC\n", + "2\tAccept\n", + "3\tAerosmith\n", + "*/\n", + "\n", + "\n", + "CREATE TABLE \"InvoiceLine\" (\n", + "\t\"InvoiceLineId\" INTEGER NOT NULL, \n", + "\t\"InvoiceId\" INTEGER NOT NULL, \n", + "\t\"TrackId\" INTEGER NOT NULL, \n", + "\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n", + "\t\"Quantity\" INTEGER NOT NULL, \n", + "\tPRIMARY KEY (\"InvoiceLineId\"), \n", + "\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n", + "\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\n", + ")\n", + "\n", + "/*\n", + "3 rows from InvoiceLine table:\n", + "InvoiceLineId\tInvoiceId\tTrackId\tUnitPrice\tQuantity\n", + "1\t1\t2\t0.99\t1\n", + "2\t1\t4\t0.99\t1\n", + "3\t2\t6\t0.99\t1\n", + "*/\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "Tool Calls:\n", + " sql_db_query (call_tVtLQIRPmCM6pukgpHFfq86A)\n", + " Call ID: call_tVtLQIRPmCM6pukgpHFfq86A\n", + " Args:\n", + " query: SELECT Artist.Name AS artist_name, SUM(InvoiceLine.Quantity) AS total_sold FROM Artist JOIN Album ON Artist.ArtistId = Album.ArtistId JOIN Track ON Album.AlbumId = Track.AlbumId JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY total_sold DESC LIMIT 3\n", + "=================================\u001b[1m Tool Message \u001b[0m=================================\n", + "Name: sql_db_query\n", + "\n", + "[('Iron Maiden', 140), ('U2', 107), ('Metallica', 91)]\n", + "==================================\u001b[1m Ai Message \u001b[0m==================================\n", + "\n", + "The top 3 best selling artists are:\n", + "1. Iron Maiden - 140 units sold\n", + "2. U2 - 107 units sold\n", + "3. Metallica - 91 units sold\n" + ] + } + ], + "source": [ + "example_query = \"Who are the top 3 best selling artists?\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "id": "73521f1b-be03-44e6-8b27-a9a46ae8e962", + "metadata": {}, + "source": [ + "## Specific functionality\n", + "\n", + "`SQLDatabaseToolkit` implements a [.get_context](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html#langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.get_context) method as a convenience for use in prompts or other contexts.\n", + "\n", + "**⚠️ Disclaimer ⚠️** : The agent may generate insert/update/delete queries. When this is not expected, use a custom prompt or create a SQL users without write permissions.\n", + "\n", + "The final user might overload your SQL database by asking a simple question such as \"run the biggest query possible\". The generated query might look like:\n", + "\n", + "```sql\n", + "SELECT * FROM \"public\".\"users\"\n", + " JOIN \"public\".\"user_permissions\" ON \"public\".\"users\".id = \"public\".\"user_permissions\".user_id\n", + " JOIN \"public\".\"projects\" ON \"public\".\"users\".id = \"public\".\"projects\".user_id\n", + " JOIN \"public\".\"events\" ON \"public\".\"projects\".id = \"public\".\"events\".project_id;\n", + "```\n", + "\n", + "For a transactional SQL database, if one of the table above contains millions of rows, the query might cause trouble to other applications using the same database.\n", + "\n", + "Most datawarehouse oriented databases support user-level quota, for limiting resource usage." + ] + }, + { + "cell_type": "markdown", + "id": "1aa8a7e3-87ca-4963-a224-0cbdc9d88714", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all SQLDatabaseToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" } - ], - "source": [ - "example_query = \"Who are the top 3 best selling artists?\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " event[\"messages\"][-1].pretty_print()" - ] }, - { - "cell_type": "markdown", - "id": "73521f1b-be03-44e6-8b27-a9a46ae8e962", - "metadata": {}, - "source": [ - "## Specific functionality\n", - "\n", - "`SQLDatabaseToolkit` implements a [.get_context](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html#langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.get_context) method as a convenience for use in prompts or other contexts.\n", - "\n", - "**⚠️ Disclaimer ⚠️** : The agent may generate insert/update/delete queries. When this is not expected, use a custom prompt or create a SQL users without write permissions.\n", - "\n", - "The final user might overload your SQL database by asking a simple question such as \"run the biggest query possible\". The generated query might look like:\n", - "\n", - "```sql\n", - "SELECT * FROM \"public\".\"users\"\n", - " JOIN \"public\".\"user_permissions\" ON \"public\".\"users\".id = \"public\".\"user_permissions\".user_id\n", - " JOIN \"public\".\"projects\" ON \"public\".\"users\".id = \"public\".\"projects\".user_id\n", - " JOIN \"public\".\"events\" ON \"public\".\"projects\".id = \"public\".\"events\".project_id;\n", - "```\n", - "\n", - "For a transactional SQL database, if one of the table above contains millions of rows, the query might cause trouble to other applications using the same database.\n", - "\n", - "Most datawarehouse oriented databases support user-level quota, for limiting resource usage." - ] - }, - { - "cell_type": "markdown", - "id": "1aa8a7e3-87ca-4963-a224-0cbdc9d88714", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all SQLDatabaseToolkit features and configurations head to the [API reference](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/vectorstores/redis.ipynb b/docs/docs/integrations/vectorstores/redis.ipynb index 603a1e4527f..a3f4dafadee 100644 --- a/docs/docs/integrations/vectorstores/redis.ipynb +++ b/docs/docs/integrations/vectorstores/redis.ipynb @@ -1,1134 +1,1132 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "1957f5cb", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: Redis\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "ef1f0986", - "metadata": {}, - "source": [ - "# Redis Vector Store\n", - "\n", - "This notebook covers how to get started with the Redis vector store.\n", - "\n", - ">[Redis](https://redis.io/docs/stack/vectorsearch/) is a popular open-source, in-memory data structure store that can be used as a database, cache, message broker, and queue. It now includes vector similarity search capabilities, making it suitable for use as a vector store." - ] - }, - { - "cell_type": "markdown", - "id": "a717e10e-c8d7-41bc-9dd0-447d11f90b68", - "metadata": {}, - "source": [ - "### What is Redis?\n", - "\n", - "Most developers are familiar with `Redis`. At its core, `Redis` is a NoSQL Database in the key-value family that can used as a cache, message broker, stream processing and a primary database. Developers choose `Redis` because it is fast, has a large ecosystem of client libraries, and has been deployed by major enterprises for years.\n", - "\n", - "On top of these traditional use cases, `Redis` provides additional capabilities like the Search and Query capability that allows users to create secondary index structures within `Redis`. This allows `Redis` to be a Vector Database, at the speed of a cache. \n", - "\n", - "\n", - "### Redis as a Vector Database\n", - "\n", - "`Redis` uses compressed, inverted indexes for fast indexing with a low memory footprint. It also supports a number of advanced features such as:\n", - "\n", - "* Indexing of multiple fields in Redis hashes and `JSON`\n", - "* Vector similarity search (with `HNSW` (ANN) or `FLAT` (KNN))\n", - "* Vector Range Search (e.g. find all vectors within a radius of a query vector)\n", - "* Incremental indexing without performance loss\n", - "* Document ranking (using [tf-idf](https://en.wikipedia.org/wiki/Tf%E2%80%93idf), with optional user-provided weights)\n", - "* Field weighting\n", - "* Complex boolean queries with `AND`, `OR`, and `NOT` operators\n", - "* Prefix matching, fuzzy matching, and exact-phrase queries\n", - "* Support for [double-metaphone phonetic matching](https://redis.io/docs/stack/search/reference/phonetic_matching/)\n", - "* Auto-complete suggestions (with fuzzy prefix suggestions)\n", - "* Stemming-based query expansion in [many languages](https://redis.io/docs/stack/search/reference/stemming/) (using [Snowball](http://snowballstem.org/))\n", - "* Support for Chinese-language tokenization and querying (using [Friso](https://github.com/lionsoul2014/friso))\n", - "* Numeric filters and ranges\n", - "* Geospatial searches using Redis geospatial indexing\n", - "* A powerful aggregations engine\n", - "* Supports for all `utf-8` encoded text\n", - "* Retrieve full documents, selected fields, or only the document IDs\n", - "* Sorting results (for example, by creation date)\n", - "\n", - "### Clients\n", - "\n", - "Since `Redis` is much more than just a vector database, there are often use cases that demand the usage of a `Redis` client besides just the `LangChain` integration. You can use any standard `Redis` client library to run Search and Query commands, but it's easiest to use a library that wraps the Search and Query API. Below are a few examples, but you can find more client libraries [here](https://redis.io/resources/clients/).\n", - "\n", - "| Project | Language | License | Author | Stars |\n", - "|----------|---------|--------|---------|-------|\n", - "| [jedis][jedis-url] | Java | MIT | [Redis][redis-url] | ![Stars][jedis-stars] |\n", - "| [redisvl][redisvl-url] | Python | MIT | [Redis][redis-url] | ![Stars][redisvl-stars] |\n", - "| [redis-py][redis-py-url] | Python | MIT | [Redis][redis-url] | ![Stars][redis-py-stars] |\n", - "| [node-redis][node-redis-url] | Node.js | MIT | [Redis][redis-url] | ![Stars][node-redis-stars] |\n", - "| [nredisstack][nredisstack-url] | .NET | MIT | [Redis][redis-url] | ![Stars][nredisstack-stars] |\n", - "\n", - "[redis-url]: https://redis.com\n", - "\n", - "[redisvl-url]: https://github.com/redis/redis-vl-python\n", - "[redisvl-stars]: https://img.shields.io/github/stars/redis/redisvl.svg?style=social&label=Star&maxAge=2592000\n", - "[redisvl-package]: https://pypi.python.org/pypi/redisvl\n", - "\n", - "[redis-py-url]: https://github.com/redis/redis-py\n", - "[redis-py-stars]: https://img.shields.io/github/stars/redis/redis-py.svg?style=social&label=Star&maxAge=2592000\n", - "[redis-py-package]: https://pypi.python.org/pypi/redis\n", - "\n", - "[jedis-url]: https://github.com/redis/jedis\n", - "[jedis-stars]: https://img.shields.io/github/stars/redis/jedis.svg?style=social&label=Star&maxAge=2592000\n", - "[Jedis-package]: https://search.maven.org/artifact/redis.clients/jedis\n", - "\n", - "[nredisstack-url]: https://github.com/redis/nredisstack\n", - "[nredisstack-stars]: https://img.shields.io/github/stars/redis/nredisstack.svg?style=social&label=Star&maxAge=2592000\n", - "[nredisstack-package]: https://www.nuget.org/packages/nredisstack/\n", - "\n", - "[node-redis-url]: https://github.com/redis/node-redis\n", - "[node-redis-stars]: https://img.shields.io/github/stars/redis/node-redis.svg?style=social&label=Star&maxAge=2592000\n", - "[node-redis-package]: https://www.npmjs.com/package/redis\n", - "\n", - "[redis-om-python-url]: https://github.com/redis/redis-om-python\n", - "[redis-om-python-author]: https://redis.com\n", - "[redis-om-python-stars]: https://img.shields.io/github/stars/redis/redis-om-python.svg?style=social&label=Star&maxAge=2592000\n", - "\n", - "[redisearch-go-url]: https://github.com/RediSearch/redisearch-go\n", - "[redisearch-go-author]: https://redis.com\n", - "[redisearch-go-stars]: https://img.shields.io/github/stars/RediSearch/redisearch-go.svg?style=social&label=Star&maxAge=2592000\n", - "\n", - "[redisearch-api-rs-url]: https://github.com/RediSearch/redisearch-api-rs\n", - "[redisearch-api-rs-author]: https://redis.com\n", - "[redisearch-api-rs-stars]: https://img.shields.io/github/stars/RediSearch/redisearch-api-rs.svg?style=social&label=Star&maxAge=2592000\n", - "\n", - "\n", - "### Deployment options\n", - "\n", - "There are many ways to deploy Redis with RediSearch. The easiest way to get started is to use Docker, but there are are many potential options for deployment such as\n", - "\n", - "- [Redis Cloud](https://redis.com/redis-enterprise-cloud/overview/)\n", - "- [Docker (Redis Stack)](https://hub.docker.com/r/redis/redis-stack)\n", - "- Cloud marketplaces: [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-e6y7ork67pjwg?sr=0-2&ref_=beagle&applicationId=AWSMPContessa), [Google Marketplace](https://console.cloud.google.com/marketplace/details/redislabs-public/redis-enterprise?pli=1), or [Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/garantiadata.redis_enterprise_1sp_public_preview?tab=Overview)\n", - "- On-premise: [Redis Enterprise Software](https://redis.com/redis-enterprise-software/overview/)\n", - "- Kubernetes: [Redis Enterprise Software on Kubernetes](https://docs.redis.com/latest/kubernetes/)\n", - " \n", - "### Redis connection Url schemas\n", - "\n", - "Valid Redis Url schemas are:\n", - "1. `redis://` - Connection to Redis standalone, unencrypted\n", - "2. `rediss://` - Connection to Redis standalone, with TLS encryption\n", - "3. `redis+sentinel://` - Connection to Redis server via Redis Sentinel, unencrypted\n", - "4. `rediss+sentinel://` - Connection to Redis server via Redis Sentinel, booth connections with TLS encryption\n", - "\n", - "More information about additional connection parameters can be found in the [redis-py documentation](https://redis-py.readthedocs.io/en/stable/connections.html)." - ] - }, - { - "cell_type": "markdown", - "id": "36fdc060", - "metadata": {}, - "source": [ - "## Setup\n", - "\n", - "To use the RedisVectorStore, you'll need to install the `langchain-redis` partner package, as well as the other packages used throughout this notebook." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "64e28aa6", - "metadata": {}, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n" - ] - } - ], - "source": [ - "%pip install -qU langchain-redis langchain-huggingface sentence-transformers scikit-learn" - ] - }, - { - "cell_type": "markdown", - "id": "37d388a3-1a56-453e-8f84-e24a72d682eb", - "metadata": {}, - "source": [ - "### Credentials\n", - "\n", - "Redis connection credentials are passed as part of the Redis Connection URL. Redis Connection URLs are versatile and can accommodate various Redis server topologies and authentication methods. These URLs follow a specific format that includes the connection protocol, authentication details, host, port, and database information.\n", - "The basic structure of a Redis Connection URL is:\n", - "\n", - "```\n", - "[protocol]://[auth]@[host]:[port]/[database]\n", - "```\n", - "\n", - "Where:\n", - "\n", - "* protocol can be redis for standard connections, rediss for SSL/TLS connections, or redis+sentinel for Sentinel connections.\n", - "* auth includes username and password (if applicable).\n", - "* host is the Redis server hostname or IP address.\n", - "* port is the Redis server port.\n", - "* database is the Redis database number.\n", - "\n", - "Redis Connection URLs support various configurations, including:\n", - "\n", - "* Standalone Redis servers (with or without authentication)\n", - "* Redis Sentinel setups\n", - "* SSL/TLS encrypted connections\n", - "* Different authentication methods (password-only or username-password)\n", - "\n", - "Below are examples of Redis Connection URLs for different configurations:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "b1b1eb90-5155-44ca-a8a7-b04b02d5e77c", - "metadata": {}, - "outputs": [], - "source": [ - "# connection to redis standalone at localhost, db 0, no password\n", - "redis_url = \"redis://localhost:6379\"\n", - "# connection to host \"redis\" port 7379 with db 2 and password \"secret\" (old style authentication scheme without username / pre 6.x)\n", - "redis_url = \"redis://:secret@redis:7379/2\"\n", - "# connection to host redis on default port with user \"joe\", pass \"secret\" using redis version 6+ ACLs\n", - "redis_url = \"redis://joe:secret@redis/0\"\n", - "\n", - "# connection to sentinel at localhost with default group mymaster and db 0, no password\n", - "redis_url = \"redis+sentinel://localhost:26379\"\n", - "# connection to sentinel at host redis with default port 26379 and user \"joe\" with password \"secret\" with default group mymaster and db 0\n", - "redis_url = \"redis+sentinel://joe:secret@redis\"\n", - "# connection to sentinel, no auth with sentinel monitoring group \"zone-1\" and database 2\n", - "redis_url = \"redis+sentinel://redis:26379/zone-1/2\"\n", - "\n", - "# connection to redis standalone at localhost, db 0, no password but with TLS support\n", - "redis_url = \"rediss://localhost:6379\"\n", - "# connection to redis sentinel at localhost and default port, db 0, no password\n", - "# but with TLS support for booth Sentinel and Redis server\n", - "redis_url = \"rediss+sentinel://localhost\"" - ] - }, - { - "cell_type": "markdown", - "id": "9695dee7", - "metadata": {}, - "source": [ - "### Launching a Redis Instance with Docker\n", - "\n", - "To use Redis with LangChain, you need a running Redis instance. You can start one using Docker with:\n", - "\n", - "```bash\n", - "docker run -d -p 6379:6379 redis/redis-stack:latest\n", - "```\n", - "\n", - "For this example, we'll use a local Redis instance. If you're using a remote instance, you'll need to modify the Redis URL accordingly." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "894c30e4", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Connecting to Redis at: redis://redis:6379\n" - ] - } - ], - "source": [ - "import os\n", - "\n", - "REDIS_URL = os.getenv(\"REDIS_URL\", \"redis://localhost:6379\")\n", - "print(f\"Connecting to Redis at: {REDIS_URL}\")" - ] - }, - { - "cell_type": "markdown", - "id": "7f98392b", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "e7b6a6e0", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "63dd7f8d-df13-45d8-8e13-892b29803e96", - "metadata": {}, - "source": [ - "Let's check that Redis is up an running by pinging it:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "37fc2d36-f5bf-465f-9774-510bdc134b62", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "True" + "cell_type": "raw", + "id": "1957f5cb", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: Redis\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "import redis\n", - "\n", - "redis_client = redis.from_url(REDIS_URL)\n", - "redis_client.ping()" - ] - }, - { - "cell_type": "markdown", - "id": "4e388814-4188-4f29-8f24-cc67d4048ebe", - "metadata": {}, - "source": [ - "### Sample Data\n", - "\n", - "The 20 newsgroups dataset comprises around 18000 newsgroups posts on 20 topics. We'll use a subset for this demonstration and focus on two categories: 'alt.atheism' and 'sci.space':" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "869a4726-1e24-48fd-9ffd-c62a589d0bb1", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "250" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain.docstore.document import Document\n", - "from sklearn.datasets import fetch_20newsgroups\n", - "\n", - "categories = [\"alt.atheism\", \"sci.space\"]\n", - "newsgroups = fetch_20newsgroups(\n", - " subset=\"train\", categories=categories, shuffle=True, random_state=42\n", - ")\n", - "\n", - "# Use only the first 250 documents\n", - "texts = newsgroups.data[:250]\n", - "metadata = [\n", - " {\"category\": newsgroups.target_names[target]} for target in newsgroups.target[:250]\n", - "]\n", - "\n", - "len(texts)" - ] - }, - { - "cell_type": "markdown", - "id": "93df377e", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "The RedisVectorStore instance can be initialized in several ways:\n", - "\n", - "- `RedisVectorStore.__init__` - Initialize directly\n", - "- `RedisVectorStore.from_texts` - Initialize from a list of texts (optionally with metadata)\n", - "- `RedisVectorStore.from_documents` - Initialize from a list of `langchain_core.documents.Document` objects\n", - "- `RedisVectorStore.from_existing_index` - Initialize from an existing Redis index\n", - "\n", - "Below we will use the `RedisVectorStore.__init__` method using a `RedisConfig` instance.\n", - "\n", - "import EmbeddingTabs from \"@theme/EmbeddingTabs\";\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "7a95c110-015b-4300-93b7-c0100d55d024", - "metadata": {}, - "outputs": [], - "source": [ - "%%capture\n", - "# | output: false\n", - "# | echo: false\n", - "os.environ[\"TOKENIZERS_PARALLELISM\"] = \"false\"\n", - "from langchain_huggingface import HuggingFaceEmbeddings\n", - "from tqdm.auto import tqdm\n", - "\n", - "embeddings = HuggingFaceEmbeddings(model_name=\"msmarco-distilbert-base-v4\")" - ] - }, - { - "cell_type": "markdown", - "id": "ec110186-d5e4-4eaa-a4e3-899f405f719f", - "metadata": {}, - "source": [ - "We'll use the SentenceTransformer model to create embeddings. This model runs locally and doesn't require an API key." - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "dc37144c-208d-4ab3-9f3a-0407a69fe052", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain_redis import RedisConfig, RedisVectorStore\n", - "\n", - "config = RedisConfig(\n", - " index_name=\"newsgroups\",\n", - " redis_url=REDIS_URL,\n", - " metadata_schema=[\n", - " {\"name\": \"category\", \"type\": \"tag\"},\n", - " ],\n", - ")\n", - "\n", - "vector_store = RedisVectorStore(embeddings, config=config)" - ] - }, - { - "cell_type": "markdown", - "id": "ac6071d4", - "metadata": {}, - "source": [ - "## Manage vector store\n", - "\n", - "### Add items to vector store" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "17f5efc0", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "['newsgroups:f1e788ee61fe410daa8ef941dd166223', 'newsgroups:80b39032181f4299a359a9aaed6e2401', 'newsgroups:99a3efc1883647afba53d115b49e6e92', 'newsgroups:503a6c07cd71418eb71e11b42589efd7', 'newsgroups:7351210e32d1427bbb3c7426cf93a44f', 'newsgroups:4e79fdf67abe471b8ee98ba0e8a1a055', 'newsgroups:03559a1d574e4f9ca0479d7b3891402e', 'newsgroups:9a1c2a7879b8409a805db72feac03580', 'newsgroups:3578a1e129f5435f9743cf803413f37a', 'newsgroups:9f68baf4d6b04f1683d6b871ce8ad92d']\n" - ] - } - ], - "source": [ - "ids = vector_store.add_texts(texts, metadata)\n", - "\n", - "print(ids[0:10])" - ] - }, - { - "cell_type": "markdown", - "id": "f8822e55-40d5-48aa-8e29-79101feb645a", - "metadata": {}, - "source": [ - "Let's inspect the first document:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "ca27e394-ae1e-4fdb-b79a-4a6b45a953a8", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "('From: bil@okcforum.osrhe.edu (Bill Conner)\\nSubject: Re: Not the Omni!\\nNntp-Posting-Host: okcforum.osrhe.edu\\nOrganization: Okcforum Unix Users Group\\nX-Newsreader: TIN [version 1.1 PL6]\\nLines: 18\\n\\nCharley Wingate (mangoe@cs.umd.edu) wrote:\\n: \\n: >> Please enlighten me. How is omnipotence contradictory?\\n: \\n: >By definition, all that can occur in the universe is governed by the rules\\n: >of nature. Thus god cannot break them. Anything that god does must be allowed\\n: >in the rules somewhere. Therefore, omnipotence CANNOT exist! It contradicts\\n: >the rules of nature.\\n: \\n: Obviously, an omnipotent god can change the rules.\\n\\nWhen you say, \"By definition\", what exactly is being defined;\\ncertainly not omnipotence. You seem to be saying that the \"rules of\\nnature\" are pre-existant somehow, that they not only define nature but\\nactually cause it. If that\\'s what you mean I\\'d like to hear your\\nfurther thoughts on the question.\\n\\nBill\\n',\n", - " {'category': 'alt.atheism'})" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "texts[0], metadata[0]" - ] - }, - { - "cell_type": "markdown", - "id": "dcf1b905", - "metadata": {}, - "source": [ - "### Delete items from vector store" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "ef61e188", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "1" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Delete documents by passing one or more keys/ids\n", - "vector_store.index.drop_keys(ids[0])" - ] - }, - { - "cell_type": "markdown", - "id": "021e2e3a-8f87-4d62-a1f3-dc291c6b26be", - "metadata": {}, - "source": [ - "### Inspecting the created Index\n", - "\n", - "Once the ``Redis`` VectorStore object has been constructed, an index will have been created in Redis if it did not already exist. The index can be inspected with both the ``rvl``and the ``redis-cli`` command line tool. If you installed ``redisvl`` above, you can use the ``rvl`` command line tool to inspect the index." - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "787d9cbf-8942-4e6f-b030-f404d4632972", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Using Redis address from environment variable, REDIS_URL\n", - "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Indices:\n", - "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m 1. newsgroups\n" - ] - } - ], - "source": [ - "# assumes you're running Redis locally (use --host, --port, --password, --username, to change this)\n", - "!rvl index listall --port 6379" - ] - }, - { - "cell_type": "markdown", - "id": "b869de6f-e3da-4bfc-a267-102df1165521", - "metadata": {}, - "source": [ - "The ``Redis`` VectorStore implementation will attempt to generate index schema (fields for filtering) for any metadata passed through the ``from_texts``, ``from_texts_return_keys``, and ``from_documents`` methods. This way, whatever metadata is passed will be indexed into the Redis search index allowing\n", - "for filtering on those fields.\n", - "\n", - "Below we show what fields were created from the metadata we defined above" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "0eb45eb1-492f-487d-a8a7-7d2d301c7bdb", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Using Redis address from environment variable, REDIS_URL\n", - "\n", - "\n", - "Index Information:\n", - "╭──────────────┬────────────────┬────────────────┬─────────────────┬────────────╮\n", - "│ Index Name │ Storage Type │ Prefixes │ Index Options │ Indexing │\n", - "├──────────────┼────────────────┼────────────────┼─────────────────┼────────────┤\n", - "│ newsgroups │ HASH │ ['newsgroups'] │ [] │ 0 │\n", - "╰──────────────┴────────────────┴────────────────┴─────────────────┴────────────╯\n", - "Index Fields:\n", - "╭───────────┬─────────────┬────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬─────────────────┬────────────────╮\n", - "│ Name │ Attribute │ Type │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │\n", - "├───────────┼─────────────┼────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼─────────────────┼────────────────┤\n", - "│ text │ text │ TEXT │ WEIGHT │ 1 │ │ │ │ │ │ │\n", - "│ embedding │ embedding │ VECTOR │ algorithm │ FLAT │ data_type │ FLOAT32 │ dim │ 768 │ distance_metric │ COSINE │\n", - "│ category │ category │ TAG │ SEPARATOR │ | │ │ │ │ │ │ │\n", - "╰───────────┴─────────────┴────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴─────────────────┴────────────────╯\n" - ] - } - ], - "source": [ - "!rvl index info -i newsgroups --port 6379" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "84f9a77c-41b1-4515-97f4-2635998dc0dd", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[32m17:54:51\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Using Redis address from environment variable, REDIS_URL\n", - "\n", - "Statistics:\n", - "╭─────────────────────────────┬────────────╮\n", - "│ Stat Key │ Value │\n", - "├─────────────────────────────┼────────────┤\n", - "│ num_docs │ 249 │\n", - "│ num_terms │ 16178 │\n", - "│ max_doc_id │ 250 │\n", - "│ num_records │ 50394 │\n", - "│ percent_indexed │ 1 │\n", - "│ hash_indexing_failures │ 0 │\n", - "│ number_of_uses │ 2 │\n", - "│ bytes_per_record_avg │ 38.2743 │\n", - "│ doc_table_size_mb │ 0.0263586 │\n", - "│ inverted_sz_mb │ 1.83944 │\n", - "│ key_table_size_mb │ 0.00932026 │\n", - "│ offset_bits_per_record_avg │ 10.6699 │\n", - "│ offset_vectors_sz_mb │ 0.089057 │\n", - "│ offsets_per_term_avg │ 1.38937 │\n", - "│ records_per_doc_avg │ 202.386 │\n", - "│ sortable_values_size_mb │ 0 │\n", - "│ total_indexing_time │ 72.444 │\n", - "│ total_inverted_index_blocks │ 16207 │\n", - "│ vector_index_sz_mb │ 3.01776 │\n", - "╰─────────────────────────────┴────────────╯\n" - ] - } - ], - "source": [ - "!rvl stats -i newsgroups --port 6379" - ] - }, - { - "cell_type": "markdown", - "id": "c3620501", - "metadata": {}, - "source": [ - "## Query vector store\n", - "\n", - "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. \n", - "\n", - "### Query directly\n", - "\n", - "Performing a simple similarity search can be done as follows:" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "aa0a16fa", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Simple Similarity Search Results:\n", - "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", - "Subject: A flawed propulsion system: Space Shuttle\n", - "X-Ad...\n", - "Metadata: {'category': 'sci.space'}\n", - "\n", - "Content: From: nsmca@aurora.alaska.edu\n", - "Subject: Space Design Movies?\n", - "Article-I.D.: aurora.1993Apr23.124722.1\n", - "...\n", - "Metadata: {'category': 'sci.space'}\n", - "\n" - ] - } - ], - "source": [ - "query = \"Tell me about space exploration\"\n", - "results = vector_store.similarity_search(query, k=2)\n", - "\n", - "print(\"Simple Similarity Search Results:\")\n", - "for doc in results:\n", - " print(f\"Content: {doc.page_content[:100]}...\")\n", - " print(f\"Metadata: {doc.metadata}\")\n", - " print()" - ] - }, - { - "cell_type": "markdown", - "id": "3ed9d733", - "metadata": {}, - "source": [ - "If you want to execute a similarity search and receive the corresponding scores you can run:" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "id": "5efd2eaa", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Similarity Search with Score Results:\n", - "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", - "Subject: A flawed propulsion system: Space Shuttle\n", - "X-Ad...\n", - "Metadata: {'category': 'sci.space'}\n", - "Score: 0.569670975208\n", - "\n", - "Content: From: nsmca@aurora.alaska.edu\n", - "Subject: Space Design Movies?\n", - "Article-I.D.: aurora.1993Apr23.124722.1\n", - "...\n", - "Metadata: {'category': 'sci.space'}\n", - "Score: 0.590400338173\n", - "\n" - ] - } - ], - "source": [ - "# Similarity search with score and filter\n", - "scored_results = vector_store.similarity_search_with_score(query, k=2)\n", - "\n", - "print(\"Similarity Search with Score Results:\")\n", - "for doc, score in scored_results:\n", - " print(f\"Content: {doc.page_content[:100]}...\")\n", - " print(f\"Metadata: {doc.metadata}\")\n", - " print(f\"Score: {score}\")\n", - " print()" - ] - }, - { - "cell_type": "markdown", - "id": "0c235cdc", - "metadata": {}, - "source": [ - "### Query by turning into retriever\n", - "\n", - "You can also transform the vector store into a retriever for easier usage in your chains." - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "id": "f3460093", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[Document(metadata={'category': 'sci.space'}, page_content='Subject: Re: Comet in Temporary Orbit Around Jupiter?\\nFrom: Robert Coe \\nDistribution: world\\nOrganization: 1776 Enterprises, Sudbury MA\\nLines: 23\\n\\njgarland@kean.ucs.mun.ca writes:\\n\\n> >> Also, perihelions of Gehrels3 were:\\n> >> \\n> >> April 1973 83 jupiter radii\\n> >> August 1970 ~3 jupiter radii\\n> > \\n> > Where 1 Jupiter radius = 71,000 km = 44,000 mi = 0.0005 AU. So the\\n> > 1970 figure seems unlikely to actually be anything but a perijove.\\n> > Is that the case for the 1973 figure as well?\\n> > -- \\n> Sorry, _perijoves_...I\\'m not used to talking this language.\\n\\nHmmmm.... The prefix \"peri-\" is Greek, not Latin, so it\\'s usually used\\nwith the Greek form of the name of the body being orbited. (That\\'s why\\nit\\'s \"perihelion\" rather than \"perisol\", \"perigee\" rather than \"periterr\",\\nand \"pericynthion\" rather than \"perilune\".) So for Jupiter I\\'d expect it\\nto be something like \"perizeon\".) :^)\\n\\n ___ _ - Bob\\n /__) _ / / ) _ _\\n(_/__) (_)_(_) (___(_)_(/_______________________________________ bob@1776.COM\\nRobert K. Coe ** 14 Churchill St, Sudbury, Massachusetts 01776 ** 508-443-3265\\n'),\n", - " Document(metadata={'category': 'sci.space'}, page_content='From: pyron@skndiv.dseg.ti.com (Dillon Pyron)\\nSubject: Re: Why not give $1 billion to first year-long moon residents?\\nLines: 42\\nNntp-Posting-Host: skndiv.dseg.ti.com\\nReply-To: pyron@skndiv.dseg.ti.com\\nOrganization: TI/DSEG VAX Support\\n\\n\\nIn article <1qve4kINNpas@sal-sun121.usc.edu>, schaefer@sal-sun121.usc.edu (Peter Schaefer) writes:\\n>In article <1993Apr19.130503.1@aurora.alaska.edu>, nsmca@aurora.alaska.edu writes:\\n>|> In article <6ZV82B2w165w@theporch.raider.net>, gene@theporch.raider.net (Gene Wright) writes:\\n>|> > With the continuin talk about the \"End of the Space Age\" and complaints \\n>|> > by government over the large cost, why not try something I read about \\n>|> > that might just work.\\n>|> > \\n>|> > Announce that a reward of $1 billion would go to the first corporation \\n>|> > who successfully keeps at least 1 person alive on the moon for a year. \\n>|> > Then you\\'d see some of the inexpensive but not popular technologies begin \\n>|> > to be developed. THere\\'d be a different kind of space race then!\\n>|> > \\n>|> > --\\n>|> > gene@theporch.raider.net (Gene Wright)\\n>|> > theporch.raider.net 615/297-7951 The MacInteresteds of Nashville\\n>|> ====\\n>|> If that were true, I\\'d go for it.. I have a few friends who we could pool our\\n>|> resources and do it.. Maybe make it a prize kind of liek the \"Solar Car Race\"\\n>|> in Australia..\\n>|> Anybody game for a contest!\\n>|> \\n>|> ==\\n>|> Michael Adams, nsmca@acad3.alaska.edu -- I\\'m not high, just jacked\\n>\\n>\\n>Oh gee, a billion dollars! That\\'d be just about enough to cover the cost of the\\n>feasability study! Happy, Happy, JOY! JOY!\\n>\\n\\nFeasability study?? What a wimp!! While you are studying, others would be\\ndoing. Too damn many engineers doing way too little engineering.\\n\\n\"He who sits on his arse sits on his fortune\" - Sir Richard Francis Burton\\n--\\nDillon Pyron | The opinions expressed are those of the\\nTI/DSEG Lewisville VAX Support | sender unless otherwise stated.\\n(214)462-3556 (when I\\'m here) |\\n(214)492-4656 (when I\\'m home) |Texans: Vote NO on Robin Hood. We need\\npyron@skndiv.dseg.ti.com |solutions, not gestures.\\nPADI DM-54909 |\\n\\n')]" - ] - }, - "execution_count": 17, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "retriever = vector_store.as_retriever(search_type=\"similarity\", search_kwargs={\"k\": 2})\n", - "retriever.invoke(\"What planet in the solar system has the largest number of moons?\")" - ] - }, - { - "cell_type": "markdown", - "id": "901c75dc", - "metadata": {}, - "source": [ - "## Usage for retrieval-augmented generation\n", - "\n", - "For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n", - "\n", - "- [Tutorials](/docs/tutorials/)\n", - "- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n", - "- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/retrieval)" - ] - }, - { - "cell_type": "markdown", - "id": "069f1b5f", - "metadata": {}, - "source": [ - "## Redis-specific functionality\n", - "\n", - "Redis offers some unique features for vector search:" - ] - }, - { - "cell_type": "markdown", - "id": "8a627d3a-af78-46e2-b314-007e641b4d1d", - "metadata": {}, - "source": [ - "### Similarity search with metadata filtering\n", - "We can filter our search results based on metadata:" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "id": "23d6e6fe-8aee-4cee-bf05-59cf3fba36ae", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Filtered Similarity Search Results:\n", - "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", - "Subject: A flawed propulsion system: Space Shuttle\n", - "X-Ad...\n", - "Metadata: {'category': 'sci.space'}\n", - "\n", - "Content: From: nsmca@aurora.alaska.edu\n", - "Subject: Space Design Movies?\n", - "Article-I.D.: aurora.1993Apr23.124722.1\n", - "...\n", - "Metadata: {'category': 'sci.space'}\n", - "\n" - ] - } - ], - "source": [ - "from redisvl.query.filter import Tag\n", - "\n", - "query = \"Tell me about space exploration\"\n", - "\n", - "# Create a RedisVL filter expression\n", - "filter_condition = Tag(\"category\") == \"sci.space\"\n", - "\n", - "filtered_results = vector_store.similarity_search(query, k=2, filter=filter_condition)\n", - "\n", - "print(\"Filtered Similarity Search Results:\")\n", - "for doc in filtered_results:\n", - " print(f\"Content: {doc.page_content[:100]}...\")\n", - " print(f\"Metadata: {doc.metadata}\")\n", - " print()" - ] - }, - { - "cell_type": "markdown", - "id": "f35b9ebf-6419-4402-a066-c1d5287ed38e", - "metadata": {}, - "source": [ - "### Maximum marginal relevance search\n", - "Maximum marginal relevance search helps in getting diverse results:" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "id": "5be2afeb-d0a3-4075-bd3c-4cbe409dfb3a", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Maximum Marginal Relevance Search Results:\n", - "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", - "Subject: A flawed propulsion system: Space Shuttle\n", - "X-Ad...\n", - "Metadata: {'category': 'sci.space'}\n", - "\n", - "Content: From: moroney@world.std.com (Michael Moroney)\n", - "Subject: Re: Vulcan? (No, not the guy with the ears!)\n", - "...\n", - "Metadata: {'category': 'sci.space'}\n", - "\n" - ] - } - ], - "source": [ - "# Maximum marginal relevance search with filter\n", - "mmr_results = vector_store.max_marginal_relevance_search(\n", - " query, k=2, fetch_k=10, filter=filter_condition\n", - ")\n", - "\n", - "print(\"Maximum Marginal Relevance Search Results:\")\n", - "for doc in mmr_results:\n", - " print(f\"Content: {doc.page_content[:100]}...\")\n", - " print(f\"Metadata: {doc.metadata}\")\n", - " print()" - ] - }, - { - "cell_type": "markdown", - "id": "09c3343c-6af4-4151-ba0a-50800fc34855", - "metadata": {}, - "source": [ - "## Chain usage\n", - "The code below shows how to use the vector store as a retriever in a simple RAG chain:\n", - "\n", - "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", - "\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "id": "9f6658f8-45b7-4004-a0b3-893bd23bff41", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "OpenAI API key not found in environment variables.\n" - ] }, { - "name": "stdin", - "output_type": "stream", - "text": [ - "Please enter your OpenAI API key: ········\n" - ] + "cell_type": "markdown", + "id": "ef1f0986", + "metadata": {}, + "source": [ + "# Redis Vector Store\n", + "\n", + "This notebook covers how to get started with the Redis vector store.\n", + "\n", + ">[Redis](https://redis.io/docs/stack/vectorsearch/) is a popular open-source, in-memory data structure store that can be used as a database, cache, message broker, and queue. It now includes vector similarity search capabilities, making it suitable for use as a vector store." + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "OpenAI API key has been set for this session.\n" - ] - } - ], - "source": [ - "# | output: false\n", - "# | echo: false\n", - "from getpass import getpass\n", - "\n", - "from langchain_openai import ChatOpenAI\n", - "\n", - "# Check if OPENAI_API_KEY is already set in the environment\n", - "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n", - "\n", - "if not openai_api_key:\n", - " print(\"OpenAI API key not found in environment variables.\")\n", - " openai_api_key = getpass(\"Please enter your OpenAI API key: \")\n", - "\n", - " # Set the API key for the current session\n", - " os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n", - " print(\"OpenAI API key has been set for this session.\")\n", - "else:\n", - " print(\"OpenAI API key found in environment variables.\")\n", - "\n", - "llm = ChatOpenAI(model=\"gpt-4o-mini\")" - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "id": "d0ac614c-3f80-4839-8451-d3322a870809", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "'The Space Shuttle program was a NASA initiative that enabled reusable spacecraft to transport astronauts and cargo to and from low Earth orbit. It conducted a variety of missions, including satellite deployment, scientific research, and assembly of the International Space Station, and typically carried a crew of five astronauts. Although it achieved many successes, the program faced criticism for its safety concerns and the complexity of its propulsion system.'" + "cell_type": "markdown", + "id": "a717e10e-c8d7-41bc-9dd0-447d11f90b68", + "metadata": {}, + "source": [ + "### What is Redis?\n", + "\n", + "Most developers are familiar with `Redis`. At its core, `Redis` is a NoSQL Database in the key-value family that can used as a cache, message broker, stream processing and a primary database. Developers choose `Redis` because it is fast, has a large ecosystem of client libraries, and has been deployed by major enterprises for years.\n", + "\n", + "On top of these traditional use cases, `Redis` provides additional capabilities like the Search and Query capability that allows users to create secondary index structures within `Redis`. This allows `Redis` to be a Vector Database, at the speed of a cache.\n", + "\n", + "\n", + "### Redis as a Vector Database\n", + "\n", + "`Redis` uses compressed, inverted indexes for fast indexing with a low memory footprint. It also supports a number of advanced features such as:\n", + "\n", + "* Indexing of multiple fields in Redis hashes and `JSON`\n", + "* Vector similarity search (with `HNSW` (ANN) or `FLAT` (KNN))\n", + "* Vector Range Search (e.g. find all vectors within a radius of a query vector)\n", + "* Incremental indexing without performance loss\n", + "* Document ranking (using [tf-idf](https://en.wikipedia.org/wiki/Tf%E2%80%93idf), with optional user-provided weights)\n", + "* Field weighting\n", + "* Complex boolean queries with `AND`, `OR`, and `NOT` operators\n", + "* Prefix matching, fuzzy matching, and exact-phrase queries\n", + "* Support for [double-metaphone phonetic matching](https://redis.io/docs/stack/search/reference/phonetic_matching/)\n", + "* Auto-complete suggestions (with fuzzy prefix suggestions)\n", + "* Stemming-based query expansion in [many languages](https://redis.io/docs/stack/search/reference/stemming/) (using [Snowball](http://snowballstem.org/))\n", + "* Support for Chinese-language tokenization and querying (using [Friso](https://github.com/lionsoul2014/friso))\n", + "* Numeric filters and ranges\n", + "* Geospatial searches using Redis geospatial indexing\n", + "* A powerful aggregations engine\n", + "* Supports for all `utf-8` encoded text\n", + "* Retrieve full documents, selected fields, or only the document IDs\n", + "* Sorting results (for example, by creation date)\n", + "\n", + "### Clients\n", + "\n", + "Since `Redis` is much more than just a vector database, there are often use cases that demand the usage of a `Redis` client besides just the `LangChain` integration. You can use any standard `Redis` client library to run Search and Query commands, but it's easiest to use a library that wraps the Search and Query API. Below are a few examples, but you can find more client libraries [here](https://redis.io/resources/clients/).\n", + "\n", + "| Project | Language | License | Author | Stars |\n", + "|----------|---------|--------|---------|-------|\n", + "| [jedis][jedis-url] | Java | MIT | [Redis][redis-url] | ![Stars][jedis-stars] |\n", + "| [redisvl][redisvl-url] | Python | MIT | [Redis][redis-url] | ![Stars][redisvl-stars] |\n", + "| [redis-py][redis-py-url] | Python | MIT | [Redis][redis-url] | ![Stars][redis-py-stars] |\n", + "| [node-redis][node-redis-url] | Node.js | MIT | [Redis][redis-url] | ![Stars][node-redis-stars] |\n", + "| [nredisstack][nredisstack-url] | .NET | MIT | [Redis][redis-url] | ![Stars][nredisstack-stars] |\n", + "\n", + "[redis-url]: https://redis.com\n", + "\n", + "[redisvl-url]: https://github.com/redis/redis-vl-python\n", + "[redisvl-stars]: https://img.shields.io/github/stars/redis/redisvl.svg?style=social&label=Star&maxAge=2592000\n", + "[redisvl-package]: https://pypi.python.org/pypi/redisvl\n", + "\n", + "[redis-py-url]: https://github.com/redis/redis-py\n", + "[redis-py-stars]: https://img.shields.io/github/stars/redis/redis-py.svg?style=social&label=Star&maxAge=2592000\n", + "[redis-py-package]: https://pypi.python.org/pypi/redis\n", + "\n", + "[jedis-url]: https://github.com/redis/jedis\n", + "[jedis-stars]: https://img.shields.io/github/stars/redis/jedis.svg?style=social&label=Star&maxAge=2592000\n", + "[Jedis-package]: https://search.maven.org/artifact/redis.clients/jedis\n", + "\n", + "[nredisstack-url]: https://github.com/redis/nredisstack\n", + "[nredisstack-stars]: https://img.shields.io/github/stars/redis/nredisstack.svg?style=social&label=Star&maxAge=2592000\n", + "[nredisstack-package]: https://www.nuget.org/packages/nredisstack/\n", + "\n", + "[node-redis-url]: https://github.com/redis/node-redis\n", + "[node-redis-stars]: https://img.shields.io/github/stars/redis/node-redis.svg?style=social&label=Star&maxAge=2592000\n", + "[node-redis-package]: https://www.npmjs.com/package/redis\n", + "\n", + "[redis-om-python-url]: https://github.com/redis/redis-om-python\n", + "[redis-om-python-author]: https://redis.com\n", + "[redis-om-python-stars]: https://img.shields.io/github/stars/redis/redis-om-python.svg?style=social&label=Star&maxAge=2592000\n", + "\n", + "[redisearch-go-url]: https://github.com/RediSearch/redisearch-go\n", + "[redisearch-go-author]: https://redis.com\n", + "[redisearch-go-stars]: https://img.shields.io/github/stars/RediSearch/redisearch-go.svg?style=social&label=Star&maxAge=2592000\n", + "\n", + "[redisearch-api-rs-url]: https://github.com/RediSearch/redisearch-api-rs\n", + "[redisearch-api-rs-author]: https://redis.com\n", + "[redisearch-api-rs-stars]: https://img.shields.io/github/stars/RediSearch/redisearch-api-rs.svg?style=social&label=Star&maxAge=2592000\n", + "\n", + "\n", + "### Deployment options\n", + "\n", + "There are many ways to deploy Redis with RediSearch. The easiest way to get started is to use Docker, but there are are many potential options for deployment such as\n", + "\n", + "- [Redis Cloud](https://redis.com/redis-enterprise-cloud/overview/)\n", + "- [Docker (Redis Stack)](https://hub.docker.com/r/redis/redis-stack)\n", + "- Cloud marketplaces: [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-e6y7ork67pjwg?sr=0-2&ref_=beagle&applicationId=AWSMPContessa), [Google Marketplace](https://console.cloud.google.com/marketplace/details/redislabs-public/redis-enterprise?pli=1), or [Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/garantiadata.redis_enterprise_1sp_public_preview?tab=Overview)\n", + "- On-premise: [Redis Enterprise Software](https://redis.com/redis-enterprise-software/overview/)\n", + "- Kubernetes: [Redis Enterprise Software on Kubernetes](https://docs.redis.com/latest/kubernetes/)\n", + "\n", + "### Redis connection Url schemas\n", + "\n", + "Valid Redis Url schemas are:\n", + "1. `redis://` - Connection to Redis standalone, unencrypted\n", + "2. `rediss://` - Connection to Redis standalone, with TLS encryption\n", + "3. `redis+sentinel://` - Connection to Redis server via Redis Sentinel, unencrypted\n", + "4. `rediss+sentinel://` - Connection to Redis server via Redis Sentinel, booth connections with TLS encryption\n", + "\n", + "More information about additional connection parameters can be found in the [redis-py documentation](https://redis-py.readthedocs.io/en/stable/connections.html)." ] - }, - "execution_count": 21, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from langchain_core.output_parsers import StrOutputParser\n", - "from langchain_core.prompts import ChatPromptTemplate\n", - "from langchain_core.runnables import RunnablePassthrough\n", - "\n", - "# Prompt\n", - "prompt = ChatPromptTemplate.from_messages(\n", - " [\n", - " (\n", - " \"human\",\n", - " \"\"\"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n", - "Question: {question} \n", - "Context: {context} \n", - "Answer:\"\"\",\n", - " ),\n", - " ]\n", - ")\n", - "\n", - "\n", - "def format_docs(docs):\n", - " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", - "\n", - "\n", - "rag_chain = (\n", - " {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n", - " | prompt\n", - " | llm\n", - " | StrOutputParser()\n", - ")\n", - "\n", - "rag_chain.invoke(\"Describe the Space Shuttle program?\")" - ] - }, - { - "cell_type": "markdown", - "id": "8ad3e6e4-36ef-494a-be50-4bf8e374b077", - "metadata": {}, - "source": [ - "## Connect to an existing Index\n", - "\n", - "In order to have the same metadata indexed when using the ``Redis`` VectorStore. You will need to have the same ``index_schema`` passed in either as a path to a yaml file or as a dictionary. The following shows how to obtain the schema from an index and connect to an existing index." - ] - }, - { - "cell_type": "code", - "execution_count": 24, - "id": "6a0e7a49-8271-44b2-abb2-0ef499546b28", - "metadata": {}, - "outputs": [], - "source": [ - "# write the schema to a yaml file\n", - "vector_store.index.schema.to_yaml(\"redis_schema.yaml\")" - ] - }, - { - "cell_type": "code", - "execution_count": 33, - "id": "e3588805-b3d9-4af8-8786-b57fc640ebb0", - "metadata": {}, - "outputs": [ + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "18:19:58 redisvl.index.index INFO Index already exists, not overwriting.\n", - "page_content='From: aa429@freenet.carleton.ca (Terry Ford)\n", - "Subject: A flawed propulsion system: Space Shuttle\n", - "X-Added: Forwarded by Space Digest\n", - "Organization: [via International Space University]\n", - "Original-Sender: isu@VACATION.VENARI.CS.CMU.EDU\n", - "Distribution: sci\n", - "Lines: 13\n", - "\n", - "\n", - "\n", - "For an essay, I am writing about the space shuttle and a need for a better\n", - "propulsion system. Through research, I have found that it is rather clumsy \n", - "(i.e. all the checks/tests before launch), the safety hazards (\"sitting\n", - "on a hydrogen bomb\"), etc.. If you have any beefs about the current\n", - "space shuttle program Re: propulsion, please send me your ideas.\n", - "\n", - "Thanks a lot.\n", - "\n", - "--\n", - "Terry Ford [aa429@freenet.carleton.ca]\n", - "Nepean, Ontario, Canada.\n", - "' metadata={'category': 'sci.space'}\n" - ] - } - ], - "source": [ - "# now we can connect to our existing index as follows\n", - "\n", - "new_rdvs = RedisVectorStore(\n", - " embeddings,\n", - " redis_url=REDIS_URL,\n", - " schema_path=\"redis_schema.yaml\",\n", - ")\n", - "\n", - "results = new_rdvs.similarity_search(\"Space Shuttle Propulsion System\", k=3)\n", - "print(results[0])" - ] - }, - { - "cell_type": "code", - "execution_count": 35, - "id": "4d7ff456-de2a-4c58-9a3f-a9a3cfdca492", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "True" + "cell_type": "markdown", + "id": "36fdc060", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "To use the RedisVectorStore, you'll need to install the `langchain-redis` partner package, as well as the other packages used throughout this notebook." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "64e28aa6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install -qU langchain-redis langchain-huggingface sentence-transformers scikit-learn" + ] + }, + { + "cell_type": "markdown", + "id": "37d388a3-1a56-453e-8f84-e24a72d682eb", + "metadata": {}, + "source": [ + "### Credentials\n", + "\n", + "Redis connection credentials are passed as part of the Redis Connection URL. Redis Connection URLs are versatile and can accommodate various Redis server topologies and authentication methods. These URLs follow a specific format that includes the connection protocol, authentication details, host, port, and database information.\n", + "The basic structure of a Redis Connection URL is:\n", + "\n", + "```\n", + "[protocol]://[auth]@[host]:[port]/[database]\n", + "```\n", + "\n", + "Where:\n", + "\n", + "* protocol can be redis for standard connections, rediss for SSL/TLS connections, or redis+sentinel for Sentinel connections.\n", + "* auth includes username and password (if applicable).\n", + "* host is the Redis server hostname or IP address.\n", + "* port is the Redis server port.\n", + "* database is the Redis database number.\n", + "\n", + "Redis Connection URLs support various configurations, including:\n", + "\n", + "* Standalone Redis servers (with or without authentication)\n", + "* Redis Sentinel setups\n", + "* SSL/TLS encrypted connections\n", + "* Different authentication methods (password-only or username-password)\n", + "\n", + "Below are examples of Redis Connection URLs for different configurations:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "b1b1eb90-5155-44ca-a8a7-b04b02d5e77c", + "metadata": {}, + "outputs": [], + "source": [ + "# connection to redis standalone at localhost, db 0, no password\n", + "redis_url = \"redis://localhost:6379\"\n", + "# connection to host \"redis\" port 7379 with db 2 and password \"secret\" (old style authentication scheme without username / pre 6.x)\n", + "redis_url = \"redis://:secret@redis:7379/2\"\n", + "# connection to host redis on default port with user \"joe\", pass \"secret\" using redis version 6+ ACLs\n", + "redis_url = \"redis://joe:secret@redis/0\"\n", + "\n", + "# connection to sentinel at localhost with default group mymaster and db 0, no password\n", + "redis_url = \"redis+sentinel://localhost:26379\"\n", + "# connection to sentinel at host redis with default port 26379 and user \"joe\" with password \"secret\" with default group mymaster and db 0\n", + "redis_url = \"redis+sentinel://joe:secret@redis\"\n", + "# connection to sentinel, no auth with sentinel monitoring group \"zone-1\" and database 2\n", + "redis_url = \"redis+sentinel://redis:26379/zone-1/2\"\n", + "\n", + "# connection to redis standalone at localhost, db 0, no password but with TLS support\n", + "redis_url = \"rediss://localhost:6379\"\n", + "# connection to redis sentinel at localhost and default port, db 0, no password\n", + "# but with TLS support for booth Sentinel and Redis server\n", + "redis_url = \"rediss+sentinel://localhost\"" + ] + }, + { + "cell_type": "markdown", + "id": "9695dee7", + "metadata": {}, + "source": [ + "### Launching a Redis Instance with Docker\n", + "\n", + "To use Redis with LangChain, you need a running Redis instance. You can start one using Docker with:\n", + "\n", + "```bash\n", + "docker run -d -p 6379:6379 redis/redis-stack:latest\n", + "```\n", + "\n", + "For this example, we'll use a local Redis instance. If you're using a remote instance, you'll need to modify the Redis URL accordingly." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "894c30e4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Connecting to Redis at: redis://redis:6379\n" + ] + } + ], + "source": [ + "import os\n", + "\n", + "REDIS_URL = os.getenv(\"REDIS_URL\", \"redis://localhost:6379\")\n", + "print(f\"Connecting to Redis at: {REDIS_URL}\")" + ] + }, + { + "cell_type": "markdown", + "id": "7f98392b", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "e7b6a6e0", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "63dd7f8d-df13-45d8-8e13-892b29803e96", + "metadata": {}, + "source": [ + "Let's check that Redis is up an running by pinging it:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "37fc2d36-f5bf-465f-9774-510bdc134b62", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import redis\n", + "\n", + "redis_client = redis.from_url(REDIS_URL)\n", + "redis_client.ping()" + ] + }, + { + "cell_type": "markdown", + "id": "4e388814-4188-4f29-8f24-cc67d4048ebe", + "metadata": {}, + "source": [ + "### Sample Data\n", + "\n", + "The 20 newsgroups dataset comprises around 18000 newsgroups posts on 20 topics. We'll use a subset for this demonstration and focus on two categories: 'alt.atheism' and 'sci.space':" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "869a4726-1e24-48fd-9ffd-c62a589d0bb1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "250" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain.docstore.document import Document\n", + "from sklearn.datasets import fetch_20newsgroups\n", + "\n", + "categories = [\"alt.atheism\", \"sci.space\"]\n", + "newsgroups = fetch_20newsgroups(\n", + " subset=\"train\", categories=categories, shuffle=True, random_state=42\n", + ")\n", + "\n", + "# Use only the first 250 documents\n", + "texts = newsgroups.data[:250]\n", + "metadata = [\n", + " {\"category\": newsgroups.target_names[target]} for target in newsgroups.target[:250]\n", + "]\n", + "\n", + "len(texts)" + ] + }, + { + "cell_type": "markdown", + "id": "93df377e", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "The RedisVectorStore instance can be initialized in several ways:\n", + "\n", + "- `RedisVectorStore.__init__` - Initialize directly\n", + "- `RedisVectorStore.from_texts` - Initialize from a list of texts (optionally with metadata)\n", + "- `RedisVectorStore.from_documents` - Initialize from a list of `langchain_core.documents.Document` objects\n", + "- `RedisVectorStore.from_existing_index` - Initialize from an existing Redis index\n", + "\n", + "Below we will use the `RedisVectorStore.__init__` method using a `RedisConfig` instance.\n", + "\n", + "import EmbeddingTabs from \"@theme/EmbeddingTabs\";\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "7a95c110-015b-4300-93b7-c0100d55d024", + "metadata": {}, + "outputs": [], + "source": [ + "%%capture\n", + "# | output: false\n", + "# | echo: false\n", + "os.environ[\"TOKENIZERS_PARALLELISM\"] = \"false\"\n", + "from langchain_huggingface import HuggingFaceEmbeddings\n", + "from tqdm.auto import tqdm\n", + "\n", + "embeddings = HuggingFaceEmbeddings(model_name=\"msmarco-distilbert-base-v4\")" + ] + }, + { + "cell_type": "markdown", + "id": "ec110186-d5e4-4eaa-a4e3-899f405f719f", + "metadata": {}, + "source": [ + "We'll use the SentenceTransformer model to create embeddings. This model runs locally and doesn't require an API key." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "dc37144c-208d-4ab3-9f3a-0407a69fe052", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain_redis import RedisConfig, RedisVectorStore\n", + "\n", + "config = RedisConfig(\n", + " index_name=\"newsgroups\",\n", + " redis_url=REDIS_URL,\n", + " metadata_schema=[\n", + " {\"name\": \"category\", \"type\": \"tag\"},\n", + " ],\n", + ")\n", + "\n", + "vector_store = RedisVectorStore(embeddings, config=config)" + ] + }, + { + "cell_type": "markdown", + "id": "ac6071d4", + "metadata": {}, + "source": [ + "## Manage vector store\n", + "\n", + "### Add items to vector store" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "17f5efc0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['newsgroups:f1e788ee61fe410daa8ef941dd166223', 'newsgroups:80b39032181f4299a359a9aaed6e2401', 'newsgroups:99a3efc1883647afba53d115b49e6e92', 'newsgroups:503a6c07cd71418eb71e11b42589efd7', 'newsgroups:7351210e32d1427bbb3c7426cf93a44f', 'newsgroups:4e79fdf67abe471b8ee98ba0e8a1a055', 'newsgroups:03559a1d574e4f9ca0479d7b3891402e', 'newsgroups:9a1c2a7879b8409a805db72feac03580', 'newsgroups:3578a1e129f5435f9743cf803413f37a', 'newsgroups:9f68baf4d6b04f1683d6b871ce8ad92d']\n" + ] + } + ], + "source": [ + "ids = vector_store.add_texts(texts, metadata)\n", + "\n", + "print(ids[0:10])" + ] + }, + { + "cell_type": "markdown", + "id": "f8822e55-40d5-48aa-8e29-79101feb645a", + "metadata": {}, + "source": [ + "Let's inspect the first document:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "ca27e394-ae1e-4fdb-b79a-4a6b45a953a8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "('From: bil@okcforum.osrhe.edu (Bill Conner)\\nSubject: Re: Not the Omni!\\nNntp-Posting-Host: okcforum.osrhe.edu\\nOrganization: Okcforum Unix Users Group\\nX-Newsreader: TIN [version 1.1 PL6]\\nLines: 18\\n\\nCharley Wingate (mangoe@cs.umd.edu) wrote:\\n: \\n: >> Please enlighten me. How is omnipotence contradictory?\\n: \\n: >By definition, all that can occur in the universe is governed by the rules\\n: >of nature. Thus god cannot break them. Anything that god does must be allowed\\n: >in the rules somewhere. Therefore, omnipotence CANNOT exist! It contradicts\\n: >the rules of nature.\\n: \\n: Obviously, an omnipotent god can change the rules.\\n\\nWhen you say, \"By definition\", what exactly is being defined;\\ncertainly not omnipotence. You seem to be saying that the \"rules of\\nnature\" are pre-existant somehow, that they not only define nature but\\nactually cause it. If that\\'s what you mean I\\'d like to hear your\\nfurther thoughts on the question.\\n\\nBill\\n',\n", + " {'category': 'alt.atheism'})" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "texts[0], metadata[0]" + ] + }, + { + "cell_type": "markdown", + "id": "dcf1b905", + "metadata": {}, + "source": [ + "### Delete items from vector store" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "ef61e188", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Delete documents by passing one or more keys/ids\n", + "vector_store.index.drop_keys(ids[0])" + ] + }, + { + "cell_type": "markdown", + "id": "021e2e3a-8f87-4d62-a1f3-dc291c6b26be", + "metadata": {}, + "source": [ + "### Inspecting the created Index\n", + "\n", + "Once the ``Redis`` VectorStore object has been constructed, an index will have been created in Redis if it did not already exist. The index can be inspected with both the ``rvl``and the ``redis-cli`` command line tool. If you installed ``redisvl`` above, you can use the ``rvl`` command line tool to inspect the index." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "787d9cbf-8942-4e6f-b030-f404d4632972", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Using Redis address from environment variable, REDIS_URL\n", + "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Indices:\n", + "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m 1. newsgroups\n" + ] + } + ], + "source": [ + "# assumes you're running Redis locally (use --host, --port, --password, --username, to change this)\n", + "!rvl index listall --port 6379" + ] + }, + { + "cell_type": "markdown", + "id": "b869de6f-e3da-4bfc-a267-102df1165521", + "metadata": {}, + "source": [ + "The ``Redis`` VectorStore implementation will attempt to generate index schema (fields for filtering) for any metadata passed through the ``from_texts``, ``from_texts_return_keys``, and ``from_documents`` methods. This way, whatever metadata is passed will be indexed into the Redis search index allowing\n", + "for filtering on those fields.\n", + "\n", + "Below we show what fields were created from the metadata we defined above" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "0eb45eb1-492f-487d-a8a7-7d2d301c7bdb", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32m17:54:50\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Using Redis address from environment variable, REDIS_URL\n", + "\n", + "\n", + "Index Information:\n", + "╭──────────────┬────────────────┬────────────────┬─────────────────┬────────────╮\n", + "│ Index Name │ Storage Type │ Prefixes │ Index Options │ Indexing │\n", + "├──────────────┼────────────────┼────────────────┼─────────────────┼────────────┤\n", + "│ newsgroups │ HASH │ ['newsgroups'] │ [] │ 0 │\n", + "╰──────────────┴────────────────┴────────────────┴─────────────────┴────────────╯\n", + "Index Fields:\n", + "╭───────────┬─────────────┬────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬────────────────┬─────────────────┬────────────────╮\n", + "│ Name │ Attribute │ Type │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │ Field Option │ Option Value │\n", + "├───────────┼─────────────┼────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼─────────────────┼────────────────┤\n", + "│ text │ text │ TEXT │ WEIGHT │ 1 │ │ │ │ │ │ │\n", + "│ embedding │ embedding │ VECTOR │ algorithm │ FLAT │ data_type │ FLOAT32 │ dim │ 768 │ distance_metric │ COSINE │\n", + "│ category │ category │ TAG │ SEPARATOR │ | │ │ │ │ │ │ │\n", + "╰───────────┴─────────────┴────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴─────────────────┴────────────────╯\n" + ] + } + ], + "source": [ + "!rvl index info -i newsgroups --port 6379" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "84f9a77c-41b1-4515-97f4-2635998dc0dd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32m17:54:51\u001b[0m \u001b[34m[RedisVL]\u001b[0m \u001b[1;30mINFO\u001b[0m Using Redis address from environment variable, REDIS_URL\n", + "\n", + "Statistics:\n", + "╭─────────────────────────────┬────────────╮\n", + "│ Stat Key │ Value │\n", + "├─────────────────────────────┼────────────┤\n", + "│ num_docs │ 249 │\n", + "│ num_terms │ 16178 │\n", + "│ max_doc_id │ 250 │\n", + "│ num_records │ 50394 │\n", + "│ percent_indexed │ 1 │\n", + "│ hash_indexing_failures │ 0 │\n", + "│ number_of_uses │ 2 │\n", + "│ bytes_per_record_avg │ 38.2743 │\n", + "│ doc_table_size_mb │ 0.0263586 │\n", + "│ inverted_sz_mb │ 1.83944 │\n", + "│ key_table_size_mb │ 0.00932026 │\n", + "│ offset_bits_per_record_avg │ 10.6699 │\n", + "│ offset_vectors_sz_mb │ 0.089057 │\n", + "│ offsets_per_term_avg │ 1.38937 │\n", + "│ records_per_doc_avg │ 202.386 │\n", + "│ sortable_values_size_mb │ 0 │\n", + "│ total_indexing_time │ 72.444 │\n", + "│ total_inverted_index_blocks │ 16207 │\n", + "│ vector_index_sz_mb │ 3.01776 │\n", + "╰─────────────────────────────┴────────────╯\n" + ] + } + ], + "source": [ + "!rvl stats -i newsgroups --port 6379" + ] + }, + { + "cell_type": "markdown", + "id": "c3620501", + "metadata": {}, + "source": [ + "## Query vector store\n", + "\n", + "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent.\n", + "\n", + "### Query directly\n", + "\n", + "Performing a simple similarity search can be done as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "aa0a16fa", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Simple Similarity Search Results:\n", + "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", + "Subject: A flawed propulsion system: Space Shuttle\n", + "X-Ad...\n", + "Metadata: {'category': 'sci.space'}\n", + "\n", + "Content: From: nsmca@aurora.alaska.edu\n", + "Subject: Space Design Movies?\n", + "Article-I.D.: aurora.1993Apr23.124722.1\n", + "...\n", + "Metadata: {'category': 'sci.space'}\n", + "\n" + ] + } + ], + "source": [ + "query = \"Tell me about space exploration\"\n", + "results = vector_store.similarity_search(query, k=2)\n", + "\n", + "print(\"Simple Similarity Search Results:\")\n", + "for doc in results:\n", + " print(f\"Content: {doc.page_content[:100]}...\")\n", + " print(f\"Metadata: {doc.metadata}\")\n", + " print()" + ] + }, + { + "cell_type": "markdown", + "id": "3ed9d733", + "metadata": {}, + "source": [ + "If you want to execute a similarity search and receive the corresponding scores you can run:" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "5efd2eaa", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Similarity Search with Score Results:\n", + "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", + "Subject: A flawed propulsion system: Space Shuttle\n", + "X-Ad...\n", + "Metadata: {'category': 'sci.space'}\n", + "Score: 0.569670975208\n", + "\n", + "Content: From: nsmca@aurora.alaska.edu\n", + "Subject: Space Design Movies?\n", + "Article-I.D.: aurora.1993Apr23.124722.1\n", + "...\n", + "Metadata: {'category': 'sci.space'}\n", + "Score: 0.590400338173\n", + "\n" + ] + } + ], + "source": [ + "# Similarity search with score and filter\n", + "scored_results = vector_store.similarity_search_with_score(query, k=2)\n", + "\n", + "print(\"Similarity Search with Score Results:\")\n", + "for doc, score in scored_results:\n", + " print(f\"Content: {doc.page_content[:100]}...\")\n", + " print(f\"Metadata: {doc.metadata}\")\n", + " print(f\"Score: {score}\")\n", + " print()" + ] + }, + { + "cell_type": "markdown", + "id": "0c235cdc", + "metadata": {}, + "source": [ + "### Query by turning into retriever\n", + "\n", + "You can also transform the vector store into a retriever for easier usage in your chains." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "f3460093", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Document(metadata={'category': 'sci.space'}, page_content='Subject: Re: Comet in Temporary Orbit Around Jupiter?\\nFrom: Robert Coe \\nDistribution: world\\nOrganization: 1776 Enterprises, Sudbury MA\\nLines: 23\\n\\njgarland@kean.ucs.mun.ca writes:\\n\\n> >> Also, perihelions of Gehrels3 were:\\n> >> \\n> >> April 1973 83 jupiter radii\\n> >> August 1970 ~3 jupiter radii\\n> > \\n> > Where 1 Jupiter radius = 71,000 km = 44,000 mi = 0.0005 AU. So the\\n> > 1970 figure seems unlikely to actually be anything but a perijove.\\n> > Is that the case for the 1973 figure as well?\\n> > -- \\n> Sorry, _perijoves_...I\\'m not used to talking this language.\\n\\nHmmmm.... The prefix \"peri-\" is Greek, not Latin, so it\\'s usually used\\nwith the Greek form of the name of the body being orbited. (That\\'s why\\nit\\'s \"perihelion\" rather than \"perisol\", \"perigee\" rather than \"periterr\",\\nand \"pericynthion\" rather than \"perilune\".) So for Jupiter I\\'d expect it\\nto be something like \"perizeon\".) :^)\\n\\n ___ _ - Bob\\n /__) _ / / ) _ _\\n(_/__) (_)_(_) (___(_)_(/_______________________________________ bob@1776.COM\\nRobert K. Coe ** 14 Churchill St, Sudbury, Massachusetts 01776 ** 508-443-3265\\n'),\n", + " Document(metadata={'category': 'sci.space'}, page_content='From: pyron@skndiv.dseg.ti.com (Dillon Pyron)\\nSubject: Re: Why not give $1 billion to first year-long moon residents?\\nLines: 42\\nNntp-Posting-Host: skndiv.dseg.ti.com\\nReply-To: pyron@skndiv.dseg.ti.com\\nOrganization: TI/DSEG VAX Support\\n\\n\\nIn article <1qve4kINNpas@sal-sun121.usc.edu>, schaefer@sal-sun121.usc.edu (Peter Schaefer) writes:\\n>In article <1993Apr19.130503.1@aurora.alaska.edu>, nsmca@aurora.alaska.edu writes:\\n>|> In article <6ZV82B2w165w@theporch.raider.net>, gene@theporch.raider.net (Gene Wright) writes:\\n>|> > With the continuin talk about the \"End of the Space Age\" and complaints \\n>|> > by government over the large cost, why not try something I read about \\n>|> > that might just work.\\n>|> > \\n>|> > Announce that a reward of $1 billion would go to the first corporation \\n>|> > who successfully keeps at least 1 person alive on the moon for a year. \\n>|> > Then you\\'d see some of the inexpensive but not popular technologies begin \\n>|> > to be developed. THere\\'d be a different kind of space race then!\\n>|> > \\n>|> > --\\n>|> > gene@theporch.raider.net (Gene Wright)\\n>|> > theporch.raider.net 615/297-7951 The MacInteresteds of Nashville\\n>|> ====\\n>|> If that were true, I\\'d go for it.. I have a few friends who we could pool our\\n>|> resources and do it.. Maybe make it a prize kind of liek the \"Solar Car Race\"\\n>|> in Australia..\\n>|> Anybody game for a contest!\\n>|> \\n>|> ==\\n>|> Michael Adams, nsmca@acad3.alaska.edu -- I\\'m not high, just jacked\\n>\\n>\\n>Oh gee, a billion dollars! That\\'d be just about enough to cover the cost of the\\n>feasability study! Happy, Happy, JOY! JOY!\\n>\\n\\nFeasability study?? What a wimp!! While you are studying, others would be\\ndoing. Too damn many engineers doing way too little engineering.\\n\\n\"He who sits on his arse sits on his fortune\" - Sir Richard Francis Burton\\n--\\nDillon Pyron | The opinions expressed are those of the\\nTI/DSEG Lewisville VAX Support | sender unless otherwise stated.\\n(214)462-3556 (when I\\'m here) |\\n(214)492-4656 (when I\\'m home) |Texans: Vote NO on Robin Hood. We need\\npyron@skndiv.dseg.ti.com |solutions, not gestures.\\nPADI DM-54909 |\\n\\n')]" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "retriever = vector_store.as_retriever(search_type=\"similarity\", search_kwargs={\"k\": 2})\n", + "retriever.invoke(\"What planet in the solar system has the largest number of moons?\")" + ] + }, + { + "cell_type": "markdown", + "id": "901c75dc", + "metadata": {}, + "source": [ + "## Usage for retrieval-augmented generation\n", + "\n", + "For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n", + "\n", + "- [Tutorials](/docs/tutorials/)\n", + "- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n", + "- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/retrieval)" + ] + }, + { + "cell_type": "markdown", + "id": "069f1b5f", + "metadata": {}, + "source": [ + "## Redis-specific functionality\n", + "\n", + "Redis offers some unique features for vector search:" + ] + }, + { + "cell_type": "markdown", + "id": "8a627d3a-af78-46e2-b314-007e641b4d1d", + "metadata": {}, + "source": [ + "### Similarity search with metadata filtering\n", + "We can filter our search results based on metadata:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "23d6e6fe-8aee-4cee-bf05-59cf3fba36ae", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Filtered Similarity Search Results:\n", + "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", + "Subject: A flawed propulsion system: Space Shuttle\n", + "X-Ad...\n", + "Metadata: {'category': 'sci.space'}\n", + "\n", + "Content: From: nsmca@aurora.alaska.edu\n", + "Subject: Space Design Movies?\n", + "Article-I.D.: aurora.1993Apr23.124722.1\n", + "...\n", + "Metadata: {'category': 'sci.space'}\n", + "\n" + ] + } + ], + "source": [ + "from redisvl.query.filter import Tag\n", + "\n", + "query = \"Tell me about space exploration\"\n", + "\n", + "# Create a RedisVL filter expression\n", + "filter_condition = Tag(\"category\") == \"sci.space\"\n", + "\n", + "filtered_results = vector_store.similarity_search(query, k=2, filter=filter_condition)\n", + "\n", + "print(\"Filtered Similarity Search Results:\")\n", + "for doc in filtered_results:\n", + " print(f\"Content: {doc.page_content[:100]}...\")\n", + " print(f\"Metadata: {doc.metadata}\")\n", + " print()" + ] + }, + { + "cell_type": "markdown", + "id": "f35b9ebf-6419-4402-a066-c1d5287ed38e", + "metadata": {}, + "source": [ + "### Maximum marginal relevance search\n", + "Maximum marginal relevance search helps in getting diverse results:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "5be2afeb-d0a3-4075-bd3c-4cbe409dfb3a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Maximum Marginal Relevance Search Results:\n", + "Content: From: aa429@freenet.carleton.ca (Terry Ford)\n", + "Subject: A flawed propulsion system: Space Shuttle\n", + "X-Ad...\n", + "Metadata: {'category': 'sci.space'}\n", + "\n", + "Content: From: moroney@world.std.com (Michael Moroney)\n", + "Subject: Re: Vulcan? (No, not the guy with the ears!)\n", + "...\n", + "Metadata: {'category': 'sci.space'}\n", + "\n" + ] + } + ], + "source": [ + "# Maximum marginal relevance search with filter\n", + "mmr_results = vector_store.max_marginal_relevance_search(\n", + " query, k=2, fetch_k=10, filter=filter_condition\n", + ")\n", + "\n", + "print(\"Maximum Marginal Relevance Search Results:\")\n", + "for doc in mmr_results:\n", + " print(f\"Content: {doc.page_content[:100]}...\")\n", + " print(f\"Metadata: {doc.metadata}\")\n", + " print()" + ] + }, + { + "cell_type": "markdown", + "id": "09c3343c-6af4-4151-ba0a-50800fc34855", + "metadata": {}, + "source": [ + "## Chain usage\n", + "The code below shows how to use the vector store as a retriever in a simple RAG chain:\n", + "\n", + "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "9f6658f8-45b7-4004-a0b3-893bd23bff41", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "OpenAI API key not found in environment variables.\n" + ] + }, + { + "name": "stdin", + "output_type": "stream", + "text": [ + "Please enter your OpenAI API key: ········\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "OpenAI API key has been set for this session.\n" + ] + } + ], + "source": [ + "# | output: false\n", + "# | echo: false\n", + "from getpass import getpass\n", + "\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "# Check if OPENAI_API_KEY is already set in the environment\n", + "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n", + "\n", + "if not openai_api_key:\n", + " print(\"OpenAI API key not found in environment variables.\")\n", + " openai_api_key = getpass(\"Please enter your OpenAI API key: \")\n", + "\n", + " # Set the API key for the current session\n", + " os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n", + " print(\"OpenAI API key has been set for this session.\")\n", + "else:\n", + " print(\"OpenAI API key found in environment variables.\")\n", + "\n", + "llm = ChatOpenAI(model=\"gpt-4o-mini\")" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "d0ac614c-3f80-4839-8451-d3322a870809", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The Space Shuttle program was a NASA initiative that enabled reusable spacecraft to transport astronauts and cargo to and from low Earth orbit. It conducted a variety of missions, including satellite deployment, scientific research, and assembly of the International Space Station, and typically carried a crew of five astronauts. Although it achieved many successes, the program faced criticism for its safety concerns and the complexity of its propulsion system.'" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "from langchain_core.runnables import RunnablePassthrough\n", + "\n", + "# Prompt\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"human\",\n", + " \"\"\"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n", + "Question: {question}\n", + "Context: {context}\n", + "Answer:\"\"\",\n", + " ),\n", + " ]\n", + ")\n", + "\n", + "\n", + "def format_docs(docs):\n", + " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", + "\n", + "\n", + "rag_chain = (\n", + " {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")\n", + "\n", + "rag_chain.invoke(\"Describe the Space Shuttle program?\")" + ] + }, + { + "cell_type": "markdown", + "id": "8ad3e6e4-36ef-494a-be50-4bf8e374b077", + "metadata": {}, + "source": [ + "## Connect to an existing Index\n", + "\n", + "In order to have the same metadata indexed when using the ``Redis`` VectorStore. You will need to have the same ``index_schema`` passed in either as a path to a yaml file or as a dictionary. The following shows how to obtain the schema from an index and connect to an existing index." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "6a0e7a49-8271-44b2-abb2-0ef499546b28", + "metadata": {}, + "outputs": [], + "source": [ + "# write the schema to a yaml file\n", + "vector_store.index.schema.to_yaml(\"redis_schema.yaml\")" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "id": "e3588805-b3d9-4af8-8786-b57fc640ebb0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "18:19:58 redisvl.index.index INFO Index already exists, not overwriting.\n", + "page_content='From: aa429@freenet.carleton.ca (Terry Ford)\n", + "Subject: A flawed propulsion system: Space Shuttle\n", + "X-Added: Forwarded by Space Digest\n", + "Organization: [via International Space University]\n", + "Original-Sender: isu@VACATION.VENARI.CS.CMU.EDU\n", + "Distribution: sci\n", + "Lines: 13\n", + "\n", + "\n", + "\n", + "For an essay, I am writing about the space shuttle and a need for a better\n", + "propulsion system. Through research, I have found that it is rather clumsy \n", + "(i.e. all the checks/tests before launch), the safety hazards (\"sitting\n", + "on a hydrogen bomb\"), etc.. If you have any beefs about the current\n", + "space shuttle program Re: propulsion, please send me your ideas.\n", + "\n", + "Thanks a lot.\n", + "\n", + "--\n", + "Terry Ford [aa429@freenet.carleton.ca]\n", + "Nepean, Ontario, Canada.\n", + "' metadata={'category': 'sci.space'}\n" + ] + } + ], + "source": [ + "# now we can connect to our existing index as follows\n", + "\n", + "new_rdvs = RedisVectorStore(\n", + " embeddings,\n", + " redis_url=REDIS_URL,\n", + " schema_path=\"redis_schema.yaml\",\n", + ")\n", + "\n", + "results = new_rdvs.similarity_search(\"Space Shuttle Propulsion System\", k=3)\n", + "print(results[0])" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "id": "4d7ff456-de2a-4c58-9a3f-a9a3cfdca492", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# compare the two schemas to verify they are the same\n", + "new_rdvs.index.schema == vector_store.index.schema" + ] + }, + { + "cell_type": "markdown", + "id": "044a2a8c-cb25-453b-a439-38fcb06081ab", + "metadata": {}, + "source": [ + "## Cleanup vector store" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "bb24ab8b-1040-489d-bef6-9137dd2215f3", + "metadata": {}, + "outputs": [], + "source": [ + "# Clear vector store\n", + "vector_store.index.delete(drop=True)" + ] + }, + { + "cell_type": "markdown", + "id": "8a27244f", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all RedisVectorStore features and configurations head to the API reference: https://python.langchain.com/api_reference/redis/vectorstores/langchain_redis.vectorstores.RedisVectorStore.html" ] - }, - "execution_count": 35, - "metadata": {}, - "output_type": "execute_result" } - ], - "source": [ - "# compare the two schemas to verify they are the same\n", - "new_rdvs.index.schema == vector_store.index.schema" - ] + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "id": "044a2a8c-cb25-453b-a439-38fcb06081ab", - "metadata": {}, - "source": [ - "## Cleanup vector store" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "id": "bb24ab8b-1040-489d-bef6-9137dd2215f3", - "metadata": {}, - "outputs": [], - "source": [ - "# Clear vector store\n", - "vector_store.index.delete(drop=True)" - ] - }, - { - "cell_type": "markdown", - "id": "8a27244f", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all RedisVectorStore features and configurations head to the API reference: https://python.langchain.com/api_reference/redis/vectorstores/langchain_redis.vectorstores.RedisVectorStore.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/docs/docs/integrations/vectorstores/vdms.ipynb b/docs/docs/integrations/vectorstores/vdms.ipynb index f8fe1b69c01..f79aa2c592c 100644 --- a/docs/docs/integrations/vectorstores/vdms.ipynb +++ b/docs/docs/integrations/vectorstores/vdms.ipynb @@ -1,974 +1,974 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "1957f5cb", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: VDMS\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "ef1f0986", - "metadata": {}, - "source": [ - "# Intel's Visual Data Management System (VDMS)\n", - "\n", - "This notebook covers how to get started with VDMS as a vector store.\n", - "\n", - ">Intel's [Visual Data Management System (VDMS)](https://github.com/IntelLabs/vdms) is a storage solution for efficient access of big-”visual”-data that aims to achieve cloud scale by searching for relevant visual data via visual metadata stored as a graph and enabling machine friendly enhancements to visual data for faster access. VDMS is licensed under MIT. For more information on `VDMS`, visit [this page](https://github.com/IntelLabs/vdms/wiki), and find the LangChain API reference [here](https://python.langchain.com/api_reference/community/vectorstores/langchain_community.vectorstores.vdms.VDMS.html).\n", - "\n", - "VDMS supports:\n", - "* K nearest neighbor search\n", - "* Euclidean distance (L2) and inner product (IP)\n", - "* Libraries for indexing and computing distances: FaissFlat (Default), FaissHNSWFlat, FaissIVFFlat, Flinng, TileDBDense, TileDBSparse\n", - "* Embeddings for text, images, and video\n", - "* Vector and metadata searches" - ] - }, - { - "cell_type": "markdown", - "id": "36fdc060", - "metadata": {}, - "source": [ - "## Setup\n", - "\n", - "To access VDMS vector stores you'll need to install the `langchain-vdms` integration package and deploy a VDMS server via the publicly available Docker image.\n", - "For simplicity, this notebook will deploy a VDMS server on local host using port 55555." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "id": "64e28aa6", - "metadata": { - "vscode": { - "languageId": "shellscript" - } - }, - "outputs": [ + "cells": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Note: you may need to restart the kernel to use updated packages.\n", - "c464076e292613df27241765184a673b00c775cecb7792ef058591c2cbf0bde8\n" - ] - } - ], - "source": [ - "%pip install -qU \"langchain-vdms>=0.1.3\"\n", - "!docker run --no-healthcheck --rm -d -p 55555:55555 --name vdms_vs_test_nb intellabs/vdms:latest\n", - "!sleep 5" - ] - }, - { - "cell_type": "markdown", - "id": "9695dee7", - "metadata": {}, - "source": [ - "### Credentials\n" - ] - }, - { - "cell_type": "markdown", - "id": "7f98392b", - "metadata": {}, - "source": [ - "You can use `VDMS` without any credentials.\n", - "\n", - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "e7b6a6e0", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "93df377e", - "metadata": {}, - "source": [ - "## Initialization\n", - "Use the VDMS Client to connect to a VDMS vectorstore using FAISS IndexFlat indexing (default) and Euclidean distance (default) as the distance metric for similarity search.\n", - "\n", - "import EmbeddingTabs from \"@theme/EmbeddingTabs\";\n", - "\n", - "" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "01d25284", - "metadata": {}, - "outputs": [], - "source": [ - "# | output: false\n", - "# | echo: false\n", - "\n", - "! pip install -qU langchain-huggingface\n", - "from langchain_huggingface import HuggingFaceEmbeddings\n", - "\n", - "embeddings = HuggingFaceEmbeddings(model_name=\"sentence-transformers/all-mpnet-base-v2\")" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "dc37144c-208d-4ab3-9f3a-0407a69fe052", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain_vdms.vectorstores import VDMS, VDMS_Client\n", - "\n", - "collection_name = \"test_collection_faiss_L2\"\n", - "\n", - "vdms_client = VDMS_Client(host=\"localhost\", port=55555)\n", - "\n", - "vector_store = VDMS(\n", - " client=vdms_client,\n", - " embedding=embeddings,\n", - " collection_name=collection_name,\n", - " engine=\"FaissFlat\",\n", - " distance_strategy=\"L2\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "ac6071d4", - "metadata": {}, - "source": [ - "## Manage vector store\n", - "\n", - "### Add items to vector store" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "17f5efc0", - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']" + "cell_type": "raw", + "id": "1957f5cb", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: VDMS\n", + "---" ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "import logging\n", - "\n", - "logging.basicConfig()\n", - "logging.getLogger(\"langchain_vdms.vectorstores\").setLevel(logging.INFO)\n", - "\n", - "from langchain_core.documents import Document\n", - "\n", - "document_1 = Document(\n", - " page_content=\"I had chocolate chip pancakes and scrambled eggs for breakfast this morning.\",\n", - " metadata={\"source\": \"tweet\"},\n", - " id=1,\n", - ")\n", - "\n", - "document_2 = Document(\n", - " page_content=\"The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees.\",\n", - " metadata={\"source\": \"news\"},\n", - " id=2,\n", - ")\n", - "\n", - "document_3 = Document(\n", - " page_content=\"Building an exciting new project with LangChain - come check it out!\",\n", - " metadata={\"source\": \"tweet\"},\n", - " id=3,\n", - ")\n", - "\n", - "document_4 = Document(\n", - " page_content=\"Robbers broke into the city bank and stole $1 million in cash.\",\n", - " metadata={\"source\": \"news\"},\n", - " id=4,\n", - ")\n", - "\n", - "document_5 = Document(\n", - " page_content=\"Wow! That was an amazing movie. I can't wait to see it again.\",\n", - " metadata={\"source\": \"tweet\"},\n", - " id=5,\n", - ")\n", - "\n", - "document_6 = Document(\n", - " page_content=\"Is the new iPhone worth the price? Read this review to find out.\",\n", - " metadata={\"source\": \"website\"},\n", - " id=6,\n", - ")\n", - "\n", - "document_7 = Document(\n", - " page_content=\"The top 10 soccer players in the world right now.\",\n", - " metadata={\"source\": \"website\"},\n", - " id=7,\n", - ")\n", - "\n", - "document_8 = Document(\n", - " page_content=\"LangGraph is the best framework for building stateful, agentic applications!\",\n", - " metadata={\"source\": \"tweet\"},\n", - " id=8,\n", - ")\n", - "\n", - "document_9 = Document(\n", - " page_content=\"The stock market is down 500 points today due to fears of a recession.\",\n", - " metadata={\"source\": \"news\"},\n", - " id=9,\n", - ")\n", - "\n", - "document_10 = Document(\n", - " page_content=\"I have a bad feeling I am going to get deleted :(\",\n", - " metadata={\"source\": \"tweet\"},\n", - " id=10,\n", - ")\n", - "\n", - "documents = [\n", - " document_1,\n", - " document_2,\n", - " document_3,\n", - " document_4,\n", - " document_5,\n", - " document_6,\n", - " document_7,\n", - " document_8,\n", - " document_9,\n", - " document_10,\n", - "]\n", - "\n", - "doc_ids = [str(i) for i in range(1, 11)]\n", - "vector_store.add_documents(documents=documents, ids=doc_ids)" - ] - }, - { - "cell_type": "markdown", - "id": "37f4ad56", - "metadata": {}, - "source": [ - "If an id is provided multiple times, `add_documents` does not check whether the ids are unique. For this reason, use `upsert` to delete existing id entries prior to adding." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "id": "cb6a9f86", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "{'succeeded': ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'],\n", - " 'failed': []}" + "cell_type": "markdown", + "id": "ef1f0986", + "metadata": {}, + "source": [ + "# Intel's Visual Data Management System (VDMS)\n", + "\n", + "This notebook covers how to get started with VDMS as a vector store.\n", + "\n", + ">Intel's [Visual Data Management System (VDMS)](https://github.com/IntelLabs/vdms) is a storage solution for efficient access of big-”visual”-data that aims to achieve cloud scale by searching for relevant visual data via visual metadata stored as a graph and enabling machine friendly enhancements to visual data for faster access. VDMS is licensed under MIT. For more information on `VDMS`, visit [this page](https://github.com/IntelLabs/vdms/wiki), and find the LangChain API reference [here](https://python.langchain.com/api_reference/community/vectorstores/langchain_community.vectorstores.vdms.VDMS.html).\n", + "\n", + "VDMS supports:\n", + "* K nearest neighbor search\n", + "* Euclidean distance (L2) and inner product (IP)\n", + "* Libraries for indexing and computing distances: FaissFlat (Default), FaissHNSWFlat, FaissIVFFlat, Flinng, TileDBDense, TileDBSparse\n", + "* Embeddings for text, images, and video\n", + "* Vector and metadata searches" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "vector_store.upsert(documents, ids=doc_ids)" - ] - }, - { - "cell_type": "markdown", - "id": "c738c3e0", - "metadata": {}, - "source": [ - "### Update items in vector store\n" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "id": "f0aa8b71", - "metadata": {}, - "outputs": [], - "source": [ - "updated_document_1 = Document(\n", - " page_content=\"I had chocolate chip pancakes and fried eggs for breakfast this morning.\",\n", - " metadata={\"source\": \"tweet\"},\n", - " id=1,\n", - ")\n", - "\n", - "updated_document_2 = Document(\n", - " page_content=\"The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees.\",\n", - " metadata={\"source\": \"news\"},\n", - " id=2,\n", - ")\n", - "\n", - "vector_store.update_documents(\n", - " ids=doc_ids[:2],\n", - " documents=[updated_document_1, updated_document_2],\n", - " batch_size=2,\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "dcf1b905", - "metadata": {}, - "source": [ - "### Delete items from vector store\n" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "id": "ef61e188", - "metadata": {}, - "outputs": [ + }, { - "data": { - "text/plain": [ - "True" + "cell_type": "markdown", + "id": "36fdc060", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "To access VDMS vector stores you'll need to install the `langchain-vdms` integration package and deploy a VDMS server via the publicly available Docker image.\n", + "For simplicity, this notebook will deploy a VDMS server on local host using port 55555." ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "vector_store.delete(ids=doc_ids[-1])" - ] - }, - { - "cell_type": "markdown", - "id": "c3620501", - "metadata": {}, - "source": [ - "## Query vector store\n", - "\n", - "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. \n", - "\n", - "### Query directly\n", - "\n", - "Performing a simple similarity search can be done as follows:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "aa0a16fa", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0063 seconds\n" - ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* ID=3: Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", - "* ID=8: LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n" - ] - } - ], - "source": [ - "results = vector_store.similarity_search(\n", - " \"LangChain provides abstractions to make working with LLMs easy\",\n", - " k=2,\n", - " filter={\"source\": [\"==\", \"tweet\"]},\n", - ")\n", - "for doc in results:\n", - " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "3ed9d733", - "metadata": {}, - "source": [ - "If you want to execute a similarity search and receive the corresponding scores you can run:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "id": "5efd2eaa", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0460 seconds\n" - ] + "cell_type": "code", + "execution_count": 1, + "id": "64e28aa6", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Note: you may need to restart the kernel to use updated packages.\n", + "c464076e292613df27241765184a673b00c775cecb7792ef058591c2cbf0bde8\n" + ] + } + ], + "source": [ + "%pip install -qU \"langchain-vdms>=0.1.3\"\n", + "!docker run --no-healthcheck --rm -d -p 55555:55555 --name vdms_vs_test_nb intellabs/vdms:latest\n", + "!sleep 5" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* [SIM=0.753577] The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees. [{'source': 'news'}]\n" - ] - } - ], - "source": [ - "results = vector_store.similarity_search_with_score(\n", - " \"Will it be hot tomorrow?\", k=1, filter={\"source\": [\"==\", \"news\"]}\n", - ")\n", - "for doc, score in results:\n", - " print(f\"* [SIM={score:3f}] {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "496501e8", - "metadata": {}, - "source": [ - "If you want to execute a similarity search using an embedding you can run:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "dfa010e5", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0044 seconds\n" - ] + "cell_type": "markdown", + "id": "9695dee7", + "metadata": {}, + "source": [ + "### Credentials\n" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees. [{'source': 'news'}]\n" - ] - } - ], - "source": [ - "results = vector_store.similarity_search_by_vector(\n", - " embedding=embeddings.embed_query(\"I love green eggs and ham!\"), k=1\n", - ")\n", - "for doc in results:\n", - " print(f\"* {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "0c235cdc", - "metadata": {}, - "source": [ - "### Query by turning into retriever\n", - "\n", - "You can also transform the vector store into a retriever for easier usage in your chains. " - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "id": "bf66cf31", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0042 seconds\n" - ] + "cell_type": "markdown", + "id": "7f98392b", + "metadata": {}, + "source": [ + "You can use `VDMS` without any credentials.\n", + "\n", + "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n", - "* The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}]\n", - "* Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n" - ] - } - ], - "source": [ - "retriever = vector_store.as_retriever(\n", - " search_type=\"similarity\",\n", - " search_kwargs={\"k\": 3},\n", - ")\n", - "results = retriever.invoke(\"Stealing from the bank is a crime\")\n", - "for doc in results:\n", - " print(f\"* {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "id": "f3460093", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0042 seconds\n" - ] + "cell_type": "code", + "execution_count": 2, + "id": "e7b6a6e0", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n" - ] - } - ], - "source": [ - "retriever = vector_store.as_retriever(\n", - " search_type=\"similarity_score_threshold\",\n", - " search_kwargs={\n", - " \"k\": 1,\n", - " \"score_threshold\": 0.0, # >= score_threshold\n", - " },\n", - ")\n", - "results = retriever.invoke(\"Stealing from the bank is a crime\")\n", - "for doc in results:\n", - " print(f\"* {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "id": "6e971ae8", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:VDMS mmr search took 0.0042 secs\n" - ] + "cell_type": "markdown", + "id": "93df377e", + "metadata": {}, + "source": [ + "## Initialization\n", + "Use the VDMS Client to connect to a VDMS vectorstore using FAISS IndexFlat indexing (default) and Euclidean distance (default) as the distance metric for similarity search.\n", + "\n", + "import EmbeddingTabs from \"@theme/EmbeddingTabs\";\n", + "\n", + "" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n" - ] - } - ], - "source": [ - "retriever = vector_store.as_retriever(\n", - " search_type=\"mmr\",\n", - " search_kwargs={\"k\": 1, \"fetch_k\": 10},\n", - ")\n", - "results = retriever.invoke(\n", - " \"Stealing from the bank is a crime\", filter={\"source\": [\"==\", \"news\"]}\n", - ")\n", - "for doc in results:\n", - " print(f\"* {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "5f790d1b", - "metadata": {}, - "source": [ - "### Delete collection\n", - "Previously, we removed documents based on its `id`. Here, all documents are removed since no ID is provided." - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "id": "4bfac767", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Documents before deletion: 10\n", - "Documents after deletion: 0\n" - ] - } - ], - "source": [ - "print(\"Documents before deletion: \", vector_store.count())\n", - "\n", - "vector_store.delete(collection_name=collection_name)\n", - "\n", - "print(\"Documents after deletion: \", vector_store.count())" - ] - }, - { - "cell_type": "markdown", - "id": "901c75dc", - "metadata": {}, - "source": [ - "## Usage for retrieval-augmented generation\n", - "\n", - "For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n", - "\n", - "- [Multi-modal RAG using VDMS](https://github.com/langchain-ai/langchain/blob/master/cookbook/multi_modal_RAG_vdms.ipynb)\n", - "- [Visual RAG using VDMS](https://github.com/langchain-ai/langchain/blob/master/cookbook/visual_RAG_vdms.ipynb)\n", - "- [Tutorials](/docs/tutorials/)\n", - "- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n", - "- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/#retrieval)" - ] - }, - { - "cell_type": "markdown", - "id": "069f1b5f", - "metadata": {}, - "source": [ - "## Similarity Search using other engines\n", - "\n", - "VDMS supports various libraries for indexing and computing distances: FaissFlat (Default), FaissHNSWFlat, FaissIVFFlat, Flinng, TileDBDense, and TileDBSparse.\n", - "By default, the vectorstore uses FaissFlat. Below we show a few examples using the other engines." - ] - }, - { - "cell_type": "markdown", - "id": "68ab4d5d", - "metadata": {}, - "source": [ - "### Similarity Search using Faiss HNSWFlat and Euclidean Distance\n", - "\n", - "Here, we add the documents to VDMS using Faiss IndexHNSWFlat indexing and L2 as the distance metric for similarity search. We search for three documents (`k=3`) related to a query and also return the score along with the document." - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "id": "75af55fa", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:Descriptor set my_collection_FaissHNSWFlat_L2 created\n", - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.1272 seconds\n" - ] + "cell_type": "code", + "execution_count": null, + "id": "01d25284", + "metadata": {}, + "outputs": [], + "source": [ + "# | output: false\n", + "# | echo: false\n", + "\n", + "! pip install -qU langchain-huggingface\n", + "from langchain_huggingface import HuggingFaceEmbeddings\n", + "\n", + "embeddings = HuggingFaceEmbeddings(model_name=\"sentence-transformers/all-mpnet-base-v2\")" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* [SIM=0.716791] Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", - "* [SIM=0.936718] LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n", - "* [SIM=1.834110] Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n" - ] - } - ], - "source": [ - "db_FaissHNSWFlat = VDMS.from_documents(\n", - " documents,\n", - " client=vdms_client,\n", - " ids=doc_ids,\n", - " collection_name=\"my_collection_FaissHNSWFlat_L2\",\n", - " embedding=embeddings,\n", - " engine=\"FaissHNSWFlat\",\n", - " distance_strategy=\"L2\",\n", - ")\n", - "# Query\n", - "k = 3\n", - "query = \"LangChain provides abstractions to make working with LLMs easy\"\n", - "docs_with_score = db_FaissHNSWFlat.similarity_search_with_score(query, k=k, filter=None)\n", - "\n", - "for res, score in docs_with_score:\n", - " print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "37f62525", - "metadata": {}, - "source": [ - "### Similarity Search using Faiss IVFFlat and Inner Product (IP) Distance\n", - "\n", - "We add the documents to VDMS using Faiss IndexIVFFlat indexing and IP as the distance metric for similarity search. We search for three documents (`k=3`) related to a query and also return the score along with the document." - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "id": "b6c07d2f", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:Descriptor set my_collection_FaissIVFFlat_IP created\n", - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0052 seconds\n" - ] + "cell_type": "code", + "execution_count": 4, + "id": "dc37144c-208d-4ab3-9f3a-0407a69fe052", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain_vdms.vectorstores import VDMS, VDMS_Client\n", + "\n", + "collection_name = \"test_collection_faiss_L2\"\n", + "\n", + "vdms_client = VDMS_Client(host=\"localhost\", port=55555)\n", + "\n", + "vector_store = VDMS(\n", + " client=vdms_client,\n", + " embedding=embeddings,\n", + " collection_name=collection_name,\n", + " engine=\"FaissFlat\",\n", + " distance_strategy=\"L2\",\n", + ")" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* [SIM=0.641605] Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", - "* [SIM=0.531641] LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n", - "* [SIM=0.082945] Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n" - ] - } - ], - "source": [ - "db_FaissIVFFlat = VDMS.from_documents(\n", - " documents,\n", - " client=vdms_client,\n", - " ids=doc_ids,\n", - " collection_name=\"my_collection_FaissIVFFlat_IP\",\n", - " embedding=embeddings,\n", - " engine=\"FaissIVFFlat\",\n", - " distance_strategy=\"IP\",\n", - ")\n", - "\n", - "k = 3\n", - "query = \"LangChain provides abstractions to make working with LLMs easy\"\n", - "docs_with_score = db_FaissIVFFlat.similarity_search_with_score(query, k=k, filter=None)\n", - "for res, score in docs_with_score:\n", - " print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "f8efce5d", - "metadata": {}, - "source": [ - "### Similarity Search using FLINNG and IP Distance\n", - "\n", - "In this section, we add the documents to VDMS using Filters to Identify Near-Neighbor Groups (FLINNG) indexing and IP as the distance metric for similarity search. We search for three documents (`k=3`) related to a query and also return the score along with the document." - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "id": "69154f31", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "INFO:langchain_vdms.vectorstores:Descriptor set my_collection_Flinng_IP created\n", - "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0042 seconds\n" - ] + "cell_type": "markdown", + "id": "ac6071d4", + "metadata": {}, + "source": [ + "## Manage vector store\n", + "\n", + "### Add items to vector store" + ] }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* [SIM=0.000000] I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n", - "* [SIM=0.000000] I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n", - "* [SIM=0.000000] I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n" - ] - } - ], - "source": [ - "db_Flinng = VDMS.from_documents(\n", - " documents,\n", - " client=vdms_client,\n", - " ids=doc_ids,\n", - " collection_name=\"my_collection_Flinng_IP\",\n", - " embedding=embeddings,\n", - " engine=\"Flinng\",\n", - " distance_strategy=\"IP\",\n", - ")\n", - "# Query\n", - "k = 3\n", - "query = \"LangChain provides abstractions to make working with LLMs easy\"\n", - "docs_with_score = db_Flinng.similarity_search_with_score(query, k=k, filter=None)\n", - "for res, score in docs_with_score:\n", - " print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "375a9eef", - "metadata": {}, - "source": [ - "## Filtering on metadata\n", - "\n", - "It can be helpful to narrow down the collection before working with it.\n", - "\n", - "For example, collections can be filtered on metadata using the `get_by_constraints` method. A dictionary is used to filter metadata. Here we retrieve the document where `langchain_id = \"2\"` and remove it from the vector store. \n", - "\n", - "***NOTE:*** `id` was generated as additional metadata as an integer while `langchain_id` (the internal ID) is an unique string for each entry. " - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "id": "fea51565", - "metadata": {}, - "outputs": [ + "cell_type": "code", + "execution_count": 5, + "id": "17f5efc0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import logging\n", + "\n", + "logging.basicConfig()\n", + "logging.getLogger(\"langchain_vdms.vectorstores\").setLevel(logging.INFO)\n", + "\n", + "from langchain_core.documents import Document\n", + "\n", + "document_1 = Document(\n", + " page_content=\"I had chocolate chip pancakes and scrambled eggs for breakfast this morning.\",\n", + " metadata={\"source\": \"tweet\"},\n", + " id=1,\n", + ")\n", + "\n", + "document_2 = Document(\n", + " page_content=\"The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees.\",\n", + " metadata={\"source\": \"news\"},\n", + " id=2,\n", + ")\n", + "\n", + "document_3 = Document(\n", + " page_content=\"Building an exciting new project with LangChain - come check it out!\",\n", + " metadata={\"source\": \"tweet\"},\n", + " id=3,\n", + ")\n", + "\n", + "document_4 = Document(\n", + " page_content=\"Robbers broke into the city bank and stole $1 million in cash.\",\n", + " metadata={\"source\": \"news\"},\n", + " id=4,\n", + ")\n", + "\n", + "document_5 = Document(\n", + " page_content=\"Wow! That was an amazing movie. I can't wait to see it again.\",\n", + " metadata={\"source\": \"tweet\"},\n", + " id=5,\n", + ")\n", + "\n", + "document_6 = Document(\n", + " page_content=\"Is the new iPhone worth the price? Read this review to find out.\",\n", + " metadata={\"source\": \"website\"},\n", + " id=6,\n", + ")\n", + "\n", + "document_7 = Document(\n", + " page_content=\"The top 10 soccer players in the world right now.\",\n", + " metadata={\"source\": \"website\"},\n", + " id=7,\n", + ")\n", + "\n", + "document_8 = Document(\n", + " page_content=\"LangGraph is the best framework for building stateful, agentic applications!\",\n", + " metadata={\"source\": \"tweet\"},\n", + " id=8,\n", + ")\n", + "\n", + "document_9 = Document(\n", + " page_content=\"The stock market is down 500 points today due to fears of a recession.\",\n", + " metadata={\"source\": \"news\"},\n", + " id=9,\n", + ")\n", + "\n", + "document_10 = Document(\n", + " page_content=\"I have a bad feeling I am going to get deleted :(\",\n", + " metadata={\"source\": \"tweet\"},\n", + " id=10,\n", + ")\n", + "\n", + "documents = [\n", + " document_1,\n", + " document_2,\n", + " document_3,\n", + " document_4,\n", + " document_5,\n", + " document_6,\n", + " document_7,\n", + " document_8,\n", + " document_9,\n", + " document_10,\n", + "]\n", + "\n", + "doc_ids = [str(i) for i in range(1, 11)]\n", + "vector_store.add_documents(documents=documents, ids=doc_ids)" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "Deleted entry:\n", - "* ID=2: The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees. [{'source': 'news'}]\n" - ] - } - ], - "source": [ - "response, response_array = db_FaissIVFFlat.get_by_constraints(\n", - " db_FaissIVFFlat.collection_name,\n", - " limit=1,\n", - " include=[\"metadata\", \"embeddings\"],\n", - " constraints={\"langchain_id\": [\"==\", \"2\"]},\n", - ")\n", - "\n", - "# Delete id=2\n", - "db_FaissIVFFlat.delete(collection_name=db_FaissIVFFlat.collection_name, ids=[\"2\"])\n", - "\n", - "print(\"Deleted entry:\")\n", - "for doc in response:\n", - " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "id": "af7bffc1", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "37f4ad56", + "metadata": {}, + "source": [ + "If an id is provided multiple times, `add_documents` does not check whether the ids are unique. For this reason, use `upsert` to delete existing id entries prior to adding." + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* ID=10: I have a bad feeling I am going to get deleted :( [{'source': 'tweet'}]\n", - "* ID=9: The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}]\n", - "* ID=8: LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n", - "* ID=7: The top 10 soccer players in the world right now. [{'source': 'website'}]\n", - "* ID=6: Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n", - "* ID=5: Wow! That was an amazing movie. I can't wait to see it again. [{'source': 'tweet'}]\n", - "* ID=4: Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n", - "* ID=3: Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", - "* ID=1: I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n" - ] - } - ], - "source": [ - "response, response_array = db_FaissIVFFlat.get_by_constraints(\n", - " db_FaissIVFFlat.collection_name,\n", - " include=[\"metadata\"],\n", - ")\n", - "for doc in response:\n", - " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "3c3edde4", - "metadata": {}, - "source": [ - "Here we use `id` to filter for a range of IDs since it is an integer." - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "id": "6cacfcc6", - "metadata": {}, - "outputs": [ + "cell_type": "code", + "execution_count": 6, + "id": "cb6a9f86", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'succeeded': ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'],\n", + " 'failed': []}" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "vector_store.upsert(documents, ids=doc_ids)" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "* ID=9: The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}]\n", - "* ID=4: Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n" - ] - } - ], - "source": [ - "response, response_array = db_FaissIVFFlat.get_by_constraints(\n", - " db_FaissIVFFlat.collection_name,\n", - " include=[\"metadata\", \"embeddings\"],\n", - " constraints={\"source\": [\"==\", \"news\"]},\n", - ")\n", - "for doc in response:\n", - " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "f42c279e", - "metadata": {}, - "source": [ - "## Stop VDMS Server" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "id": "a838c50b", - "metadata": {}, - "outputs": [ + "cell_type": "markdown", + "id": "c738c3e0", + "metadata": {}, + "source": [ + "### Update items in vector store\n" + ] + }, { - "name": "stdout", - "output_type": "stream", - "text": [ - "vdms_vs_test_nb\n" - ] + "cell_type": "code", + "execution_count": 7, + "id": "f0aa8b71", + "metadata": {}, + "outputs": [], + "source": [ + "updated_document_1 = Document(\n", + " page_content=\"I had chocolate chip pancakes and fried eggs for breakfast this morning.\",\n", + " metadata={\"source\": \"tweet\"},\n", + " id=1,\n", + ")\n", + "\n", + "updated_document_2 = Document(\n", + " page_content=\"The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees.\",\n", + " metadata={\"source\": \"news\"},\n", + " id=2,\n", + ")\n", + "\n", + "vector_store.update_documents(\n", + " ids=doc_ids[:2],\n", + " documents=[updated_document_1, updated_document_2],\n", + " batch_size=2,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "dcf1b905", + "metadata": {}, + "source": [ + "### Delete items from vector store\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "ef61e188", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "vector_store.delete(ids=doc_ids[-1])" + ] + }, + { + "cell_type": "markdown", + "id": "c3620501", + "metadata": {}, + "source": [ + "## Query vector store\n", + "\n", + "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent.\n", + "\n", + "### Query directly\n", + "\n", + "Performing a simple similarity search can be done as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "aa0a16fa", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0063 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* ID=3: Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", + "* ID=8: LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n" + ] + } + ], + "source": [ + "results = vector_store.similarity_search(\n", + " \"LangChain provides abstractions to make working with LLMs easy\",\n", + " k=2,\n", + " filter={\"source\": [\"==\", \"tweet\"]},\n", + ")\n", + "for doc in results:\n", + " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "3ed9d733", + "metadata": {}, + "source": [ + "If you want to execute a similarity search and receive the corresponding scores you can run:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "5efd2eaa", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0460 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* [SIM=0.753577] The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees. [{'source': 'news'}]\n" + ] + } + ], + "source": [ + "results = vector_store.similarity_search_with_score(\n", + " \"Will it be hot tomorrow?\", k=1, filter={\"source\": [\"==\", \"news\"]}\n", + ")\n", + "for doc, score in results:\n", + " print(f\"* [SIM={score:3f}] {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "496501e8", + "metadata": {}, + "source": [ + "If you want to execute a similarity search using an embedding you can run:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "dfa010e5", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0044 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* The weather forecast for tomorrow is sunny and warm, with a high of 82 degrees. [{'source': 'news'}]\n" + ] + } + ], + "source": [ + "results = vector_store.similarity_search_by_vector(\n", + " embedding=embeddings.embed_query(\"I love green eggs and ham!\"), k=1\n", + ")\n", + "for doc in results:\n", + " print(f\"* {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "0c235cdc", + "metadata": {}, + "source": [ + "### Query by turning into retriever\n", + "\n", + "You can also transform the vector store into a retriever for easier usage in your chains." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "bf66cf31", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0042 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n", + "* The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}]\n", + "* Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n" + ] + } + ], + "source": [ + "retriever = vector_store.as_retriever(\n", + " search_type=\"similarity\",\n", + " search_kwargs={\"k\": 3},\n", + ")\n", + "results = retriever.invoke(\"Stealing from the bank is a crime\")\n", + "for doc in results:\n", + " print(f\"* {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "f3460093", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0042 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n" + ] + } + ], + "source": [ + "retriever = vector_store.as_retriever(\n", + " search_type=\"similarity_score_threshold\",\n", + " search_kwargs={\n", + " \"k\": 1,\n", + " \"score_threshold\": 0.0, # >= score_threshold\n", + " },\n", + ")\n", + "results = retriever.invoke(\"Stealing from the bank is a crime\")\n", + "for doc in results:\n", + " print(f\"* {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "6e971ae8", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:VDMS mmr search took 0.0042 secs\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n" + ] + } + ], + "source": [ + "retriever = vector_store.as_retriever(\n", + " search_type=\"mmr\",\n", + " search_kwargs={\"k\": 1, \"fetch_k\": 10},\n", + ")\n", + "results = retriever.invoke(\n", + " \"Stealing from the bank is a crime\", filter={\"source\": [\"==\", \"news\"]}\n", + ")\n", + "for doc in results:\n", + " print(f\"* {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "5f790d1b", + "metadata": {}, + "source": [ + "### Delete collection\n", + "Previously, we removed documents based on its `id`. Here, all documents are removed since no ID is provided." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "4bfac767", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Documents before deletion: 10\n", + "Documents after deletion: 0\n" + ] + } + ], + "source": [ + "print(\"Documents before deletion: \", vector_store.count())\n", + "\n", + "vector_store.delete(collection_name=collection_name)\n", + "\n", + "print(\"Documents after deletion: \", vector_store.count())" + ] + }, + { + "cell_type": "markdown", + "id": "901c75dc", + "metadata": {}, + "source": [ + "## Usage for retrieval-augmented generation\n", + "\n", + "For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n", + "\n", + "- [Multi-modal RAG using VDMS](https://github.com/langchain-ai/langchain/blob/master/cookbook/multi_modal_RAG_vdms.ipynb)\n", + "- [Visual RAG using VDMS](https://github.com/langchain-ai/langchain/blob/master/cookbook/visual_RAG_vdms.ipynb)\n", + "- [Tutorials](/docs/tutorials/)\n", + "- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n", + "- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/#retrieval)" + ] + }, + { + "cell_type": "markdown", + "id": "069f1b5f", + "metadata": {}, + "source": [ + "## Similarity Search using other engines\n", + "\n", + "VDMS supports various libraries for indexing and computing distances: FaissFlat (Default), FaissHNSWFlat, FaissIVFFlat, Flinng, TileDBDense, and TileDBSparse.\n", + "By default, the vectorstore uses FaissFlat. Below we show a few examples using the other engines." + ] + }, + { + "cell_type": "markdown", + "id": "68ab4d5d", + "metadata": {}, + "source": [ + "### Similarity Search using Faiss HNSWFlat and Euclidean Distance\n", + "\n", + "Here, we add the documents to VDMS using Faiss IndexHNSWFlat indexing and L2 as the distance metric for similarity search. We search for three documents (`k=3`) related to a query and also return the score along with the document." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "75af55fa", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:Descriptor set my_collection_FaissHNSWFlat_L2 created\n", + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.1272 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* [SIM=0.716791] Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", + "* [SIM=0.936718] LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n", + "* [SIM=1.834110] Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n" + ] + } + ], + "source": [ + "db_FaissHNSWFlat = VDMS.from_documents(\n", + " documents,\n", + " client=vdms_client,\n", + " ids=doc_ids,\n", + " collection_name=\"my_collection_FaissHNSWFlat_L2\",\n", + " embedding=embeddings,\n", + " engine=\"FaissHNSWFlat\",\n", + " distance_strategy=\"L2\",\n", + ")\n", + "# Query\n", + "k = 3\n", + "query = \"LangChain provides abstractions to make working with LLMs easy\"\n", + "docs_with_score = db_FaissHNSWFlat.similarity_search_with_score(query, k=k, filter=None)\n", + "\n", + "for res, score in docs_with_score:\n", + " print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "37f62525", + "metadata": {}, + "source": [ + "### Similarity Search using Faiss IVFFlat and Inner Product (IP) Distance\n", + "\n", + "We add the documents to VDMS using Faiss IndexIVFFlat indexing and IP as the distance metric for similarity search. We search for three documents (`k=3`) related to a query and also return the score along with the document." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "b6c07d2f", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:Descriptor set my_collection_FaissIVFFlat_IP created\n", + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0052 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* [SIM=0.641605] Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", + "* [SIM=0.531641] LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n", + "* [SIM=0.082945] Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n" + ] + } + ], + "source": [ + "db_FaissIVFFlat = VDMS.from_documents(\n", + " documents,\n", + " client=vdms_client,\n", + " ids=doc_ids,\n", + " collection_name=\"my_collection_FaissIVFFlat_IP\",\n", + " embedding=embeddings,\n", + " engine=\"FaissIVFFlat\",\n", + " distance_strategy=\"IP\",\n", + ")\n", + "\n", + "k = 3\n", + "query = \"LangChain provides abstractions to make working with LLMs easy\"\n", + "docs_with_score = db_FaissIVFFlat.similarity_search_with_score(query, k=k, filter=None)\n", + "for res, score in docs_with_score:\n", + " print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "f8efce5d", + "metadata": {}, + "source": [ + "### Similarity Search using FLINNG and IP Distance\n", + "\n", + "In this section, we add the documents to VDMS using Filters to Identify Near-Neighbor Groups (FLINNG) indexing and IP as the distance metric for similarity search. We search for three documents (`k=3`) related to a query and also return the score along with the document." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "69154f31", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO:langchain_vdms.vectorstores:Descriptor set my_collection_Flinng_IP created\n", + "INFO:langchain_vdms.vectorstores:VDMS similarity search took 0.0042 seconds\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* [SIM=0.000000] I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n", + "* [SIM=0.000000] I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n", + "* [SIM=0.000000] I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n" + ] + } + ], + "source": [ + "db_Flinng = VDMS.from_documents(\n", + " documents,\n", + " client=vdms_client,\n", + " ids=doc_ids,\n", + " collection_name=\"my_collection_Flinng_IP\",\n", + " embedding=embeddings,\n", + " engine=\"Flinng\",\n", + " distance_strategy=\"IP\",\n", + ")\n", + "# Query\n", + "k = 3\n", + "query = \"LangChain provides abstractions to make working with LLMs easy\"\n", + "docs_with_score = db_Flinng.similarity_search_with_score(query, k=k, filter=None)\n", + "for res, score in docs_with_score:\n", + " print(f\"* [SIM={score:3f}] {res.page_content} [{res.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "375a9eef", + "metadata": {}, + "source": [ + "## Filtering on metadata\n", + "\n", + "It can be helpful to narrow down the collection before working with it.\n", + "\n", + "For example, collections can be filtered on metadata using the `get_by_constraints` method. A dictionary is used to filter metadata. Here we retrieve the document where `langchain_id = \"2\"` and remove it from the vector store.\n", + "\n", + "***NOTE:*** `id` was generated as additional metadata as an integer while `langchain_id` (the internal ID) is an unique string for each entry." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "fea51565", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Deleted entry:\n", + "* ID=2: The weather forecast for tomorrow is cloudy and overcast, with a high of 62 degrees. [{'source': 'news'}]\n" + ] + } + ], + "source": [ + "response, response_array = db_FaissIVFFlat.get_by_constraints(\n", + " db_FaissIVFFlat.collection_name,\n", + " limit=1,\n", + " include=[\"metadata\", \"embeddings\"],\n", + " constraints={\"langchain_id\": [\"==\", \"2\"]},\n", + ")\n", + "\n", + "# Delete id=2\n", + "db_FaissIVFFlat.delete(collection_name=db_FaissIVFFlat.collection_name, ids=[\"2\"])\n", + "\n", + "print(\"Deleted entry:\")\n", + "for doc in response:\n", + " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "af7bffc1", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* ID=10: I have a bad feeling I am going to get deleted :( [{'source': 'tweet'}]\n", + "* ID=9: The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}]\n", + "* ID=8: LangGraph is the best framework for building stateful, agentic applications! [{'source': 'tweet'}]\n", + "* ID=7: The top 10 soccer players in the world right now. [{'source': 'website'}]\n", + "* ID=6: Is the new iPhone worth the price? Read this review to find out. [{'source': 'website'}]\n", + "* ID=5: Wow! That was an amazing movie. I can't wait to see it again. [{'source': 'tweet'}]\n", + "* ID=4: Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n", + "* ID=3: Building an exciting new project with LangChain - come check it out! [{'source': 'tweet'}]\n", + "* ID=1: I had chocolate chip pancakes and scrambled eggs for breakfast this morning. [{'source': 'tweet'}]\n" + ] + } + ], + "source": [ + "response, response_array = db_FaissIVFFlat.get_by_constraints(\n", + " db_FaissIVFFlat.collection_name,\n", + " include=[\"metadata\"],\n", + ")\n", + "for doc in response:\n", + " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "3c3edde4", + "metadata": {}, + "source": [ + "Here we use `id` to filter for a range of IDs since it is an integer." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "6cacfcc6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* ID=9: The stock market is down 500 points today due to fears of a recession. [{'source': 'news'}]\n", + "* ID=4: Robbers broke into the city bank and stole $1 million in cash. [{'source': 'news'}]\n" + ] + } + ], + "source": [ + "response, response_array = db_FaissIVFFlat.get_by_constraints(\n", + " db_FaissIVFFlat.collection_name,\n", + " include=[\"metadata\", \"embeddings\"],\n", + " constraints={\"source\": [\"==\", \"news\"]},\n", + ")\n", + "for doc in response:\n", + " print(f\"* ID={doc.id}: {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "f42c279e", + "metadata": {}, + "source": [ + "## Stop VDMS Server" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "a838c50b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "vdms_vs_test_nb\n" + ] + } + ], + "source": [ + "!docker kill vdms_vs_test_nb" + ] + }, + { + "cell_type": "markdown", + "id": "8a27244f", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "TODO: add API reference" + ] + }, + { + "cell_type": "markdown", + "id": "af4a2189", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".langchain-venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.10" } - ], - "source": [ - "!docker kill vdms_vs_test_nb" - ] }, - { - "cell_type": "markdown", - "id": "8a27244f", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "TODO: add API reference" - ] - }, - { - "cell_type": "markdown", - "id": "af4a2189", - "metadata": {}, - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": ".langchain-venv", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.10" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/libs/cli/langchain_cli/integration_template/docs/chat.ipynb b/libs/cli/langchain_cli/integration_template/docs/chat.ipynb index e6b89ecf937..ae55c6f188f 100644 --- a/libs/cli/langchain_cli/integration_template/docs/chat.ipynb +++ b/libs/cli/langchain_cli/integration_template/docs/chat.ipynb @@ -1,262 +1,260 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: __ModuleName__\n", - "---" - ] + "cells": [ + { + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: __ModuleName__\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# Chat__ModuleName__\n", + "\n", + "- TODO: Make sure API reference link is correct.\n", + "\n", + "This will help you getting started with __ModuleName__ [chat models](/docs/concepts/chat_models). For detailed documentation of all Chat__ModuleName__ features and configurations head to the [API reference](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.Chat__ModuleName__.html).\n", + "\n", + "- TODO: Add any other relevant links, like information about models, prices, context windows, etc. See https://python.langchain.com/docs/integrations/chat/openai/ for an example.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "- TODO: Fill in table features.\n", + "- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n", + "- TODO: Make sure API reference links are correct.\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/__package_name_short_snake__) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [Chat__ModuleName__](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.Chat__ModuleName__.html) | [__package_name__](https://python.langchain.com/api_reference/__package_name_short_snake__/) | ✅/❌ | beta/❌ | ✅/❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/__package_name__?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/__package_name__?style=flat-square&label=%20) |\n", + "\n", + "### Model features\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ |\n", + "\n", + "## Setup\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "To access __ModuleName__ models you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", + " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain __ModuleName__ integration lives in the `__package_name__` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU __package_name__" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from __module_name__ import Chat__ModuleName__\n", + "\n", + "llm = Chat__ModuleName__(\n", + " model=\"model-name\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation\n", + "\n", + "- TODO: Run cells so output can be seen." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "messages = [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + " ),\n", + " (\"human\", \"I love programming.\"),\n", + "]\n", + "ai_msg = llm.invoke(messages)\n", + "ai_msg" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [], + "source": [ + "print(ai_msg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:\n", + "\n", + "- TODO: Run cells so output can be seen." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "prompt = ChatPromptTemplate(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ),\n", + " (\"human\", \"{input}\"),\n", + " ]\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"input_language\": \"English\",\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## TODO: Any functionality specific to this model provider\n", + "\n", + "E.g. creating/using finetuned models via this provider. Delete if not relevant." + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all Chat__ModuleName__ features and configurations head to the [API reference](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.Chat__ModuleName__.html)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# Chat__ModuleName__\n", - "\n", - "- TODO: Make sure API reference link is correct.\n", - "\n", - "This will help you getting started with __ModuleName__ [chat models](/docs/concepts/chat_models). For detailed documentation of all Chat__ModuleName__ features and configurations head to the [API reference](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.Chat__ModuleName__.html).\n", - "\n", - "- TODO: Add any other relevant links, like information about models, prices, context windows, etc. See https://python.langchain.com/docs/integrations/chat/openai/ for an example.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "- TODO: Fill in table features.\n", - "- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n", - "- TODO: Make sure API reference links are correct.\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/__package_name_short_snake__) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [Chat__ModuleName__](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.Chat__ModuleName__.html) | [__package_name__](https://python.langchain.com/api_reference/__package_name_short_snake__/) | ✅/❌ | beta/❌ | ✅/❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/__package_name__?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/__package_name__?style=flat-square&label=%20) |\n", - "\n", - "### Model features\n", - "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", - "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", - "| ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | ✅/❌ | \n", - "\n", - "## Setup\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "To access __ModuleName__ models you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", - " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain __ModuleName__ integration lives in the `__package_name__` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU __package_name__" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from __module_name__ import Chat__ModuleName__\n", - "\n", - "llm = Chat__ModuleName__(\n", - " model=\"model-name\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "2b4f3e15", - "metadata": {}, - "source": [ - "## Invocation\n", - "\n", - "- TODO: Run cells so output can be seen." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "62e0dbc3", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "messages = [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", - " ),\n", - " (\"human\", \"I love programming.\"),\n", - "]\n", - "ai_msg = llm.invoke(messages)\n", - "ai_msg" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", - "metadata": {}, - "outputs": [], - "source": [ - "print(ai_msg.content)" - ] - }, - { - "cell_type": "markdown", - "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:\n", - "\n", - "- TODO: Run cells so output can be seen." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.prompts import ChatPromptTemplate\n", - "\n", - "prompt = ChatPromptTemplate(\n", - " [\n", - " (\n", - " \"system\",\n", - " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", - " ),\n", - " (\"human\", \"{input}\"),\n", - " ]\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"input_language\": \"English\",\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## TODO: Any functionality specific to this model provider\n", - "\n", - "E.g. creating/using finetuned models via this provider. Delete if not relevant." - ] - }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all Chat__ModuleName__ features and configurations head to the [API reference](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.Chat__ModuleName__.html)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/libs/cli/langchain_cli/integration_template/docs/document_loaders.ipynb b/libs/cli/langchain_cli/integration_template/docs/document_loaders.ipynb index 67ac60d249f..072ba47466f 100644 --- a/libs/cli/langchain_cli/integration_template/docs/document_loaders.ipynb +++ b/libs/cli/langchain_cli/integration_template/docs/document_loaders.ipynb @@ -1,219 +1,217 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: __ModuleName__\n", - "---" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: __ModuleName__\n", + "---" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# __ModuleName__Loader\n", + "\n", + "- TODO: Make sure API reference link is correct.\n", + "\n", + "This notebook provides a quick overview for getting started with __ModuleName__ [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all __ModuleName__Loader features and configurations head to the [API reference](https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.__module_name___loader.__ModuleName__Loader.html).\n", + "\n", + "- TODO: Add any other relevant links, like information about underlying API, etc.\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "- TODO: Fill in table features.\n", + "- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n", + "- TODO: Make sure API reference links are correct.\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/web_loaders/__module_name___loader)|\n", + "| :--- | :--- | :---: | :---: | :---: |\n", + "| [__ModuleName__Loader](https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.__module_name__loader.__ModuleName__Loader.html) | [langchain_community](https://api.python.langchain.com/en/latest/community_api_reference.html) | ✅/❌ | beta/❌ | ✅/❌ | \n", + "### Loader features\n", + "| Source | Document Lazy Loading | Native Async Support\n", + "| :---: | :---: | :---: | \n", + "| __ModuleName__Loader | ✅/❌ | ✅/❌ | \n", + "\n", + "## Setup\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "To access __ModuleName__ document loader you'll need to install the `__package_name__` integration package, and create a **ModuleName** account and get an API key.\n", + "\n", + "### Credentials\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "Install **langchain_community**.\n", + "\n", + "- TODO: Add any other required packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU langchain_community" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "Now we can instantiate our model object and load documents:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_community.document_loaders import __ModuleName__Loader\n", + "\n", + "loader = __ModuleName__Loader(\n", + " # required params = ...\n", + " # optional params = ...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load\n", + "\n", + "- TODO: Run cells to show loading capabilities" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "docs = loader.load()\n", + "docs[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lazy Load\n", + "\n", + "- TODO: Run cells to show lazy loading capabilities. Delete if lazy loading is not implemented." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "page = []\n", + "for doc in loader.lazy_load():\n", + " page.append(doc)\n", + " if len(page) >= 10:\n", + " # do some paged operation, e.g.\n", + " # index.upsert(page)\n", + "\n", + " page = []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## TODO: Any functionality specific to this document loader\n", + "\n", + "E.g. using specific configs for different loading behavior. Delete if not relevant." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all __ModuleName__Loader features and configurations head to the API reference: https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.__module_name___loader.__ModuleName__Loader.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# __ModuleName__Loader\n", - "\n", - "- TODO: Make sure API reference link is correct.\n", - "\n", - "This notebook provides a quick overview for getting started with __ModuleName__ [document loader](https://python.langchain.com/docs/concepts/document_loaders). For detailed documentation of all __ModuleName__Loader features and configurations head to the [API reference](https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.__module_name___loader.__ModuleName__Loader.html).\n", - "\n", - "- TODO: Add any other relevant links, like information about underlying API, etc.\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "- TODO: Fill in table features.\n", - "- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n", - "- TODO: Make sure API reference links are correct.\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/web_loaders/__module_name___loader)|\n", - "| :--- | :--- | :---: | :---: | :---: |\n", - "| [__ModuleName__Loader](https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.__module_name__loader.__ModuleName__Loader.html) | [langchain_community](https://api.python.langchain.com/en/latest/community_api_reference.html) | ✅/❌ | beta/❌ | ✅/❌ | \n", - "### Loader features\n", - "| Source | Document Lazy Loading | Native Async Support\n", - "| :---: | :---: | :---: | \n", - "| __ModuleName__Loader | ✅/❌ | ✅/❌ | \n", - "\n", - "## Setup\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "To access __ModuleName__ document loader you'll need to install the `__package_name__` integration package, and create a **ModuleName** account and get an API key.\n", - "\n", - "### Credentials\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to get automated best in-class tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "Install **langchain_community**.\n", - "\n", - "- TODO: Add any other required packages" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU langchain_community" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "Now we can instantiate our model object and load documents:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import __ModuleName__Loader\n", - "\n", - "loader = __ModuleName__Loader(\n", - " # required params = ...\n", - " # optional params = ...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load\n", - "\n", - "- TODO: Run cells to show loading capabilities" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "docs = loader.load()\n", - "docs[0]" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "print(docs[0].metadata)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Lazy Load\n", - "\n", - "- TODO: Run cells to show lazy loading capabilities. Delete if lazy loading is not implemented." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "page = []\n", - "for doc in loader.lazy_load():\n", - " page.append(doc)\n", - " if len(page) >= 10:\n", - " # do some paged operation, e.g.\n", - " # index.upsert(page)\n", - "\n", - " page = []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## TODO: Any functionality specific to this document loader\n", - "\n", - "E.g. using specific configs for different loading behavior. Delete if not relevant." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all __ModuleName__Loader features and configurations head to the API reference: https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.__module_name___loader.__ModuleName__Loader.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/libs/cli/langchain_cli/integration_template/docs/llms.ipynb b/libs/cli/langchain_cli/integration_template/docs/llms.ipynb index 3a2bedfd3ca..4700ced811b 100644 --- a/libs/cli/langchain_cli/integration_template/docs/llms.ipynb +++ b/libs/cli/langchain_cli/integration_template/docs/llms.ipynb @@ -1,238 +1,236 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "67db2992", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: __ModuleName__\n", - "---" - ] + "cells": [ + { + "cell_type": "raw", + "id": "67db2992", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: __ModuleName__\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "9597802c", + "metadata": {}, + "source": [ + "# __ModuleName__LLM\n", + "\n", + "- [ ] TODO: Make sure API reference link is correct\n", + "\n", + "This will help you get started with __ModuleName__ completion models (LLMs) using LangChain. For detailed documentation on `__ModuleName__LLM` features and configuration options, please refer to the [API reference](https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "- TODO: Fill in table features.\n", + "- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n", + "- TODO: Make sure API reference links are correct.\n", + "\n", + "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/llms/__package_name_short_snake__) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [__ModuleName__LLM](https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html) | [__package_name__](https://api.python.langchain.com/en/latest/__package_name_short_snake___api_reference.html) | ✅/❌ | beta/❌ | ✅/❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/__package_name__?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/__package_name__?style=flat-square&label=%20) |\n", + "\n", + "## Setup\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "To access __ModuleName__ models you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bc51e756", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", + " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "4b6e1ca6", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "196c2b41", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "809c6577", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain __ModuleName__ integration lives in the `__package_name__` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "59c710c4", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU __package_name__" + ] + }, + { + "cell_type": "markdown", + "id": "0a760037", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a0562a13", + "metadata": {}, + "outputs": [], + "source": [ + "from __module_name__ import __ModuleName__LLM\n", + "\n", + "llm = __ModuleName__LLM(\n", + " model=\"model-name\",\n", + " temperature=0,\n", + " max_tokens=None,\n", + " timeout=None,\n", + " max_retries=2,\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0ee90032", + "metadata": {}, + "source": [ + "## Invocation\n", + "\n", + "- [ ] TODO: Run cells so output can be seen." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "035dea0f", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "input_text = \"__ModuleName__ is an AI company that \"\n", + "\n", + "completion = llm.invoke(input_text)\n", + "completion" + ] + }, + { + "cell_type": "markdown", + "id": "add38532", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our completion model with a prompt template like so:\n", + "\n", + "- TODO: Run cells so output can be seen." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "078e9db2", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.prompts import PromptTemplate\n", + "\n", + "prompt = PromptTemplate(\n", + " \"How to say {input} in {output_language}:\\n\"\n", + ")\n", + "\n", + "chain = prompt | llm\n", + "chain.invoke(\n", + " {\n", + " \"output_language\": \"German\",\n", + " \"input\": \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "e99eef30", + "metadata": {}, + "source": [ + "## TODO: Any functionality specific to this model provider\n", + "\n", + "E.g. creating/using finetuned models via this provider. Delete if not relevant" + ] + }, + { + "cell_type": "markdown", + "id": "e9bdfcef", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `__ModuleName__LLM` features and configurations head to the API reference: https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3.11.1 64-bit", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.7" + }, + "vscode": { + "interpreter": { + "hash": "e971737741ff4ec9aff7dc6155a1060a59a8a6d52c757dbbe66bf8ee389494b1" + } + } }, - { - "cell_type": "markdown", - "id": "9597802c", - "metadata": {}, - "source": [ - "# __ModuleName__LLM\n", - "\n", - "- [ ] TODO: Make sure API reference link is correct\n", - "\n", - "This will help you get started with __ModuleName__ completion models (LLMs) using LangChain. For detailed documentation on `__ModuleName__LLM` features and configuration options, please refer to the [API reference](https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "- TODO: Fill in table features.\n", - "- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n", - "- TODO: Make sure API reference links are correct.\n", - "\n", - "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/llms/__package_name_short_snake__) | Package downloads | Package latest |\n", - "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", - "| [__ModuleName__LLM](https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html) | [__package_name__](https://api.python.langchain.com/en/latest/__package_name_short_snake___api_reference.html) | ✅/❌ | beta/❌ | ✅/❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/__package_name__?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/__package_name__?style=flat-square&label=%20) |\n", - "\n", - "## Setup\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "To access __ModuleName__ models you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "bc51e756", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", - " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "4b6e1ca6", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "196c2b41", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "809c6577", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain __ModuleName__ integration lives in the `__package_name__` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "59c710c4", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU __package_name__" - ] - }, - { - "cell_type": "markdown", - "id": "0a760037", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a0562a13", - "metadata": {}, - "outputs": [], - "source": [ - "from __module_name__ import __ModuleName__LLM\n", - "\n", - "llm = __ModuleName__LLM(\n", - " model=\"model-name\",\n", - " temperature=0,\n", - " max_tokens=None,\n", - " timeout=None,\n", - " max_retries=2,\n", - " # other params...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "0ee90032", - "metadata": {}, - "source": [ - "## Invocation\n", - "\n", - "- [ ] TODO: Run cells so output can be seen." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "035dea0f", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "input_text = \"__ModuleName__ is an AI company that \"\n", - "\n", - "completion = llm.invoke(input_text)\n", - "completion" - ] - }, - { - "cell_type": "markdown", - "id": "add38532", - "metadata": {}, - "source": [ - "## Chaining\n", - "\n", - "We can [chain](/docs/how_to/sequence/) our completion model with a prompt template like so:\n", - "\n", - "- TODO: Run cells so output can be seen." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "078e9db2", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.prompts import PromptTemplate\n", - "\n", - "prompt = PromptTemplate(\n", - " \"How to say {input} in {output_language}:\\n\"\n", - ")\n", - "\n", - "chain = prompt | llm\n", - "chain.invoke(\n", - " {\n", - " \"output_language\": \"German\",\n", - " \"input\": \"I love programming.\",\n", - " }\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "e99eef30", - "metadata": {}, - "source": [ - "## TODO: Any functionality specific to this model provider\n", - "\n", - "E.g. creating/using finetuned models via this provider. Delete if not relevant" - ] - }, - { - "cell_type": "markdown", - "id": "e9bdfcef", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all `__ModuleName__LLM` features and configurations head to the API reference: https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3.11.1 64-bit", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.7" - }, - "vscode": { - "interpreter": { - "hash": "e971737741ff4ec9aff7dc6155a1060a59a8a6d52c757dbbe66bf8ee389494b1" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/libs/cli/langchain_cli/integration_template/docs/text_embedding.ipynb b/libs/cli/langchain_cli/integration_template/docs/text_embedding.ipynb index 81369c29ca3..306512bde29 100644 --- a/libs/cli/langchain_cli/integration_template/docs/text_embedding.ipynb +++ b/libs/cli/langchain_cli/integration_template/docs/text_embedding.ipynb @@ -1,246 +1,244 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: __ModuleName__\n", - "---" - ] + "cells": [ + { + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: __ModuleName__\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# __ModuleName__Embeddings\n", + "\n", + "- [ ] TODO: Make sure API reference link is correct\n", + "\n", + "This will help you get started with __ModuleName__ embedding models using LangChain. For detailed documentation on `__ModuleName__Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/v0.2/api_reference/__package_name_short__/embeddings/__module_name__.embeddings__ModuleName__Embeddings.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Provider | Package |\n", + "|:--------:|:-------:|\n", + "| [__ModuleName__](/docs/integrations/providers/__package_name_short__/) | [__package_name__](https://python.langchain.com/v0.2/api_reference/__module_name__/embeddings/__module_name__.embeddings__ModuleName__Embeddings.html) |\n", + "\n", + "## Setup\n", + "\n", + "- [ ] TODO: Update with relevant info.\n", + "\n", + "To access __ModuleName__ embedding models you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "36521c2a", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", + " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "c84fb993", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39a4953b", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "d9664366", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "The LangChain __ModuleName__ integration lives in the `__package_name__` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64853226", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU __package_name__" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "from __module_name__ import __ModuleName__Embeddings\n", + "\n", + "embeddings = __ModuleName__Embeddings(\n", + " model=\"model-name\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d817716b", + "metadata": {}, + "outputs": [], + "source": [ + "# Create a vector store with a sample text\n", + "from langchain_core.vectorstores import InMemoryVectorStore\n", + "\n", + "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", + "\n", + "vectorstore = InMemoryVectorStore.from_texts(\n", + " [text],\n", + " embedding=embeddings,\n", + ")\n", + "\n", + "# Use the vectorstore as a retriever\n", + "retriever = vectorstore.as_retriever()\n", + "\n", + "# Retrieve the most similar text\n", + "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", + "\n", + "# show the retrieved document's content\n", + "retrieved_documents[0].page_content" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed single texts or documents with `embed_query`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0d2befcd", + "metadata": {}, + "outputs": [], + "source": [ + "single_vector = embeddings.embed_query(text)\n", + "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts with `embed_documents`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [], + "source": [ + "text2 = (\n", + " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", + ")\n", + "two_vectors = embeddings.embed_documents([text, text2])\n", + "for vector in two_vectors:\n", + " print(str(vector)[:100]) # Show the first 100 characters of the vector" + ] + }, + { + "cell_type": "markdown", + "id": "98785c12", + "metadata": {}, + "source": [ + "## API Reference\n", + "\n", + "For detailed documentation on `__ModuleName__Embeddings` features and configuration options, please refer to the [API reference](https://api.python.langchain.com/en/latest/embeddings/__module_name__.embeddings.__ModuleName__Embeddings.html).\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.5" + } }, - { - "cell_type": "markdown", - "id": "9a3d6f34", - "metadata": {}, - "source": [ - "# __ModuleName__Embeddings\n", - "\n", - "- [ ] TODO: Make sure API reference link is correct\n", - "\n", - "This will help you get started with __ModuleName__ embedding models using LangChain. For detailed documentation on `__ModuleName__Embeddings` features and configuration options, please refer to the [API reference](https://python.langchain.com/v0.2/api_reference/__package_name_short__/embeddings/__module_name__.embeddings__ModuleName__Embeddings.html).\n", - "\n", - "## Overview\n", - "### Integration details\n", - "\n", - "| Provider | Package |\n", - "|:--------:|:-------:|\n", - "| [__ModuleName__](/docs/integrations/providers/__package_name_short__/) | [__package_name__](https://python.langchain.com/v0.2/api_reference/__module_name__/embeddings/__module_name__.embeddings__ModuleName__Embeddings.html) |\n", - "\n", - "## Setup\n", - "\n", - "- [ ] TODO: Update with relevant info.\n", - "\n", - "To access __ModuleName__ embedding models you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package.\n", - "\n", - "### Credentials\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "36521c2a", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", - " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "c84fb993", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "39a4953b", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n", - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "d9664366", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "The LangChain __ModuleName__ integration lives in the `__package_name__` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "64853226", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU __package_name__" - ] - }, - { - "cell_type": "markdown", - "id": "45dd1724", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our model object and generate chat completions:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9ea7a09b", - "metadata": {}, - "outputs": [], - "source": [ - "from __module_name__ import __ModuleName__Embeddings\n", - "\n", - "embeddings = __ModuleName__Embeddings(\n", - " model=\"model-name\",\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "77d271b6", - "metadata": {}, - "source": [ - "## Indexing and Retrieval\n", - "\n", - "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our [RAG tutorials](/docs/tutorials/).\n", - "\n", - "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document in the `InMemoryVectorStore`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "d817716b", - "metadata": {}, - "outputs": [], - "source": [ - "# Create a vector store with a sample text\n", - "from langchain_core.vectorstores import InMemoryVectorStore\n", - "\n", - "text = \"LangChain is the framework for building context-aware reasoning applications\"\n", - "\n", - "vectorstore = InMemoryVectorStore.from_texts(\n", - " [text],\n", - " embedding=embeddings,\n", - ")\n", - "\n", - "# Use the vectorstore as a retriever\n", - "retriever = vectorstore.as_retriever()\n", - "\n", - "# Retrieve the most similar text\n", - "retrieved_documents = retriever.invoke(\"What is LangChain?\")\n", - "\n", - "# show the retrieved document's content\n", - "retrieved_documents[0].page_content" - ] - }, - { - "cell_type": "markdown", - "id": "e02b9855", - "metadata": {}, - "source": [ - "## Direct Usage\n", - "\n", - "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embed_documents(...)` and `embeddings.embed_query(...)` to create embeddings for the text(s) used in `from_texts` and retrieval `invoke` operations, respectively.\n", - "\n", - "You can directly call these methods to get embeddings for your own use cases.\n", - "\n", - "### Embed single texts\n", - "\n", - "You can embed single texts or documents with `embed_query`:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0d2befcd", - "metadata": {}, - "outputs": [], - "source": [ - "single_vector = embeddings.embed_query(text)\n", - "print(str(single_vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "1b5a7d03", - "metadata": {}, - "source": [ - "### Embed multiple texts\n", - "\n", - "You can embed multiple texts with `embed_documents`:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2f4d6e97", - "metadata": {}, - "outputs": [], - "source": [ - "text2 = (\n", - " \"LangGraph is a library for building stateful, multi-actor applications with LLMs\"\n", - ")\n", - "two_vectors = embeddings.embed_documents([text, text2])\n", - "for vector in two_vectors:\n", - " print(str(vector)[:100]) # Show the first 100 characters of the vector" - ] - }, - { - "cell_type": "markdown", - "id": "98785c12", - "metadata": {}, - "source": [ - "## API Reference\n", - "\n", - "For detailed documentation on `__ModuleName__Embeddings` features and configuration options, please refer to the [API reference](https://api.python.langchain.com/en/latest/embeddings/__module_name__.embeddings.__ModuleName__Embeddings.html).\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.5" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/libs/cli/langchain_cli/integration_template/docs/toolkits.ipynb b/libs/cli/langchain_cli/integration_template/docs/toolkits.ipynb index 23fbd2903cd..50d661223cc 100644 --- a/libs/cli/langchain_cli/integration_template/docs/toolkits.ipynb +++ b/libs/cli/langchain_cli/integration_template/docs/toolkits.ipynb @@ -1,201 +1,199 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "afaf8039", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: __ModuleName__\n", - "---" - ] + "cells": [ + { + "cell_type": "raw", + "id": "afaf8039", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: __ModuleName__\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# __ModuleName__Toolkit\n", + "\n", + "- TODO: Make sure API reference link is correct.\n", + "\n", + "This will help you getting started with the __ModuleName__ [toolkit](/docs/concepts/tools/#toolkits). For detailed documentation of all __ModuleName__Toolkit features and configurations head to the [API reference](https://api.python.langchain.com/en/latest/agent_toolkits/__module_name__.agent_toolkits.__ModuleName__.toolkit.__ModuleName__Toolkit.html).\n", + "\n", + "## Setup\n", + "\n", + "- TODO: Update with relevant info." + ] + }, + { + "cell_type": "markdown", + "id": "72ee0c4b-9764-423a-9dbf-95129e185210", + "metadata": {}, + "source": "To enable automated tracing of individual tools, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "0730d6a1-c893-4840-9817-5e5251676d5d", + "metadata": {}, + "source": [ + "### Installation\n", + "\n", + "This toolkit lives in the `__package_name__` package:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "652d6238-1f87-422a-b135-f5abbb8652fc", + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -qU __package_name__" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our toolkit:\n", + "\n", + "- TODO: Update model instantiation with relevant params." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "from __module_name__ import __ModuleName__Toolkit\n", + "\n", + "toolkit = __ModuleName__Toolkit(\n", + " # ...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "5c5f2839-4020-424e-9fc9-07777eede442", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "View available tools:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "51a60dbe-9f2e-4e04-bb62-23968f17164a", + "metadata": {}, + "outputs": [], + "source": [ + "toolkit.get_tools()" + ] + }, + { + "cell_type": "markdown", + "id": "d11245ad-3661-4405-8558-1188896347ec", + "metadata": {}, + "source": [ + "TODO: list API reference pages for individual tools." + ] + }, + { + "cell_type": "markdown", + "id": "dfe8aad4-8626-4330-98a9-7ea1ca5d2e0e", + "metadata": {}, + "source": [ + "## Use within an agent" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "310bf18e-6c9a-4072-b86e-47bc1fcca29d", + "metadata": {}, + "outputs": [], + "source": [ + "from langgraph.prebuilt import create_react_agent\n", + "\n", + "agent_executor = create_react_agent(llm, tools)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "23e11cc9-abd6-4855-a7eb-799f45ca01ae", + "metadata": {}, + "outputs": [], + "source": [ + "example_query = \"...\"\n", + "\n", + "events = agent_executor.stream(\n", + " {\"messages\": [(\"user\", example_query)]},\n", + " stream_mode=\"values\",\n", + ")\n", + "for event in events:\n", + " event[\"messages\"][-1].pretty_print()" + ] + }, + { + "cell_type": "markdown", + "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", + "metadata": {}, + "source": [ + "## TODO: Any functionality or considerations specific to this toolkit\n", + "\n", + "Fill in or delete if not relevant." + ] + }, + { + "cell_type": "markdown", + "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all __ModuleName__Toolkit features and configurations head to the [API reference](https://api.python.langchain.com/en/latest/agent_toolkits/__module_name__.agent_toolkits.__ModuleName__.toolkit.__ModuleName__Toolkit.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } }, - { - "cell_type": "markdown", - "id": "e49f1e0d", - "metadata": {}, - "source": [ - "# __ModuleName__Toolkit\n", - "\n", - "- TODO: Make sure API reference link is correct.\n", - "\n", - "This will help you getting started with the __ModuleName__ [toolkit](/docs/concepts/tools/#toolkits). For detailed documentation of all __ModuleName__Toolkit features and configurations head to the [API reference](https://api.python.langchain.com/en/latest/agent_toolkits/__module_name__.agent_toolkits.__ModuleName__.toolkit.__ModuleName__Toolkit.html).\n", - "\n", - "## Setup\n", - "\n", - "- TODO: Update with relevant info." - ] - }, - { - "cell_type": "markdown", - "id": "72ee0c4b-9764-423a-9dbf-95129e185210", - "metadata": {}, - "source": [ - "If you want to get automated tracing from runs of individual tools, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "0730d6a1-c893-4840-9817-5e5251676d5d", - "metadata": {}, - "source": [ - "### Installation\n", - "\n", - "This toolkit lives in the `__package_name__` package:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "652d6238-1f87-422a-b135-f5abbb8652fc", - "metadata": {}, - "outputs": [], - "source": [ - "%pip install -qU __package_name__" - ] - }, - { - "cell_type": "markdown", - "id": "a38cde65-254d-4219-a441-068766c0d4b5", - "metadata": {}, - "source": [ - "## Instantiation\n", - "\n", - "Now we can instantiate our toolkit:\n", - "\n", - "- TODO: Update model instantiation with relevant params." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", - "metadata": {}, - "outputs": [], - "source": [ - "from __module_name__ import __ModuleName__Toolkit\n", - "\n", - "toolkit = __ModuleName__Toolkit(\n", - " # ...\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "5c5f2839-4020-424e-9fc9-07777eede442", - "metadata": {}, - "source": [ - "## Tools\n", - "\n", - "View available tools:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "51a60dbe-9f2e-4e04-bb62-23968f17164a", - "metadata": {}, - "outputs": [], - "source": [ - "toolkit.get_tools()" - ] - }, - { - "cell_type": "markdown", - "id": "d11245ad-3661-4405-8558-1188896347ec", - "metadata": {}, - "source": [ - "TODO: list API reference pages for individual tools." - ] - }, - { - "cell_type": "markdown", - "id": "dfe8aad4-8626-4330-98a9-7ea1ca5d2e0e", - "metadata": {}, - "source": [ - "## Use within an agent" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "310bf18e-6c9a-4072-b86e-47bc1fcca29d", - "metadata": {}, - "outputs": [], - "source": [ - "from langgraph.prebuilt import create_react_agent\n", - "\n", - "agent_executor = create_react_agent(llm, tools)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "23e11cc9-abd6-4855-a7eb-799f45ca01ae", - "metadata": {}, - "outputs": [], - "source": [ - "example_query = \"...\"\n", - "\n", - "events = agent_executor.stream(\n", - " {\"messages\": [(\"user\", example_query)]},\n", - " stream_mode=\"values\",\n", - ")\n", - "for event in events:\n", - " event[\"messages\"][-1].pretty_print()" - ] - }, - { - "cell_type": "markdown", - "id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd", - "metadata": {}, - "source": [ - "## TODO: Any functionality or considerations specific to this toolkit\n", - "\n", - "Fill in or delete if not relevant." - ] - }, - { - "cell_type": "markdown", - "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all __ModuleName__Toolkit features and configurations head to the [API reference](https://api.python.langchain.com/en/latest/agent_toolkits/__module_name__.agent_toolkits.__ModuleName__.toolkit.__ModuleName__Toolkit.html)." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 } diff --git a/libs/cli/langchain_cli/integration_template/docs/vectorstores.ipynb b/libs/cli/langchain_cli/integration_template/docs/vectorstores.ipynb index bbf0b85a2fa..dcc18696549 100644 --- a/libs/cli/langchain_cli/integration_template/docs/vectorstores.ipynb +++ b/libs/cli/langchain_cli/integration_template/docs/vectorstores.ipynb @@ -1,342 +1,340 @@ { - "cells": [ - { - "cell_type": "raw", - "id": "1957f5cb", - "metadata": {}, - "source": [ - "---\n", - "sidebar_label: __ModuleName__\n", - "---" - ] - }, - { - "cell_type": "markdown", - "id": "ef1f0986", - "metadata": {}, - "source": [ - "# __ModuleName__VectorStore\n", - "\n", - "This notebook covers how to get started with the __ModuleName__ vector store." - ] - }, - { - "cell_type": "markdown", - "id": "36fdc060", - "metadata": {}, - "source": [ - "## Setup\n", - "\n", - "- TODO: Update with relevant info.\n", - "- TODO: Update minimum version to be correct.\n", - "\n", - "To access __ModuleName__ vector stores you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package." - ] - }, - { - "cell_type": "raw", - "id": "64e28aa6", - "metadata": { - "vscode": { - "languageId": "raw" + "cells": [ + { + "cell_type": "raw", + "id": "1957f5cb", + "metadata": {}, + "source": [ + "---\n", + "sidebar_label: __ModuleName__\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "ef1f0986", + "metadata": {}, + "source": [ + "# __ModuleName__VectorStore\n", + "\n", + "This notebook covers how to get started with the __ModuleName__ vector store." + ] + }, + { + "cell_type": "markdown", + "id": "36fdc060", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "- TODO: Update with relevant info.\n", + "- TODO: Update minimum version to be correct.\n", + "\n", + "To access __ModuleName__ vector stores you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package." + ] + }, + { + "cell_type": "raw", + "id": "64e28aa6", + "metadata": { + "vscode": { + "languageId": "raw" + } + }, + "source": [ + "%pip install -qU \"__package_name__>=MINIMUM_VERSION\"" + ] + }, + { + "cell_type": "markdown", + "id": "9695dee7", + "metadata": {}, + "source": [ + "### Credentials\n", + "\n", + "- TODO: Update with relevant info.\n", + "\n", + "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "894c30e4", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", + " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "7f98392b", + "metadata": {}, + "source": "To enable automated tracing of your model calls, set your [LangSmith](https://docs.smith.langchain.com/) API key:" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e7b6a6e0", + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "id": "93df377e", + "metadata": {}, + "source": [ + "## Initialization\n", + "\n", + "- TODO: Fill out with relevant init params\n", + "\n", + "\n", + "```{=mdx}\n", + "import EmbeddingTabs from \"@theme/EmbeddingTabs\";\n", + "\n", + "\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dc37144c-208d-4ab3-9f3a-0407a69fe052", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from __module_name__.vectorstores import __ModuleName__VectorStore\n", + "\n", + "vector_store = __ModuleName__VectorStore(embeddings=embeddings)" + ] + }, + { + "cell_type": "markdown", + "id": "ac6071d4", + "metadata": {}, + "source": [ + "## Manage vector store\n", + "\n", + "### Add items to vector store\n", + "\n", + "- TODO: Edit and then run code cell to generate output" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "17f5efc0", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_core.documents import Document\n", + "\n", + "document_1 = Document(\n", + " page_content=\"foo\",\n", + " metadata={\"source\": \"https://example.com\"}\n", + ")\n", + "\n", + "document_2 = Document(\n", + " page_content=\"bar\",\n", + " metadata={\"source\": \"https://example.com\"}\n", + ")\n", + "\n", + "document_3 = Document(\n", + " page_content=\"baz\",\n", + " metadata={\"source\": \"https://example.com\"}\n", + ")\n", + "\n", + "documents = [document_1, document_2, document_3]\n", + "\n", + "vector_store.add_documents(documents=documents,ids=[\"1\",\"2\",\"3\"])" + ] + }, + { + "cell_type": "markdown", + "id": "c738c3e0", + "metadata": {}, + "source": [ + "### Update items in vector store\n", + "\n", + "- TODO: Edit and then run code cell to generate output" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f0aa8b71", + "metadata": {}, + "outputs": [], + "source": [ + "updated_document = Document(\n", + " page_content=\"qux\",\n", + " metadata={\"source\": \"https://another-example.com\"}\n", + ")\n", + "\n", + "vector_store.update_documents(document_id=\"1\",document=updated_document)" + ] + }, + { + "cell_type": "markdown", + "id": "dcf1b905", + "metadata": {}, + "source": [ + "### Delete items from vector store\n", + "\n", + "- TODO: Edit and then run code cell to generate output" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ef61e188", + "metadata": {}, + "outputs": [], + "source": [ + "vector_store.delete(ids=[\"3\"])" + ] + }, + { + "cell_type": "markdown", + "id": "c3620501", + "metadata": {}, + "source": [ + "## Query vector store\n", + "\n", + "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent.\n", + "\n", + "### Query directly\n", + "\n", + "Performing a simple similarity search can be done as follows:\n", + "\n", + "- TODO: Edit and then run code cell to generate output" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aa0a16fa", + "metadata": {}, + "outputs": [], + "source": [ + "results = vector_store.similarity_search(query=\"thud\",k=1,filter={\"source\":\"https://another-example.com\"})\n", + "for doc in results:\n", + " print(f\"* {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "3ed9d733", + "metadata": {}, + "source": [ + "If you want to execute a similarity search and receive the corresponding scores you can run:\n", + "\n", + "- TODO: Edit and then run code cell to generate output" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5efd2eaa", + "metadata": {}, + "outputs": [], + "source": [ + "results = vector_store.similarity_search_with_score(query=\"thud\",k=1,filter={\"source\":\"https://example.com\"})\n", + "for doc, score in results:\n", + " print(f\"* [SIM={score:3f}] {doc.page_content} [{doc.metadata}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "0c235cdc", + "metadata": {}, + "source": [ + "### Query by turning into retriever\n", + "\n", + "You can also transform the vector store into a retriever for easier usage in your chains.\n", + "\n", + "- TODO: Edit and then run code cell to generate output" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f3460093", + "metadata": {}, + "outputs": [], + "source": [ + "retriever = vector_store.as_retriever(\n", + " search_type=\"mmr\",\n", + " search_kwargs={\"k\": 1}\n", + ")\n", + "retriever.invoke(\"thud\")" + ] + }, + { + "cell_type": "markdown", + "id": "901c75dc", + "metadata": {}, + "source": [ + "## Usage for retrieval-augmented generation\n", + "\n", + "For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n", + "\n", + "- [Tutorials](/docs/tutorials/)\n", + "- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n", + "- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/retrieval/)" + ] + }, + { + "cell_type": "markdown", + "id": "069f1b5f", + "metadata": {}, + "source": [ + "## TODO: Any functionality specific to this vector store\n", + "\n", + "E.g. creating a persisten database to save to your disk, etc." + ] + }, + { + "cell_type": "markdown", + "id": "8a27244f", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all __ModuleName__VectorStore features and configurations head to the API reference: https://api.python.langchain.com/en/latest/vectorstores/__module_name__.vectorstores.__ModuleName__VectorStore.html" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" } - }, - "source": [ - "%pip install -qU \"__package_name__>=MINIMUM_VERSION\"" - ] }, - { - "cell_type": "markdown", - "id": "9695dee7", - "metadata": {}, - "source": [ - "### Credentials\n", - "\n", - "- TODO: Update with relevant info.\n", - "\n", - "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "894c30e4", - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "import os\n", - "\n", - "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n", - " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")" - ] - }, - { - "cell_type": "markdown", - "id": "7f98392b", - "metadata": {}, - "source": [ - "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e7b6a6e0", - "metadata": {}, - "outputs": [], - "source": [ - "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", - "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" - ] - }, - { - "cell_type": "markdown", - "id": "93df377e", - "metadata": {}, - "source": [ - "## Initialization\n", - "\n", - "- TODO: Fill out with relevant init params\n", - "\n", - "\n", - "```{=mdx}\n", - "import EmbeddingTabs from \"@theme/EmbeddingTabs\";\n", - "\n", - "\n", - "```" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "dc37144c-208d-4ab3-9f3a-0407a69fe052", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from __module_name__.vectorstores import __ModuleName__VectorStore\n", - "\n", - "vector_store = __ModuleName__VectorStore(embeddings=embeddings)" - ] - }, - { - "cell_type": "markdown", - "id": "ac6071d4", - "metadata": {}, - "source": [ - "## Manage vector store\n", - "\n", - "### Add items to vector store\n", - "\n", - "- TODO: Edit and then run code cell to generate output" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "17f5efc0", - "metadata": {}, - "outputs": [], - "source": [ - "from langchain_core.documents import Document\n", - "\n", - "document_1 = Document(\n", - " page_content=\"foo\",\n", - " metadata={\"source\": \"https://example.com\"}\n", - ")\n", - "\n", - "document_2 = Document(\n", - " page_content=\"bar\",\n", - " metadata={\"source\": \"https://example.com\"}\n", - ")\n", - "\n", - "document_3 = Document(\n", - " page_content=\"baz\",\n", - " metadata={\"source\": \"https://example.com\"}\n", - ")\n", - "\n", - "documents = [document_1, document_2, document_3]\n", - "\n", - "vector_store.add_documents(documents=documents,ids=[\"1\",\"2\",\"3\"])" - ] - }, - { - "cell_type": "markdown", - "id": "c738c3e0", - "metadata": {}, - "source": [ - "### Update items in vector store\n", - "\n", - "- TODO: Edit and then run code cell to generate output" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f0aa8b71", - "metadata": {}, - "outputs": [], - "source": [ - "updated_document = Document(\n", - " page_content=\"qux\",\n", - " metadata={\"source\": \"https://another-example.com\"}\n", - ")\n", - "\n", - "vector_store.update_documents(document_id=\"1\",document=updated_document)" - ] - }, - { - "cell_type": "markdown", - "id": "dcf1b905", - "metadata": {}, - "source": [ - "### Delete items from vector store\n", - "\n", - "- TODO: Edit and then run code cell to generate output" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "ef61e188", - "metadata": {}, - "outputs": [], - "source": [ - "vector_store.delete(ids=[\"3\"])" - ] - }, - { - "cell_type": "markdown", - "id": "c3620501", - "metadata": {}, - "source": [ - "## Query vector store\n", - "\n", - "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. \n", - "\n", - "### Query directly\n", - "\n", - "Performing a simple similarity search can be done as follows:\n", - "\n", - "- TODO: Edit and then run code cell to generate output" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "aa0a16fa", - "metadata": {}, - "outputs": [], - "source": [ - "results = vector_store.similarity_search(query=\"thud\",k=1,filter={\"source\":\"https://another-example.com\"})\n", - "for doc in results:\n", - " print(f\"* {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "3ed9d733", - "metadata": {}, - "source": [ - "If you want to execute a similarity search and receive the corresponding scores you can run:\n", - "\n", - "- TODO: Edit and then run code cell to generate output" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5efd2eaa", - "metadata": {}, - "outputs": [], - "source": [ - "results = vector_store.similarity_search_with_score(query=\"thud\",k=1,filter={\"source\":\"https://example.com\"})\n", - "for doc, score in results:\n", - " print(f\"* [SIM={score:3f}] {doc.page_content} [{doc.metadata}]\")" - ] - }, - { - "cell_type": "markdown", - "id": "0c235cdc", - "metadata": {}, - "source": [ - "### Query by turning into retriever\n", - "\n", - "You can also transform the vector store into a retriever for easier usage in your chains. \n", - "\n", - "- TODO: Edit and then run code cell to generate output" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f3460093", - "metadata": {}, - "outputs": [], - "source": [ - "retriever = vector_store.as_retriever(\n", - " search_type=\"mmr\",\n", - " search_kwargs={\"k\": 1}\n", - ")\n", - "retriever.invoke(\"thud\")" - ] - }, - { - "cell_type": "markdown", - "id": "901c75dc", - "metadata": {}, - "source": [ - "## Usage for retrieval-augmented generation\n", - "\n", - "For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n", - "\n", - "- [Tutorials](/docs/tutorials/)\n", - "- [How-to: Question and answer with RAG](https://python.langchain.com/docs/how_to/#qa-with-rag)\n", - "- [Retrieval conceptual docs](https://python.langchain.com/docs/concepts/retrieval/)" - ] - }, - { - "cell_type": "markdown", - "id": "069f1b5f", - "metadata": {}, - "source": [ - "## TODO: Any functionality specific to this vector store\n", - "\n", - "E.g. creating a persisten database to save to your disk, etc." - ] - }, - { - "cell_type": "markdown", - "id": "8a27244f", - "metadata": {}, - "source": [ - "## API reference\n", - "\n", - "For detailed documentation of all __ModuleName__VectorStore features and configurations head to the API reference: https://api.python.langchain.com/en/latest/vectorstores/__module_name__.vectorstores.__ModuleName__VectorStore.html" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.12" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "nbformat": 4, + "nbformat_minor": 5 }