Merge branch 'master' into eugene/langchain_how_to_config

docs/docs/concepts.mdx

@@ -108,20 +108,39 @@ These also have corresponding async methods that should be used with [asyncio](h
 
 The **input type** and **output type** varies by component:
 
-| Component | Input Type | Output Type |
-| --- | --- | --- |
-| Prompt | Dictionary | PromptValue |
-| ChatModel | Single string, list of chat messages or a PromptValue | ChatMessage |
-| LLM | Single string, list of chat messages or a PromptValue | String |
-| OutputParser | The output of an LLM or ChatModel | Depends on the parser |
-| Retriever | Single string | List of Documents |
-| Tool | Single string or dictionary, depending on the tool | Depends on the tool |
+| Component    | Input Type                                            | Output Type           |
+|--------------|-------------------------------------------------------|-----------------------|
+| Prompt       | Dictionary                                            | PromptValue           |
+| ChatModel    | Single string, list of chat messages or a PromptValue | ChatMessage           |
+| LLM          | Single string, list of chat messages or a PromptValue | String                |
+| OutputParser | The output of an LLM or ChatModel                     | Depends on the parser |
+| Retriever    | Single string                                         | List of Documents     |
+| Tool         | Single string or dictionary, depending on the tool    | Depends on the tool   |
 
 
 All runnables expose input and output **schemas** to inspect the inputs and outputs:
+
 - `input_schema`: an input Pydantic model auto-generated from the structure of the Runnable
 - `output_schema`: an output Pydantic model auto-generated from the structure of the Runnable
 
+
+#### Runnable Config
+
+The standard runnable methods (e.g., `stream`, `invoke`, `batch`) accept a 2nd argument. This argument is a config dict that follows the [RunnableConfig](https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.config.RunnableConfig.html) format.
+
+A `RunnableConfig` can have any of the following properties defined:
+
+| Attribute       | Description                                                                                |
+|-----------------|--------------------------------------------------------------------------------------------|
+| tags            | Tags for this call and any sub-calls.                                                      |
+| metadata        | Metadata for this call and any sub-calls.                                                  |
+| callbacks       | Callbacks for this call and any sub-calls.                                                 |
+| run_name        | Name used for the given Runnable (not inherited).                                          |
+| max_concurrency | Maximum number of parallel calls to make (e.g., used by batch).                            |
+| recursion_limit | Maximum number of times a call can recurse (e.g., used by Runnables that return Runnables) |
+| configurable    | Runtime values for configurable attributes of the Runnable.                                |
+| run_id          | Unique identifier for the tracer run for this call.                                        |
+
 ## Components
 
 LangChain provides standard, extendable interfaces and external integrations for various components useful for building with LLMs.
@@ -546,10 +565,9 @@ callbacks to any child objects.
 Any `RunnableLambda`, a `RunnableGenerator`, or `Tool` that invokes other runnables
 and is running async in python<=3.10, will have to propagate callbacks to child
 objects manually. This is because LangChain cannot automatically propagate
-callbacks to child objects in this case.
-
-This is a common reason why you may fail to see events being emitted from custom
-runnables or tools.
+callbacks to child objects in this case. If you see that callbacks
+are not being triggered in your child objects, you may need to [manually
+propagate runnable config to child runnables](/docs/how_to/runnables_propagate_config/).
 :::
 
 ## Techniques

docs/docs/how_to/callbacks_async.ipynb

@@ -12,6 +12,7 @@
     "\n",
     "- [Callbacks](/docs/concepts/#callbacks)\n",
     "- [Custom callback handlers](/docs/how_to/custom_callbacks)\n",
+    "- [Custom callback handlers](/docs/how_to/propagate_runnable_config)\n",
     ":::\n",
     "\n",
     "If you are planning to use the async APIs, it is recommended to use and extend [`AsyncCallbackHandler`](https://api.python.langchain.com/en/latest/callbacks/langchain_core.callbacks.base.AsyncCallbackHandler.html) to avoid blocking the event.\n",
@@ -23,8 +24,9 @@
     "\n",
     ":::{.callout-danger}\n",
     "\n",
-    "If you're on `python<=3.10`, you need to remember to propagate `config` or `callbacks` when invoking other `runnable` from within a `RunnableLambda`, `RunnableGenerator` or `@tool`. If you do not do this,\n",
-    "the callbacks will not be propagated to the child runnables being invoked.\n",
+    "If you're on `python<=3.10`, you need to remember to propagate `config` or `callbacks` when invoking other `runnable` from within a `RunnableLambda`, `RunnableGenerator` or `@tool`. \n",
+    "\n",
+    "See this the guide on [how to propagate runnable config to child components](/docs/how_to/propagate_runnable_config) for more information.\n",
     ":::"
    ]
   },

docs/docs/how_to/index.mdx

@@ -34,6 +34,7 @@ This highlights functionality that is core to using LangChain.
 - [How to: stream runnables](/docs/how_to/streaming)
 - [How to: invoke runnables in parallel](/docs/how_to/parallel/)
 - [How to: add default invocation args to runnables](/docs/how_to/binding/)
+- [How to: propagate runnable config to child runnables](/docs/how_to/runnables_propagate_config/)
 - [How to: turn any function into a runnable](/docs/how_to/functions)
 - [How to: pass through inputs from one chain step to the next](/docs/how_to/passthrough)
 - [How to: configure runnable behavior at runtime](/docs/how_to/configure)

docs/docs/how_to/runnables_propagate_config.ipynb

@@ -0,0 +1,383 @@
+{
+ "cells": [
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "sidebar_position: 0\n",
+    "keywords: [Runnable, Runnables, LCEL, RunnableConfig, runnable config]\n",
+    "---"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# How to propagate config\n",
+    "\n",
+    ":::info Prerequisites\n",
+    "\n",
+    "This guide assumes familiarity with the following concepts:\n",
+    "\n",
+    "- [LangChain Expression Language (LCEL)](/docs/concepts/#langchain-expression-language)\n",
+    "- [LangChain RunnableConfig](/docs/concepts/#runnable-config)\n",
+    ":::\n",
+    "\n",
+    "\n",
+    "The [RunnableConfig](https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.config.RunnableConfig.html) dict contains run time configuration information that needs to be properly\n",
+    "propagated throughout the entire chain. This config contains things like `callbacks` and user provided `tags` and `metadata`.\n",
+    "\n",
+    "If you're composing a chain entirely using LCEL, then `RunnableConfig` will be propagated automatically on your behalf.\n",
+    "\n",
+    "If you're using `RunnableLambda`, `RunnableGenerator` or `@tool` to create custom components, `langchain` will attempt to propagate callbacks on your behalf from those components to any child `Runnables` when possible.\n",
+    "\n",
+    ":::{.callout-important}\n",
+    "\n",
+    "If you're working in an `async` code base and are using `python<=3.10`, you will need to propagate `config` (or `callbacks`) manually since LangChain is unable to do this automatically.\n",
+    "\n",
+    ":::"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Sync (All Python Versions)\n",
+    "\n",
+    "You should see that the custom callback is invoked twice!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "---\n",
+      "Invoking custom callback. Received inputs: hello\n",
+      "---\n",
+      "Invoking custom callback. Received inputs: goodbye\n",
+      "---\n",
+      "Inspect the config given to foo: `{'tags': ['hello'], 'metadata': {}, 'callbacks': <langchain_core.callbacks.manager.CallbackManager object at 0x7171772f1b90>, 'recursion_limit': 25}`\n"
+     ]
+    }
+   ],
+   "source": [
+    "import asyncio\n",
+    "from typing import Any, Dict, List\n",
+    "\n",
+    "from langchain_core.callbacks import BaseCallbackHandler\n",
+    "from langchain_core.runnables import RunnableLambda\n",
+    "\n",
+    "\n",
+    "class MyCustomHandler(BaseCallbackHandler):\n",
+    "    \"\"\"Sync callback handler that can be used to handle callbacks from langchain.\"\"\"\n",
+    "\n",
+    "    def on_chain_start(\n",
+    "        self,\n",
+    "        serialized: Dict[str, Any],\n",
+    "        inputs: Dict[str, Any],\n",
+    "        **kwargs: Any,\n",
+    "    ) -> None:\n",
+    "        \"\"\"Run when chain starts running.\"\"\"\n",
+    "        print(\"---\")\n",
+    "        print(f\"Invoking custom callback. Received inputs: {inputs}\")\n",
+    "\n",
+    "\n",
+    "@RunnableLambda\n",
+    "def foo(\n",
+    "    inputs, config\n",
+    "):  # You don't need to have config. Using it just to print it out.\n",
+    "    print(\"---\")\n",
+    "    print(f\"Inspect the config given to foo: `{config}`\")\n",
+    "    return \"world\"\n",
+    "\n",
+    "\n",
+    "@RunnableLambda\n",
+    "def bar(inputs):\n",
+    "    return foo.invoke(\"goodbye\")\n",
+    "\n",
+    "\n",
+    "bar.invoke(\"hello\", {\"callbacks\": [MyCustomHandler()], \"tags\": [\"hello\"]});"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Async Python >=3.11\n",
+    "\n",
+    "The custom callback will be invoked twice!\n",
+    "\n",
+    "This code will work correctly even thought `bar` does not propagate config to `foo` does not propagate config!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'3.11.4 (main, Sep 25 2023, 10:06:23) [GCC 11.4.0]'"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import sys\n",
+    "\n",
+    "sys.version"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "---\n",
+      "Invoking custom callback. Received inputs: hello\n",
+      "---\n",
+      "Invoking custom callback. Received inputs: goodbye\n",
+      "---\n",
+      "Inspect the config given to foo: `{'tags': [], 'metadata': {}, 'callbacks': <langchain_core.callbacks.manager.AsyncCallbackManager object at 0x7171772feb50>, 'recursion_limit': 25}`\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'world'"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import asyncio\n",
+    "from typing import Any, Dict, List\n",
+    "\n",
+    "from langchain_core.callbacks import AsyncCallbackHandler\n",
+    "from langchain_core.runnables import RunnableLambda\n",
+    "\n",
+    "\n",
+    "class AsyncMyCustomHandler(AsyncCallbackHandler):\n",
+    "    \"\"\"Async callback handler that can be used to handle callbacks from langchain.\"\"\"\n",
+    "\n",
+    "    async def on_chain_start(\n",
+    "        self,\n",
+    "        serialized: Dict[str, Any],\n",
+    "        inputs: Dict[str, Any],\n",
+    "        **kwargs: Any,\n",
+    "    ) -> None:\n",
+    "        \"\"\"Run when chain starts running.\"\"\"\n",
+    "        print(\"---\")\n",
+    "        print(f\"Invoking custom callback. Received inputs: {inputs}\")\n",
+    "\n",
+    "\n",
+    "@RunnableLambda\n",
+    "async def foo(inputs, config):\n",
+    "    print(\"---\")\n",
+    "    print(f\"Inspect the config given to foo: `{config}`\")\n",
+    "    return \"world\"\n",
+    "\n",
+    "\n",
+    "@RunnableLambda\n",
+    "async def bar(inputs):\n",
+    "    return await foo.ainvoke(\"goodbye\")\n",
+    "\n",
+    "\n",
+    "await bar.ainvoke(\"hello\", {\"callbacks\": [AsyncMyCustomHandler()]})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Async Python <=3.10\n",
+    "\n",
+    "### Incorrect\n",
+    "\n",
+    "This code will not work!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'3.9.6 (default, Sep 26 2023, 21:46:56) \\n[GCC 11.4.0]'"
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import sys\n",
+    "\n",
+    "sys.version"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "---\n",
+      "Invoking custom callback. Recevied inputs: hello\n",
+      "---\n",
+      "Inspect the config given to foo: `{'tags': [], 'metadata': {}, 'callbacks': <langchain_core.callbacks.manager.AsyncCallbackManager object at 0x76f8f69eb700>, 'recursion_limit': 25}`\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'world'"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import asyncio\n",
+    "from typing import Any, Dict, List\n",
+    "\n",
+    "from langchain_core.callbacks import AsyncCallbackHandler\n",
+    "from langchain_core.runnables import RunnableLambda\n",
+    "\n",
+    "\n",
+    "class AsyncMyCustomHandler(AsyncCallbackHandler):\n",
+    "    \"\"\"Async callback handler that can be used to handle callbacks from langchain.\"\"\"\n",
+    "\n",
+    "    async def on_chain_start(\n",
+    "        self,\n",
+    "        serialized: Dict[str, Any],\n",
+    "        inputs: Dict[str, Any],\n",
+    "        **kwargs: Any,\n",
+    "    ) -> None:\n",
+    "        \"\"\"Run when chain starts running.\"\"\"\n",
+    "        print(\"---\")\n",
+    "        print(f\"Invoking custom callback. Recevied inputs: {inputs}\")\n",
+    "\n",
+    "\n",
+    "@RunnableLambda\n",
+    "async def foo(inputs, config):\n",
+    "    print(\"---\")\n",
+    "    print(f\"Inspect the config given to foo: `{config}`\")\n",
+    "    return \"world\"\n",
+    "\n",
+    "\n",
+    "@RunnableLambda\n",
+    "async def bar(inputs):\n",
+    "    return await foo.ainvoke(\"goodbye\")\n",
+    "\n",
+    "\n",
+    "await bar.ainvoke(\"hello\", {\"callbacks\": [AsyncMyCustomHandler()]})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Correct\n",
+    "\n",
+    "This code will work correctly across all supported versions of python.\n",
+    "\n",
+    ":::{.callout-tip}\n",
+    "\n",
+    "This code exposes `config` and propagates it to child component as `foo.ainvoke('goodbye', config=config)`\n",
+    ":::"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "---\n",
+      "Invoking custom callback. Recevied inputs: hello\n",
+      "---\n",
+      "Invoking custom callback. Recevied inputs: goodbye\n",
+      "---\n",
+      "Inspect the config given to foo: `{'tags': ['meow'], 'metadata': {}, 'callbacks': <langchain_core.callbacks.manager.AsyncCallbackManager object at 0x76f90c796d00>, 'recursion_limit': 25}`\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'world'"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "@RunnableLambda\n",
+    "async def foo(inputs, config):\n",
+    "    print(\"---\")\n",
+    "    print(f\"Inspect the config given to foo: `{config}`\")\n",
+    "    return \"world\"\n",
+    "\n",
+    "\n",
+    "@RunnableLambda\n",
+    "async def bar(inputs, config):\n",
+    "    return await foo.ainvoke(\"goodbye\", config=config)\n",
+    "\n",
+    "\n",
+    "await bar.ainvoke(\"hello\", {\"callbacks\": [AsyncMyCustomHandler()], \"tags\": [\"meow\"]})"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}