{
 "cells": [
  {
   "cell_type": "raw",
   "id": "366a0e68-fd67-4fe5-a292-5c33733339ea",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 0\n",
    "title: Interface\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a9acd2e",
   "metadata": {},
   "source": [
    "In an effort to make it as easy as possible to create custom chains, we've implemented a [\"Runnable\"](https://api.python.langchain.com/en/latest/schema/langchain.schema.runnable.base.Runnable.html#langchain.schema.runnable.base.Runnable) protocol that most components implement. This is a standard interface with a few different methods, which makes it easy to define custom chains as well as making it possible to invoke them in a standard way. The standard interface exposed includes:\n",
    "\n",
    "- [`stream`](#stream): stream back chunks of the response\n",
    "- [`invoke`](#invoke): call the chain on an input\n",
    "- [`batch`](#batch): call the chain on a list of inputs\n",
    "\n",
    "These also have corresponding async methods:\n",
    "\n",
    "- [`astream`](#async-stream): stream back chunks of the response async\n",
    "- [`ainvoke`](#async-invoke): call the chain on an input async\n",
    "- [`abatch`](#async-batch): call the chain on a list of inputs async\n",
    "- [`astream_log`](#async-stream-intermediate-steps): stream back intermediate steps as they happen, in addition to the final response\n",
    "\n",
    "The type of the input varies by component:\n",
    "\n",
    "| Component | Input Type |\n",
    "| --- | --- |\n",
    "|Prompt|Dictionary|\n",
    "|Retriever|Single string|\n",
    "|LLM, ChatModel| Single string, list of chat messages or a PromptValue|\n",
    "|Tool|Single string, or dictionary, depending on the tool|\n",
    "|OutputParser|The output of an LLM or ChatModel|\n",
    "\n",
    "The output type also varies by component:\n",
    "\n",
    "| Component | Output Type |\n",
    "| --- | --- |\n",
    "| LLM | String |\n",
    "| ChatModel | ChatMessage |\n",
    "| Prompt | PromptValue |\n",
    "| Retriever | List of documents |\n",
    "| Tool | Depends on the tool |\n",
    "| OutputParser | Depends on the parser |\n",
    "\n",
    "All runnables expose properties to inspect the input and output types:\n",
    "- [`input_schema`](#input-schema): an input Pydantic model auto-generated from the structure of the Runnable\n",
    "- [`output_schema`](#output-schema): an output Pydantic model auto-generated from the structure of the Runnable\n",
    "\n",
    "Let's take a look at these methods! To do so, we'll create a super simple PromptTemplate + ChatModel chain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "466b65b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import ChatPromptTemplate\n",
    "from langchain.chat_models import ChatOpenAI\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3c634ef0",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ChatOpenAI()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d1850a1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = ChatPromptTemplate.from_template(\"tell me a joke about {topic}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "56d0669f",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = prompt | model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cccdf0b-2d89-4f74-9530-bf499610e9a5",
   "metadata": {},
   "source": [
    "## Input Schema\n",
    "\n",
    "A description of the inputs accepted by a Runnable.\n",
    "This is a Pydantic model dynamically generated from the structure of any Runnable.\n",
    "You can call `.schema()` on it to obtain a JSONSchema representation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "25e146d4-60da-40a2-9026-b5dfee106a3f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'title': 'PromptInput',\n",
       " 'type': 'object',\n",
       " 'properties': {'topic': {'title': 'Topic', 'type': 'string'}}}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The input schema of the chain is the input schema of its first part, the prompt.\n",
    "chain.input_schema.schema()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5059a5dc-d544-4add-85bd-78a3f2b78b9a",
   "metadata": {},
   "source": [
    "## Output Schema\n",
    "\n",
    "A description of the outputs produced by a Runnable.\n",
    "This is a Pydantic model dynamically generated from the structure of any Runnable.\n",
    "You can call `.schema()` on it to obtain a JSONSchema representation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a0e41fd3-77d8-4911-af6a-d4d3aad5f77b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'title': 'ChatOpenAIOutput',\n",
       " 'anyOf': [{'$ref': '#/definitions/HumanMessageChunk'},\n",
       "  {'$ref': '#/definitions/AIMessageChunk'},\n",
       "  {'$ref': '#/definitions/ChatMessageChunk'},\n",
       "  {'$ref': '#/definitions/FunctionMessageChunk'},\n",
       "  {'$ref': '#/definitions/SystemMessageChunk'}],\n",
       " 'definitions': {'HumanMessageChunk': {'title': 'HumanMessageChunk',\n",
       "   'description': 'A Human Message chunk.',\n",
       "   'type': 'object',\n",
       "   'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
       "    'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
       "    'type': {'title': 'Type',\n",
       "     'default': 'human',\n",
       "     'enum': ['human'],\n",
       "     'type': 'string'},\n",
       "    'example': {'title': 'Example', 'default': False, 'type': 'boolean'},\n",
       "    'is_chunk': {'title': 'Is Chunk',\n",
       "     'default': True,\n",
       "     'enum': [True],\n",
       "     'type': 'boolean'}},\n",
       "   'required': ['content']},\n",
       "  'AIMessageChunk': {'title': 'AIMessageChunk',\n",
       "   'description': 'A Message chunk from an AI.',\n",
       "   'type': 'object',\n",
       "   'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
       "    'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
       "    'type': {'title': 'Type',\n",
       "     'default': 'ai',\n",
       "     'enum': ['ai'],\n",
       "     'type': 'string'},\n",
       "    'example': {'title': 'Example', 'default': False, 'type': 'boolean'},\n",
       "    'is_chunk': {'title': 'Is Chunk',\n",
       "     'default': True,\n",
       "     'enum': [True],\n",
       "     'type': 'boolean'}},\n",
       "   'required': ['content']},\n",
       "  'ChatMessageChunk': {'title': 'ChatMessageChunk',\n",
       "   'description': 'A Chat Message chunk.',\n",
       "   'type': 'object',\n",
       "   'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
       "    'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
       "    'type': {'title': 'Type',\n",
       "     'default': 'chat',\n",
       "     'enum': ['chat'],\n",
       "     'type': 'string'},\n",
       "    'role': {'title': 'Role', 'type': 'string'},\n",
       "    'is_chunk': {'title': 'Is Chunk',\n",
       "     'default': True,\n",
       "     'enum': [True],\n",
       "     'type': 'boolean'}},\n",
       "   'required': ['content', 'role']},\n",
       "  'FunctionMessageChunk': {'title': 'FunctionMessageChunk',\n",
       "   'description': 'A Function Message chunk.',\n",
       "   'type': 'object',\n",
       "   'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
       "    'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
       "    'type': {'title': 'Type',\n",
       "     'default': 'function',\n",
       "     'enum': ['function'],\n",
       "     'type': 'string'},\n",
       "    'name': {'title': 'Name', 'type': 'string'},\n",
       "    'is_chunk': {'title': 'Is Chunk',\n",
       "     'default': True,\n",
       "     'enum': [True],\n",
       "     'type': 'boolean'}},\n",
       "   'required': ['content', 'name']},\n",
       "  'SystemMessageChunk': {'title': 'SystemMessageChunk',\n",
       "   'description': 'A System Message chunk.',\n",
       "   'type': 'object',\n",
       "   'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
       "    'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
       "    'type': {'title': 'Type',\n",
       "     'default': 'system',\n",
       "     'enum': ['system'],\n",
       "     'type': 'string'},\n",
       "    'is_chunk': {'title': 'Is Chunk',\n",
       "     'default': True,\n",
       "     'enum': [True],\n",
       "     'type': 'boolean'}},\n",
       "   'required': ['content']}}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The output schema of the chain is the output schema of its last part, in this case a ChatModel, which outputs a ChatMessage\n",
    "chain.output_schema.schema()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daf2b2b2",
   "metadata": {},
   "source": [
    "## Stream"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "bea9639d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Why don't bears wear shoes? \n",
      "\n",
      "Because they have bear feet!"
     ]
    }
   ],
   "source": [
    "for s in chain.stream({\"topic\": \"bears\"}):\n",
    "    print(s.content, end=\"\", flush=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbf1c782",
   "metadata": {},
   "source": [
    "## Invoke"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "470e483f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\")"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.invoke({\"topic\": \"bears\"})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88f0c279",
   "metadata": {},
   "source": [
    "## Batch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9685de67",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\"),\n",
       " AIMessage(content=\"Why don't cats play poker in the wild?\\n\\nToo many cheetahs!\")]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.batch([{\"topic\": \"bears\"}, {\"topic\": \"cats\"}])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2434ab15",
   "metadata": {},
   "source": [
    "You can set the number of concurrent requests by using the `max_concurrency` parameter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a08522f6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\"),\n",
       " AIMessage(content=\"Sure, here's a cat joke for you:\\n\\nWhy don't cats play poker in the wild?\\n\\nToo many cheetahs!\")]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.batch([{\"topic\": \"bears\"}, {\"topic\": \"cats\"}], config={\"max_concurrency\": 5})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b960cbfe",
   "metadata": {},
   "source": [
    "## Async Stream"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ea35eee4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sure, here's a bear joke for you:\n",
      "\n",
      "Why don't bears wear shoes?\n",
      "\n",
      "Because they have bear feet!"
     ]
    }
   ],
   "source": [
    "async for s in chain.astream({\"topic\": \"bears\"}):\n",
    "    print(s.content, end=\"\", flush=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04cb3324",
   "metadata": {},
   "source": [
    "## Async Invoke"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ef8c9b20",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\"Why don't bears wear shoes? \\n\\nBecause they have bear feet!\")"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "await chain.ainvoke({\"topic\": \"bears\"})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3da288d5",
   "metadata": {},
   "source": [
    "## Async Batch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "eba2a103",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\")]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "await chain.abatch([{\"topic\": \"bears\"}])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9cef104",
   "metadata": {},
   "source": [
    "## Async Stream Intermediate Steps\n",
    "\n",
    "All runnables also have a method `.astream_log()` which can be used to stream (as they happen) all or part of the intermediate steps of your chain/sequence. \n",
    "\n",
    "This is useful eg. to show progress to the user, to use intermediate results, or even just to debug your chain.\n",
    "\n",
    "You can choose to stream all steps (default), or include/exclude steps by name, tags or metadata.\n",
    "\n",
    "This method yields [JSONPatch](https://jsonpatch.com) ops that when applied in the same order as received build up the RunState.\n",
    "\n",
    "```python\n",
    "class LogEntry(TypedDict):\n",
    "    id: str\n",
    "    \"\"\"ID of the sub-run.\"\"\"\n",
    "    name: str\n",
    "    \"\"\"Name of the object being run.\"\"\"\n",
    "    type: str\n",
    "    \"\"\"Type of the object being run, eg. prompt, chain, llm, etc.\"\"\"\n",
    "    tags: List[str]\n",
    "    \"\"\"List of tags for the run.\"\"\"\n",
    "    metadata: Dict[str, Any]\n",
    "    \"\"\"Key-value pairs of metadata for the run.\"\"\"\n",
    "    start_time: str\n",
    "    \"\"\"ISO-8601 timestamp of when the run started.\"\"\"\n",
    "\n",
    "    streamed_output_str: List[str]\n",
    "    \"\"\"List of LLM tokens streamed by this run, if applicable.\"\"\"\n",
    "    final_output: Optional[Any]\n",
    "    \"\"\"Final output of this run.\n",
    "    Only available after the run has finished successfully.\"\"\"\n",
    "    end_time: Optional[str]\n",
    "    \"\"\"ISO-8601 timestamp of when the run ended.\n",
    "    Only available after the run has finished.\"\"\"\n",
    "\n",
    "\n",
    "class RunState(TypedDict):\n",
    "    id: str\n",
    "    \"\"\"ID of the run.\"\"\"\n",
    "    streamed_output: List[Any]\n",
    "    \"\"\"List of output chunks streamed by Runnable.stream()\"\"\"\n",
    "    final_output: Optional[Any]\n",
    "    \"\"\"Final output of the run, usually the result of aggregating (`+`) streamed_output.\n",
    "    Only available after the run has finished successfully.\"\"\"\n",
    "\n",
    "    logs: Dict[str, LogEntry]\n",
    "    \"\"\"Map of run names to sub-runs. If filters were supplied, this list will\n",
    "    contain only the runs that matched the filters.\"\"\"\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a146a5df-25be-4fa2-a7e4-df8ebe55a35e",
   "metadata": {},
   "source": [
    "### Streaming JSONPatch chunks\n",
    "\n",
    "This is useful eg. to stream the JSONPatch in an HTTP server, and then apply the ops on the client to rebuild the run state there. See [LangServe](https://github.com/langchain-ai/langserve) for tooling to make it easier to build a webserver from any Runnable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "21c9019e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RunLogPatch({'op': 'replace',\n",
      "  'path': '',\n",
      "  'value': {'final_output': None,\n",
      "            'id': 'fd6fcf62-c92c-4edf-8713-0fc5df000f62',\n",
      "            'logs': {},\n",
      "            'streamed_output': []}})\n",
      "RunLogPatch({'op': 'add',\n",
      "  'path': '/logs/Docs',\n",
      "  'value': {'end_time': None,\n",
      "            'final_output': None,\n",
      "            'id': '8c998257-1ec8-4546-b744-c3fdb9728c41',\n",
      "            'metadata': {},\n",
      "            'name': 'Docs',\n",
      "            'start_time': '2023-10-05T12:52:35.668',\n",
      "            'streamed_output_str': [],\n",
      "            'tags': ['map:key:context', 'FAISS'],\n",
      "            'type': 'retriever'}})\n",
      "RunLogPatch({'op': 'add',\n",
      "  'path': '/logs/Docs/final_output',\n",
      "  'value': {'documents': [Document(page_content='harrison worked at kensho')]}},\n",
      " {'op': 'add',\n",
      "  'path': '/logs/Docs/end_time',\n",
      "  'value': '2023-10-05T12:52:36.033'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ''})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': 'H'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': 'arrison'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ' worked'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ' at'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ' Kens'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': 'ho'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': '.'})\n",
      "RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ''})\n",
      "RunLogPatch({'op': 'replace',\n",
      "  'path': '/final_output',\n",
      "  'value': {'output': 'Harrison worked at Kensho.'}})\n"
     ]
    }
   ],
   "source": [
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.schema.output_parser import StrOutputParser\n",
    "from langchain.schema.runnable import RunnablePassthrough\n",
    "from langchain.vectorstores import FAISS\n",
    "\n",
    "template = \"\"\"Answer the question based only on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)\n",
    "\n",
    "vectorstore = FAISS.from_texts([\"harrison worked at kensho\"], embedding=OpenAIEmbeddings())\n",
    "retriever = vectorstore.as_retriever()\n",
    "\n",
    "retrieval_chain = (\n",
    "    {\"context\": retriever.with_config(run_name='Docs'), \"question\": RunnablePassthrough()}\n",
    "    | prompt \n",
    "    | model \n",
    "    | StrOutputParser()\n",
    ")\n",
    "\n",
    "async for chunk in retrieval_chain.astream_log(\"where did harrison work?\", include_names=['Docs']):\n",
    "    print(chunk)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19570f36-7126-4fe2-b209-0cc6178b4582",
   "metadata": {},
   "source": [
    "### Streaming the incremental RunState\n",
    "\n",
    "You can simply pass diff=False to get incremental values of RunState."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "5c26b731-b4eb-4967-a42a-dec813249ecb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {},\n",
      " 'streamed_output': []})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': None,\n",
      "                   'final_output': None,\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': []})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': []})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['', 'H']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['', 'H', 'arrison']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['', 'H', 'arrison', ' worked']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['', 'H', 'arrison', ' worked', ' at']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['', 'H', 'arrison', ' worked', ' at', ' Kens']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['', 'H', 'arrison', ' worked', ' at', ' Kens', 'ho']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['', 'H', 'arrison', ' worked', ' at', ' Kens', 'ho', '.']})\n",
      "RunLog({'final_output': None,\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['',\n",
      "                     'H',\n",
      "                     'arrison',\n",
      "                     ' worked',\n",
      "                     ' at',\n",
      "                     ' Kens',\n",
      "                     'ho',\n",
      "                     '.',\n",
      "                     '']})\n",
      "RunLog({'final_output': {'output': 'Harrison worked at Kensho.'},\n",
      " 'id': 'f95ccb87-31f1-48ea-a51c-d2dadde44185',\n",
      " 'logs': {'Docs': {'end_time': '2023-10-05T12:52:37.217',\n",
      "                   'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
      "                   'id': '621597dd-d716-4532-938d-debc21a453d1',\n",
      "                   'metadata': {},\n",
      "                   'name': 'Docs',\n",
      "                   'start_time': '2023-10-05T12:52:36.935',\n",
      "                   'streamed_output_str': [],\n",
      "                   'tags': ['map:key:context', 'FAISS'],\n",
      "                   'type': 'retriever'}},\n",
      " 'streamed_output': ['',\n",
      "                     'H',\n",
      "                     'arrison',\n",
      "                     ' worked',\n",
      "                     ' at',\n",
      "                     ' Kens',\n",
      "                     'ho',\n",
      "                     '.',\n",
      "                     '']})\n"
     ]
    }
   ],
   "source": [
    "async for chunk in retrieval_chain.astream_log(\"where did harrison work?\", include_names=['Docs'], diff=False):\n",
    "    print(chunk)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7006f1aa",
   "metadata": {},
   "source": [
    "## Parallelism\n",
    "\n",
    "Let's take a look at how LangChain Expression Language support parallel requests as much as possible. For example, when using a RunnableParallel (often written as a dictionary) it executes each element in parallel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0a1c409d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.schema.runnable import RunnableParallel\n",
    "chain1 = ChatPromptTemplate.from_template(\"tell me a joke about {topic}\") | model\n",
    "chain2 = ChatPromptTemplate.from_template(\"write a short (2 line) poem about {topic}\") | model\n",
    "combined = RunnableParallel(joke=chain1, poem=chain2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "08044c0a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 31.7 ms, sys: 8.59 ms, total: 40.3 ms\n",
      "Wall time: 1.05 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\"Why don't bears like fast food?\\n\\nBecause they can't catch it!\", additional_kwargs={}, example=False)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "chain1.invoke({\"topic\": \"bears\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "22c56804",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 42.9 ms, sys: 10.2 ms, total: 53 ms\n",
      "Wall time: 1.93 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\"In forest's embrace, bears roam free,\\nSilent strength, nature's majesty.\", additional_kwargs={}, example=False)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "chain2.invoke({\"topic\": \"bears\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4fff4cbb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 96.3 ms, sys: 20.4 ms, total: 117 ms\n",
      "Wall time: 1.1 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'joke': AIMessage(content=\"Why don't bears wear socks?\\n\\nBecause they have bear feet!\", additional_kwargs={}, example=False),\n",
       " 'poem': AIMessage(content=\"In forest's embrace,\\nMajestic bears leave their trace.\", additional_kwargs={}, example=False)}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "combined.invoke({\"topic\": \"bears\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fab75d1d",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
