mirror of
https://github.com/hwchase17/langchain.git
synced 2025-10-22 17:50:03 +00:00
1152 lines
66 KiB
Plaintext
1152 lines
66 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"id": "473081cc",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"sidebar_position: 1\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "16ee4216",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Streaming\n",
|
|
"\n",
|
|
"Streaming is an important UX consideration for LLM apps, and agents are no exception. Streaming with agents is made more complicated by the fact that it's not just tokens of the final answer that you will want to stream, but you may also want to stream back the intermediate steps an agent takes.\n",
|
|
"\n",
|
|
"In this notebook, we'll cover the `stream/astream` and `astream_events` for streaming.\n",
|
|
"\n",
|
|
"Our agent will use a tools API for tool invocation with the tools:\n",
|
|
"\n",
|
|
"1. `where_cat_is_hiding`: Returns a location where the cat is hiding\n",
|
|
"2. `get_items`: Lists items that can be found in a particular place\n",
|
|
"\n",
|
|
"These tools will allow us to explore streaming in a more interesting situation where the agent will have to use both tools to answer some questions (e.g., to answer the question `what items are located where the cat is hiding?`).\n",
|
|
"\n",
|
|
"Ready?🏎️"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "d40aae3d-b872-4e0f-ad54-8df6150fa863",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain import hub\n",
|
|
"from langchain.agents import AgentExecutor, create_openai_tools_agent\n",
|
|
"from langchain.prompts import ChatPromptTemplate\n",
|
|
"from langchain.tools import tool\n",
|
|
"from langchain_core.callbacks import Callbacks\n",
|
|
"from langchain_openai import ChatOpenAI"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "59502ed8-2f9f-4758-a0d5-90a0392ed33d",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Create the model\n",
|
|
"\n",
|
|
"**Attention** We're setting `streaming=True` on the LLM. This will allow us to stream tokens from the agent using the `astream_events` API. This is needed for older versions of LangChain."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "66e36d43-2c12-4cda-b591-383eb61b4f69",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"model = ChatOpenAI(temperature=0, streaming=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7ec9c5e5-34d4-4208-9f78-7f9a1ff3029b",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Tools\n",
|
|
"\n",
|
|
"We define two tools that rely on a chat model to generate output!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"id": "cd29a18c-e11c-4fbe-9fb8-b64dc9be95fd",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import random\n",
|
|
"\n",
|
|
"\n",
|
|
"@tool\n",
|
|
"async def where_cat_is_hiding() -> str:\n",
|
|
" \"\"\"Where is the cat hiding right now?\"\"\"\n",
|
|
" return random.choice([\"under the bed\", \"on the shelf\"])\n",
|
|
"\n",
|
|
"\n",
|
|
"@tool\n",
|
|
"async def get_items(place: str) -> str:\n",
|
|
" \"\"\"Use this tool to look up which items are in the given place.\"\"\"\n",
|
|
" if \"bed\" in place: # For under the bed\n",
|
|
" return \"socks, shoes and dust bunnies\"\n",
|
|
" if \"shelf\" in place: # For 'shelf'\n",
|
|
" return \"books, penciles and pictures\"\n",
|
|
" else: # if the agent decides to ask about a different place\n",
|
|
" return \"cat snacks\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "1257a508-c791-4d81-82d2-df021c560bec",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"'on the shelf'"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"await where_cat_is_hiding.ainvoke({})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "eea408ee-5260-418c-b769-5ba20e2999e1",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"'books, penciles and pictures'"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"await get_items.ainvoke({\"place\": \"shelf\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "07c08cd5-34eb-41a7-b524-7c3d1d274a67",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Initialize the agent\n",
|
|
"\n",
|
|
"Here, we'll initialize an OpenAI tools agent.\n",
|
|
"\n",
|
|
"**ATTENTION** Please note that we associated the name `Agent` with our agent using `\"run_name\"=\"Agent\"`. We'll use that fact later on with the `astream_events` API."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "adecca7a-9864-496d-a3a9-906b56ecd03b",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Get the prompt to use - you can modify this!\n",
|
|
"prompt = hub.pull(\"hwchase17/openai-tools-agent\")\n",
|
|
"# print(prompt.messages) -- to see the prompt\n",
|
|
"tools = [get_items, where_cat_is_hiding]\n",
|
|
"agent = create_openai_tools_agent(\n",
|
|
" model.with_config({\"tags\": [\"agent_llm\"]}), tools, prompt\n",
|
|
")\n",
|
|
"agent_executor = AgentExecutor(agent=agent, tools=tools).with_config(\n",
|
|
" {\"run_name\": \"Agent\"}\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cba9a9eb",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Stream Intermediate Steps\n",
|
|
"\n",
|
|
"We'll use `.stream` method of the AgentExecutor to stream the agent's intermediate steps.\n",
|
|
"\n",
|
|
"The output from `.stream` alternates between (action, observation) pairs, finally concluding with the answer if the agent achieved its objective. \n",
|
|
"\n",
|
|
"It'll look like this:\n",
|
|
"\n",
|
|
"1. actions output\n",
|
|
"2. observations output\n",
|
|
"3. actions output\n",
|
|
"4. observations output\n",
|
|
"\n",
|
|
"**... (continue until goal is reached) ...**\n",
|
|
"\n",
|
|
"Then, if the final goal is reached, the agent will output the **final answer**.\n",
|
|
"\n",
|
|
"\n",
|
|
"The contents of these outputs are summarized here:\n",
|
|
"\n",
|
|
"| Output | Contents |\n",
|
|
"|----------------------|------------------------------------------------------------------------------------------------------|\n",
|
|
"| **Actions** | `actions` `AgentAction` or a subclass, `messages` chat messages corresponding to action invocation |\n",
|
|
"| **Observations** | `steps` History of what the agent did so far, including the current action and its observation, `messages` chat message with function invocation results (aka observations)|\n",
|
|
"| **Final answer** | `output` `AgentFinish`, `messages` chat messages with the final output|"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"id": "eab4d4a0-55ed-407a-baf0-9f0eaf8c3518",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"------\n",
|
|
"{'actions': [...], 'messages': [...]}\n",
|
|
"------\n",
|
|
"{'messages': [...], 'steps': [...]}\n",
|
|
"------\n",
|
|
"{'actions': [...], 'messages': [...]}\n",
|
|
"------\n",
|
|
"{'messages': [...], 'steps': [...]}\n",
|
|
"------\n",
|
|
"{'messages': [...],\n",
|
|
" 'output': 'The items located where the cat is hiding on the shelf are books, '\n",
|
|
" 'pencils, and pictures.'}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Note: We use `pprint` to print only to depth 1, it makes it easier to see the output from a high level, before digging in.\n",
|
|
"import pprint\n",
|
|
"\n",
|
|
"chunks = []\n",
|
|
"\n",
|
|
"async for chunk in agent_executor.astream(\n",
|
|
" {\"input\": \"what's items are located where the cat is hiding?\"}\n",
|
|
"):\n",
|
|
" chunks.append(chunk)\n",
|
|
" print(\"------\")\n",
|
|
" pprint.pprint(chunk, depth=1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "76a930c7-7c6f-4602-b265-d38018f067be",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Using Messages\n",
|
|
"\n",
|
|
"You can access the underlying `messages` from the outputs. Using messages can be nice when working with chat applications - because everything is a message!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "4d5a3112-b2d4-488a-ac76-aa40dcec9cfe",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pKy4OLcBx6pR6k3GHBOlH68r', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_pKy4OLcBx6pR6k3GHBOlH68r')]"
|
|
]
|
|
},
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"chunks[0][\"actions\"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "2f5eead3-f6f0-40b7-82c7-3b485c634e94",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pKy4OLcBx6pR6k3GHBOlH68r', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})]\n",
|
|
"[FunctionMessage(content='on the shelf', name='where_cat_is_hiding')]\n",
|
|
"[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_qZTz1mRfCCXT18SUy0E07eS4', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]})]\n",
|
|
"[FunctionMessage(content='books, penciles and pictures', name='get_items')]\n",
|
|
"[AIMessage(content='The items located where the cat is hiding on the shelf are books, pencils, and pictures.')]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"for chunk in chunks:\n",
|
|
" print(chunk[\"messages\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1397f859-8595-488e-9857-c4e090a136d3",
|
|
"metadata": {},
|
|
"source": [
|
|
"In addition, they contain full logging information (`actions` and `steps`) which may be easier to process for rendering purposes."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "edd291a7",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Using AgentAction/Observation\n",
|
|
"\n",
|
|
"The outputs also contain richer structured information inside of `actions` and `steps`, which could be useful in some situations, but can also be harder to parse.\n",
|
|
"\n",
|
|
"**Attention** `AgentFinish` is not available as part of the `streaming` method. If this is something you'd like to be added, please start a discussion on github and explain why its needed."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"id": "603bff1d",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Calling Tool: `where_cat_is_hiding` with input `{}`\n",
|
|
"---\n",
|
|
"Tool Result: `on the shelf`\n",
|
|
"---\n",
|
|
"Calling Tool: `get_items` with input `{'place': 'shelf'}`\n",
|
|
"---\n",
|
|
"Tool Result: `books, penciles and pictures`\n",
|
|
"---\n",
|
|
"Final Output: The items located where the cat is hiding on the shelf are books, pencils, and pictures.\n",
|
|
"---\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"async for chunk in agent_executor.astream(\n",
|
|
" {\"input\": \"what's items are located where the cat is hiding?\"}\n",
|
|
"):\n",
|
|
" # Agent Action\n",
|
|
" if \"actions\" in chunk:\n",
|
|
" for action in chunk[\"actions\"]:\n",
|
|
" print(f\"Calling Tool: `{action.tool}` with input `{action.tool_input}`\")\n",
|
|
" # Observation\n",
|
|
" elif \"steps\" in chunk:\n",
|
|
" for step in chunk[\"steps\"]:\n",
|
|
" print(f\"Tool Result: `{step.observation}`\")\n",
|
|
" # Final result\n",
|
|
" elif \"output\" in chunk:\n",
|
|
" print(f'Final Output: {chunk[\"output\"]}')\n",
|
|
" else:\n",
|
|
" raise ValueError()\n",
|
|
" print(\"---\")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "5058f098-d8b5-4500-bd99-b972af3ecc09",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Custom Streaming With Events\n",
|
|
"\n",
|
|
"Use the `astream_events` API in case the default behavior of *stream* does not work for your application (e.g., if you need to stream individual tokens from the agent or surface steps occurring **within** tools).\n",
|
|
"\n",
|
|
"⚠️ This is a **beta** API, meaning that some details might change slightly in the future based on usage.\n",
|
|
"⚠️ To make sure all callbacks work properly, use `async` code throughout. Try avoiding mixing in sync versions of code (e.g., sync versions of tools).\n",
|
|
"\n",
|
|
"Let's use this API to stream the following events:\n",
|
|
"\n",
|
|
"1. Agent Start with inputs\n",
|
|
"1. Tool Start with inputs\n",
|
|
"1. Tool End with outputs\n",
|
|
"1. Stream the agent final anwer token by token\n",
|
|
"1. Agent End with outputs"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"id": "46c59cac-25fa-4f42-8cf2-9bcaed6d92c4",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Starting agent: Agent with input: {'input': 'where is the cat hiding? what items are in that location?'}\n",
|
|
"--\n",
|
|
"Starting tool: where_cat_is_hiding with inputs: {}\n",
|
|
"Done tool: where_cat_is_hiding\n",
|
|
"Tool output was: on the shelf\n",
|
|
"--\n",
|
|
"--\n",
|
|
"Starting tool: get_items with inputs: {'place': 'shelf'}\n",
|
|
"Done tool: get_items\n",
|
|
"Tool output was: books, penciles and pictures\n",
|
|
"--\n",
|
|
"The| cat| is| currently| hiding| on| the| shelf|.| In| that| location|,| you| can| find| books|,| pencils|,| and| pictures|.|\n",
|
|
"--\n",
|
|
"Done agent: Agent with output: The cat is currently hiding on the shelf. In that location, you can find books, pencils, and pictures.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"async for event in agent_executor.astream_events(\n",
|
|
" {\"input\": \"where is the cat hiding? what items are in that location?\"},\n",
|
|
" version=\"v1\",\n",
|
|
"):\n",
|
|
" kind = event[\"event\"]\n",
|
|
" if kind == \"on_chain_start\":\n",
|
|
" if (\n",
|
|
" event[\"name\"] == \"Agent\"\n",
|
|
" ): # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
|
|
" print(\n",
|
|
" f\"Starting agent: {event['name']} with input: {event['data'].get('input')}\"\n",
|
|
" )\n",
|
|
" elif kind == \"on_chain_end\":\n",
|
|
" if (\n",
|
|
" event[\"name\"] == \"Agent\"\n",
|
|
" ): # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
|
|
" print()\n",
|
|
" print(\"--\")\n",
|
|
" print(\n",
|
|
" f\"Done agent: {event['name']} with output: {event['data'].get('output')['output']}\"\n",
|
|
" )\n",
|
|
" if kind == \"on_chat_model_stream\":\n",
|
|
" content = event[\"data\"][\"chunk\"].content\n",
|
|
" if content:\n",
|
|
" # Empty content in the context of OpenAI means\n",
|
|
" # that the model is asking for a tool to be invoked.\n",
|
|
" # So we only print non-empty content\n",
|
|
" print(content, end=\"|\")\n",
|
|
" elif kind == \"on_tool_start\":\n",
|
|
" print(\"--\")\n",
|
|
" print(\n",
|
|
" f\"Starting tool: {event['name']} with inputs: {event['data'].get('input')}\"\n",
|
|
" )\n",
|
|
" elif kind == \"on_tool_end\":\n",
|
|
" print(f\"Done tool: {event['name']}\")\n",
|
|
" print(f\"Tool output was: {event['data'].get('output')}\")\n",
|
|
" print(\"--\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "09711ba8-f60e-4a5d-9ace-1bdc613a7c44",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Stream Events from within Tools\n",
|
|
"\n",
|
|
"If your tool leverages LangChain runnable objects (e.g., LCEL chains, LLMs, retrievers etc.) and you want to stream events from those objects as well, you'll need to make sure that callbacks are propagated correctly.\n",
|
|
"\n",
|
|
"To see how to pass callbacks, let's re-implement the `get_items` tool to make it use an LLM and pass callbacks to that LLM. Feel free to adapt this to your use case."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"id": "fdd005f4-31d3-450f-b16b-b614c26a72f3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"@tool\n",
|
|
"async def get_items(place: str, callbacks: Callbacks) -> str: # <--- Accept callbacks\n",
|
|
" \"\"\"Use this tool to look up which items are in the given place.\"\"\"\n",
|
|
" template = ChatPromptTemplate.from_messages(\n",
|
|
" [\n",
|
|
" (\n",
|
|
" \"human\",\n",
|
|
" \"Can you tell me what kind of items i might find in the following place: '{place}'. \"\n",
|
|
" \"List at least 3 such items separating them by a comma. And include a brief description of each item..\",\n",
|
|
" )\n",
|
|
" ]\n",
|
|
" )\n",
|
|
" chain = template | model.with_config(\n",
|
|
" {\n",
|
|
" \"run_name\": \"Get Items LLM\",\n",
|
|
" \"tags\": [\"tool_llm\"],\n",
|
|
" \"callbacks\": callbacks, # <-- Propagate callbacks\n",
|
|
" }\n",
|
|
" )\n",
|
|
" chunks = [chunk async for chunk in chain.astream({\"place\": place})]\n",
|
|
" return \"\".join(chunk.content for chunk in chunks)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "66828308-538f-4a06-8ed6-bf398d7a3d56",
|
|
"metadata": {},
|
|
"source": [
|
|
"^ Take a look at how the tool propagates callbacks. \n",
|
|
"\n",
|
|
"Next, let's initialize our agent, and take a look at the new output."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "095df835-ab27-4791-80e9-07cdba180822",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Starting agent: Agent with input: {'input': 'where is the cat hiding? what items are in that location?'}\n",
|
|
"--\n",
|
|
"Starting tool: where_cat_is_hiding with inputs: {}\n",
|
|
"Done tool: where_cat_is_hiding\n",
|
|
"Tool output was: on the shelf\n",
|
|
"--\n",
|
|
"--\n",
|
|
"Starting tool: get_items with inputs: {'place': 'shelf'}\n",
|
|
"In| a| shelf|,| you| might| find|:\n",
|
|
"\n",
|
|
"|1|.| Books|:| A| shelf| is| commonly| used| to| store| books|.| It| may| contain| various| genres| such| as| novels|,| textbooks|,| or| reference| books|.| Books| provide| knowledge|,| entertainment|,| and| can| transport| you| to| different| worlds| through| storytelling|.\n",
|
|
"\n",
|
|
"|2|.| Decor|ative| items|:| Sh|elves| often| display| decorative| items| like| figur|ines|,| v|ases|,| or| photo| frames|.| These| items| add| a| personal| touch| to| the| space| and| can| reflect| the| owner|'s| interests| or| memories|.\n",
|
|
"\n",
|
|
"|3|.| Storage| boxes|:| Sh|elves| can| also| hold| storage| boxes| or| baskets|.| These| containers| help| organize| and| decl|utter| the| space| by| storing| miscellaneous| items| like| documents|,| accessories|,| or| small| household| items|.| They| provide| a| neat| and| tidy| appearance| to| the| shelf|.|Done tool: get_items\n",
|
|
"Tool output was: In a shelf, you might find:\n",
|
|
"\n",
|
|
"1. Books: A shelf is commonly used to store books. It may contain various genres such as novels, textbooks, or reference books. Books provide knowledge, entertainment, and can transport you to different worlds through storytelling.\n",
|
|
"\n",
|
|
"2. Decorative items: Shelves often display decorative items like figurines, vases, or photo frames. These items add a personal touch to the space and can reflect the owner's interests or memories.\n",
|
|
"\n",
|
|
"3. Storage boxes: Shelves can also hold storage boxes or baskets. These containers help organize and declutter the space by storing miscellaneous items like documents, accessories, or small household items. They provide a neat and tidy appearance to the shelf.\n",
|
|
"--\n",
|
|
"The| cat| is| hiding| on| the| shelf|.| In| that| location|,| you| might| find| books|,| decorative| items|,| and| storage| boxes|.|\n",
|
|
"--\n",
|
|
"Done agent: Agent with output: The cat is hiding on the shelf. In that location, you might find books, decorative items, and storage boxes.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Get the prompt to use - you can modify this!\n",
|
|
"prompt = hub.pull(\"hwchase17/openai-tools-agent\")\n",
|
|
"# print(prompt.messages) -- to see the prompt\n",
|
|
"tools = [get_items, where_cat_is_hiding]\n",
|
|
"agent = create_openai_tools_agent(\n",
|
|
" model.with_config({\"tags\": [\"agent_llm\"]}), tools, prompt\n",
|
|
")\n",
|
|
"agent_executor = AgentExecutor(agent=agent, tools=tools).with_config(\n",
|
|
" {\"run_name\": \"Agent\"}\n",
|
|
")\n",
|
|
"\n",
|
|
"async for event in agent_executor.astream_events(\n",
|
|
" {\"input\": \"where is the cat hiding? what items are in that location?\"},\n",
|
|
" version=\"v1\",\n",
|
|
"):\n",
|
|
" kind = event[\"event\"]\n",
|
|
" if kind == \"on_chain_start\":\n",
|
|
" if (\n",
|
|
" event[\"name\"] == \"Agent\"\n",
|
|
" ): # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
|
|
" print(\n",
|
|
" f\"Starting agent: {event['name']} with input: {event['data'].get('input')}\"\n",
|
|
" )\n",
|
|
" elif kind == \"on_chain_end\":\n",
|
|
" if (\n",
|
|
" event[\"name\"] == \"Agent\"\n",
|
|
" ): # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
|
|
" print()\n",
|
|
" print(\"--\")\n",
|
|
" print(\n",
|
|
" f\"Done agent: {event['name']} with output: {event['data'].get('output')['output']}\"\n",
|
|
" )\n",
|
|
" if kind == \"on_chat_model_stream\":\n",
|
|
" content = event[\"data\"][\"chunk\"].content\n",
|
|
" if content:\n",
|
|
" # Empty content in the context of OpenAI means\n",
|
|
" # that the model is asking for a tool to be invoked.\n",
|
|
" # So we only print non-empty content\n",
|
|
" print(content, end=\"|\")\n",
|
|
" elif kind == \"on_tool_start\":\n",
|
|
" print(\"--\")\n",
|
|
" print(\n",
|
|
" f\"Starting tool: {event['name']} with inputs: {event['data'].get('input')}\"\n",
|
|
" )\n",
|
|
" elif kind == \"on_tool_end\":\n",
|
|
" print(f\"Done tool: {event['name']}\")\n",
|
|
" print(f\"Tool output was: {event['data'].get('output')}\")\n",
|
|
" print(\"--\")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "24386754-5cd6-4322-82f7-affb93322bad",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Other aproaches\n",
|
|
"\n",
|
|
"#### Using astream_log\n",
|
|
"\n",
|
|
"**Note** You can also use the [astream_log](/docs/expression_language/interface#async-stream-intermediate-steps) API. This API produces a granular log of all events that occur during execution. The log format is based on the [JSONPatch](https://jsonpatch.com/) standard. It's granular, but requires effort to parse. For this reason, we created the `astream_events` API instead."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"id": "01ad657f-7759-4fb3-a7ca-e2d7e7f8b28f",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"RunLogPatch({'op': 'replace',\n",
|
|
" 'path': '',\n",
|
|
" 'value': {'final_output': None,\n",
|
|
" 'id': 'c261bc30-60d1-4420-9c66-c6c0797f2c2d',\n",
|
|
" 'logs': {},\n",
|
|
" 'name': 'Agent',\n",
|
|
" 'streamed_output': [],\n",
|
|
" 'type': 'chain'}})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableSequence',\n",
|
|
" 'value': {'end_time': None,\n",
|
|
" 'final_output': None,\n",
|
|
" 'id': '183cb6f8-ed29-4967-b1ea-024050ce66c7',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'RunnableSequence',\n",
|
|
" 'start_time': '2024-01-22T20:38:43.650+00:00',\n",
|
|
" 'streamed_output': [],\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': [],\n",
|
|
" 'type': 'chain'}})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableAssign<agent_scratchpad>',\n",
|
|
" 'value': {'end_time': None,\n",
|
|
" 'final_output': None,\n",
|
|
" 'id': '7fe1bb27-3daf-492e-bc7e-28602398f008',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'RunnableAssign<agent_scratchpad>',\n",
|
|
" 'start_time': '2024-01-22T20:38:43.652+00:00',\n",
|
|
" 'streamed_output': [],\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['seq:step:1'],\n",
|
|
" 'type': 'chain'}})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableAssign<agent_scratchpad>/streamed_output/-',\n",
|
|
" 'value': {'input': 'where is the cat hiding? what items are in that '\n",
|
|
" 'location?',\n",
|
|
" 'intermediate_steps': []}})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableParallel<agent_scratchpad>',\n",
|
|
" 'value': {'end_time': None,\n",
|
|
" 'final_output': None,\n",
|
|
" 'id': 'b034e867-e6bb-4296-bfe6-752c44fba6ce',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'RunnableParallel<agent_scratchpad>',\n",
|
|
" 'start_time': '2024-01-22T20:38:43.652+00:00',\n",
|
|
" 'streamed_output': [],\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': [],\n",
|
|
" 'type': 'chain'}})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableLambda',\n",
|
|
" 'value': {'end_time': None,\n",
|
|
" 'final_output': None,\n",
|
|
" 'id': '65ceef3e-7a80-4015-8b5b-d949326872e9',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'RunnableLambda',\n",
|
|
" 'start_time': '2024-01-22T20:38:43.653+00:00',\n",
|
|
" 'streamed_output': [],\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:agent_scratchpad'],\n",
|
|
" 'type': 'chain'}})\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/logs/RunnableLambda/streamed_output/-', 'value': []})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableParallel<agent_scratchpad>/streamed_output/-',\n",
|
|
" 'value': {'agent_scratchpad': []}})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableAssign<agent_scratchpad>/streamed_output/-',\n",
|
|
" 'value': {'agent_scratchpad': []}})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableLambda/final_output',\n",
|
|
" 'value': {'output': []}},\n",
|
|
" {'op': 'add',\n",
|
|
" 'path': '/logs/RunnableLambda/end_time',\n",
|
|
" 'value': '2024-01-22T20:38:43.654+00:00'})\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/RunnableParallel<agent_scratchpad>/final_output',\n",
|
|
" 'value': {'agent_scratchpad': []}},\n",
|
|
" {'op': 'add',\n",
|
|
" 'path': '/logs/RunnableParallel<agent_scratchpad>/end_time',\n",
|
|
" 'value': '2024-01-22T20:38:43.655+00:00'})\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"i = 0\n",
|
|
"async for chunk in agent_executor.astream_log(\n",
|
|
" {\"input\": \"where is the cat hiding? what items are in that location?\"},\n",
|
|
"):\n",
|
|
" print(chunk)\n",
|
|
" i += 1\n",
|
|
" if i > 10:\n",
|
|
" break"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5763c64b-7fff-4167-9eb3-172209cef958",
|
|
"metadata": {},
|
|
"source": [
|
|
"This may require some logic to get in a workable format"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"id": "f7120cbd-6bea-4706-821a-ff3b6722bf1d",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"None\n",
|
|
"----\n",
|
|
"/logs/RunnableSequence\n",
|
|
"{'id': '22bbd5db-9578-4e3f-a6ec-9b61f08cb8a9', 'name': 'RunnableSequence', 'type': 'chain', 'tags': [], 'metadata': {}, 'start_time': '2024-01-22T20:38:43.668+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/RunnableAssign<agent_scratchpad>\n",
|
|
"{'id': 'e0c00ae2-aaa2-4a09-bc93-cb34bf3f6554', 'name': 'RunnableAssign<agent_scratchpad>', 'type': 'chain', 'tags': ['seq:step:1'], 'metadata': {}, 'start_time': '2024-01-22T20:38:43.672+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/RunnableAssign<agent_scratchpad>/streamed_output/-\n",
|
|
"{'input': 'where is the cat hiding? what items are in that location?', 'intermediate_steps': []}\n",
|
|
"----\n",
|
|
"/logs/RunnableParallel<agent_scratchpad>\n",
|
|
"{'id': '26ff576d-ff9d-4dea-98b2-943312a37f4d', 'name': 'RunnableParallel<agent_scratchpad>', 'type': 'chain', 'tags': [], 'metadata': {}, 'start_time': '2024-01-22T20:38:43.674+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/RunnableLambda\n",
|
|
"{'id': '9f343c6a-23f7-4a28-832f-d4fe3e95d1dc', 'name': 'RunnableLambda', 'type': 'chain', 'tags': ['map:key:agent_scratchpad'], 'metadata': {}, 'start_time': '2024-01-22T20:38:43.685+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/RunnableLambda/streamed_output/-\n",
|
|
"[]\n",
|
|
"----\n",
|
|
"/logs/RunnableParallel<agent_scratchpad>/streamed_output/-\n",
|
|
"{'agent_scratchpad': []}\n",
|
|
"----\n",
|
|
"/logs/RunnableAssign<agent_scratchpad>/streamed_output/-\n",
|
|
"{'input': 'where is the cat hiding? what items are in that location?', 'intermediate_steps': [], 'agent_scratchpad': []}\n",
|
|
"----\n",
|
|
"/logs/RunnableLambda/end_time\n",
|
|
"2024-01-22T20:38:43.687+00:00\n",
|
|
"----\n",
|
|
"/logs/RunnableParallel<agent_scratchpad>/end_time\n",
|
|
"2024-01-22T20:38:43.688+00:00\n",
|
|
"----\n",
|
|
"/logs/RunnableAssign<agent_scratchpad>/end_time\n",
|
|
"2024-01-22T20:38:43.688+00:00\n",
|
|
"----\n",
|
|
"/logs/ChatPromptTemplate\n",
|
|
"{'id': '7e3a84d5-46b8-4782-8eed-d1fe92be6a30', 'name': 'ChatPromptTemplate', 'type': 'prompt', 'tags': ['seq:step:2'], 'metadata': {}, 'start_time': '2024-01-22T20:38:43.689+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/ChatPromptTemplate/end_time\n",
|
|
"2024-01-22T20:38:43.689+00:00\n",
|
|
"----\n",
|
|
"/logs/ChatOpenAI\n",
|
|
"{'id': '6446f7ec-b3e4-4637-89d8-b4b34b46ea14', 'name': 'ChatOpenAI', 'type': 'llm', 'tags': ['seq:step:3', 'agent_llm'], 'metadata': {}, 'start_time': '2024-01-22T20:38:43.690+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/ChatOpenAI/streamed_output/-\n",
|
|
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_gKFg6FX8ZQ88wFUs94yx86PF', 'function': {'arguments': '', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}\n",
|
|
"----\n",
|
|
"/logs/ChatOpenAI/streamed_output/-\n",
|
|
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_gKFg6FX8ZQ88wFUs94yx86PF', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}\n",
|
|
"----\n",
|
|
"/logs/ChatOpenAI/streamed_output/-\n",
|
|
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_gKFg6FX8ZQ88wFUs94yx86PF', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}\n",
|
|
"----\n",
|
|
"/logs/ChatOpenAI/end_time\n",
|
|
"2024-01-22T20:38:44.203+00:00\n",
|
|
"----\n",
|
|
"/logs/OpenAIToolsAgentOutputParser\n",
|
|
"{'id': '65912835-8dcd-4be2-ad05-9f239a7ef704', 'name': 'OpenAIToolsAgentOutputParser', 'type': 'parser', 'tags': ['seq:step:4'], 'metadata': {}, 'start_time': '2024-01-22T20:38:44.204+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/OpenAIToolsAgentOutputParser/end_time\n",
|
|
"2024-01-22T20:38:44.205+00:00\n",
|
|
"----\n",
|
|
"/logs/RunnableSequence/streamed_output/-\n",
|
|
"[OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_gKFg6FX8ZQ88wFUs94yx86PF', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_gKFg6FX8ZQ88wFUs94yx86PF')]\n",
|
|
"----\n",
|
|
"/logs/RunnableSequence/end_time\n",
|
|
"2024-01-22T20:38:44.206+00:00\n",
|
|
"----\n",
|
|
"/final_output\n",
|
|
"None\n",
|
|
"----\n",
|
|
"/logs/where_cat_is_hiding\n",
|
|
"{'id': '21fde139-0dfa-42bb-ad90-b5b1e984aaba', 'name': 'where_cat_is_hiding', 'type': 'tool', 'tags': [], 'metadata': {}, 'start_time': '2024-01-22T20:38:44.208+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/where_cat_is_hiding/end_time\n",
|
|
"2024-01-22T20:38:44.208+00:00\n",
|
|
"----\n",
|
|
"/final_output/messages/1\n",
|
|
"content='under the bed' name='where_cat_is_hiding'\n",
|
|
"----\n",
|
|
"/logs/RunnableSequence:2\n",
|
|
"{'id': '37d52845-b689-4c18-9c10-ffdd0c4054b0', 'name': 'RunnableSequence', 'type': 'chain', 'tags': [], 'metadata': {}, 'start_time': '2024-01-22T20:38:44.210+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/RunnableAssign<agent_scratchpad>:2\n",
|
|
"{'id': '30024dea-064f-4b04-b130-671f47ac59bc', 'name': 'RunnableAssign<agent_scratchpad>', 'type': 'chain', 'tags': ['seq:step:1'], 'metadata': {}, 'start_time': '2024-01-22T20:38:44.213+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n",
|
|
"/logs/RunnableAssign<agent_scratchpad>:2/streamed_output/-\n",
|
|
"{'input': 'where is the cat hiding? what items are in that location?', 'intermediate_steps': [(OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_gKFg6FX8ZQ88wFUs94yx86PF', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_gKFg6FX8ZQ88wFUs94yx86PF'), 'under the bed')]}\n",
|
|
"----\n",
|
|
"/logs/RunnableParallel<agent_scratchpad>:2\n",
|
|
"{'id': '98906cd7-93c2-47e8-a7d7-2e8d4ab09ed0', 'name': 'RunnableParallel<agent_scratchpad>', 'type': 'chain', 'tags': [], 'metadata': {}, 'start_time': '2024-01-22T20:38:44.215+00:00', 'streamed_output': [], 'streamed_output_str': [], 'final_output': None, 'end_time': None}\n",
|
|
"----\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"i = 0\n",
|
|
"path_status = {}\n",
|
|
"async for chunk in agent_executor.astream_log(\n",
|
|
" {\"input\": \"where is the cat hiding? what items are in that location?\"},\n",
|
|
"):\n",
|
|
" for op in chunk.ops:\n",
|
|
" if op[\"op\"] == \"add\":\n",
|
|
" if op[\"path\"] not in path_status:\n",
|
|
" path_status[op[\"path\"]] = op[\"value\"]\n",
|
|
" else:\n",
|
|
" path_status[op[\"path\"]] += op[\"value\"]\n",
|
|
" print(op[\"path\"])\n",
|
|
" print(path_status.get(op[\"path\"]))\n",
|
|
" print(\"----\")\n",
|
|
" i += 1\n",
|
|
" if i > 30:\n",
|
|
" break"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d85bf6ed-8d89-46fb-bbd8-6c84de7ae18f",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Using callbacks (Legacy)\n",
|
|
"\n",
|
|
"Another approach to streaming is using callbacks. This may be useful if you're still on an older version of LangChain and cannot upgrade.\n",
|
|
"\n",
|
|
"Generall, this is **NOT** a recommended approach because:\n",
|
|
"\n",
|
|
"1. for most applications, you'll need to create two workers, write the callbacks to a queue and have another worker reading from the queue (i.e., there's hidden complexity to make this work).\n",
|
|
"2. **end** events may be missing some metadata (e.g., like run name). So if you need the additional metadata, you should inherit from `BaseTracer` instead of `AsyncCallbackHandler` to pick up the relevant information from the runs (aka traces), or else implement the aggregation logic yourself based on the `run_id`.\n",
|
|
"3. There is inconsistent behavior with the callbacks (e.g., how inputs and outputs are encoded) depending on the callback type that you'll need to workaround.\n",
|
|
"\n",
|
|
"For illustration purposes, we implement a callback below that shows how to get *token by token* streaming. Feel free to implement other callbacks based on your application needs.\n",
|
|
"\n",
|
|
"But `astream_events` does all of this you under the hood, so you don't have to!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"id": "2c577a4a-b754-4c32-a951-8003b876ea9a",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"on chain start: \n",
|
|
"{'input': 'where is the cat hiding and what items can be found there?'}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"On chain end\n",
|
|
"[]\n",
|
|
"On chain end\n",
|
|
"{'agent_scratchpad': []}\n",
|
|
"On chain end\n",
|
|
"{'input': 'where is the cat hiding and what items can be found there?', 'intermediate_steps': [], 'agent_scratchpad': []}\n",
|
|
"on chain start: \n",
|
|
"{'input': 'where is the cat hiding and what items can be found there?', 'intermediate_steps': [], 'agent_scratchpad': []}\n",
|
|
"On chain end\n",
|
|
"{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'prompts', 'chat', 'ChatPromptValue'], 'kwargs': {'messages': [{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'SystemMessage'], 'kwargs': {'content': 'You are a helpful assistant', 'additional_kwargs': {}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'HumanMessage'], 'kwargs': {'content': 'where is the cat hiding and what items can be found there?', 'additional_kwargs': {}}}]}}\n",
|
|
"agent_llm: \n",
|
|
"\n",
|
|
"on chain start: \n",
|
|
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}\n",
|
|
"On chain end\n",
|
|
"[{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'agent', 'OpenAIToolAgentAction'], 'kwargs': {'tool': 'where_cat_is_hiding', 'tool_input': {}, 'log': '\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', 'message_log': [{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'AIMessageChunk'], 'kwargs': {'example': False, 'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}}}], 'tool_call_id': 'call_pboyZTT0587rJtujUluO2OOc'}}]\n",
|
|
"On chain end\n",
|
|
"[OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_pboyZTT0587rJtujUluO2OOc')]\n",
|
|
"Tool start\n",
|
|
"{'name': 'where_cat_is_hiding', 'description': 'where_cat_is_hiding() -> str - Where is the cat hiding right now?'}\n",
|
|
"Tool end\n",
|
|
"on the shelf\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"On chain end\n",
|
|
"[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc')]\n",
|
|
"On chain end\n",
|
|
"{'agent_scratchpad': [AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc')]}\n",
|
|
"On chain end\n",
|
|
"{'input': 'where is the cat hiding and what items can be found there?', 'intermediate_steps': [(OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), 'on the shelf')], 'agent_scratchpad': [AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc')]}\n",
|
|
"on chain start: \n",
|
|
"{'input': 'where is the cat hiding and what items can be found there?', 'intermediate_steps': [(OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), 'on the shelf')], 'agent_scratchpad': [AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc')]}\n",
|
|
"On chain end\n",
|
|
"{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'prompts', 'chat', 'ChatPromptValue'], 'kwargs': {'messages': [{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'SystemMessage'], 'kwargs': {'content': 'You are a helpful assistant', 'additional_kwargs': {}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'HumanMessage'], 'kwargs': {'content': 'where is the cat hiding and what items can be found there?', 'additional_kwargs': {}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'AIMessageChunk'], 'kwargs': {'example': False, 'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'ToolMessage'], 'kwargs': {'tool_call_id': 'call_pboyZTT0587rJtujUluO2OOc', 'content': 'on the shelf', 'additional_kwargs': {'name': 'where_cat_is_hiding'}}}]}}\n",
|
|
"agent_llm: \n",
|
|
"\n",
|
|
"on chain start: \n",
|
|
"content='' additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]}\n",
|
|
"On chain end\n",
|
|
"[{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'agent', 'OpenAIToolAgentAction'], 'kwargs': {'tool': 'get_items', 'tool_input': {'place': 'shelf'}, 'log': \"\\nInvoking: `get_items` with `{'place': 'shelf'}`\\n\\n\\n\", 'message_log': [{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'AIMessageChunk'], 'kwargs': {'example': False, 'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]}}}], 'tool_call_id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh'}}]\n",
|
|
"On chain end\n",
|
|
"[OpenAIToolAgentAction(tool='get_items', tool_input={'place': 'shelf'}, log=\"\\nInvoking: `get_items` with `{'place': 'shelf'}`\\n\\n\\n\", message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]})], tool_call_id='call_vIVtgUb9Gvmc3zAGIrshnmbh')]\n",
|
|
"Tool start\n",
|
|
"{'name': 'get_items', 'description': 'get_items(place: str, callbacks: Union[List[langchain_core.callbacks.base.BaseCallbackHandler], langchain_core.callbacks.base.BaseCallbackManager, NoneType]) -> str - Use this tool to look up which items are in the given place.'}\n",
|
|
"tool_llm: In| a| shelf|,| you| might| find|:\n",
|
|
"\n",
|
|
"|1|.| Books|:| A| shelf| is| commonly| used| to| store| books|.| Books| can| be| of| various| genres|,| such| as| novels|,| textbooks|,| or| reference| books|.| They| provide| knowledge|,| entertainment|,| and| can| transport| you| to| different| worlds| through| storytelling|.\n",
|
|
"\n",
|
|
"|2|.| Decor|ative| items|:| Sh|elves| often| serve| as| a| display| area| for| decorative| items| like| figur|ines|,| v|ases|,| or| sculptures|.| These| items| add| aesthetic| value| to| the| space| and| reflect| the| owner|'s| personal| taste| and| style|.\n",
|
|
"\n",
|
|
"|3|.| Storage| boxes|:| Sh|elves| can| also| be| used| to| store| various| items| in| organized| boxes|.| These| boxes| can| hold| anything| from| office| supplies|,| craft| materials|,| or| sentimental| items|.| They| help| keep| the| space| tidy| and| provide| easy| access| to| stored| belongings|.|\n",
|
|
"\n",
|
|
"Tool end\n",
|
|
"In a shelf, you might find:\n",
|
|
"\n",
|
|
"1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\n",
|
|
"\n",
|
|
"2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\n",
|
|
"\n",
|
|
"3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"on chain start: \n",
|
|
"{'input': ''}\n",
|
|
"On chain end\n",
|
|
"[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]}), ToolMessage(content=\"In a shelf, you might find:\\n\\n1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\\n\\n2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\\n\\n3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\", additional_kwargs={'name': 'get_items'}, tool_call_id='call_vIVtgUb9Gvmc3zAGIrshnmbh')]\n",
|
|
"On chain end\n",
|
|
"{'agent_scratchpad': [AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]}), ToolMessage(content=\"In a shelf, you might find:\\n\\n1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\\n\\n2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\\n\\n3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\", additional_kwargs={'name': 'get_items'}, tool_call_id='call_vIVtgUb9Gvmc3zAGIrshnmbh')]}\n",
|
|
"On chain end\n",
|
|
"{'input': 'where is the cat hiding and what items can be found there?', 'intermediate_steps': [(OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), 'on the shelf'), (OpenAIToolAgentAction(tool='get_items', tool_input={'place': 'shelf'}, log=\"\\nInvoking: `get_items` with `{'place': 'shelf'}`\\n\\n\\n\", message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]})], tool_call_id='call_vIVtgUb9Gvmc3zAGIrshnmbh'), \"In a shelf, you might find:\\n\\n1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\\n\\n2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\\n\\n3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\")], 'agent_scratchpad': [AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]}), ToolMessage(content=\"In a shelf, you might find:\\n\\n1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\\n\\n2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\\n\\n3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\", additional_kwargs={'name': 'get_items'}, tool_call_id='call_vIVtgUb9Gvmc3zAGIrshnmbh')]}\n",
|
|
"on chain start: \n",
|
|
"{'input': 'where is the cat hiding and what items can be found there?', 'intermediate_steps': [(OpenAIToolAgentAction(tool='where_cat_is_hiding', tool_input={}, log='\\nInvoking: `where_cat_is_hiding` with `{}`\\n\\n\\n', message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]})], tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), 'on the shelf'), (OpenAIToolAgentAction(tool='get_items', tool_input={'place': 'shelf'}, log=\"\\nInvoking: `get_items` with `{'place': 'shelf'}`\\n\\n\\n\", message_log=[AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]})], tool_call_id='call_vIVtgUb9Gvmc3zAGIrshnmbh'), \"In a shelf, you might find:\\n\\n1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\\n\\n2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\\n\\n3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\")], 'agent_scratchpad': [AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}), ToolMessage(content='on the shelf', additional_kwargs={'name': 'where_cat_is_hiding'}, tool_call_id='call_pboyZTT0587rJtujUluO2OOc'), AIMessageChunk(content='', additional_kwargs={'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]}), ToolMessage(content=\"In a shelf, you might find:\\n\\n1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\\n\\n2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\\n\\n3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\", additional_kwargs={'name': 'get_items'}, tool_call_id='call_vIVtgUb9Gvmc3zAGIrshnmbh')]}\n",
|
|
"On chain end\n",
|
|
"{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'prompts', 'chat', 'ChatPromptValue'], 'kwargs': {'messages': [{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'SystemMessage'], 'kwargs': {'content': 'You are a helpful assistant', 'additional_kwargs': {}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'HumanMessage'], 'kwargs': {'content': 'where is the cat hiding and what items can be found there?', 'additional_kwargs': {}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'AIMessageChunk'], 'kwargs': {'example': False, 'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_pboyZTT0587rJtujUluO2OOc', 'function': {'arguments': '{}', 'name': 'where_cat_is_hiding'}, 'type': 'function'}]}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'ToolMessage'], 'kwargs': {'tool_call_id': 'call_pboyZTT0587rJtujUluO2OOc', 'content': 'on the shelf', 'additional_kwargs': {'name': 'where_cat_is_hiding'}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'AIMessageChunk'], 'kwargs': {'example': False, 'content': '', 'additional_kwargs': {'tool_calls': [{'index': 0, 'id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'function': {'arguments': '{\\n \"place\": \"shelf\"\\n}', 'name': 'get_items'}, 'type': 'function'}]}}}, {'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'messages', 'ToolMessage'], 'kwargs': {'tool_call_id': 'call_vIVtgUb9Gvmc3zAGIrshnmbh', 'content': \"In a shelf, you might find:\\n\\n1. Books: A shelf is commonly used to store books. Books can be of various genres, such as novels, textbooks, or reference books. They provide knowledge, entertainment, and can transport you to different worlds through storytelling.\\n\\n2. Decorative items: Shelves often serve as a display area for decorative items like figurines, vases, or sculptures. These items add aesthetic value to the space and reflect the owner's personal taste and style.\\n\\n3. Storage boxes: Shelves can also be used to store various items in organized boxes. These boxes can hold anything from office supplies, craft materials, or sentimental items. They help keep the space tidy and provide easy access to stored belongings.\", 'additional_kwargs': {'name': 'get_items'}}}]}}\n",
|
|
"agent_llm: The| cat| is| hiding| on| the| shelf|.| In| the| shelf|,| you| might| find| books|,| decorative| items|,| and| storage| boxes|.|\n",
|
|
"\n",
|
|
"on chain start: \n",
|
|
"content='The cat is hiding on the shelf. In the shelf, you might find books, decorative items, and storage boxes.'\n",
|
|
"On chain end\n",
|
|
"{'lc': 1, 'type': 'constructor', 'id': ['langchain', 'schema', 'agent', 'AgentFinish'], 'kwargs': {'return_values': {'output': 'The cat is hiding on the shelf. In the shelf, you might find books, decorative items, and storage boxes.'}, 'log': 'The cat is hiding on the shelf. In the shelf, you might find books, decorative items, and storage boxes.'}}\n",
|
|
"On chain end\n",
|
|
"return_values={'output': 'The cat is hiding on the shelf. In the shelf, you might find books, decorative items, and storage boxes.'} log='The cat is hiding on the shelf. In the shelf, you might find books, decorative items, and storage boxes.'\n",
|
|
"On chain end\n",
|
|
"{'output': 'The cat is hiding on the shelf. In the shelf, you might find books, decorative items, and storage boxes.'}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from typing import TYPE_CHECKING, Any, Dict, List, Optional, Sequence, TypeVar, Union\n",
|
|
"from uuid import UUID\n",
|
|
"\n",
|
|
"from langchain_core.callbacks.base import AsyncCallbackHandler\n",
|
|
"from langchain_core.messages import BaseMessage\n",
|
|
"from langchain_core.outputs import ChatGenerationChunk, GenerationChunk, LLMResult\n",
|
|
"\n",
|
|
"# Here is a custom handler that will print the tokens to stdout.\n",
|
|
"# Instead of printing to stdout you can send the data elsewhere; e.g., to a streaming API response\n",
|
|
"\n",
|
|
"\n",
|
|
"class TokenByTokenHandler(AsyncCallbackHandler):\n",
|
|
" def __init__(self, tags_of_interest: List[str]) -> None:\n",
|
|
" \"\"\"A custom call back handler.\n",
|
|
"\n",
|
|
" Args:\n",
|
|
" tags_of_interest: Only LLM tokens from models with these tags will be\n",
|
|
" printed.\n",
|
|
" \"\"\"\n",
|
|
" self.tags_of_interest = tags_of_interest\n",
|
|
"\n",
|
|
" async def on_chain_start(\n",
|
|
" self,\n",
|
|
" serialized: Dict[str, Any],\n",
|
|
" inputs: Dict[str, Any],\n",
|
|
" *,\n",
|
|
" run_id: UUID,\n",
|
|
" parent_run_id: Optional[UUID] = None,\n",
|
|
" tags: Optional[List[str]] = None,\n",
|
|
" metadata: Optional[Dict[str, Any]] = None,\n",
|
|
" **kwargs: Any,\n",
|
|
" ) -> None:\n",
|
|
" \"\"\"Run when chain starts running.\"\"\"\n",
|
|
" print(\"on chain start: \")\n",
|
|
" print(inputs)\n",
|
|
"\n",
|
|
" async def on_chain_end(\n",
|
|
" self,\n",
|
|
" outputs: Dict[str, Any],\n",
|
|
" *,\n",
|
|
" run_id: UUID,\n",
|
|
" parent_run_id: Optional[UUID] = None,\n",
|
|
" tags: Optional[List[str]] = None,\n",
|
|
" **kwargs: Any,\n",
|
|
" ) -> None:\n",
|
|
" \"\"\"Run when chain ends running.\"\"\"\n",
|
|
" print(\"On chain end\")\n",
|
|
" print(outputs)\n",
|
|
"\n",
|
|
" async def on_chat_model_start(\n",
|
|
" self,\n",
|
|
" serialized: Dict[str, Any],\n",
|
|
" messages: List[List[BaseMessage]],\n",
|
|
" *,\n",
|
|
" run_id: UUID,\n",
|
|
" parent_run_id: Optional[UUID] = None,\n",
|
|
" tags: Optional[List[str]] = None,\n",
|
|
" metadata: Optional[Dict[str, Any]] = None,\n",
|
|
" **kwargs: Any,\n",
|
|
" ) -> Any:\n",
|
|
" \"\"\"Run when a chat model starts running.\"\"\"\n",
|
|
" overlap_tags = self.get_overlap_tags(tags)\n",
|
|
"\n",
|
|
" if overlap_tags:\n",
|
|
" print(\",\".join(overlap_tags), end=\": \", flush=True)\n",
|
|
"\n",
|
|
" def on_tool_start(\n",
|
|
" self,\n",
|
|
" serialized: Dict[str, Any],\n",
|
|
" input_str: str,\n",
|
|
" *,\n",
|
|
" run_id: UUID,\n",
|
|
" parent_run_id: Optional[UUID] = None,\n",
|
|
" tags: Optional[List[str]] = None,\n",
|
|
" metadata: Optional[Dict[str, Any]] = None,\n",
|
|
" inputs: Optional[Dict[str, Any]] = None,\n",
|
|
" **kwargs: Any,\n",
|
|
" ) -> Any:\n",
|
|
" \"\"\"Run when tool starts running.\"\"\"\n",
|
|
" print(\"Tool start\")\n",
|
|
" print(serialized)\n",
|
|
"\n",
|
|
" def on_tool_end(\n",
|
|
" self,\n",
|
|
" output: Any,\n",
|
|
" *,\n",
|
|
" run_id: UUID,\n",
|
|
" parent_run_id: Optional[UUID] = None,\n",
|
|
" **kwargs: Any,\n",
|
|
" ) -> Any:\n",
|
|
" \"\"\"Run when tool ends running.\"\"\"\n",
|
|
" print(\"Tool end\")\n",
|
|
" print(str(output))\n",
|
|
"\n",
|
|
" async def on_llm_end(\n",
|
|
" self,\n",
|
|
" response: LLMResult,\n",
|
|
" *,\n",
|
|
" run_id: UUID,\n",
|
|
" parent_run_id: Optional[UUID] = None,\n",
|
|
" tags: Optional[List[str]] = None,\n",
|
|
" **kwargs: Any,\n",
|
|
" ) -> None:\n",
|
|
" \"\"\"Run when LLM ends running.\"\"\"\n",
|
|
" overlap_tags = self.get_overlap_tags(tags)\n",
|
|
"\n",
|
|
" if overlap_tags:\n",
|
|
" # Who can argue with beauty?\n",
|
|
" print()\n",
|
|
" print()\n",
|
|
"\n",
|
|
" def get_overlap_tags(self, tags: Optional[List[str]]) -> List[str]:\n",
|
|
" \"\"\"Check for overlap with filtered tags.\"\"\"\n",
|
|
" if not tags:\n",
|
|
" return []\n",
|
|
" return sorted(set(tags or []) & set(self.tags_of_interest or []))\n",
|
|
"\n",
|
|
" async def on_llm_new_token(\n",
|
|
" self,\n",
|
|
" token: str,\n",
|
|
" *,\n",
|
|
" chunk: Optional[Union[GenerationChunk, ChatGenerationChunk]] = None,\n",
|
|
" run_id: UUID,\n",
|
|
" parent_run_id: Optional[UUID] = None,\n",
|
|
" tags: Optional[List[str]] = None,\n",
|
|
" **kwargs: Any,\n",
|
|
" ) -> None:\n",
|
|
" \"\"\"Run on new LLM token. Only available when streaming is enabled.\"\"\"\n",
|
|
" overlap_tags = self.get_overlap_tags(tags)\n",
|
|
"\n",
|
|
" if token and overlap_tags:\n",
|
|
" print(token, end=\"|\", flush=True)\n",
|
|
"\n",
|
|
"\n",
|
|
"handler = TokenByTokenHandler(tags_of_interest=[\"tool_llm\", \"agent_llm\"])\n",
|
|
"\n",
|
|
"result = await agent_executor.ainvoke(\n",
|
|
" {\"input\": \"where is the cat hiding and what items can be found there?\"},\n",
|
|
" {\"callbacks\": [handler]},\n",
|
|
")"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.4"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|