mirror of
https://github.com/hwchase17/langchain.git
synced 2025-06-08 07:57:07 +00:00
580fbd9ada
``` https://api\.python\.langchain\.com/en/latest/([^/]*)/langchain_([^.]*)\.(.*)\.html([^"]*) https://python.langchain.com/v0.2/api_reference/$2/$1/langchain_$2.$3.html$4 ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>
388 lines
14 KiB
Plaintext
388 lines
14 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"metadata": {
|
|
"vscode": {
|
|
"languageId": "raw"
|
|
}
|
|
},
|
|
"source": [
|
|
"---\n",
|
|
"keywords: [tool calling, tool call]\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# How to use chat models to call tools\n",
|
|
"\n",
|
|
":::info Prerequisites\n",
|
|
"\n",
|
|
"This guide assumes familiarity with the following concepts:\n",
|
|
"\n",
|
|
"- [Chat models](/docs/concepts/#chat-models)\n",
|
|
"- [Tool calling](/docs/concepts/#functiontool-calling)\n",
|
|
"- [Tools](/docs/concepts/#tools)\n",
|
|
"- [Output parsers](/docs/concepts/#output-parsers)\n",
|
|
":::\n",
|
|
"\n",
|
|
"[Tool calling](/docs/concepts/#functiontool-calling) allows a chat model to respond to a given prompt by \"calling a tool\".\n",
|
|
"\n",
|
|
"Remember, while the name \"tool calling\" implies that the model is directly performing some action, this is actually not the case! The model only generates the arguments to a tool, and actually running the tool (or not) is up to the user.\n",
|
|
"\n",
|
|
"Tool calling is a general technique that generates structured output from a model, and you can use it even when you don't intend to invoke any tools. An example use-case of that is [extraction from unstructured text](/docs/tutorials/extraction/).\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"If you want to see how to use the model-generated tool call to actually run a tool [check out this guide](/docs/how_to/tool_results_pass_to_model/).\n",
|
|
"\n",
|
|
":::note Supported models\n",
|
|
"\n",
|
|
"Tool calling is not universal, but is supported by many popular LLM providers. You can find a [list of all models that support tool calling here](/docs/integrations/chat/).\n",
|
|
"\n",
|
|
":::\n",
|
|
"\n",
|
|
"LangChain implements standard interfaces for defining tools, passing them to LLMs, and representing tool calls.\n",
|
|
"This guide will cover how to bind tools to an LLM, then invoke the LLM to generate these arguments."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Defining tool schemas\n",
|
|
"\n",
|
|
"For a model to be able to call tools, we need to pass in tool schemas that describe what the tool does and what it's arguments are. Chat models that support tool calling features implement a `.bind_tools()` method for passing tool schemas to the model. Tool schemas can be passed in as Python functions (with typehints and docstrings), Pydantic models, TypedDict classes, or LangChain [Tool objects](https://python.langchain.com/v0.2/api_reference/core/tools/langchain_core.tools.BaseTool.html#langchain_core.tools.BaseTool). Subsequent invocations of the model will pass in these tool schemas along with the prompt.\n",
|
|
"\n",
|
|
"### Python functions\n",
|
|
"Our tool schemas can be Python functions:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# The function name, type hints, and docstring are all part of the tool\n",
|
|
"# schema that's passed to the model. Defining good, descriptive schemas\n",
|
|
"# is an extension of prompt engineering and is an important part of\n",
|
|
"# getting models to perform well.\n",
|
|
"def add(a: int, b: int) -> int:\n",
|
|
" \"\"\"Add two integers.\n",
|
|
"\n",
|
|
" Args:\n",
|
|
" a: First integer\n",
|
|
" b: Second integer\n",
|
|
" \"\"\"\n",
|
|
" return a + b\n",
|
|
"\n",
|
|
"\n",
|
|
"def multiply(a: int, b: int) -> int:\n",
|
|
" \"\"\"Multiply two integers.\n",
|
|
"\n",
|
|
" Args:\n",
|
|
" a: First integer\n",
|
|
" b: Second integer\n",
|
|
" \"\"\"\n",
|
|
" return a * b"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### LangChain Tool\n",
|
|
"\n",
|
|
"LangChain also implements a `@tool` decorator that allows for further control of the tool schema, such as tool names and argument descriptions. See the how-to guide [here](/docs/how_to/custom_tools/#creating-tools-from-functions) for details.\n",
|
|
"\n",
|
|
"### Pydantic class\n",
|
|
"\n",
|
|
"You can equivalently define the schemas without the accompanying functions using [Pydantic](https://docs.pydantic.dev).\n",
|
|
"\n",
|
|
"Note that all fields are `required` unless provided a default value.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_core.pydantic_v1 import BaseModel, Field\n",
|
|
"\n",
|
|
"\n",
|
|
"class add(BaseModel):\n",
|
|
" \"\"\"Add two integers.\"\"\"\n",
|
|
"\n",
|
|
" a: int = Field(..., description=\"First integer\")\n",
|
|
" b: int = Field(..., description=\"Second integer\")\n",
|
|
"\n",
|
|
"\n",
|
|
"class multiply(BaseModel):\n",
|
|
" \"\"\"Multiply two integers.\"\"\"\n",
|
|
"\n",
|
|
" a: int = Field(..., description=\"First integer\")\n",
|
|
" b: int = Field(..., description=\"Second integer\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### TypedDict class\n",
|
|
"\n",
|
|
":::info Requires `langchain-core>=0.2.25`\n",
|
|
":::\n",
|
|
"\n",
|
|
"Or using TypedDicts and annotations:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from typing_extensions import Annotated, TypedDict\n",
|
|
"\n",
|
|
"\n",
|
|
"class add(TypedDict):\n",
|
|
" \"\"\"Add two integers.\"\"\"\n",
|
|
"\n",
|
|
" # Annotations must have the type and can optionally include a default value and description (in that order).\n",
|
|
" a: Annotated[int, ..., \"First integer\"]\n",
|
|
" b: Annotated[int, ..., \"Second integer\"]\n",
|
|
"\n",
|
|
"\n",
|
|
"class multiply(BaseModel):\n",
|
|
" \"\"\"Multiply two integers.\"\"\"\n",
|
|
"\n",
|
|
" a: Annotated[int, ..., \"First integer\"]\n",
|
|
" b: Annotated[int, ..., \"Second integer\"]\n",
|
|
"\n",
|
|
"\n",
|
|
"tools = [add, multiply]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"To actually bind those schemas to a chat model, we'll use the `.bind_tools()` method. This handles converting\n",
|
|
"the `add` and `multiply` schemas to the proper format for the model. The tool schema will then be passed it in each time the model is invoked.\n",
|
|
"\n",
|
|
"```{=mdx}\n",
|
|
"import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
|
|
"\n",
|
|
"<ChatModelTabs\n",
|
|
" customVarName=\"llm\"\n",
|
|
" fireworksParams={`model=\"accounts/fireworks/models/firefunction-v1\", temperature=0`}\n",
|
|
"/>\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# | output: false\n",
|
|
"# | echo: false\n",
|
|
"\n",
|
|
"%pip install -qU langchain_openai\n",
|
|
"\n",
|
|
"import os\n",
|
|
"from getpass import getpass\n",
|
|
"\n",
|
|
"from langchain_openai import ChatOpenAI\n",
|
|
"\n",
|
|
"os.environ[\"OPENAI_API_KEY\"] = getpass()\n",
|
|
"\n",
|
|
"llm = ChatOpenAI(model=\"gpt-4o-mini\", temperature=0)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_BwYJ4UgU5pRVCBOUmiu7NhF9', 'function': {'arguments': '{\"a\":3,\"b\":12}', 'name': 'multiply'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 17, 'prompt_tokens': 80, 'total_tokens': 97}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_ba606877f9', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-7f05e19e-4561-40e2-a2d0-8f4e28e9a00f-0', tool_calls=[{'name': 'multiply', 'args': {'a': 3, 'b': 12}, 'id': 'call_BwYJ4UgU5pRVCBOUmiu7NhF9', 'type': 'tool_call'}], usage_metadata={'input_tokens': 80, 'output_tokens': 17, 'total_tokens': 97})"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"llm_with_tools = llm.bind_tools(tools)\n",
|
|
"\n",
|
|
"query = \"What is 3 * 12?\"\n",
|
|
"\n",
|
|
"llm_with_tools.invoke(query)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"As we can see our LLM generated arguments to a tool! You can look at the docs for [bind_tools()](https://python.langchain.com/v0.2/api_reference/openai/chat_models/langchain_openai.chat_models.base.BaseChatOpenAI.html#langchain_openai.chat_models.base.BaseChatOpenAI.bind_tools) to learn about all the ways to customize how your LLM selects tools, as well as [this guide on how to force the LLM to call a tool](/docs/how_to/tool_choice/) rather than letting it decide."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Tool calls\n",
|
|
"\n",
|
|
"If tool calls are included in a LLM response, they are attached to the corresponding \n",
|
|
"[message](https://python.langchain.com/v0.2/api_reference/core/messages/langchain_core.messages.ai.AIMessage.html#langchain_core.messages.ai.AIMessage) \n",
|
|
"or [message chunk](https://python.langchain.com/v0.2/api_reference/core/messages/langchain_core.messages.ai.AIMessageChunk.html#langchain_core.messages.ai.AIMessageChunk) \n",
|
|
"as a list of [tool call](https://python.langchain.com/v0.2/api_reference/core/messages/langchain_core.messages.tool.ToolCall.html#langchain_core.messages.tool.ToolCall) \n",
|
|
"objects in the `.tool_calls` attribute.\n",
|
|
"\n",
|
|
"Note that chat models can call multiple tools at once.\n",
|
|
"\n",
|
|
"A `ToolCall` is a typed dict that includes a \n",
|
|
"tool name, dict of argument values, and (optionally) an identifier. Messages with no \n",
|
|
"tool calls default to an empty list for this attribute."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[{'name': 'multiply',\n",
|
|
" 'args': {'a': 3, 'b': 12},\n",
|
|
" 'id': 'call_rcdMie7E89Xx06lEKKxJyB5N',\n",
|
|
" 'type': 'tool_call'},\n",
|
|
" {'name': 'add',\n",
|
|
" 'args': {'a': 11, 'b': 49},\n",
|
|
" 'id': 'call_nheGN8yfvSJsnIuGZaXihou3',\n",
|
|
" 'type': 'tool_call'}]"
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"query = \"What is 3 * 12? Also, what is 11 + 49?\"\n",
|
|
"\n",
|
|
"llm_with_tools.invoke(query).tool_calls"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The `.tool_calls` attribute should contain valid tool calls. Note that on occasion, \n",
|
|
"model providers may output malformed tool calls (e.g., arguments that are not \n",
|
|
"valid JSON). When parsing fails in these cases, instances \n",
|
|
"of [InvalidToolCall](https://python.langchain.com/v0.2/api_reference/core/messages/langchain_core.messages.tool.InvalidToolCall.html#langchain_core.messages.tool.InvalidToolCall) \n",
|
|
"are populated in the `.invalid_tool_calls` attribute. An `InvalidToolCall` can have \n",
|
|
"a name, string arguments, identifier, and error message.\n",
|
|
"\n",
|
|
"\n",
|
|
"## Parsing\n",
|
|
"\n",
|
|
"If desired, [output parsers](/docs/how_to#output-parsers) can further process the output. For example, we can convert existing values populated on the `.tool_calls` to Pydantic objects using the\n",
|
|
"[PydanticToolsParser](https://python.langchain.com/v0.2/api_reference/core/output_parsers/langchain_core.output_parsers.openai_tools.PydanticToolsParser.html):"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[multiply(a=3, b=12), add(a=11, b=49)]"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_core.output_parsers import PydanticToolsParser\n",
|
|
"from langchain_core.pydantic_v1 import BaseModel, Field\n",
|
|
"\n",
|
|
"\n",
|
|
"class add(BaseModel):\n",
|
|
" \"\"\"Add two integers.\"\"\"\n",
|
|
"\n",
|
|
" a: int = Field(..., description=\"First integer\")\n",
|
|
" b: int = Field(..., description=\"Second integer\")\n",
|
|
"\n",
|
|
"\n",
|
|
"class multiply(BaseModel):\n",
|
|
" \"\"\"Multiply two integers.\"\"\"\n",
|
|
"\n",
|
|
" a: int = Field(..., description=\"First integer\")\n",
|
|
" b: int = Field(..., description=\"Second integer\")\n",
|
|
"\n",
|
|
"\n",
|
|
"chain = llm_with_tools | PydanticToolsParser(tools=[add, multiply])\n",
|
|
"chain.invoke(query)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Next steps\n",
|
|
"\n",
|
|
"Now you've learned how to bind tool schemas to a chat model and have the model call the tool.\n",
|
|
"\n",
|
|
"Next, check out this guide on actually using the tool by invoking the function and passing the results back to the model:\n",
|
|
"\n",
|
|
"- Pass [tool results back to model](/docs/how_to/tool_results_pass_to_model)\n",
|
|
"\n",
|
|
"You can also check out some more specific uses of tool calling:\n",
|
|
"\n",
|
|
"- Getting [structured outputs](/docs/how_to/structured_output/) from models\n",
|
|
"- Few shot prompting [with tools](/docs/how_to/tools_few_shot/)\n",
|
|
"- Stream [tool calls](/docs/how_to/tool_streaming/)\n",
|
|
"- Pass [runtime values to tools](/docs/how_to/tool_runtime)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "poetry-venv-311",
|
|
"language": "python",
|
|
"name": "poetry-venv-311"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.9"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|