langchain/docs/docs/how_to/tool_calling.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "---\n",
    "keywords: [tool calling, tool call]\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to use chat models to call tools\n",
    "\n",
    ":::info Prerequisites\n",
    "\n",
    "This guide assumes familiarity with the following concepts:\n",
    "- [Chat models](/docs/concepts/#chat-models)\n",
    "- [LangChain Tools](/docs/concepts/#tools)\n",
    "- [Output parsers](/docs/concepts/#output-parsers)\n",
    "\n",
    ":::\n",
    "\n",
    ":::info Tool calling vs function calling\n",
    "\n",
    "We use the term tool calling interchangeably with function calling. Although\n",
    "function calling is sometimes meant to refer to invocations of a single function,\n",
    "we treat all models as though they can return multiple tool or function calls in \n",
    "each message.\n",
    "\n",
    ":::\n",
    "\n",
    ":::info Supported models\n",
    "\n",
    "You can find a [list of all models that support tool calling](/docs/integrations/chat/).\n",
    "\n",
    ":::\n",
    "\n",
    "Tool calling allows a chat model to respond to a given prompt by \"calling a tool\".\n",
    "While the name implies that the model is performing \n",
    "some action, this is actually not the case! The model generates the \n",
    "arguments to a tool, and actually running the tool (or not) is up to the user.\n",
    "For example, if you want to [extract output matching some schema](/docs/how_to/structured_output/) \n",
    "from unstructured text, you could give the model an \"extraction\" tool that takes \n",
    "parameters matching the desired schema, then treat the generated output as your final \n",
    "result.\n",
    "\n",
    ":::note\n",
    "\n",
    "If you only need formatted values, try the [.with_structured_output()](/docs/how_to/structured_output/#the-with_structured_output-method) chat model method as a simpler entrypoint.\n",
    "\n",
    ":::\n",
    "\n",
    "However, tool calling goes beyond [structured output](/docs/how_to/structured_output/)\n",
    "since you can pass responses from called tools back to the model to create longer interactions.\n",
    "For instance, given a search engine tool, an LLM might handle a \n",
    "query by first issuing a call to the search engine with arguments. The system calling the LLM can \n",
    "receive the tool call, execute it, and return the output to the LLM to inform its \n",
    "response. LangChain includes a suite of [built-in tools](/docs/integrations/tools/) \n",
    "and supports several methods for defining your own [custom tools](/docs/how_to/custom_tools). \n",
    "\n",
    "Tool calling is not universal, but many popular LLM providers, including [Anthropic](https://www.anthropic.com/), \n",
    "[Cohere](https://cohere.com/), [Google](https://cloud.google.com/vertex-ai), \n",
    "[Mistral](https://mistral.ai/), [OpenAI](https://openai.com/), and others, \n",
    "support variants of a tool calling feature.\n",
    "\n",
    "LangChain implements standard interfaces for defining tools, passing them to LLMs, \n",
    "and representing tool calls. This guide and the other How-to pages in the Tool section will show you how to use tools with LangChain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Passing tools to chat models\n",
    "\n",
    "Chat models that support tool calling features implement a `.bind_tools` method, which \n",
    "receives a list of LangChain [tool objects](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.BaseTool.html#langchain_core.tools.BaseTool) \n",
    "and binds them to the chat model in its expected format. Subsequent invocations of the \n",
    "chat model will include tool schemas in its calls to the LLM.\n",
    "\n",
    "For example, we can define the schema for custom tools using the `@tool` decorator \n",
    "on Python functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.tools import tool\n",
    "\n",
    "\n",
    "@tool\n",
    "def add(a: int, b: int) -> int:\n",
    "    \"\"\"Adds a and b.\"\"\"\n",
    "    return a + b\n",
    "\n",
    "\n",
    "@tool\n",
    "def multiply(a: int, b: int) -> int:\n",
    "    \"\"\"Multiplies a and b.\"\"\"\n",
    "    return a * b\n",
    "\n",
    "\n",
    "tools = [add, multiply]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or below, we define the schema using [Pydantic](https://docs.pydantic.dev):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.pydantic_v1 import BaseModel, Field\n",
    "\n",
    "\n",
    "# Note that the docstrings here are crucial, as they will be passed along\n",
    "# to the model along with the class name.\n",
    "class Add(BaseModel):\n",
    "    \"\"\"Add two integers together.\"\"\"\n",
    "\n",
    "    a: int = Field(..., description=\"First integer\")\n",
    "    b: int = Field(..., description=\"Second integer\")\n",
    "\n",
    "\n",
    "class Multiply(BaseModel):\n",
    "    \"\"\"Multiply two integers together.\"\"\"\n",
    "\n",
    "    a: int = Field(..., description=\"First integer\")\n",
    "    b: int = Field(..., description=\"Second integer\")\n",
    "\n",
    "\n",
    "tools = [Add, Multiply]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can bind them to chat models as follows:\n",
    "\n",
    "```{=mdx}\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs\n",
    "  customVarName=\"llm\"\n",
    "  fireworksParams={`model=\"accounts/fireworks/models/firefunction-v1\", temperature=0`}\n",
    "/>\n",
    "```\n",
    "\n",
    "We'll use the `.bind_tools()` method to handle converting\n",
    "`Multiply` to the proper format for the model, then and bind it (i.e.,\n",
    "passing it in each time the model is invoked)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# | output: false\n",
    "# | echo: false\n",
    "\n",
    "%pip install -qU langchain_openai\n",
    "\n",
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = getpass()\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_g4RuAijtDcSeM96jXyCuiLSN', 'function': {'arguments': '{\"a\":3,\"b\":12}', 'name': 'Multiply'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 18, 'prompt_tokens': 95, 'total_tokens': 113}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-5157d15a-7e0e-4ab1-af48-3d98010cd152-0', tool_calls=[{'name': 'Multiply', 'args': {'a': 3, 'b': 12}, 'id': 'call_g4RuAijtDcSeM96jXyCuiLSN'}], usage_metadata={'input_tokens': 95, 'output_tokens': 18, 'total_tokens': 113})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_with_tools = llm.bind_tools(tools)\n",
    "\n",
    "query = \"What is 3 * 12?\"\n",
    "\n",
    "llm_with_tools.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, even though the prompt didn't really suggest a tool call, our LLM made one since it was forced to do so. You can look at the docs for [bind_tools()](https://api.python.langchain.com/en/latest/chat_models/langchain_openai.chat_models.base.BaseChatOpenAI.html#langchain_openai.chat_models.base.BaseChatOpenAI.bind_tools) to learn about all the ways to customize how your LLM selects tools."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tool calls\n",
    "\n",
    "If tool calls are included in a LLM response, they are attached to the corresponding \n",
    "[message](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.ai.AIMessage.html#langchain_core.messages.ai.AIMessage) \n",
    "or [message chunk](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.ai.AIMessageChunk.html#langchain_core.messages.ai.AIMessageChunk) \n",
    "as a list of [tool call](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.tool.ToolCall.html#langchain_core.messages.tool.ToolCall) \n",
    "objects in the `.tool_calls` attribute.\n",
    "\n",
    "Note that chat models can call multiple tools at once.\n",
    "\n",
    "A `ToolCall` is a typed dict that includes a \n",
    "tool name, dict of argument values, and (optionally) an identifier. Messages with no \n",
    "tool calls default to an empty list for this attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'name': 'Multiply',\n",
       "  'args': {'a': 3, 'b': 12},\n",
       "  'id': 'call_TnadLbWJu9HwDULRb51RNSMw'},\n",
       " {'name': 'Add',\n",
       "  'args': {'a': 11, 'b': 49},\n",
       "  'id': 'call_Q9vt1up05sOQScXvUYWzSpCg'}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = \"What is 3 * 12? Also, what is 11 + 49?\"\n",
    "\n",
    "llm_with_tools.invoke(query).tool_calls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `.tool_calls` attribute should contain valid tool calls. Note that on occasion, \n",
    "model providers may output malformed tool calls (e.g., arguments that are not \n",
    "valid JSON). When parsing fails in these cases, instances \n",
    "of [InvalidToolCall](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.tool.InvalidToolCall.html#langchain_core.messages.tool.InvalidToolCall) \n",
    "are populated in the `.invalid_tool_calls` attribute. An `InvalidToolCall` can have \n",
    "a name, string arguments, identifier, and error message.\n",
    "\n",
    "If desired, [output parsers](/docs/how_to#output-parsers) can further \n",
    "process the output. For example, we can convert existing values populated on the `.tool_calls` attribute back to the original Pydantic class using the\n",
    "[PydanticToolsParser](https://api.python.langchain.com/en/latest/output_parsers/langchain_core.output_parsers.openai_tools.PydanticToolsParser.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Multiply(a=3, b=12), Add(a=11, b=49)]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.output_parsers import PydanticToolsParser\n",
    "\n",
    "chain = llm_with_tools | PydanticToolsParser(tools=[Multiply, Add])\n",
    "chain.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "Now you've learned how to bind tool schemas to a chat model and to call those tools. Next, you can learn more about how to use tools:\n",
    "\n",
    "- Few shot promting [with tools](/docs/how_to/tools_few_shot/)\n",
    "- Stream [tool calls](/docs/how_to/tool_streaming/)\n",
    "- Bind [model-specific tools](/docs/how_to/tools_model_specific/)\n",
    "- Pass [runtime values to tools](/docs/how_to/tool_runtime)\n",
    "- Pass [tool results back to model](/docs/how_to/tool_results_pass_to_model)\n",
    "\n",
    "You can also check out some more specific uses of tool calling:\n",
    "\n",
    "- Building [tool-using chains and agents](/docs/how_to#tools)\n",
    "- Getting [structured outputs](/docs/how_to/structured_output/) from models"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}