diff --git a/docs/docs/concepts.mdx b/docs/docs/concepts.mdx index 6b592328e89..32b25b89b2c 100644 --- a/docs/docs/concepts.mdx +++ b/docs/docs/concepts.mdx @@ -246,13 +246,18 @@ This property returns a list of dictionaries. Each dictionary has the following This represents a system message, which tells the model how to behave. Not every model provider supports this. -#### FunctionMessage - -This represents the result of a function call. In addition to `role` and `content`, this message has a `name` parameter which conveys the name of the function that was called to produce this result. - #### ToolMessage -This represents the result of a tool call. This is distinct from a FunctionMessage in order to match OpenAI's `function` and `tool` message types. In addition to `role` and `content`, this message has a `tool_call_id` parameter which conveys the id of the call to the tool that was called to produce this result. +This represents the result of a tool call. In addition to `role` and `content`, this message has: + +- a `tool_call_id` field which conveys the id of the call to the tool that was called to produce this result. +- an `artifact` field which can be used to pass along arbitrary artifacts of the tool execution which are useful to track but which should not be sent to the model. + +#### (Legacy) FunctionMessage + +This is a legacy message type, corresponding to OpenAI's legacy function-calling API. ToolMessage should be used instead to correspond to the updated tool-calling API. + +This represents the result of a function call. In addition to `role` and `content`, this message has a `name` parameter which conveys the name of the function that was called to produce this result. ### Prompt templates @@ -496,35 +501,96 @@ For specifics on how to use retrievers, see the [relevant how-to guides here](/d ### Tools -Tools are interfaces that an agent, a chain, or a chat model / LLM can use to interact with the world. +Tools are utilities designed to be called by a model: their inputs are designed to be generated by models, and their outputs are designed to be passed back to models. +Tools are needed whenever you want a model to control parts of your code or call out to external APIs. -A tool consists of the following components: +A tool consists of: -1. The name of the tool -2. A description of what the tool does -3. JSON schema of what the inputs to the tool are -4. The function to call -5. Whether the result of a tool should be returned directly to the user (only relevant for agents) +1. The name of the tool. +2. A description of what the tool does. +3. A JSON schema defining the inputs to the tool. +4. A function (and, optionally, an async variant of the function). -The name, description and JSON schema are provided as context -to the LLM, allowing the LLM to determine how to use the tool -appropriately. +When a tool is bound to a model, the name, description and JSON schema are provided as context to the model. +Given a list of tools and a set of instructions, a model can request to call one or more tools with specific inputs. +Once the chosen tools are invoked, the results can be passed back to the model so that it can complete whatever task +it's performing. -Given a list of available tools and a prompt, an LLM can request -that one or more tools be invoked with appropriate arguments. +#### Tool inputs -Generally, when designing tools to be used by a chat model or LLM, it is important to keep in mind the following: +A tool can take arbitrary arguments as input. At runtime, these arguments can be passed in either: -- Chat models that have been fine-tuned for tool calling will be better at tool calling than non-fine-tuned models. -- Non fine-tuned models may not be able to use tools at all, especially if the tools are complex or require multiple tool calls. -- Models will perform better if the tools have well-chosen names, descriptions, and JSON schemas. -- Simpler tools are generally easier for models to use than more complex tools. +1. As a dict of just the arguments, +2. As a `ToolCall`, which contains the arguments along with other metadata like the tool call ID. -For specifics on how to use tools, see the [relevant how-to guides here](/docs/how_to/#tools). +```python +tool = ... +llm_with_tools = llm.bind_tools([tool]) +ai_msg = llm_with_tools.invoke("do xyz...") # AIMessage(tool_calls=[ToolCall(...), ...], ...) +tool_call = ai_msg.tool_calls[0] # ToolCall(args={...}, id=..., ...) -To use an existing pre-built tool, see [here](/docs/integrations/tools/) for a list of pre-built tools. +# 1. pass in args directly +tool.invoke(tool_call["args"]) + +# 2. pass in the whole ToolCall +tool.invoke(tool_call) +``` + +A tool also has access to the `RunnableConfig` that's passed into whatever chain the tool is a part of. This allows you to write tool logic that can be parameterized by the chain config. + +```python +config = {"configurable": {"tool_param_foo": ...}} +tool.invoke(tool_call, config) +``` + +See the how-to guide for [passing in configs here](/docs/how_to/tool_configure/). + +#### Tool outputs + +The format of a tool's output depends on the format of the input. If a tool is called: + +1. With a dict of its arguments then it will produce an arbitrary output that we assume can be passed to a model as the `ToolMessage.content` field, +2. A `ToolCall` then it will produce a `ToolMessage(content=..., ...)` where the tool output has already been assigned to the `ToolMessage.content` field. + +```python +# 1. pass in args directly + +tool.invoke(tool_call["args"]) +# -> "tool result foobar..." + +# 2. pass in the whole ToolCall + +tool.invoke(tool_call) +# -> ToolMessage(content="tool result foobar...", tool_call_id=..., name="tool_name") +``` + +A tool can also be defined to include an artifact when invoked with a `ToolCall`. An artifact is some element of the +tool's execution which is useful to return but shouldn't be sent to the model. The artifact can *only* be returned +when the tool input is a `ToolCall`: + +```python +tool_with_artifact.invoke(tool_call) +# -> ToolMessage(content="tool result foobar...", tool_call_id=..., name="tool_name", artifact=...). +``` + +Learn about [`ToolMessage.artifact` here](/docs/concepts/#toolmessage) and about [defining tools that return artifacts here](/docs/how_to/tool_artifacts/). + +#### Best practices + +When designing tools to be used by a model, it is important to keep in mind that: + +- Chat models that have explicit [tool-calling APIs](/docs/concepts/#functiontool-calling) will be better at tool calling than non-fine-tuned models. +- Models will perform better if the tools have well-chosen names, descriptions, and JSON schemas. This another form of prompt engineering. +- Simple, narrowly scoped tools are easier for models to use than complex tools. + +#### Related + +For specifics on how to use tools, see the [tools how-to guides](/docs/how_to/#tools). + +To use a pre-built tool, see the [tool integration docs](/docs/integrations/tools/). ### Toolkits + Toolkits are collections of tools that are designed to be used together for specific tasks. They have convenient loading methods.