github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-04 04:28:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

anthropic[minor]: tool use (#20016)

Browse Source
This commit is contained in:
Bagatur
2024-04-04 13:22:48 -07:00
committed by GitHub
parent 3aacd11846
commit 209de0a561
13 changed files with 1021 additions and 196 deletions
Download Patch File Download Diff File Expand all files Collapse all files

414
docs/docs/integrations/chat/anthropic.ipynb
View File

@@ -61,7 +61,10 @@
"cell_type": "markdown",
"id": "d1f9df276476f0bc",
"metadata": {
"collapsed": false
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"source": [
"The code provided assumes that your ANTHROPIC_API_KEY is set in your environment variables. If you would like to manually specify your API key and also choose a different model, you can use the following code:\n",
@@ -75,6 +78,17 @@
"You can check the model comparison doc [here](https://docs.anthropic.com/claude/docs/models-overview#model-comparison)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "238bdbaa-526a-4130-89e9-523aa44bb196",
"metadata": {},
"outputs": [],
"source": [
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.prompts import ChatPromptTemplate"
]
},
{
"cell_type": "code",
"execution_count": 5,
@@ -99,9 +113,6 @@
}
],
"source": [
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"\n",
"chat = ChatAnthropic(temperature=0, model_name=\"claude-3-opus-20240229\")\n",
"\n",
"system = (\n",
@@ -288,13 +299,400 @@
"chat.invoke(messages)"
]
},
{
"cell_type": "markdown",
"id": "ab0174d8-7140-413c-80a9-7cf3a8b81bb4",
"metadata": {},
"source": [
"## [Beta] Tool-calling\n",
"\n",
"With Anthropic's [tool-calling, or tool-use, API](https://docs.anthropic.com/claude/docs/functions-external-tools), you can define tools for the model to invoke. This is extremely useful for building tool-using chains and agents, as well as for getting structured outputs from a model.\n",
"\n",
"::: {.callout-note}\n",
"\n",
"Anthropic's tool-calling functionality is still in beta.\n",
"\n",
":::\n",
"\n",
"### bind_tools()\n",
"\n",
"With `ChatAnthropic.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an Anthropic tool schemas, which looks like:\n",
"```\n",
"{\n",
" \"name\": \"...\",\n",
" \"description\": \"...\",\n",
" \"input_schema\": {...} # JSONSchema\n",
"}\n",
"```\n",
"and passed in every model invocation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3a381f8e",
"execution_count": 4,
"id": "42f87466-cb8e-490d-a9f8-aa0f8e9b4217",
"metadata": {},
"outputs": [],
"source": []
"source": [
"from langchain_core.pydantic_v1 import BaseModel, Field\n",
"\n",
"llm = ChatAnthropic(\n",
" model=\"claude-3-opus-20240229\",\n",
" default_headers={\"anthropic-beta\": \"tools-2024-04-04\"},\n",
")\n",
"\n",
"\n",
"class GetWeather(BaseModel):\n",
" \"\"\"Get the current weather in a given location\"\"\"\n",
"\n",
" location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n",
"\n",
"\n",
"llm_with_tools = llm.bind_tools([GetWeather])"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "997be6ff-3fd3-4b1c-b7e3-2e5fed4ac964",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content=[{'text': '<thinking>\\nBased on the user\\'s question, the relevant function to call is GetWeather, which requires the \"location\" parameter.\\n\\nThe user has directly specified the location as \"San Francisco\". Since San Francisco is a well known city, I can reasonably infer they mean San Francisco, CA without needing the state specified.\\n\\nAll the required parameters are provided, so I can proceed with the API call.\\n</thinking>', 'type': 'text'}, {'text': None, 'type': 'tool_use', 'id': 'toolu_01SCgExKzQ7eqSkMHfygvYuu', 'name': 'GetWeather', 'input': {'location': 'San Francisco, CA'}}], response_metadata={'id': 'msg_01GM3zQtoFv8jGQMW7abLnhi', 'model': 'claude-3-opus-20240229', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 487, 'output_tokens': 145}}, id='run-87b1331e-9251-4a68-acef-f0a018b639cc-0')"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ai_msg = llm_with_tools.invoke(\n",
" \"what is the weather like in San Francisco\",\n",
")\n",
"ai_msg"
]
},
{
"cell_type": "markdown",
"id": "1e63ac67-8c42-4468-8178-e54f13c3c5c3",
"metadata": {},
"source": [
"Notice that the output message content is a list that contains a text block and then a tool_use block:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "7c4cd4c4-1c78-4d6c-8607-759e32a8903b",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'text': '<thinking>\\nBased on the user\\'s question, the relevant function to call is GetWeather, which requires the \"location\" parameter.\\n\\nThe user has directly specified the location as \"San Francisco\". Since San Francisco is a well known city, I can reasonably infer they mean San Francisco, CA without needing the state specified.\\n\\nAll the required parameters are provided, so I can proceed with the API call.\\n</thinking>',\n",
" 'type': 'text'}"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ai_msg.content[0]"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "5b92d91d-37cb-4843-8b2e-e337d2eec53e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'text': None,\n",
" 'type': 'tool_use',\n",
" 'id': 'toolu_01SCgExKzQ7eqSkMHfygvYuu',\n",
" 'name': 'GetWeather',\n",
" 'input': {'location': 'San Francisco, CA'}}"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ai_msg.content[1]"
]
},
{
"cell_type": "markdown",
"id": "90e015e0-c6e5-4ff5-8fb9-be0cd3c86395",
"metadata": {},
"source": [
"::: {.callout-tip}\n",
"\n",
"ChatAnthropic model outputs are always a single AI message that can have either a single string or a list of content blocks. The content blocks can be text blocks or tool-duse blocks. There can be multiple of each and they can be interspersed.\n",
"\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "8652ee98-814c-4ed6-9def-275eeaa9651e",
"metadata": {},
"source": [
"### Parsing tool calls\n",
"\n",
"The `langchain_anthropic.output_parsers.ToolsOutputParser` makes it easy to extract just the tool calls from an Anthropic AI message:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "59c175b1-0929-4ed4-a608-f0006031a3c2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'name': 'GetWeather',\n",
" 'args': {'location': 'New York City, NY'},\n",
" 'id': 'toolu_01UK2AEWa75PUGA3DpiaHfBN',\n",
" 'index': 1},\n",
" {'name': 'GetWeather',\n",
" 'args': {'location': 'Los Angeles, CA'},\n",
" 'id': 'toolu_01M84DY7xWg9bLoX6JCArczx',\n",
" 'index': 2},\n",
" {'name': 'GetWeather',\n",
" 'args': {'location': 'San Francisco, CA'},\n",
" 'id': 'toolu_01FEasmxGpxFPwf9SF3nCTeb',\n",
" 'index': 3},\n",
" {'name': 'GetWeather',\n",
" 'args': {'location': 'Cleveland, OH'},\n",
" 'id': 'toolu_01B8fZdiyPbzWyj5cDCzGSTe',\n",
" 'index': 4}]"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_anthropic.output_parsers import ToolsOutputParser\n",
"\n",
"parser = ToolsOutputParser()\n",
"chain = llm_with_tools | parser\n",
"chain.invoke(\"What is the weather like in nyc, la, sf and cleveland\")"
]
},
{
"cell_type": "markdown",
"id": "c4394c23-8d79-4f2c-b0fe-7b877eaac7c7",
"metadata": {},
"source": [
"The `index` tells us where in the original list of content blocks each tool call was.\n",
"\n",
"We can pass in Pydantic classes to parse our tool calls into pydantic objects:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "08f6c62c-923b-400e-9bc8-8aff417466b2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[GetWeather(location='New York City, NY'),\n",
" GetWeather(location='Los Angeles, CA'),\n",
" GetWeather(location='San Francisco, CA'),\n",
" GetWeather(location='Cleveland, OH')]"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"parser = ToolsOutputParser(pydantic_schemas=[GetWeather])\n",
"chain = llm_with_tools | parser\n",
"chain.invoke(\"What is the weather like in nyc, la, sf and cleveland\")"
]
},
{
"cell_type": "markdown",
"id": "8ccdc039-d8ce-4460-bb2f-543753aac016",
"metadata": {},
"source": [
"If we want we can return only the first tool call:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "7746c643-851f-4908-ac34-8ddbb949454d",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'name': 'GetWeather',\n",
" 'args': {'location': 'New York City, NY'},\n",
" 'id': 'toolu_01EjFAADbpdrML1uaSMr9tN3',\n",
" 'index': 1}"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"parser = ToolsOutputParser(first_tool_only=True)\n",
"chain = llm_with_tools | parser\n",
"chain.invoke(\"What is the weather like in nyc\")"
]
},
{
"cell_type": "markdown",
"id": "ab05dd51-0a9e-4b7b-b182-65cec44941ac",
"metadata": {},
"source": [
"### with_structured_output()\n",
"\n",
"The [BaseChatModel.with_structured_output interface](/docs/modules/model_io/chat/structured_output) makes it easy to get structured output from chat models. You can use `ChatAnthropic.with_structured_output`, which uses tool-calling under the hood), to get the model to more reliably return an output in a specific format:"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "e047b831-2338-4c2d-9ee4-0763f74e80e1",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"GetWeather(location='San Francisco, CA')"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"structured_llm = llm.with_structured_output(GetWeather)\n",
"structured_llm.invoke(\n",
" \"what is the weather like in San Francisco\",\n",
")"
]
},
{
"cell_type": "markdown",
"id": "5b61884e-3e4e-4145-b10d-188987ae1eb6",
"metadata": {},
"source": [
"### Passing tool results to model\n",
"\n",
"We can use `ToolMessage`s with the appropriate `tool_call_id`s to pass tool results back to the model:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "9d07a1c1-4542-440e-a1fb-392542267fb8",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content='Based on calling the GetWeather function, the weather in San Francisco, CA is:\\nRain with a high temperature of 54°F and winds from the southwest at 15-25 mph. There is a 100% chance of rain.', response_metadata={'id': 'msg_01J7nWVRPPTgae4eDpf9yR3M', 'model': 'claude-3-opus-20240229', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 670, 'output_tokens': 56}}, id='run-44fcd34f-9c24-464f-94dd-63bd0d22870d-0')"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_core.messages import AIMessage, HumanMessage, ToolMessage\n",
"\n",
"messages = [\n",
" HumanMessage(\"What is the weather like in San Francisco\"),\n",
" AIMessage(\n",
" content=[\n",
" {\n",
" \"text\": '<thinking>\\nBased on the user\\'s question, the relevant function to call is GetWeather, which requires the \"location\" parameter.\\n\\nThe user has directly specified the location as \"San Francisco\". Since San Francisco is a well known city, I can reasonably infer they mean San Francisco, CA without needing the state specified.\\n\\nAll the required parameters are provided, so I can proceed with the API call.\\n</thinking>',\n",
" \"type\": \"text\",\n",
" },\n",
" {\n",
" \"type\": \"tool_use\",\n",
" \"id\": \"toolu_01SCgExKzQ7eqSkMHfygvYuu\",\n",
" \"name\": \"GetWeather\",\n",
" \"input\": {\"location\": \"San Francisco, CA\"},\n",
" \"text\": None,\n",
" },\n",
" ],\n",
" ),\n",
" ToolMessage(\n",
" \"Rain. High 54F. Winds SW at 15 to 25 mph. Chance of rain 100%.\",\n",
" tool_call_id=\"toolu_01SCgExKzQ7eqSkMHfygvYuu\",\n",
" ),\n",
"]\n",
"llm_with_tools.invoke(messages)"
]
},
{
"cell_type": "markdown",
"id": "1c82d198-77ce-4d5a-a65b-a98fd3c10740",
"metadata": {},
"source": [
"### Streaming\n",
"\n",
"::: {.callout-warning}\n",
"\n",
"Anthropic does not currently support streaming tool calls. Attempting to stream will yield a single final message.\n",
"\n",
":::"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "d1284ddc-eb82-44be-b034-5046809536de",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/bagatur/langchain/libs/partners/anthropic/langchain_anthropic/chat_models.py:328: UserWarning: stream: Tool use is not yet supported in streaming mode.\n",
" warnings.warn(\"stream: Tool use is not yet supported in streaming mode.\")\n"
]
},
{
"data": {
"text/plain": [
"[AIMessage(content=[{'text': '<thinking>\\nThe user is asking for the current weather in a specific location, San Francisco. The GetWeather function is the relevant tool to answer this request, as it returns the current weather for a given location.\\n\\nThe GetWeather function has one required parameter:\\nlocation: The city and state, e.g. San Francisco, CA\\n\\nThe user provided the city San Francisco in their request. They did not specify the state, but it can be reasonably inferred that they are referring to San Francisco, California since that is the most well known city with that name.\\n\\nSince the required location parameter has been provided by the user, we can proceed with calling the GetWeather function.\\n</thinking>', 'type': 'text'}, {'text': None, 'type': 'tool_use', 'id': 'toolu_01V9ZripoQzuY8HubspJy6fP', 'name': 'GetWeather', 'input': {'location': 'San Francisco, CA'}}], id='run-b825206b-5b6b-48bc-ad8d-802dee310c7f')]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"list(llm_with_tools.stream(\"What's the weather in san francisco\"))"
]
}
],
"metadata": {
@@ -313,7 +711,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
"version": "3.9.1"
}
},
"nbformat": 4,

2
docs/docs/integrations/platforms/anthropic.mdx
View File

@@ -31,7 +31,7 @@ model = ChatAnthropic(model='claude-3-opus-20240229')
Read more in the [ChatAnthropic documentation](/docs/integrations/chat/anthropic).
## `AnthropicLLM`
## [Legacy] `AnthropicLLM`
`AnthropicLLM` is a subclass of LangChain's `LLM`. It is a wrapper around Anthropic's
text-based completion endpoints.

1
docs/docs/modules/model_io/chat/function_calling.mdx
View File

@@ -72,6 +72,7 @@ import ChatModelTabs from "@theme/ChatModelTabs";
customVarName="llm"
fireworksParams={`model="accounts/fireworks/models/firefunction-v1", temperature=0`}
hideGoogle={true}
hideAnthropic={true}
/>
We can use the `bind_tools()` method to handle converting

2
docs/docs/modules/model_io/chat/quick_start.ipynb
View File

@@ -783,7 +783,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.1"
"version": "3.9.1"
}
},
"nbformat": 4,

44
docs/docs/modules/model_io/chat/structured_output.ipynb
View File

@@ -39,7 +39,7 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 2,
"id": "08029f4e",
"metadata": {},
"outputs": [],
@@ -49,7 +49,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 3,
"id": "070bf702",
"metadata": {},
"outputs": [],
@@ -481,6 +481,44 @@
" \"Tell me a joke about cats, respond in JSON with `setup` and `punchline` keys\"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "f94e9c7a-bfbd-409c-b3a6-59e485e4ea5b",
"metadata": {},
"source": [
"## Anthropic\n",
"\n",
"Anthropic's tool-calling API can be used for structuring outputs. Note that there is currently no way to force a tool-call via the API, so prompting the model correctly is still important."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "12682237-6689-4408-88b1-3595feac447f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Joke(setup='What do you call a cat that loves to bowl?', punchline='An alley cat!')"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_anthropic import ChatAnthropic\n",
"\n",
"model = ChatAnthropic(\n",
" model=\"claude-3-opus-20240229\",\n",
" default_headers={\"anthropic-beta\": \"tools-2024-04-04\"},\n",
")\n",
"model_with_structure = model.with_structured_output(Joke)\n",
"model_with_structure.invoke(\"Tell me a joke about cats\")"
]
}
],
"metadata": {
@@ -499,7 +537,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.1"
"version": "3.9.1"
}
},
"nbformat": 4,