diff --git a/docs/docs/concepts/structured_outputs.mdx b/docs/docs/concepts/structured_outputs.mdx index d6d31a5d215..2c0d31d632b 100644 --- a/docs/docs/concepts/structured_outputs.mdx +++ b/docs/docs/concepts/structured_outputs.mdx @@ -29,6 +29,22 @@ model_with_structure = model.with_structured_output(schema) structured_output = model_with_structure.invoke(user_input) ``` +:::warning[Tool Order Matters] + +When combining structured output with additional tools, bind tools **first**, then apply structured output: + +```python +# Correct +model_with_tools = model.bind_tools([tool1, tool2]) +structured_model = model_with_tools.with_structured_output(schema) + +# Incorrect - will cause tool resolution errors +structured_model = model.with_structured_output(schema) +broken_model = structured_model.bind_tools([tool1, tool2]) +``` + +::: + ## Schema definition The central concept is that the output structure of model responses needs to be represented in some way. diff --git a/docs/docs/how_to/structured_output.ipynb b/docs/docs/how_to/structured_output.ipynb index ae647d70781..fa2fe54b1f1 100644 --- a/docs/docs/how_to/structured_output.ipynb +++ b/docs/docs/how_to/structured_output.ipynb @@ -998,6 +998,91 @@ "\n", "chain.invoke({\"query\": query})" ] + }, + { + "cell_type": "markdown", + "id": "xfejabhtn2", + "metadata": {}, + "source": [ + "## Combining with Additional Tools\n", + "\n", + "When you need to use both structured output and additional tools (like web search), note the order of operations:\n", + "\n", + "**Correct Order**:\n", + "```python\n", + "# 1. Bind tools first\n", + "llm_with_tools = llm.bind_tools([web_search_tool, calculator_tool])\n", + "\n", + "# 2. Apply structured output\n", + "structured_llm = llm_with_tools.with_structured_output(MySchema)\n", + "```\n", + "\n", + "**Incorrect Order**:\n", + "\n", + "```python\n", + "# This will fail with \"Tool 'MySchema' not found\" error\n", + "structured_llm = llm.with_structured_output(MySchema)\n", + "broken_llm = structured_llm.bind_tools([web_search_tool])\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "653798ca", + "metadata": {}, + "source": [ + "**Why Order Matters:**\n", + "`with_structured_output()` internally uses tool calling to enforce the schema. When you bind additional tools afterward, it creates a conflict in the tool resolution system." + ] + }, + { + "cell_type": "markdown", + "id": "1345f4a4", + "metadata": {}, + "source": [ + "**Complete Example:**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0835637b", + "metadata": {}, + "outputs": [], + "source": [ + "from pydantic import BaseModel, Field\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "\n", + "class SearchResult(BaseModel):\n", + " \"\"\"Structured search result.\"\"\"\n", + "\n", + " query: str = Field(description=\"The search query\")\n", + " findings: str = Field(description=\"Summary of findings\")\n", + "\n", + "\n", + "# Define tools\n", + "search_tool = {\n", + " \"type\": \"function\",\n", + " \"function\": {\n", + " \"name\": \"web_search\",\n", + " \"description\": \"Search the web for information\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\"query\": {\"type\": \"string\", \"description\": \"Search query\"}},\n", + " \"required\": [\"query\"],\n", + " },\n", + " },\n", + "}\n", + "\n", + "# Correct approach\n", + "llm = ChatOpenAI()\n", + "llm_with_search = llm.bind_tools([search_tool])\n", + "structured_search_llm = llm_with_search.with_structured_output(SearchResult)\n", + "\n", + "# Now you can use both search and get structured output\n", + "result = structured_search_llm.invoke(\"Search for latest AI research and summarize\")" + ] } ], "metadata": { diff --git a/docs/docs/integrations/chat/anthropic.ipynb b/docs/docs/integrations/chat/anthropic.ipynb index 6e2c0283862..bcd6637ce5e 100644 --- a/docs/docs/integrations/chat/anthropic.ipynb +++ b/docs/docs/integrations/chat/anthropic.ipynb @@ -1240,6 +1240,58 @@ "response = llm_with_tools.invoke(\"How do I update a web app to TypeScript 5.5?\")" ] }, + { + "cell_type": "markdown", + "id": "kloc4rvd1w", + "metadata": {}, + "source": [ + "#### Web search + structured output\n", + "\n", + "When combining web search tools with structured output, it's important to **bind the tools first and then apply structured output**:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "rjjergy6ef", + "metadata": {}, + "outputs": [], + "source": [ + "from pydantic import BaseModel, Field\n", + "from langchain_anthropic import ChatAnthropic\n", + "\n", + "\n", + "# Define structured output schema\n", + "class ResearchResult(BaseModel):\n", + " \"\"\"Structured research result from web search.\"\"\"\n", + "\n", + " topic: str = Field(description=\"The research topic\")\n", + " summary: str = Field(description=\"Summary of key findings\")\n", + " key_points: list[str] = Field(description=\"List of important points discovered\")\n", + "\n", + "\n", + "# Configure web search tool\n", + "websearch_tools = [\n", + " {\n", + " \"type\": \"web_search_20250305\",\n", + " \"name\": \"web_search\",\n", + " \"max_uses\": 10,\n", + " }\n", + "]\n", + "\n", + "llm = ChatAnthropic(model=\"claude-3-5-sonnet-20241022\")\n", + "\n", + "# Correct order: bind tools first, then structured output\n", + "llm_with_search = llm.bind_tools(websearch_tools)\n", + "research_llm = llm_with_search.with_structured_output(ResearchResult)\n", + "\n", + "# Now you can use both web search and get structured output\n", + "result = research_llm.invoke(\"Research the latest developments in quantum computing\")\n", + "print(f\"Topic: {result.topic}\")\n", + "print(f\"Summary: {result.summary}\")\n", + "print(f\"Key Points: {result.key_points}\")" + ] + }, { "cell_type": "markdown", "id": "1478cdc6-2e52-4870-80f9-b4ddf88f2db2",