mirror of
https://github.com/hwchase17/langchain.git
synced 2026-07-11 09:26:29 +00:00
fix(langchain): make write_todos calls tool-only to stop duplicated plan text
Add a boundary rule to the `TodoListMiddleware` default prompt and tool description: when calling `write_todos`, the assistant message must be tool-only. User-facing plan text, status updates, approval questions, and final answers should not appear in the same message as the call — they are sent once, after the tool result returns. This stops streaming consumers from showing the same plan/question twice (before and after the call) while preserving #37643's contract that the final answer lands after the call. Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
This commit is contained in:
@@ -112,9 +112,13 @@ It is important to skip using this tool when:
|
||||
Being proactive with task management ensures you complete all requirements successfully
|
||||
Remember: If you only need to make a few tool calls to complete a task, and it is clear what you need to do, it is better to just do the task directly and NOT call this tool at all.
|
||||
|
||||
## Keep `write_todos` Calls Tool-Only
|
||||
|
||||
A `write_todos` call updates the todo state — it does not communicate with the user. When you call `write_todos`, the assistant message must contain the tool call only. Do NOT include any user-facing content — plan text, status updates, approval questions, or final answers — in the same message as the call. Send that user-facing content once, in a message AFTER the tool result returns, never before or alongside the call. Putting the same plan or question both before and after the tool call shows the user a duplicated message.
|
||||
|
||||
## When You Finish
|
||||
|
||||
`write_todos` tracks your work; it does not deliver the answer. Whatever the user asked for — computations, summaries, comparisons, data — must appear as text content in a message after your final `write_todos` call. Marking the last todo complete is not itself an answer to the user.""" # noqa: E501
|
||||
`write_todos` tracks your work; it does not deliver the answer. Whatever the user asked for — computations, summaries, comparisons, data — must appear as text content in a message after your final `write_todos` call, not before or alongside it. Marking the last todo complete is not itself an answer to the user.""" # noqa: E501
|
||||
|
||||
WRITE_TODOS_SYSTEM_PROMPT = """## `write_todos`
|
||||
|
||||
@@ -130,10 +134,11 @@ Writing todos takes time and tokens, use it when it is helpful for managing comp
|
||||
|
||||
- The `write_todos` tool should never be called multiple times in parallel.
|
||||
- Don't be afraid to revise the To-Do list as you go. New information may reveal new tasks that need to be done, or old tasks that are irrelevant.
|
||||
- Keep `write_todos` calls tool-only. When you call `write_todos`, do NOT put user-facing plan text, status updates, approval questions, or final answers in the same assistant message as the tool call. The tool call updates the todo state; the user-facing message comes once, after the tool result returns. Emitting the same plan or question both before and alongside the call and again afterward shows the user a duplicated message.
|
||||
|
||||
## Finishing a task
|
||||
|
||||
When you finish all work, write your final answer in the message AFTER your last `write_todos` call — not in the same turn as that call. Start the final message with the substantive content the user asked for — the data, computation, summary, or analysis. The user wants the result, not confirmation that the work is done.""" # noqa: E501
|
||||
When you finish all work, write your final answer in the message AFTER your last `write_todos` call — not before or in the same turn as that call. Start the final message with the substantive content the user asked for — the data, computation, summary, or analysis. The user wants the result, not confirmation that the work is done.""" # noqa: E501
|
||||
|
||||
|
||||
@tool(description=WRITE_TODOS_TOOL_DESCRIPTION)
|
||||
|
||||
@@ -79,6 +79,30 @@ def test_todo_middleware_default_prompts() -> None:
|
||||
assert tool.description == WRITE_TODOS_TOOL_DESCRIPTION
|
||||
|
||||
|
||||
def test_system_prompt_mentions_tool_only_boundary() -> None:
|
||||
"""System prompt tells the model not to duplicate user-facing text around the call."""
|
||||
prompt = WRITE_TODOS_SYSTEM_PROMPT.lower()
|
||||
# The tool-only boundary rule must be present (fixes duplicated plan text).
|
||||
assert "tool-only" in prompt
|
||||
# It must forbid user-facing content in the same message as the call.
|
||||
assert "same assistant message as the tool call" in prompt
|
||||
# It must preserve the #37643 contract: final answer lands after the call.
|
||||
assert "AFTER your last `write_todos` call" in WRITE_TODOS_SYSTEM_PROMPT
|
||||
assert "not before or in the same turn" in WRITE_TODOS_SYSTEM_PROMPT
|
||||
|
||||
|
||||
def test_tool_description_mentions_tool_only_boundary() -> None:
|
||||
"""Tool description tells the model the call is tool-only with respect to text."""
|
||||
description = WRITE_TODOS_TOOL_DESCRIPTION
|
||||
# The dedicated tool-only section must exist.
|
||||
assert "Keep `write_todos` Calls Tool-Only" in description
|
||||
# It must call out each kind of user-facing content that should not be duplicated.
|
||||
for fragment in ("plan text", "status updates", "approval questions", "final answers"):
|
||||
assert fragment in description
|
||||
# It must preserve the #37643 contract: answer after the tool result, not before.
|
||||
assert "after your final `write_todos` call, not before or alongside it" in description
|
||||
|
||||
|
||||
def test_adds_system_prompt_when_none_exists() -> None:
|
||||
"""Test that middleware adds system prompt when request has none."""
|
||||
middleware = TodoListMiddleware()
|
||||
|
||||
Reference in New Issue
Block a user