github/privateGPT
1
0
Fork 1
mirror of https://github.com/imartinez/privateGPT.git synced 2026-07-16 17:00:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

perf(prompts): use date-level current_date to preserve prompt-prefix caching (#2271)

Browse Source 
create_chat_header_prompt rendered current_date with sub-second
datetime.now().isoformat() into the system prompt header, before the
guidelines and retrieved-context blocks. Since it changes every request,
the system-prompt prefix is never byte-stable, which defeats LLM
prompt-prefix caching (OpenAI automatic prefix caching, Anthropic
cache_control, local KV-cache reuse).

Use date-level granularity; the model only needs the calendar date for
relative-date reasoning and the prefix stays stable within a day.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
rokieg77-alt
2026-06-16 16:58:47 +08:00
committed by GitHub
parent 4c0c500bbd
commit a64f1c9cf0
1 changed files with 6 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
private_gpt/components/prompts/prompt_builder.py
View File

@@ -117,7 +117,12 @@ class PromptBuilderService:
assistant_name=assistant_name or settings().chat.assistant_name,
assistant_description=assistant_description
or settings().chat.assistant_description,
current_date=current_date.astimezone().isoformat(),
# Use date-level granularity: a sub-second timestamp here changes on
# every request and sits at the front of the system prompt (before the
# guidelines and retrieved context), which defeats LLM prompt-prefix
# caching (e.g. OpenAI automatic prefix caching, local KV-cache reuse).
# The model only needs the calendar date for relative-date reasoning.
current_date=current_date.astimezone().date().isoformat(),
)
def create_chat_context_for_system_prompt(