mirror of
https://github.com/hwchase17/langchain.git
synced 2026-07-01 22:59:06 +00:00
feat(model-profiles): plain-English summary for profile refresh PRs (#38218)
Automated model-profile refresh PRs (e.g. #38210) ship a static template body, so a reviewer has to open *Files changed* and read large blocks of generated data to learn what actually moved. Because the underlying profile data is fully structured, we can describe the changes deterministically — no LLM, no hallucination risk. This adds a `langchain-profiles summarize` subcommand that compares the working-tree `_profiles.py` files against a git ref and renders a skimmable Markdown summary: models added (with a short capability descriptor), models removed, and per-field capability changes (context/output tokens, modalities, tool calling, reasoning, etc.), grouped by provider and capped so huge refreshes stay readable. Profiles are read with `ast.literal_eval` rather than imported, so the generated data file is never executed. Example output for a refresh that adds a model and bumps an output limit: ``` ## Summary of changes **1 added · 0 removed · 1 changed** across 1 provider(s). ### openai **➕ 1 added** - `gpt-6-preview` — 1,000,000 ctx, 128,000 out, text+image+audio in, reasoning, tools **✏️ 1 changed** - `gpt-3.5-turbo`: max output tokens 4,096 → 16,384 ``` Made by [Open SWE](https://openswe.vercel.app/agents/9bcbf182-effc-ba9b-0df3-afac620ad152) --------- Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
This commit is contained in:
28
.github/workflows/_refresh_model_profiles.yml
vendored
28
.github/workflows/_refresh_model_profiles.yml
vendored
@@ -167,6 +167,32 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: "📝 Build PR body with change summary"
|
||||
id: pr-body
|
||||
env:
|
||||
PROVIDERS_JSON: ${{ inputs.providers }}
|
||||
PR_BODY: ${{ inputs.pr-body }}
|
||||
run: |
|
||||
# The refresh step modified the working tree without committing, so
|
||||
# comparing against HEAD yields exactly the refresh's changes.
|
||||
cli_dir="${GITHUB_WORKSPACE}/${{ steps.cli.outputs.dir }}"
|
||||
body_file="${RUNNER_TEMP}/pr_body.md"
|
||||
printf '%s\n\n' "${PR_BODY}" > "${body_file}"
|
||||
# `summarize` builds the whole summary in memory and prints it once,
|
||||
# so a failure exits non-zero before any stdout reaches the append —
|
||||
# the body keeps only the static note, never a half-written summary.
|
||||
if ! uv run --frozen --project "${cli_dir}" \
|
||||
langchain-profiles summarize \
|
||||
--providers "${PROVIDERS_JSON}" \
|
||||
--base-ref HEAD \
|
||||
--repo-root "${GITHUB_WORKSPACE}" >> "${body_file}"; then
|
||||
echo "::warning::Could not generate change summary; see job log."
|
||||
# Surface the degradation in the PR body too: the warning above only
|
||||
# lands in the Actions log, which a PR reviewer won't see.
|
||||
printf '\n> [!NOTE]\n> Automated change summary unavailable — see the workflow run log.\n' >> "${body_file}"
|
||||
fi
|
||||
echo "path=${body_file}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: "🔑 Generate GitHub App token"
|
||||
id: app-token
|
||||
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3
|
||||
@@ -182,7 +208,7 @@ jobs:
|
||||
branch: ${{ inputs.pr-branch }}
|
||||
commit-message: ${{ inputs.pr-title }}
|
||||
title: ${{ inputs.pr-title }}
|
||||
body: ${{ inputs.pr-body }}
|
||||
body-path: ${{ steps.pr-body.outputs.path }}
|
||||
labels: ${{ inputs.pr-labels }}
|
||||
add-paths: ${{ inputs.add-paths }}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user