ci: reduce pr-severity bot comment spam

Only post a severity classification comment when the bot hasn't
commented before or when the severity actually changed. Previously
every push (synchronize event) would post a new comment even if the
classification was identical.

The prompt now instructs the classifier to:
- Check for existing bot comments via the pr-severity-bot marker
- Compare the new severity against the existing severity label
- Skip commenting if both match, while still ensuring labels are correct
- Include a severity changed banner when re-commenting due to a change
This commit is contained in:
Olaoluwa Osuntokun committed 2026-02-04 10:02:53 -08:00
1 parent 4d775457c5
commit 0871193cdf
1 file changed
+30 -7
+30 -7
View File
@@ -125,21 +125,37 @@ jobs:
## Steps ## Steps
1. First, check for existing override labels: 1. First, check for existing override labels AND existing severity labels:
``` ```
gh pr view ${{ github.event.pull_request.number }} --json labels --jq '.labels[].name' gh pr view ${{ github.event.pull_request.number }} --json labels --jq '.labels[].name'
``` ```
Note which `severity-*` label (if any) is currently applied. This is
the "previous severity".
2. If an override label exists (severity-override-*), use that level and skip classification. 2. If an override label exists (severity-override-*), use that level and skip classification.
3. Get the list of changed files: 3. Check for existing bot comments. Look for the HTML marker `<!-- pr-severity-bot -->`:
```
gh pr view ${{ github.event.pull_request.number }} --json comments --jq '.comments[].body' | grep -c 'pr-severity-bot' || true
```
This tells you whether the bot has commented before.
4. Get the list of changed files:
``` ```
gh pr view ${{ github.event.pull_request.number }} --json files,additions,deletions gh pr view ${{ github.event.pull_request.number }} --json files,additions,deletions
``` ```
4. Classify each file and determine overall severity. 5. Classify each file and determine the new overall severity.
5. Remove any existing severity-* labels (not override labels): 6. **Decide whether to comment.** Only post a comment if EITHER:
- The bot has NOT commented before (no existing comment with `<!-- pr-severity-bot -->`), OR
- The newly determined severity is DIFFERENT from the previous severity label.
If the bot already commented AND the severity has NOT changed, just
stop here — do NOT post another comment. Still update the label if
needed (step 7-8), but skip the comment.
7. Remove any existing severity-* labels (not override labels):
``` ```
gh pr edit ${{ github.event.pull_request.number }} --remove-label "severity-critical" 2>/dev/null || true gh pr edit ${{ github.event.pull_request.number }} --remove-label "severity-critical" 2>/dev/null || true
gh pr edit ${{ github.event.pull_request.number }} --remove-label "severity-high" 2>/dev/null || true gh pr edit ${{ github.event.pull_request.number }} --remove-label "severity-high" 2>/dev/null || true
@@ -147,12 +163,16 @@ jobs:
gh pr edit ${{ github.event.pull_request.number }} --remove-label "severity-low" 2>/dev/null || true gh pr edit ${{ github.event.pull_request.number }} --remove-label "severity-low" 2>/dev/null || true
``` ```
6. Apply the new severity label: 8. Apply the new severity label:
``` ```
gh pr edit ${{ github.event.pull_request.number }} --add-label "severity-<level>" gh pr edit ${{ github.event.pull_request.number }} --add-label "severity-<level>"
``` ```
7. Post a comment with your analysis. Use this format: 9. If you determined in step 6 that a comment should be posted, post it
with your analysis. Use this format:
If this is a severity CHANGE (previous label existed but differs),
prepend: `> ⚠️ Severity changed: **<OLD>** → **<NEW>** (files changed since last classification)`
```markdown ```markdown
## <emoji> PR Severity: **<LEVEL>** ## <emoji> PR Severity: **<LEVEL>**
@@ -178,11 +198,14 @@ jobs:
<!-- pr-severity-bot --> <!-- pr-severity-bot -->
``` ```
8. Post the comment using `gh pr comment`: 10. Post the comment using `gh pr comment`:
``` ```
gh pr comment ${{ github.event.pull_request.number }} --body "YOUR_COMMENT_HERE" gh pr comment ${{ github.event.pull_request.number }} --body "YOUR_COMMENT_HERE"
``` ```
11. If you decided in step 6 to SKIP commenting, do NOT post any comment.
Just ensure the label is correct and exit.
## Emoji Mapping ## Emoji Mapping
- critical: 🔴 - critical: 🔴
- high: 🟠 - high: 🟠