Skip to content

fix(cli): re-fetch live dogfood writes to catch silent no-ops - #5334

Merged
mergify[bot] merged 3 commits into
mainfrom
tmchow/live-dogfood-write-readback-0f54
Nov 3, 2026
Merged

mergify[bot] merged 3 commits into
mainfrom
tmchow/live-dogfood-write-readback-0f54

Conversation

@tmchow

@tmchow tmchow commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Intent

Issue: Closes #4234

Live dogfood treated a mutating create/update's success status as proof the write persisted. A Redmine print's issues update returned {"status":204,"success":true} while the server ignored a flat body, and the acceptance report still claimed the write lifecycle was tested.

Approach

After a live create/update exits 0, dogfood records a separate write_readback row. When it can identify a sibling get/show and the resource id, it re-fetches and checks that one submitted field now holds the submitted value. For updates it reads the resource first, so a field that still holds its previous value fails with mutation reported success but read-back shows field "<name>" unchanged. An advanced modification timestamp counts as the write having landed.

The check stays skipped, and does not fail the matrix, when no sibling read exists, the id cannot be determined, the field is absent, the value was only normalized or transformed, another submitted field shows the write landed, the read itself fails, the response has more than one JSON value, or the write's connection target cannot be copied onto the read. Dry-run, preview, delete, and action commands are not checked. One matching field is enough; partial multi-field writes are not flagged.

Sibling reads reuse the write's connection and scope flags (config, profile, base URL, auth, TLS, and data source). Scalar flags follow pflag: the last value wins, a value-taking flag consumes the next argument even when that argument is --, and a boolean shorthand takes an explicit value only as -c=false (a stuck-on word is more shorthand letters; a cluster that would not parse is dropped). Combined shorthand clusters set each boolean letter and stop at a value-taking letter. Numeric equality is exact decimal or base-10 scientific text, including numeric strings; fractions, base prefixes, underscores, and binary exponents do not match.

Scope

Primary area: live dogfood (scorer/verify)

Why this belongs in this repo: the false pass is in the Printing Press live-dogfood runner, so every future print inherits the check.

Risk

A correct CLI fails this row only when a pre-read shows every comparable submitted field still holding its old value. Missing siblings, unrecognized transforms, conflicting JSON, an uncopied connection target, and failed reads are skips. Dry-run and preview rows are unchanged. Generated CLI output, MCP schemas, and goldens are unchanged.

Output Contract

N/A. This does not change templates, generated files, manifests, command output, MCP schemas, or scorecard output.

Verification

Self-review:

  • Comments that only restate the code were not added. The comments that remain explain why the baseline is captured before the write, why a mismatch fails only when every comparable field is unchanged, why a value flag takes the next argv even when it looks like a flag, why a connection flag with no value skips instead of reading another server, why a boolean shorthand value is only the = form, and why non-decimal spellings are not treated as the same number.
  • No generator template changed, so no generated module was compiled for this change and goldens were not regenerated.
  • GOOS=linux, GOOS=darwin, and GOOS=windows go vet ./internal/pipeline succeeded. Shared code and tests do not use Linux-only paths, permissions, sudo, or signals.
  • New branches and error paths have tests, including the negative case where a 204 update does not persist a flat body. Follow-up coverage: a repeated scalar flag keeps the last value; -- and other flag-looking tokens are flag values; a bool flag does not consume the next token; a repeated slice is not one comparable field; a missing connection value skips; neighboring integers above 2^53 do not match; newline-separated JSON records are inconclusive. Boolean shorthand = values, mixed long/short repetition, and clusters follow pflag; -cfalse is not the value false. Fractions, base prefixes, underscores, and p exponents do not match stored decimals or numeric strings.
  • Bypass attempts: an echoed id is not a submitted field; disagreeing id/ID values and trailing JSON are inconclusive; dry-run and preview do not run the check; a transformed sibling field does not fail an otherwise unchanged field.

Commands on this head:

  • go test ./internal/pipeline -count=1 passed.
  • golangci-lint run ./internal/pipeline (v2.11.4) passed.
  • GOOS=linux, GOOS=darwin, and GOOS=windows go vet ./internal/pipeline passed.

AI / Automation Disclosure

  • Fully automated: generated and submitted without human review for this specific change
Open in Web Open in Cursor 

Live dogfood treated a mutating command's success status as proof the
write landed. A sibling get/show now checks that one submitted field
persisted, and fails when a pre-read shows the field unchanged.

Closes #4234

Co-authored-by: Trevin Chow <tmchow@users.noreply.github.com>
@mergify

mergify Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 require-ready-label-and-ci

  • #changes-requested-reviews-by = 0
  • #review-threads-unresolved = 0
  • check-success = build-and-test
  • check-success = generated-test
  • check-success = go-lint
  • check-success = golden
  • check-success = pr-title
  • check-success = test
  • any of:
    • label = ready-to-merge
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release
  • any of:
    • -files ~= ^(\.github/workflows/|\.github/scripts/|scripts/|\.github/CODEOWNERS$)
    • author = tmchow
    • approved-reviews-by = mvanhorn
    • approved-reviews-by = tmchow
    • author = mvanhorn
  • any of:
    • check-success = Greptile Review
    • label = queued
    • check-neutral = Greptile Review
    • check-skipped = Greptile Review
    • head ~= ^mergify/merge-queue/
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release

🟢 🚦 Auto-queue

When all merge protections are satisfied and these conditions match, this pull request will be queued automatically.

  • label = ready-to-merge

@greptile-apps

greptile-apps Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; both numbered findings are fixed and no new blocking issue was found.

Summary

Adds a separate write_readback check after successful live creates and updates.

  • Live dogfood checks whether create and update commands actually save their fields.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Live create or update] --> B{Update?}
  B -->|Yes| C[Read resource before write]
  B -->|No| D[Run write]
  C --> D
  D --> E{Write passed?}
  E -->|No| F[No read-back row]
  E -->|Yes| G[Read resource after write]
  G --> H{Enough evidence?}
  H -->|No| I[Skip read-back row]
  H -->|Yes| J[Compare submitted fields]
  J --> K[Pass or fail read-back row]
Loading

Reviews (3) · Last reviewed commit: "fix(cli): match pflag boolean shorthands..." · Reviewed by Greptile

Scalar flags follow pflag: the last value wins, and a value flag takes
the next argv even when it is "--" or looks like a flag. Sibling reads
carry the write's connection flags, or skip when that target cannot be
copied. Number checks stay exact past 2^53, more than one JSON value is
inconclusive, and create/update are typed kinds.

Co-authored-by: Trevin Chow <tmchow@users.noreply.github.com>
Explicit shorthand values such as -c=false were dropped, so read-back
kept an earlier true and failed a write that had persisted. Numeric
comparison also treated fractions and other Rat spellings as equal to
different text.

Parse single-dash clusters with pflag's rules, and accept only decimal
and base-10 scientific text when comparing numbers.

Co-authored-by: Trevin Chow <tmchow@users.noreply.github.com>
@tmchow tmchow added the ready-to-merge Queue when Mergify passes; scripts/.github PRs also need Approve from tmchow/mvanhorn. label Nov 3, 2026
@mergify mergify Bot added the queued PR is in the Mergify merge queue label Nov 3, 2026
@mergify
mergify Bot merged commit 7f49e40 into main Nov 3, 2026
42 of 49 checks passed
@mergify
mergify Bot deleted the tmchow/live-dogfood-write-readback-0f54 branch November 3, 2026 16:30
@mergify

mergify Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

✅ Merged into main as 7f49e40 at 16:30 UTC

15 min in the queue, 14 min of it CI.

So far

13 merge conditions met
  • check-success = build-and-test · required by Merge Protections rule require-ready-label-and-ci
  • check-success = generated-test · required by Merge Protections rule require-ready-label-and-ci
  • check-success = go-lint · required by Merge Protections rule require-ready-label-and-ci
  • check-success = golden · required by Merge Protections rule require-ready-label-and-ci
  • check-success = test · required by Merge Protections rule require-ready-label-and-ci
  • github-review-approved · required by GitHub branch protection
  • #changes-requested-reviews-by = 0 · required by Merge Protections rule require-ready-label-and-ci
  • #review-threads-unresolved = 0 · required by Merge Protections rule require-ready-label-and-ci
  • check-success = pr-title · required by Merge Protections rule require-ready-label-and-ci
  • any of · required by Merge Protections rule require-ready-label-and-ci:
    • label = ready-to-merge
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release
  • any of · required by Merge Protections rule require-ready-label-and-ci:
    • -files ~= ^(\.github/workflows/|\.github/scripts/|scripts/|\.github/CODEOWNERS$)
    • author = tmchow
    • approved-reviews-by = mvanhorn
    • approved-reviews-by = tmchow
    • author = mvanhorn
  • any of · required by Merge Protections rule require-ready-label-and-ci:
    • label = queued
    • check-neutral = Greptile Review
    • check-skipped = Greptile Review
    • check-success = Greptile Review
    • head ~= ^mergify/merge-queue/
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release
  • any of · required by GitHub branch protection:
    • check-neutral = Mergify Merge Protections
    • check-skipped = Mergify Merge Protections
    • check-success = Mergify Merge Protections

Times in UTC · dashboard · how the merge queue works

@mergify mergify Bot removed the queued PR is in the Mergify merge queue label Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Queue when Mergify passes; scripts/.github PRs also need Approve from tmchow/mvanhorn.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Dogfood's write-lifecycle check trusts HTTP status code, not actual persistence

2 participants