Skip to content

fix(cli): sample only cobra command files for terminal UX - #5343

Merged
mergify[bot] merged 2 commits into
mainfrom
tmchow/terminal-ux-command-sample-9e06
Nov 5, 2026
Merged

mergify[bot] merged 2 commits into
mainfrom
tmchow/terminal-ux-command-sample-9e06

Conversation

@tmchow

@tmchow tmchow commented Nov 5, 2026

Copy link
Copy Markdown
Owner

Intent

terminal_ux sampled the wrong files, so a correct CLI lost description and example points with no fix that did not make the CLI worse.

Closes #4278

Approach

sampleCommandFiles now skips *_test.go and keeps a file only when it contains a cobra.Command composite literal. The check uses go/parser, so a comment, a string, a *cobra.Command signature, or another package's Command type does not count. infraAllFiles skips are unchanged. Description and example thresholds are unchanged.

The same helper also feeds path validity, type fidelity, and the agent-native prompt/example scan, so those callers now see command files only. On a fresh golden-api print, those dimension scores did not move. Unit fixtures that scored flags or paths in a file with no command literal now include one, so they still exercise the original assertions.

Already fixed: mcp_token_efficiency

The other half of #4278 is already on main and was left alone. estimateMCPTokens in internal/pipeline/mcp_size.go counts string literals only inside each mcplib.NewTool(...) registration, bounded by mcpNewToolCallEnd, not handler bodies. That landed in #4686 (fix(cli): count only NewTool catalog text for MCP tokens). TestEstimateMCPTokens_IgnoresHandlerLiterals pins a 4000-character handler string at 10/10, and the golden-api scorecard already records mcp_token_efficiency: 10 when the registration-region average is inside the <=80 tokens/tool band.

Scope

Primary area: scorer (internal/pipeline/scorecard.go)

Why this belongs in this repo: the sample is taken from every printed CLI. An early *_test.go and a non-command helper such as auto_refresh.go were universal, so the lost point was not API-specific.

Risk

A bounded sample can now reach later command files, so terminal_ux rises when the previous sample was filled by tests or non-commands. The golden-api scorecard moves terminal_ux from 9 to 10. The normalized total stays 97. Path validity, type fidelity, and agent-native scores on that print did not change.

Output Contract

Scorecard JSON for the golden-api fixture: terminal_ux 9 → 10. No template or generated Go changes.

Verification

  • go test ./internal/pipeline/ passed
  • GOOS=linux, GOOS=darwin, and GOOS=windows go build and go vet of ./internal/pipeline/ passed
  • Regenerated the golden-api scorecard and diffed it; the only change is terminal_ux

Self-review gate

Ran before opening this PR:

  • Removed comments that only restate the code. The remaining comments say why tests and non-literals are skipped.
  • No generator templates changed, so no generated-module compile check applies.
  • Golden scorecard updated only because scorecard output changed (terminal_ux 9 → 10). Templates were not touched.
  • Cross-platform vet/build of the touched package passed for linux, darwin, and windows.
  • Tests cover the new branches: skip *_test.go, require a cobra.Command composite, keep infra skips, parse failure, empty directory, comment-only mention, string literal, *cobra.Command signature, and a different package's Command type. Positive case: five well-described command files score the full description band (+2) even when test files and non-command files sort first. Negative cases: one-word Short: fields lose the description points (score 3, examples only); three described files stay on the partial band (score 2); one command file scores 0.

AI / Automation Disclosure

  • Fully automated: generated and submitted without human review for this specific change
Open in Web Open in Cursor 

*_test.go files and helpers with no cobra.Command literal were filling
the five-file description and example sample, so a correct CLI lost
points it could not earn back.

Co-authored-by: Trevin Chow <tmchow@users.noreply.github.com>
@mergify

mergify Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 require-ready-label-and-ci

  • #changes-requested-reviews-by = 0
  • #review-threads-unresolved = 0
  • check-success = build-and-test
  • check-success = generated-test
  • check-success = go-lint
  • check-success = golden
  • check-success = pr-title
  • check-success = test
  • any of:
    • label = ready-to-merge
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release
  • any of:
    • -files ~= ^(\.github/workflows/|\.github/scripts/|scripts/|\.github/CODEOWNERS$)
    • author = tmchow
    • approved-reviews-by = mvanhorn
    • approved-reviews-by = tmchow
    • author = mvanhorn
  • any of:
    • check-success = Greptile Review
    • label = queued
    • check-neutral = Greptile Review
    • check-skipped = Greptile Review
    • head ~= ^mergify/merge-queue/
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release

🟢 🚦 Auto-queue

When all merge protections are satisfied and these conditions match, this pull request will be queued automatically.

  • label = ready-to-merge

@greptile-apps

greptile-apps Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the previous alias issue is fixed and no new blocking issues were found.

Summary

The PR limits command samples to non-test files containing a Cobra command literal. The latest update resolves the imported package rather than assuming its name is cobra.

  • Scorecard samples now use files that define Cobra commands.

Reviews (2) · Last reviewed commit: "fix(cli): resolve cobra command literals..." · Reviewed by Greptile

Comment thread internal/pipeline/scorecard.go Resolved
sampleCommandFiles matched the selector name cobra instead of the
github.com/spf13/cobra import, so an aliased import was skipped and a
different package imported as cobra was counted.

Co-authored-by: Trevin Chow <tmchow@users.noreply.github.com>
@tmchow tmchow added the ready-to-merge Queue when Mergify passes; scripts/.github PRs also need Approve from tmchow/mvanhorn. label Nov 5, 2026
@mergify mergify Bot added the queued PR is in the Mergify merge queue label Nov 5, 2026
@mergify
mergify Bot merged commit 26d1e3f into main Nov 5, 2026
56 of 59 checks passed
@mergify
mergify Bot deleted the tmchow/terminal-ux-command-sample-9e06 branch November 5, 2026 15:12
@mergify

mergify Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

✅ Merged into main as 26d1e3f at 15:12 UTC

12 min in the queue, 11 min of it CI.

So far

13 merge conditions met
  • check-success = build-and-test · required by Merge Protections rule require-ready-label-and-ci
  • check-success = generated-test · required by Merge Protections rule require-ready-label-and-ci
  • check-success = go-lint · required by Merge Protections rule require-ready-label-and-ci
  • check-success = golden · required by Merge Protections rule require-ready-label-and-ci
  • check-success = test · required by Merge Protections rule require-ready-label-and-ci
  • github-review-approved · required by GitHub branch protection
  • #changes-requested-reviews-by = 0 · required by Merge Protections rule require-ready-label-and-ci
  • #review-threads-unresolved = 0 · required by Merge Protections rule require-ready-label-and-ci
  • check-success = pr-title · required by Merge Protections rule require-ready-label-and-ci
  • any of · required by Merge Protections rule require-ready-label-and-ci:
    • label = ready-to-merge
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release
  • any of · required by Merge Protections rule require-ready-label-and-ci:
    • -files ~= ^(\.github/workflows/|\.github/scripts/|scripts/|\.github/CODEOWNERS$)
    • author = tmchow
    • approved-reviews-by = mvanhorn
    • approved-reviews-by = tmchow
    • author = mvanhorn
  • any of · required by Merge Protections rule require-ready-label-and-ci:
    • label = queued
    • check-neutral = Greptile Review
    • check-skipped = Greptile Review
    • check-success = Greptile Review
    • head ~= ^mergify/merge-queue/
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release
  • any of · required by GitHub branch protection:
    • check-neutral = Mergify Merge Protections
    • check-skipped = Mergify Merge Protections
    • check-success = Mergify Merge Protections

Times in UTC · dashboard · how the merge queue works

@mergify mergify Bot removed the queued PR is in the Mergify merge queue label Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Queue when Mergify passes; scripts/.github PRs also need Approve from tmchow/mvanhorn.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scorer: mcp_token_efficiency and terminal_ux measure the wrong input, penalizing correct CLIs with no achievable fix

2 participants