Skip to content

feat: add failed_summary option to list failed tests with logs - #1622

Merged
mikepenz merged 3 commits into
mikepenz:mainfrom
tchiotludo:feat/failed-summary
Oct 5, 2026
Merged

mikepenz merged 3 commits into
mikepenz:mainfrom
tchiotludo:feat/failed-summary

Conversation

@tchiotludo

@tchiotludo tchiotludo commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Adds optional failed-test logs to the detailed summary. Existing configurations keep failed tests in the table by default.

  • failed_summary, default true, controls whether failed tests appear in the detailed summary. Requires detailed_summary.
  • failed_summary_logs, default false, moves failed tests from table rows into collapsible log blocks. Requires detailed_summary and failed_summary. Durations follow include_time_in_summary.

Applies to the job summary, PR comment, and detailed_summary HTML output. Multiple failures of one testcase share a block, and each block retains its check name. The failure section is capped at 50,000 characters and respects the remaining PR comment space. Titles take priority over logs; logs are rendered within the remaining budget before HTML escaping. Nested failures and Windows-qualified titles are covered.

Validation: 102 tests passed, build, lint, changed-file formatting, packaging, and git diff --check. Compared 8,192 default configurations against main and verified byte-identical default comment and job-summary output. Regression coverage includes oversized logs, narrow comment budgets, 10,000 hidden-entry counts, Windows paths, nested-only failures, and identical failures from different checks.

Copilot AI balanced review requested due to automatic review settings October 4, 2026 21:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Failed-test grouping, cross-platform paths, and unbounded log payloads can produce incorrect output or publishing failures.

Review effort: Balanced
Findings: 1 High severity Β· 2 Medium severity Β· 1 Low severity

Open (4)
What changed in this PR

Adds configurable failed-test details to summaries, comments, and outputs.

Changes:

  • Adds failed_summary and failed_summary_logs inputs.
  • Renders escaped failed-test details and logs.
  • Adds table tests and rebuilds the distribution bundle.
File Description
src/​utils.ts Adds HTML escaping.
src/​table.ts Builds failed-test sections.
src/​main.ts Reads inputs and publishes sections.
src/​annotator.ts Adds sections to summaries/comments.
README.md Documents new inputs.
action.yml Declares new action inputs.
__tests__/​table.test.ts Tests failed-summary rendering.
dist/​index.js Updates bundled action code.

πŸ’‘ Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/table.ts Resolved
Comment thread src/table.ts Resolved
Comment thread src/table.ts Resolved
Comment thread README.md Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 21:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum diff size. Try reducing the number of changed files and lines, and requesting a review from Copilot again.

Copilot AI balanced review requested due to automatic review settings October 4, 2026 22:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Failed-test filtering and grouping currently produce incorrect summary behavior.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)

Comment thread src/table.ts Resolved
Comment thread src/table.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 22:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Failed-test filtering and grouping currently produce incorrect output for disabled summaries, failure-only suites, flaky failures, and multi-path failures.

Review effort: Balanced
Findings: 3 Medium severity

Open (3)

Comment thread src/table.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 22:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Comment-size budgeting remains unsafe, and directory-qualified test titles are rendered redundantly.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
Resolved since last review (3)

Comment thread src/table.ts Resolved
Comment thread src/table.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 22:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Comment truncation can exceed the API limit and report an incorrect number of hidden failures.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (2)

Comment thread src/annotator.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 22:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Comment truncation can still exceed GitHub's size limit when little space remains.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread src/annotator.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 22:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Nested failures can be omitted, verbose rows can be erased, and PR comments can still exceed the size limit.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)

Comment thread src/table.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 22:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Comment sizing and detail rendering still contain reproducible edge-case bugs.

Review effort: Balanced
Findings: 4 Medium severity Β· 1 Low severity

Open (5)

Comment thread src/table.ts Resolved
Comment thread src/table.ts Resolved
Comment thread __tests__/table.test.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 4, 2026 23:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Detail cleanup can discard valid rows, and comment fitting can still exceed GitHub’s size limit.

Review effort: Balanced
Findings: 5 Medium severity Β· 1 Low severity

Open (6)

Comment thread src/annotator.ts Resolved
@mikepenz
mikepenz changed the base branch from main to fix/npm-audit-vulnerabilities October 5, 2026 08:18
@mikepenz
mikepenz deleted the branch mikepenz:main October 5, 2026 08:18
@mikepenz mikepenz closed this Oct 5, 2026
@mikepenz

mikepenz commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Oh. the closing of this PR was not intentional.

@mikepenz mikepenz reopened this Oct 5, 2026
@mikepenz
mikepenz changed the base branch from fix/npm-audit-vulnerabilities to main October 5, 2026 08:20
@mikepenz
mikepenz force-pushed the feat/failed-summary branch from ddea25c to d9fa2e4 Compare October 5, 2026 08:21
@tchiotludo

Copy link
Copy Markdown
Contributor Author

Thanks for managing so quickly! really appreciate!
Feel free to squash everything at the end, it was my first battle with copilot reviewer!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟑 Changes recommended

Windows paths are rendered incorrectly, and log rendering performs unbounded intermediate allocations.

Review effort: Balanced
Findings: 1 High severity Β· 3 Medium severity Β· 1 Low severity

Open (5)

Comment thread src/table.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 5, 2026 08:22
@mikepenz

mikepenz commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Thanks for managing so quickly! really appreciate! Feel free to squash everything at the end, it was my first battle with copilot reviewer!

Yeah copilot loves to do endless loops of feedback :D

Thank you very much for the PR and the proposed new feature

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread README.md Resolved
Comment thread action.yml Resolved
Copilot AI balanced review requested due to automatic review settings October 5, 2026 08:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/table.ts Resolved
mikepenz and others added 3 commits October 5, 2026 11:01
Preserve existing summary defaults and check context. Group testcase failures and fit titles before logs within each destination limit.

Co-authored-by: Ludovic DEHON <tchiot.ludo@gmail.com>
Copilot AI balanced review requested due to automatic review settings October 5, 2026 09:01
@mikepenz
mikepenz force-pushed the feat/failed-summary branch from 10ad22b to 767bd31 Compare October 5, 2026 09:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟒 Approval recommended

The implementation is bounded, backward-compatible by default, and thoroughly tested across relevant edge cases.

Review effort: Balanced
Findings: None

Resolved since last review (5)

@mikepenz
mikepenz merged commit b0100d5 into mikepenz:main Oct 5, 2026
6 checks passed
@tchiotludo

Copy link
Copy Markdown
Contributor Author

You are fast !! ⏩
Do you plan to release it soon ? (no rush, just to plan on my side if I will use develop in a meantime)

@mikepenz

mikepenz commented Oct 5, 2026

Copy link
Copy Markdown
Owner

You are fast !! ⏩ Do you plan to release it soon ? (no rush, just to plan on my side if I will use develop in a meantime)

Yes, can't say when exactly, but maybe even today.

@tchiotludo

Copy link
Copy Markdown
Contributor Author

thanks for the really fast release, btw !

@mikepenz

mikepenz commented Oct 5, 2026

Copy link
Copy Markdown
Owner

You are welcome! thanks again for the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants