Skip to content

Add /review performance GitHub Agentic Workflow for PR benchmarks - #38324

Merged
kubaflo merged 31 commits into
dotnet:mainfrom
morning4coffe-dev:feature/perf-analysis-ai-followup
Sep 30, 2026
Merged

kubaflo merged 31 commits into
dotnet:mainfrom
morning4coffe-dev:feature/perf-analysis-ai-followup

Conversation

@morning4coffe-dev

@morning4coffe-dev morning4coffe-dev commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description of Change

Adds a maintainer-triggered GitHub Agentic Workflow (gh-aw) for /review performance. The command runs selected managed benchmarks against pinned merge-base/head revisions, asks GPT to interpret the evidence and coverage gaps, and publishes one independently rendered and validated performance report.

It follows the /review tests command entrypoint and #38807's separation between untrusted execution and credentialed reporting. The PR is now scoped to this hosted workflow and its required managed benchmarking/reporting dependencies.

Hosted-only scope

Reduced from 63 changed files to 24, all under .github/. There are no remaining changes to framework source, device tests/runners, eng/, or docs/.

Retained: the workflow source/compiled lock, trusted orchestration, managed selector/ABBA runner/comparator, trusted managed benchmark overlays, coverage catalogs, GPT interpretation skill, deterministic decision/renderer/validator, focused managed/report checks, and command documentation/routing.

Removed: native-device infrastructure and drivers, the local check-pr-performance skill, native request/evidence ingestion, the history writer, duplicate benchmark sources under src/, and their dedicated tests/documentation. This PR no longer depends on #38274. The platform catalog only describes coverage gaps; it does not execute native scenarios or promise an available device driver.

Merged main without rewriting history. Preserved its removal of legacy rerun automation and moved the performance-command exclusion into the new shared recovery helper.

New GitHub Agentic Workflow

  • Source: .github/workflows/copilot-review-performance.md.
  • Compiled workflow: .github/workflows/copilot-review-performance.lock.yml, generated with gh-aw v0.86.2.
  • Trusted orchestrator: .github/scripts/Review-Performance.ps1 (Gather, Measure, Prepare, Render).
  • Interpretation skill/model: perf-analysis, using gpt-5.6-sol.

A current write/maintain/admin collaborator requests a review with a newly created comment on an open PR:

/review performance

Only this exact subcommand activates the performance workflow. Other subcommands, unauthorized comments, issues, closed PRs, and changes without product files do not start measurements. The full /review trigger and missed-command recovery explicitly exclude /review performance.

  1. Trusted intake: executes from the default branch, pins PR merge-base/head and harness SHAs, and selects coverage without executing PR code.
  2. Isolated measurements: a disposable GitHub-hosted Linux job runs managed BenchmarkDotNet benchmarks in ABBA order. Base/head use separate unprivileged users and an explicit child-environment allowlist. No Copilot PAT or posting token reaches benchmark builds/runs.
  3. Independent evidence and GPT interpretation: a fresh job imports bounded structured evidence and computes the decision baseline. GPT reads that bundle and the pinned diff and supplies narrative only, with no authority to execute more measurements or select another publication target.
  4. Validated gh-aw publication: a separate safe-output job downloads independent evidence, recomputes the decision, renders and validates the report, and rechecks caller permission plus live PR base/head identities before publishing. Older performance reports are hidden. Automation failures produce a run link, not a fabricated performance verdict.

Production uses the existing copilot-pat-pool environment for GPT credentials and the repository's GitHub Actions token for scoped publication. There is no new Azure pipeline, service connection, device queue, or scheduled recovery.

Manual dispatch and retries

After deployment to the default branch:

# Measure, interpret, render, and validate without posting.
gh workflow run copilot-review-performance.lock.yml \
  --repo dotnet/maui --ref main \
  -f pr_number=32382 -f suppress_output=true

# Run the same workflow and publish the validated report.
gh workflow run copilot-review-performance.lock.yml \
  --repo dotnet/maui --ref main \
  -f pr_number=32382 -f suppress_output=false

pr_number must identify an open PR. Dry-run mode suppresses publication, not measurements, GPT interpretation, or report validation. Context, structured evidence, reports, and separate SDK-install/build/benchmark diagnostics are retained for seven days. Diagnostic logs are not admitted as measurement evidence.

Retry with a fresh dispatch or a new command, not Re-run failed jobs. Repeated attempts can leave same-named gh-aw agent artifacts in one run, allowing a downstream job to select an earlier attempt's empty output. The decision resolver recommends a managed retry only for missing/incomplete execution or benchmark data; repeating completed unchanged filters cannot fill excluded benchmark-input or native coverage gaps.

Evidence policy

Evidence Treatment
Managed allocations Repeated non-overlapping ranges; default materiality thresholds of 8 bytes and 0.5%
Shared-host timing Advisory; requires non-overlapping ranges and at least 15% median movement
Sampled benchmark families Supplemental evidence, not proof the changed path executed
Native paths Explicit unmeasured coverage gaps; no native execution or local evidence admission
Missing/incompatible evidence Incomplete or inconclusive, never a whole-PR pass

The deterministic scripts own verdicts, coverage, and permitted actions. GPT cannot erase a measured regression or clear missing/sampled evidence. Recommendations distinguish measurements, static support, and hypotheses. There is no numeric worth-it score, automatic merge decision, code/label mutation, issue closure, or automatic follow-up.

Successful final fork verification

The fresh final fork canary completed successfully against #32382, using the final production implementation from 867ffde2fe.

  • Intake, activation, measurements, evidence preparation, GPT interpretation, detection, safe outputs, and conclusion all succeeded.
  • Both Core builds succeeded under LinuxUsers isolation with credentialsSanitized: true.
  • Seven benchmarks completed in each of the four ABBA runs, with matching benchmark sets and complete measurement data.
  • Changed binding/XAML workloads remained excluded. Coverage was explicitly incomplete, with the deterministic next action needs_human_discussion; no native measurements or whole-PR clearance were claimed.
  • The downloaded report-validation artifact contains valid: true and errors: [].

Canary commit e2550e87ba differs from production only in the two workflow files: it uses the existing personal Copilot secret, restricts dispatch to the dedicated fork branch, targets the selected upstream PR, and forces staged output. No new upstream report was posted, and no upstream environment protection was changed. Production PAT-pool configuration and normal publication behavior remain intact on the PR branch.

The staged run retains a non-fatal gh-aw conclusion annotation about the absent safe-outputs-items artifact. The actual safe-output rendering/validation job and overall run succeeded; this is not a claim that live upstream publication was exercised by the dry run.

Additional validation and CI caveats

All five retained managed/decision/report suites passed again on the final revision. The 69 existing command-routing/recovery checks passed. PowerShell/JSON syntax, scoped formatting, whitespace checks, strict gh-aw compilation, and native actionlint checks passed. Existing shared-workflow actionlint diagnostics were verified against main and left unchanged.

Normal PR checks are not all green: the Pester check has 2,201 passes and two stale assertions expecting gpt-5.6-sol while Review-Tests.ps1 uses gpt-6-astra. Both test files and their subject are byte-identical to the merged main. Skill Validation fails before inspecting PR content because the unchanged checkout action refuses fork checkout in pull_request_target. These unrelated shared checks were not weakened or modified. maui-pr is neutral on the final GitHub-only revision.

Earlier real hosted runs exposed and verified fixes for assigning to PowerShell's read-only $HOME and using the Windows-only SDK installer on Linux. Those fixes and retained diagnostic artifacts remain in the reduced implementation.

Deployment remains pending: the PR is still a draft and unmerged. The workflow, lock, skill, and trusted scripts must reach dotnet/maui's default branch before the upstream slash command operates. Fork validation does not imply production activation or replace normal CI and human review. Operating instructions are in .github/docs/pr-review-workflow.md.

Issues Fixed

No product issue is closed. This PR adds an explicitly authorized, hosted managed-performance review workflow.

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 38324

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 38324"

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Skill Validation Results

@morning4coffe-dev — new skill validation results are available based on this last commit: 867ffde.
To request a fresh validation after new comments or commits, comment /evaluate-skills.

Overall Failed Static Failed LLM Skipped Skills 0 Agents 0

❌ Skill Validation Results — 867ffde · Add /review performance GitHub Agentic Workflow for PR benchmarks · 2026-09-30T19:22:52Z

❌ Static Checks Failed

Skills: 0 | Eval specs linted:

⏭️ LLM Evaluation: Skipped

💡 LLM evaluation was not run for this external PR.
A repository contributor can post /evaluate-skills on this PR to trigger full evaluation.

🔍 Full results and investigation steps

@dotnet-policy-service dotnet-policy-service Bot added the community ✨ Community Contribution label Sep 8, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Hey there @@morning4coffe-dev! Thank you so much for your PR! Someone from the team will get assigned to your PR shortly and we'll get it reviewed.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Hey there @morning4coffe-dev! Thank you so much for your PR! Someone from the team will get assigned to your PR shortly and we'll get it reviewed.

AdrianoGuzzo pushed a commit to AdrianoGuzzo/MyTaskApp that referenced this pull request Oct 26, 2026
Updated [Microsoft.Maui.Controls](https://github.com/dotnet/maui) from
10.0.110 to 10.0.120.

<details>
<summary>Release notes</summary>

_Sourced from [Microsoft.Maui.Controls's
releases](https://github.com/dotnet/maui/releases)._

## 10.0.120

# .NET MAUI 10.0.120 Release Notes

**Release source:**
[6aca5366139ed0e1cd05cca3e32b6a887d5a8dd0](dotnet/maui@6aca536)

.NET MAUI 10.0.120 introduces significant improvements across all
platforms with focus on quality, performance, and developer experience.
This release includes 213 commits with various improvements, bug fixes,
and enhancements.


## Ai Agents
- Remove only legacy /review rerun queue and scanner by @​kubaflo in
dotnet/maui#39029

- Preserve packed-ref access in isolated Vally workspace by @​PureWeen
in dotnet/maui#39025

- Add /review performance GitHub Agentic Workflow for PR benchmarks by
@​morning4coffe-dev in dotnet/maui#38324

- Use GPT-6.1 Sol for gh-aw workflows by @​kubaflo in
dotnet/maui#39036

- Add maintainer-triggered issue regression tracing by @​kubaflo in
dotnet/maui#38710

- Issue regression tracing: Bound evidence and preflight version gaps by
@​kubaflo in dotnet/maui#39121

- Add /issue triage agentic workflow for manual issue labels by
@​kubaflo in dotnet/maui#39065

- Keep /issue triage reports artifact-only while applying real labels by
@​kubaflo in dotnet/maui#39201

- Expose citation-safe prose to /issue triage by @​kubaflo in
dotnet/maui#39276

- Add /review gate without replacing full review by @​kubaflo in
dotnet/maui#39189

- Fix unattended CI-fixer evidence retrieval by @​PureWeen in
dotnet/maui#39237

## Animation
- Fix AnimationManager.Dispose — retained ticker keeps disposed manager
and callbacks alive by @​HarishwaranVijayakumar in
dotnet/maui#37639
  <details>
  <summary>🔧 Fixes</summary>

- [[leak-scan] AnimationManager.Dispose — retained ticker keeps disposed
manager and callbacks
alive](dotnet/maui#37228)
  </details>

## Blazor
- Fix BlazorWebView root component subscriptions on disconnect by
@​sheiksyedm in dotnet/maui#39247
  <details>
  <summary>🔧 Fixes</summary>

- [[Android] BlazorWebViewHandler never unsubscribes from
RootComponents.CollectionChanged on disconnect, retaining the old
MauiContext and finished
Activity](dotnet/maui#39227)
  </details>

- [Windows] Enable external file drops in BlazorWebView by @​danroth27
in dotnet/maui#37904
  <details>
  <summary>🔧 Fixes</summary>

 ... (truncated)

Commits viewable in [compare
view](dotnet/maui@10.0.110...10.0.120).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=Microsoft.Maui.Controls&package-manager=nuget&previous-version=10.0.110&new-version=10.0.120)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This was referenced Oct 27, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 31, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-ai-agents Copilot CLI agents, agent skills, AI-assisted development community ✨ Community Contribution platform/android platform/ios platform/macos macOS / Mac Catalyst platform/windows s/agent-changes-requested AI agent recommends changes - found a better alternative or issues s/agent-fix-win AI found a better alternative fix than the PR s/agent-reviewed PR was reviewed by AI agent workflow (full 4-phase review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants