Skip to content

release: train PGO and BOLT on --fast checks too - #976

Merged
Boshen merged 1 commit into
mainfrom
perf/pgo-fast-training
Nov 12, 2026
Merged

Boshen merged 1 commit into
mainfrom
perf/pgo-fast-training

Conversation

@maschwenk

Copy link
Copy Markdown
Collaborator

release binaries are PGO trained (plus BOLT on linux) on the conformance suite, fourslash, and tsrs -p . / tsrs --lint on xstate-main and webpack. none of it passes --fast, so everything the fast preset switches on (lazy member tables, lazy d.ts members, locality scheduling + work stealing, the extra checker caches, leaf regions) had zero counts and got laid out as cold. in a --fast profile of a large private monorepo the lazy member functions are visibly hot, and in the shipped profile get_ready_lazy_member_table_worker, create_lazy_member_table and get_lazy_declared_member literally have all-zero block counts

this adds tsrs -p . --fast on the same two projects to pgo-train.sh and to the instrumented tsrs runs in bolt.sh, and a --fast run of the existing cli fixture to the windows training. the end-of-script profile checks now require the new profiles too. default-mode runs stay as they were

measured on a graviton4 box (m8g.4xlarge), two PGO builds from one instrumented build that differ only in which raw profiles get merged, then BOLT on each with main's vs this branch's bolt.sh. 15 alternating pairs per workload, CPU time B vs A:

workload PGO 8 cores PGO 4 cores PGO+BOLT 8 cores PGO+BOLT 4 cores
--fast private monorepo -1.5% -0.8% -1.0% -1.8%
--fast vscode -0.5% -0.4% -1.0% -0.8%
--fast mikro-orm -1.7% -2.0% -2.2% -2.3%
--fast mui-docs -1.6% -1.6% -2.1% -2.1%
--fast supabase-studio -1.2% -1.1% -1.6% -1.5%
--fast webpack (trained) -1.4% -0.8% -1.4% -0.8%
default vscode +0.3% +0.1% +0.3% +0.3%
default mui-docs -0.2% -0.4% +0.3% +0.0%

22 of the 24 --fast cells have intervals that exclude 0. cycles and wall move the same way, peak RSS moves under 1% everywhere, and output is byte identical between arms everywhere. the win is fetch: frontend stalls -4 to -5%, l1i refills -2 to -7%

tradeoff is tiny: two more sub-second runs per training, and default mode wobbles by at most +0.3% CPU because the merged hot set grows. i also tried --fast --checkers 1, the lint run under TSRS_FAST=1, --fast at 8 cores, and a second pair of projects on top. none of them moved cycles, so they're not in here

caveats: only measured on linux aarch64. the go fourslash suite was left out of both arms (no go on the box, and it only trains the language server). macos/windows get the change unmeasured, and the windows run is a tiny fixture. full numbers, method and repro are in notes/perf-pgo-fast-training.md

The fast preset runs code default mode never does (lazy members, locality
scheduling, the extra checker caches, leaf-file regions), so the release
profiles had no counts for it. Add --fast runs of xstate-main and webpack to
pgo-train.sh and bolt.sh, and a --fast CLI run to the Windows training.
Measurements in notes/perf-pgo-fast-training.md.
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Performance change vs main

Workflow run Β· main 221ee7bf9d6c β†’ PR merge fc2e7ba88a28 (head 33879b3810a1)

Clef gate: Clef estimated 13.1% default and single-threaded wall time, 11.2% peak RSS, 14.6% single-threaded instruction count impact probability.

The main and PR tsrs release binaries are byte-for-byte identical. No compiler performance difference to measure.

tsrs-lint

The main and PR tsrs-lint binaries are byte-for-byte identical. No lint performance difference to measure.

@codspeed

codspeed Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

1 warning

⚠️ 4 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

βœ… 8 untouched benchmarks
⏩ 8 skipped benchmarks1


Comparing perf/pgo-fast-training (33879b3) with main (221ee7b)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@Boshen
Boshen marked this pull request as ready for review November 12, 2026 08:49
@Boshen
Boshen added this pull request to the merge queue Nov 12, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: βœ… Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
πŸ“ Code Review βœ… Completed 2026-11-12T08:52:25.256842Z 33879b3 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with πŸ‘€ while any review is running, comments if it has suggestions, and reacts with πŸ‘ once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸ’‘ Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33879b3810

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with πŸ‘.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread notes/README.md
- [perf-icount-round4](perf-icount-round4.md)
- [perf-mimalloc-safe](perf-mimalloc-safe.md)
- [perf-pgo](perf-pgo.md)
- [perf-pgo-fast-training](perf-pgo-fast-training.md)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove the unrequested README edit

The root AGENTS.md explicitly prohibits updating any README.md file unless the user requests it. This notes-index addition modifies notes/README.md without such a request. Revert the index change; the performance note and training-script updates can remain independently.

AGENTS.md reference: AGENTS.md:L6-L6

Useful? React with πŸ‘Β / πŸ‘Ž.

Merged via the queue into main with commit 62ca32b Nov 12, 2026
36 checks passed
@Boshen
Boshen deleted the perf/pgo-fast-training branch November 12, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants