Repository navigation
Turbopack: don't grow effect vectors inside the analyzer arena - #101480
Merged
Merged
Conversation
Contributor
Tests PassedCommit: f94710e |
The analyzer collects effects into bump-allocated vectors while it walks a module. A bump arena can't free memory, so every time one of those vectors grows, its old buffer stays allocated until the analysis finishes. Effect vectors grow constantly, so a large part of the arena ends up being abandoned buffers. Collect effects in heap vectors instead, and copy each finished block into the arena at its exact size. The arena then only holds what is actually kept, as intended when effects moved into the arena in #94614. No change in output. Measured on analyze_ecmascript_module alone, release build, 5 alternating rounds against the parent commit, as median paired changes: | Input | Peak memory | Bytes allocated | Allocations | CPU | | --------------------- | ----------- | --------------- | ----------- | ----- | | 8,000 modules | -20.1% | -8.4% | +3.3% | -0.9% | | Generated stress | -22.3% | -14.9% | +0.0% | -1.0% | | Four large benchmarks | +0.0% | +6.7% | +3.1% | -4.1% | Peak memory is the sum of each module's peak live allocations. On the 8,000 modules it goes from 7,652 MB to 6,118 MB. The CPU changes are within noise. The four large benchmark files gain bytes from the heap vectors' own growth and the final copy, without a change in peak.
bgw
approved these changes
Nov 9, 2026
jimmyhmiller
force-pushed
the
jimmym/analyzer-effect-vecs
branch
from
November 10, 2026 15:24
f0d61da to
f94710e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The analyzer collects effects into bump-allocated vectors while it walks a module. A bump arena can't free memory, so every time one of those vectors grows, its old buffer stays allocated until the analysis finishes. Effect vectors grow constantly, so a large part of the arena ends up being abandoned buffers.
Collect effects in heap vectors instead, and copy each finished block into the arena at its exact size. The arena then only holds what is actually kept, as intended when effects moved into the arena in #94614.
No change in output. Measured on analyze_ecmascript_module alone, release build, 5 alternating rounds against the parent commit, as median paired changes:
Peak memory is the sum of each module's peak live allocations. On the 8,000 modules it goes from 7,652 MB to 6,118 MB. The CPU changes are within noise. The four large benchmark files gain bytes from the heap vectors' own growth and the final copy, without a change in peak.