Repository navigation
perf(core): check bytes with SIMD where memchr does not fit - #863
Merged
Merged
Conversation
- Validate loaded CommonJS sources with simdutf8 (2x-6.7x faster than std::str::from_utf8 on 2 KB-300 KB files). - Find bytes to percent-encode in file URL paths, and spaces or controls in specifiers, 16 bytes at a time with vector compares (3x-12x faster on typical paths than Iterator::any). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
from
October 18, 2026 15:03
af76f2b to
952ebf4
Compare
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
10 times, most recently
from
October 19, 2026 04:18
c5bd0b6 to
347aae6
Compare
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
3 times, most recently
from
October 19, 2026 06:58
e1d318b to
74050d3
Compare
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
from
October 19, 2026 07:03
74050d3 to
fda7487
Compare
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
7 times, most recently
from
October 19, 2026 08:07
ee6c3eb to
6b92f9b
Compare
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
from
October 19, 2026 08:16
6b92f9b to
de14e41
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: de14e4142a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
simdutf8's basic validator checks the whole input before it reports an error, so a large binary was scanned in full, twice. The compat validator stops at the first bad block, as std did. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
3 times, most recently
from
October 19, 2026 08:44
df6b70b to
7635a37
Compare
Brooooooklyn
force-pushed
the
perf/simd-byte-checks
branch
from
October 19, 2026 09:13
7635a37 to
c731278
Compare
Brooooooklyn
added a commit
that referenced
this pull request
Oct 20, 2026
Bottom of stack #867: #861 and #863 build on it, so their CI gets working emnapi versions. #865 on main moved `emnapi` and `@emnapi/runtime` to `2.0.0-beta.2` but left `@emnapi/core` at `2.0.0-alpha.6`. `@napi-rs/cli` refuses the mismatch, so `stable - wasm32-wasip1-threads - node@24` fails on main, and the 5 test jobs after it are skipped: ``` Internal Error: emnapi version mismatch: emnapi@2.0.0-beta.2, @emnapi/core@2.0.0-alpha.6, @emnapi/runtime@2.0.0-beta.2. ``` Moving all three to beta.2 is not the fix: `@emnapi/core` beta.2 breaks `@napi-rs/wasm-runtime` 1.2.5, the latest, and `Test WASI target` fails to load the binding (`Cannot set properties of undefined (setting 'initWorker')`). So this pins all three to exactly `2.0.0-alpha.6`, the versions main used before #865 (where both wasm jobs passed). Move them together once `@napi-rs/wasm-runtime` supports emnapi beta. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #860. An audit of every byte and string scan left in
src/, keeping only the changes a benchmark shows are faster.Changed
TryAsStr::try_as_str: every CommonJS source the load hook checks, up to twice per filestd::str::from_utf8simdutf8(compatwhere the error text is shown)encode_file_url_path: "does this path need%XX?", on every resolved file URLbytes().any(encoded)any_byte(file_url_encoded)is_bare_specifier: "any byte ≤ space?"bytes().any(..)any_byte(..)any_bytechecks 16 bytes at a time with no early exit inside a chunk, so LLVM emits vector compares (NEON, SSE2). The encode set is rewritten as ranges and compares joined by|: the oldmatches!compiles to a bit test per byte, which does not vectorize (that form measured 0.8x, slower thanany).Looked at, left as is
contains("/node_modules/")(5 places)memmemwith a staticFindermeasured 0.5x–1.3x againststr::containson 34–117 byte paths: no winwindows_file_url.rshost and segment checksreplace/splitwith one charmemchrcheck, or run once at startupTests
file_url_encoded_is_the_path_to_file_url_set: the compare form matches the oldmatches!set for all 256 bytes.any_byte_is_iterator_any: same answer asIterator::anyfor lengths 0–40, with the hit at every position.cargo test --all-features(49), clippy with warnings denied, andvp run test(272 passed, 7 skipped) all pass.The numbers come from a standalone micro-benchmark (release, LTO,
codegen-units = 1) on Apple Silicon. They are not measured on x86_64.🤖 Generated with Claude Code