Skip to content

ci: cut fixed setup overhead in test jobs - #3269

Merged
messense merged 1 commit into
PyO3:mainfrom
messense:ci-setup-overhead
Jul 19, 2026
Merged

messense merged 1 commit into
PyO3:mainfrom
messense:ci-setup-overhead

Conversation

@messense

Copy link
Copy Markdown
Member

Summary

Third round for #2189. After #3267 and #3268, warm-run step timings show the test phase is no longer the bottleneck in several cells — fixed setup steps are. Per matrix cell the remaining overhead concentrates in exactly two steps; this PR addresses both.

zig cross-compile step: use the prebuilt maturin (was 1m39–3m54 per cell)

The step ran cargo run build ..., which rebuilt maturin with default features — the cargo test step builds it with --features password-storage,faster-tests, so the feature sets differ and cargo rebuilds the binary and its dependency tree. The PYO3_CONFIG_FILE and maturin new steps that follow already use target/debug/maturin directly; the zig step now does the same (zig is in the default feature set via cross-compile, so the nextest-built binary supports --zig). Most of the step's time was this rebuild, not the zig builds themselves.

setup-miniconda → setup-micromamba (was up to 2m on Windows)

setup-micromamba in binary-only mode installs in seconds. The test harness now falls back to micromamba when conda is absent: conda and micromamba share the create -n NAME python=X.Y -q -y --json CLI, and micromamba's JSON output has the same top-level prefix/success fields the harness already parses. An explicit --root-prefix under test-crates/venvs keeps it working when MAMBA_ROOT_PREFIX isn't set.

Verified locally with conda removed from PATH: develop_pyo3_pure_conda passes through the micromamba path — and runs ~2.5× faster (36s vs ~90s), since micromamba also creates the conda envs much faster than conda does. Repeat create over an existing env is idempotent.

Expected effect

Windows 3.9 (currently the long pole at ~17m53 warm with only 6m36 of actual tests): roughly −2m from the zig step rebuild and −1m40 from miniconda, plus faster conda env creation inside the conda tests. Similar cuts on the other cells running those steps.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NNNLg7XZnxM8Aiqrzo6c5q

Warm-run step timings show the remaining fixed overhead per matrix cell
is concentrated in two steps:

- The zig cross-compile step ran 'cargo run build', which rebuilt
  maturin with default features because the nextest step builds it with
  --features password-storage,faster-tests. Use the already-built
  target/debug/maturin instead, like the subsequent steps do (zig is in
  the default feature set). This was 1m39-3m54 per cell.
- setup-miniconda took up to 2 minutes on Windows. Switch to
  setup-micromamba (binary-only install, seconds) and teach the test
  harness to fall back to micromamba when conda is absent; conda and
  micromamba share the create CLI and the --json output fields the
  harness parses, and micromamba also creates the test envs ~2.5x
  faster. An explicit --root-prefix under test-crates keeps it working
  when MAMBA_ROOT_PREFIX is not set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NNNLg7XZnxM8Aiqrzo6c5q
@messense
messense merged commit f7c38a0 into PyO3:main Jul 19, 2026
85 checks passed
@messense
messense deleted the ci-setup-overhead branch July 19, 2026 07:34
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
maturin 1.15.0

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre>## What's Changed
* set `UV_PYTHON` when running `maturin develop` with uv by @davidhewitt in PyO3/maturin#3236
* Set UID and GID headers in pax by @konstin in PyO3/maturin#3241
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3238
* Replace unguarded panics with explicit errors on fallible input by @messense in PyO3/maturin#3242
* Harden PEP 517 build-wheel and fix unreachable OIDC 404 fallback by @messense in PyO3/maturin#3244
* docs: add pdfcrate and quebec to examples by @ratazzi in PyO3/maturin#3245
* Fix abi3t PyModExport entrypoint check by @messense in PyO3/maturin#3246
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3247
* fix `maturin build --pgo -i <version>` with `uv` install by @davidhewitt in PyO3/maturin#3237
* refactor: apply some mechanical cleanups by @messense in PyO3/maturin#3250
* refactor: BuildOrchestrator pipeline deduplication by @messense in PyO3/maturin#3251
* refactor: improve context and environment hygiene by @messense in PyO3/maturin#3252
* fix(ci): honor ~= and ==X.* in requires-python for generate-ci by @teddytennant in PyO3/maturin#3249
* refactor: type cleanup for options, tags, and wheel metadata by @messense in PyO3/maturin#3253
* refactor: InstallDest, single BridgeModel, and ui output facade by @messense in PyO3/maturin#3254
* Don’t specify generate-import-lib for PyO3 0.29 by @musicinmybrain in PyO3/maturin#3258
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3259
* ci: fix clippy errors on rust 1.97 by @davidhewitt in PyO3/maturin#3263
* refactor build so that pgo is part of normal pathway by @davidhewitt in PyO3/maturin#3257
* fix: verify bindings detection with cargo tree in workspaces by @messense in PyO3/maturin#3264
* fix: harden cargo tree verification of bindings detection by @messense in PyO3/maturin#3266
* ci: cut test wall time without losing coverage by @messense in PyO3/maturin#3267
* test: share cargo target dirs per fixture family by @messense in PyO3/maturin#3268
* ci: cut fixed setup overhead in test jobs by @messense in PyO3/maturin#3269
* feat: support --pgo for maturin develop by @messense in PyO3/maturin#3270
* feat: support MATURIN_PGO env var to activate --pgo by @messense in PyO3/maturin#3271
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3272
* feat: type the bindings input with a single-source-of-truth enum by @messense in PyO3/maturin#3273
* Don't force export of the PyInit_{module_name} in UniFfi mode by @badboy in PyO3/maturin#3275
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3279
* switch musllinux images to musllinux_1_2 as musllinux_1_1 is EOL by @Joshix-1 in PyO3/maturin#3280
* fix(generate-stubs): write stubs into the module's package directory by @jonasdedden in PyO3/maturin#3282
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3286
* Fix group not cwd by @kraktus in PyO3/maturin#3284
* docs: document missing [tool.maturin] config fields by @teddytennant in PyO3/maturin#3287
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3288
* docs: fix cross compiling heading level by @BD103 in PyO3/maturin#3290
* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in PyO3/maturin#3292
* Release v1.15.0 by @messense in PyO3/maturin#3294

## New Contributors
* @ratazzi made their first contribution in PyO3/maturin#3245
* @teddytennant made their first contribution in PyO3/maturin#3249
* @jonasdedden made their first contribution in PyO3/maturin#3282
* @kraktus made their first contribution in PyO3/maturin#3284
* @BD103 made their first contribution in PyO3/maturin#3290

**Full Changelog**: https://github.com/PyO3/maturin/compare/v1.14.1...v1.15.0</pre>
  <p>View the full release notes at <a href="https://github.com/PyO3/maturin/releases/tag/v1.15.0">https://github.com/PyO3/maturin/releases/tag/v1.15.0</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!17599
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant