Repository navigation
✨ feat(cli): machine-readable build report, and metadata from a wheel - #1080
Conversation
db5d10d to
7467525
Compare
|
I like this, but I think we should make it work with |
7467525 to
0376bda
Compare
c9cf8aa to
9d66ae8
Compare
d3508d2 to
b82d46c
Compare
|
I'm just thinking if this shouldn't be co-located with the built dists. I feel like having to pass a path to |
1ba9f57 to
96cd2f3
Compare
|
Good point on the friction — reworked. I kept it opt-in rather than always-on, since writing a file the user didn't ask for felt surprising — but it's now zero-config when you do want it. On concurrency: the write is atomic (temp file + |
|
I'm not sure putting it in the outdir is always the best (I'm at least happy there's a report path option!) I'd prefer the path be required, I think. I can't think of a good reason to put it in the (If twine does ignore non-wheel non-sdist files even when passed explicitly via shell expansion, that would be better. Also if it was prefixed by a dot, then shell expansion wouldn't pick it up, that's another option). |
|
That's a good point. Maybe we should discuss this with the twine team, to see if there's any appetite for uploading files from a manifest. |
|
My initial design was to let the report flag take a directory, and now that another maintainer also prefered that, I'll revert to it. I think is a cleaner approach 😊 |
|
Why not just make it take the full path ( |
After a build, downstream steps such as uploading, signing, or recording checksums need the exact artifact filenames. Those names are dynamic since they encode the version and the wheel's Python, ABI and platform tags, so today users glob dist/ (and often clear it first) or scrape the success line off stdout. Neither is reliable. Add --report PATH, which writes a JSON document listing each artifact's name, path, kind, size and SHA-256 hash, selected with --output-format. The report goes to a file build owns rather than stdout because the backend subprocess may write to stdout itself, so a structured payload there could not be parsed. Including hashes also covers the request to surface digests for provenance.
Building on --report, let --metadata take a .whl as its source argument and read the wheel's METADATA directly instead of building. A wheel path lifted from the report can then be inspected without a rebuild. A wheel passed without --metadata is rejected, since build cannot build from a wheel. Also drop typing from __main__'s lazy modules: the report's TypedDict subclasses are evaluated at import, so typing must load eagerly.
Address review: the name is too generic while JSON is the only format. Reintroduce under a clearer name if more formats are added later. The report stays JSON.
The schema sample had placeholder sizes and the empty-string SHA-256 on both artifacts. Replace it with the actual JSON from a real build, including the expanded hashes object json.dumps emits.
Writing build-report.json into the output directory by default put a non-distribution file where `twine upload dist/*` style workflows would trip over it, and keeping the outdir clean was the whole argument for the default outdir behavior in pypa#455. Review on pypa#1080 settled on always asking the caller where the report goes. Drop the optional-argument form so --report takes the report file path, and steer the docs examples away from the output directory. A flag-less default can return later if twine learns to skip non-distribution files.
96cd2f3 to
68bd564
Compare
|
Done — |
Closes #198.
Anyone wiring
buildinto a release pipeline hits the same wall: once the build finishes, they need to know which files came out so they can upload, sign, or checksum them. The filenames are not predictable, since they carry the version and, for wheels, the Python, ABI and platform tags. So today people reach for workarounds that each have a sharp edge. They globdist/*and trust that the next tool expands it (onlytwinedoes;pip install dist/*does not). Theyrm -rf distbefore every build so a stale artifact from last time does not get published by accident. They scrape the colouredSuccessfully built …line off the terminal. Or they drop down to the Python API and reimplement a chunk of the CLI. 📦 None of these is something we want to be the recommended path.This adds a first-class answer:
python -m build --report dist/report.jsonwrites a small JSON file describing exactly what was produced — each artifact's name, path, kind (sdistorwheel), size, and SHA-256 hash. A pipeline can then dotwine upload $(jq -r '.artifacts[].path' dist/report.json)and be certain it is shipping this build's output and nothing else. Including the hash also gives projects a ready hook for provenance and checksums (the ask from #789, folded in here). ✨The report goes to a file the frontend owns rather than to stdout, because the build backend runs as a subprocess that can print to stdout itself — so a structured payload there could never be parsed safely. The report is JSON and its
versionfield is a schema version, so the format can grow later without breaking readers. The defaultpython -m buildexperience is untouched and the feature is entirely opt-in.Building on that,
--metadatanow also accepts a built wheel:python -m build --metadata dist/mypackage-1.0.0-py3-none-any.whlreads the wheel'sMETADATAand prints it as JSON without building anything. A pipeline can lift a wheel's path straight out ofreport.jsonand inspect what it shipped — no rebuild, no unzip.Workarounds people use today
The fragile stopgaps people reach for in the issue threads until this lands:
rm -rf distbefore each build, thentwine upload dist/*relying on twine's own glob expansion to avoid shipping stale artifacts.pip install dist/*does not glob-expand, so CI still has to discover the dynamically named files some other way.jqinto the installer.dist/, which then breaks becausetwine dist/*tries to upload the JSON to PyPI.Successfully built …line for digests: build logs are the wrong place; machine-readable output is the real fix.Seen in the wild on GitHub
Release pipelines wipe
dist/first, then re-discover the produced files by globbing, because nothing tells them the names:noxfile.py(L400-L406) — afterpython -m build, reconstructs the expecteddist/{name}-{version}-...whl/.tar.gznames by hand (rmtree("dist")at L420). Even pypa's own tooling guesses the filenames.noxfile.py(L198-L205) —shutil.rmtreeonbuild/anddist/beforepython -m buildto tell new output from stale.makefile(L21-L24) —rm -rf build dist, thenpython -m build, thentwine upload dist/*.tasks/publish.mk(L14-L16) —python -m build, thenfind dist -name "*.whl"to locate the wheel to install.release.sh(L531-L536) —find dist -name "*.whl"to install the just-built wheel, afterrm -rf dist(L78) andpython -m build(L349).