Repository navigation
feat(debug): make --debug produce one diagnostic bundle safe to attach - #1330
Conversation
Implements OWASP#1309. --debug now writes a timestamped cve-lite-debug-<ts>/ directory containing environment.json, events.jsonl, project.json, and scan.json (when --json was used). All paths are redacted: project-relative, <home> placeholder, or <outside-project>. Test asserts no home-directory path survives in any bundle file.
f35555f to
055e88d
Compare
sonukapoor
left a comment
There was a problem hiding this comment.
The shape of this is right: four files, one builder each, the effective-flags block reusing the precedent from your own #1297 so it records that an option was set rather than its value, and events that parse line by line. The property the issue cared most about holds, and I went looking for ways to break it: scanning a project that lives inside my home directory, from a working directory somewhere else, the bundle came out with zero home-directory paths in any file. The redaction does what it was built to do.
The problem is what it was not built for. I can get two things into the bundle that must never be published.
A registry password. With a .npmrc whose registry carries basic auth, https://ciuser:s3cr3tpw@npm.corp.internal/..., the password appears 34 times in events.jsonl, through the registry config event, every fetch url, and the error message and stack when a fetch fails. Your token handling is actually fine, and worth saying: an _authToken in the same file never reaches the bundle, so you clearly thought about credentials. The gap is specifically credentials carried inside a URL.
The registry hostname. Separately from the password, npm.corp.internal is in there 34 times, and on my other fixture an internal Nexus host appeared 12 times. The issue asked for configured registry and proxy hosts to be treated as sensitive, and that part is not implemented. For an enterprise running a private mirror, the hostname alone is infrastructure disclosure, and those are exactly the people most likely to be told to attach a bundle to a public issue.
The redaction helper only understands filesystem paths and the home directory string, so a URL passes through untouched wherever it appears. I think the answer is the one your own #1297 already established: record that a registry was configured and whether it is the default or a custom one, and never the value. Same for the OSV url. If you want to keep hosts for diagnosis, strip userinfo at minimum, but presence-only is the safer default and it is consistent with what we already do.
Second blocker, and this one surprised me. The bundle can contain a scan result from a completely different project. On a run with no --json at all, it found the newest file in the working directory matching the scan-output name and bundled it as scan.json. In my test that file was a CycloneDX SBOM from an unrelated run, so the bundle both published another project's data and labelled an SBOM as the scan result, which would send whoever reads it down the wrong path. The name match needs to exclude the .cdx and .spdx variants, and more importantly the bundle should only take a file this run actually wrote. Having the writer hand back the path it wrote is cleaner than searching for it afterwards.
Third thing, and this one is my fault rather than yours. The workspace members come out wrong. On our own workspace example, whose real members are client and server, project.json reports 27 members named husky, npm-run-all, @angular/core and so on, and neither real member is in the list. Those are declared dependencies.
You implemented the issue faithfully. The issue told you to name the contents of that workspace map, and the issue was wrong about what that map holds: it is keyed by dependency name, not by member. So the count that log line was already printing was never a member count either, and I wrote "workspace members are counted but not named" on the strength of it. That is a pre-existing mislabelling you inherited and reasonably trusted.
It still cannot merge like this, because a diagnostic artefact that states 27 members with confident names is worse than the count it replaces, and attribution questions are exactly what this field exists to answer. But the fix needs a decision from me about where member names should come from rather than more work from you guessing. Give me a day and I will put the answer on the issue. If you would rather land everything else first and leave this field out, say so and I will take it as a follow-up.
Three smaller things.
The run produces two artefacts, not one. The old text log is still written beside the bundle with a name one second apart, it is still announced, and it still carries five home-directory paths on the project I tested. The "attach this" line sits under both of them. I can see from your comment that keeping it is deliberate for backward compatibility, which is fair, so this is a decision rather than a defect: either redact that log too, or stop announcing it, or drop it and say so in the release notes. My preference is to stop announcing it, since the whole point of the feature is that there is one obvious thing to attach. Note the command reference still describes the old behaviour either way.
The bundle write is not guarded. If writing it fails, the throw escapes and replaces a completed scan's exit code, so a diagnostic failing takes the product down with it. It is also only called on the two paths where the scan succeeds, which means no bundle when the scan itself throws, and that is when a reporter most needs one. Worth a try/catch that warns, and calling it on the failure path too.
Path-valued arguments keep everything below the home directory. Running with a cache directory under <home>/SECRETCLIENT/cache, the bundle redacts the home part and keeps the rest, three times. The same run records that same option in environment.json as merely set, which is the right treatment, so one file is careful about it and another prints it.
And one trivial one: the new line uses an em dash, and #1285 has just finished taking those out of everything the CLI prints.
Not yours, and I am filing it separately. Running --debug from a read-only directory crashes with a raw Node stack trace and no useful message. I checked and that reproduces on main, so it predates this PR.
For the record on the rest: merge is clean against current main, the suite is green at 169 suites and 2391 tests, nothing under src/remediation/ is touched as the issue asked, package names and versions are preserved, and no environment variables are captured.
Redact endpoint credentials, private hosts and sensitive paths across bundle files. Attach only JSON written by the current run, keep diagnostic failures non-fatal, and attempt bundles on scan failures. Use accurate finding counts and evidence-backed project metadata, leaving workspace members omitted pending maintainer clarification. Announce only the safe bundle and avoid em dashes in CLI output. Validation: test hygiene, production build and full Jest suite passed (171 suites, 2398 tests).
sonukapoor
left a comment
There was a problem hiding this comment.
Every blocker is closed, and I checked each by re-running the thing that found it rather than reading the diff.
The credential is gone. On a project with basic auth in the configured registry URL, the bundle now contains zero occurrences of the password, the username, the registry host, my home directory, or even the project's own directory name, which is further than I asked. The stale-file problem is fixed properly rather than by removing the feature: a foreign scan file sitting in the working directory is ignored, while a real --json run still bundles its own payload with the project path reduced to a dot.
The bundle write is guarded on both paths and says why it failed rather than dying. From a read-only working directory the run now prints that it could not write the log and could not write the bundle, and exits with the scan's own code. That also fixes #1357, which I filed this morning against main for the unguarded log write, so that issue closes with this.
project.json is much better: the counts are renamed so they no longer read as package counts when they are finding counts, and the workspace globs and link-entry count are both there now.
Two things stay open and both are mine, not yours. The workspace member names are the decision I owe you, and dropping the field rather than guessing was the right call. And the skipped-rules-with-reasons field from the issue's shape is still absent; I am not holding the PR for it, I will pick it up separately, because whether a rule ran at all is the question behind a whole class of report.
Approving. The suite is green at 2423 tests on the merge result, up 29, and the structure you pulled out into separate redaction and project modules is better than what I reviewed.
|
Merged, thank you @pintu544. I ran the bundle on a workspace project sitting inside a home directory, with an Two things I am picking up myself rather than asking you to extend this, both recorded as follow-ups. On a workspace declaring |
Closes #1309.
What
--debugnow produces a single timestamped bundle directory (cve-lite-debug-<ts>/) containing:environment.json— CLI version, node/platform/arch, package manager, lockfileVersion, node_modules presence, effective flagsevents.jsonl— debug log as one JSON object per lineproject.json— package counts, workspace member names (not just counts), glob patterns, skipped rulesscan.json— scan result when--jsonwas used (omitted otherwise)Redaction
All paths are project-relative,
<home>placeholder, or<outside-project>. Follows the #1297 precedent: sensitive fields as presence-only. A test asserts zero home-directory paths survive in any bundle file.