Repository navigation
fix(config): stop warning about non-yaml files in scanned dirs - #4514
Conversation
|
@ci-robbot fix CI |
|
@mudler root cause is not in this PR: #4497 bumped top-level Fix in #4517 β tidies |
3460469 to
3e26fc6
Compare
|
moving to QA |
|
Picking this up for QA. |
QA Result: PASS
Summary: The reported noise is gone and nothing else in the scan changed. One residual, pre-existing and outside this diff, is noted below. A. Real
|
total warning: lines |
(extension) |
no valid header |
|
|---|---|---|---|
baseline ca70432b |
23 | 22 | 1 |
patched 3e26fc63 |
1 | 0 | 1 |
C. Regressions checked
- Valid cloud-config still ingested on the live system:
kairos-agent config get qa_marker->present. /oemdatasource still applied: theusers:block created the user (uid=1000(kairos) ... 900(admin)).kairos-agent upgradeon live media still stops atcannot upgrade from live media/unknown boot state(unchanged), with no(extension)lines during its scan.CGO_ENABLED=0 go test ./sdk/collector/... ./agent/pkg/config/...->okboth.
Observation (not a blocker, not introduced here)
parseFiles checks the 1MB size guard before the extension check, so non-yaml files larger than 1MB still warn. On a current ISO that leaves 4 lines about initrd, kernel, uefi.img and rootfs.squashfs, i.e. non-yaml files in a scanned dir that are still warned about. The flood the issue was filed for is gone, and this is a different branch that predates the PR, so it does not block. Worth a follow-up if the intent is zero noise on live media.
CI on this head is still authorize / pending, so the hosted suites have not run against 3e26fc63 yet.
|
@ci-robbot shouldn't be hard to fix it in this same PR. A file should only be reported as skipped because of size, only if it would otherwise be included (thus it has the right extension). |
Head branch was pushed to by a user without write access
bfc3552
|
@jimmykarily agreed, and that is the residual the QA pass flagged. Fixed here in if ext := filepath.Ext(f); ext != ".yml" && ext != ".yaml" {
continue
}
if fileSize(f) > 1.0 {
if !nologs {
fmt.Printf("warning: skipping %s. too big (>1MB)\n", f)
}
continue
}Concretely, on a live ISO the old order named New spec covers it, and it does fail against the previous order:
|
|
Picking this up for QA. |
QA Result: PASS
Summary: The residual my previous pass flagged is gone. A. Real
|
| file | base | 3e26fc63 |
bfc35528 |
|---|---|---|---|
good.yaml (valid) |
ingested | ingested | ingested |
nohdr.yaml |
no valid header |
no valid header |
no valid header |
big.yaml (3 MB) |
too big |
too big |
too big |
blob.squashfs (3 MB) |
too big |
too big |
silent |
shim.efi, acpi.mod |
(extension) x2 |
silent | silent |
Only blob.squashfs changed, which is the intent. Merged output on this head still resolves good.yaml only:
MERGED=#cloud-config
# Sources:
# - /tmp/bd/good.yaml
qa: ok
C. Regressions checked
- Live system boots and logs in normally; cloud-config from the datasource still applied (
id kairos->uid=1000(kairos) ... 900(admin)). - Valid config still ingested end to end:
kairos-agent config get qa_marker->present. kairos-agent upgradeon live media still refuses withcannot upgrade from live media/unknown boot state.CGO_ENABLED=0 go test ./sdk/collector/... ./agent/pkg/config/...->okboth.- Mutation check on the new spec: putting the size check back before the extension check makes
reports the size only for a file it would otherwise have readfail (53 Passed | 1 Failed), so the spec really does pin the ordering.
Notes, neither introduced nor changed here
fileSizedivides twice with integer arithmetic (kilobytes := bytes / 1024; megabytes := float64(kilobytes / 1024)), so the> 1.0guard only trips at 2 MB. A 1.4 MB.yamlis read rather than reported. Pre-existing, unrelated to this diff, mentioned only because it makes the "1MB" in the message inaccurate.kairos-agent configreaches the collector with logs suppressed, so the warnings above are not visible through that command on a booted system; they surface on the paths that scan with logs enabled. This is why the A/B above drives the collector directly. Also pre-existing.
CI on this head is still authorize / pending, so the hosted suites have not run against bfc35528 yet.
|
@ci-robbot rebase based of master |
The config collector walks /run/initramfs/live (and the other GetUserConfigDirs paths) recursively and, for every file whose extension is not .yml or .yaml, prints a 'warning: skipping <path> (extension).' line. In practice that means kairos-agent upgrade prints hundreds of warnings for grub.efi, grub.cfg, *.mod, *.pf2 and every other file that lives next to the cloud-configs. Silently skip non-yaml extensions instead. The header-missing warning on a real .yaml file (the one jimmykarily explicitly asked to keep in kairos-io#2064) is untouched. Fixes kairos-io#2064 Signed-off-by: ci-robbot <ci-robbot@users.noreply.github.com>
bfc3552 to
4a77042
Compare
|
Rebased onto master ( #4527 landed while this was open and took the ordering fix with it: the extension check now runs before the size check on master already, which is what @jimmykarily asked for here. So I dropped my second commit ( What is left is the one thing #2064 actually asks for and master still does not do: skip a non-yaml file silently. Master still prints The header warning on a real |
Fixes #2064.
Symptom
kairos-agent upgrade(and every other command that scans the user config dirs) prints hundreds of these lines:because
sdk/collector/collector.gowalks/run/initramfs/liverecursively and warns on every file whose extension is not.ymlor.yamlβ which, in that directory, is almost every file.Fix
Silently skip non-yaml extensions. A non-yaml file in a scanned directory is never actionable for the user (the ISO ships them by design), so the warning was pure noise.
The other three warnings in the same loop stay β including the header-missing one that @jimmykarily explicitly wanted to keep visible on the issue thread, because that one is actionable ("sometimes people forget to put the header at the top of the file").
Alternative considered
Plumbing
collector.NoLogsthrough theupgradepath (as suggested in the 2024 thread). That would also silence the header-missing warning, which is the regression @jimmykarily was worried about. Dropping the(extension)message avoids the trade-off entirely and needs no changes outsidecollector.go.Test plan
sdk/collector/collector_test.goseeds a directory with a valid cloud-config, a header-less yaml, and four non-yaml siblings (grub.cfg,bootx64.efi,unicode.pf2,acpi.mod), captures stdout, and asserts:(extension)lineno valid headerwarning for the header-less yamlgo build ./sdk/... ./agent/...andgo vet ./sdk/... ./agent/...(existing warning insdk/loop/loop_test.gois unrelated)go test ./sdk/collector/...green