Skip to content

fix(config): stop warning about non-yaml files in scanned dirs - #4514

Merged
mudler merged 1 commit into
kairos-io:masterfrom
ci-forks:fix/2064-drop-extension-warning
Sep 10, 2026
Merged

mudler merged 1 commit into
kairos-io:masterfrom
ci-forks:fix/2064-drop-extension-warning

Conversation

@ci-robbot

Copy link
Copy Markdown
Collaborator

Fixes #2064.

Symptom

kairos-agent upgrade (and every other command that scans the user config dirs) prints hundreds of these lines:

warning: skipping /run/initramfs/live/EFI/BOOT/MokManager.efi (extension).
warning: skipping /run/initramfs/live/EFI/BOOT/bootx64.efi (extension).
warning: skipping /run/initramfs/live/boot/grub2/i386-pc/acpi.mod (extension).
...

because sdk/collector/collector.go walks /run/initramfs/live recursively and warns on every file whose extension is not .yml or .yaml β€” which, in that directory, is almost every file.

Fix

Silently skip non-yaml extensions. A non-yaml file in a scanned directory is never actionable for the user (the ISO ships them by design), so the warning was pure noise.

The other three warnings in the same loop stay β€” including the header-missing one that @jimmykarily explicitly wanted to keep visible on the issue thread, because that one is actionable ("sometimes people forget to put the header at the top of the file").

Alternative considered

Plumbing collector.NoLogs through the upgrade path (as suggested in the 2024 thread). That would also silence the header-missing warning, which is the regression @jimmykarily was worried about. Dropping the (extension) message avoids the trade-off entirely and needs no changes outside collector.go.

Test plan

  • New spec under sdk/collector/collector_test.go seeds a directory with a valid cloud-config, a header-less yaml, and four non-yaml siblings (grub.cfg, bootx64.efi, unicode.pf2, acpi.mod), captures stdout, and asserts:
    • the valid config is parsed
    • stdout contains no (extension) line
    • stdout still contains the no valid header warning for the header-less yaml
  • go build ./sdk/... ./agent/... and go vet ./sdk/... ./agent/... (existing warning in sdk/loop/loop_test.go is unrelated)
  • go test ./sdk/collector/... green

@ci-robbot
ci-robbot requested a review from a team as a code owner September 6, 2026 20:10
Itxaka
Itxaka previously approved these changes Sep 7, 2026
@Itxaka
Itxaka enabled auto-merge (squash) September 7, 2026 07:56
mudler
mudler previously approved these changes Sep 7, 2026
@mudler

mudler commented Sep 7, 2026

Copy link
Copy Markdown
Member

@ci-robbot fix CI

@ci-robbot

Copy link
Copy Markdown
Collaborator Author

@mudler root cause is not in this PR: #4497 bumped top-level quic-go/webtransport-go to v0.11.1 (and transitively quic-go/quic-go to v0.60.0) without running go mod tidy inside tests/. The tests module replaces kairos to ../, so MVS resolves quic-go to v0.60.0 but tests/go.sum is still pinned at v0.59.1, and every go build/go test inside tests/ fails with missing go.sum entry for module providing package github.com/quic-go/quic-go. Every recently-triggered PR (e.g. #4515, this one) is hit for the same reason.

Fix in #4517 β€” tidies tests/. Once it merges, a rebase here will go green.

jimmykarily
jimmykarily previously approved these changes Sep 7, 2026
@jimmykarily
jimmykarily force-pushed the fix/2064-drop-extension-warning branch from 3460469 to 3e26fc6 Compare September 7, 2026 09:40
@jimmykarily jimmykarily moved this to Under review πŸ” in πŸ§™Issue tracking board Sep 7, 2026
@mudler mudler moved this from Under review πŸ” to QA in πŸ§™Issue tracking board Sep 7, 2026
@mudler

mudler commented Sep 7, 2026

Copy link
Copy Markdown
Member

moving to QA

@ci-robbot

Copy link
Copy Markdown
Collaborator Author

Picking this up for QA.

@ci-robbot

Copy link
Copy Markdown
Collaborator Author

QA Result: PASS

Check Result
Environment PR head 3e26fc63 (rebased on master, incl. #4516 go.sum fix). ISO built locally with scripts/build-iso.sh: kairos-hadron-v0.5.1-standard-amd64-generic-v4.3.0-rc7-8-g3e26fc638-k3sv1.36.4+k3s1.iso, booted live in QEMU/KVM (BIOS, 4 vCPU, 4G) with a cidata datasource
Reproduction Booted the live ISO and, from a boot stage, ran the same config scan through both the shipped (patched) kairos-agent and a binary built from the PR base commit ca70432b, against (a) the real /run/initramfs/live and (b) a directory mirroring the file list from #2064
Expected No (extension) warnings; the missing-header warning still printed; valid cloud-config still ingested
Actual (extension) warnings: 5 -> 0 on the real live dir, 22 -> 0 on the #2064-shaped dir. Header warning kept in both. Valid yaml still read.

Summary: The reported noise is gone and nothing else in the scan changed. One residual, pre-existing and outside this diff, is noted below.

A. Real /run/initramfs/live on the booted live ISO

Baseline (ca70432b):

warning: skipping /run/initramfs/live/EFI/BOOT/grub.cfg (extension).
warning: skipping /run/initramfs/live/boot/boot_hybrid.img (extension).
warning: skipping /run/initramfs/live/boot/eltorito.img (extension).
warning: skipping /run/initramfs/live/boot/grub/grub.cfg (extension).
warning: skipping /run/initramfs/live/boot/grub2/grub.cfg (extension).
warning: skipping /run/initramfs/live/boot/initrd. too big (>1MB)
warning: skipping /run/initramfs/live/boot/kernel. too big (>1MB)
warning: skipping /run/initramfs/live/boot/uefi.img. too big (>1MB)
warning: skipping /run/initramfs/live/config.yaml because it has no valid header
warning: skipping /run/initramfs/live/rootfs.squashfs. too big (>1MB)

Patched (3e26fc63), same boot, same directory:

warning: skipping /run/initramfs/live/boot/initrd. too big (>1MB)
warning: skipping /run/initramfs/live/boot/kernel. too big (>1MB)
warning: skipping /run/initramfs/live/boot/uefi.img. too big (>1MB)
warning: skipping /run/initramfs/live/config.yaml because it has no valid header
warning: skipping /run/initramfs/live/rootfs.squashfs. too big (>1MB)

B. Directory mirroring the file list from #2064

24 files, 22 of them non-yaml (EFI/BOOT/*.efi, boot/grub2/i386-pc/*.mod, grub.cfg, unicode.pf2), plus one valid cloud-config and one header-less yaml:

total warning: lines (extension) no valid header
baseline ca70432b 23 22 1
patched 3e26fc63 1 0 1

C. Regressions checked

  • Valid cloud-config still ingested on the live system: kairos-agent config get qa_marker -> present.
  • /oem datasource still applied: the users: block created the user (uid=1000(kairos) ... 900(admin)).
  • kairos-agent upgrade on live media still stops at cannot upgrade from live media/unknown boot state (unchanged), with no (extension) lines during its scan.
  • CGO_ENABLED=0 go test ./sdk/collector/... ./agent/pkg/config/... -> ok both.

Observation (not a blocker, not introduced here)

parseFiles checks the 1MB size guard before the extension check, so non-yaml files larger than 1MB still warn. On a current ISO that leaves 4 lines about initrd, kernel, uefi.img and rootfs.squashfs, i.e. non-yaml files in a scanned dir that are still warned about. The flood the issue was filed for is gone, and this is a different branch that predates the PR, so it does not block. Worth a follow-up if the intent is zero noise on live media.

CI on this head is still authorize / pending, so the hosted suites have not run against 3e26fc63 yet.

@jimmykarily

Copy link
Copy Markdown
Contributor

@ci-robbot shouldn't be hard to fix it in this same PR. A file should only be reported as skipped because of size, only if it would otherwise be included (thus it has the right extension).

auto-merge was automatically disabled September 7, 2026 12:10

Head branch was pushed to by a user without write access

@ci-robbot
ci-robbot dismissed stale reviews from Itxaka, mudler, and jimmykarily via bfc3552 September 7, 2026 12:10
@ci-robbot

Copy link
Copy Markdown
Collaborator Author

@jimmykarily agreed, and that is the residual the QA pass flagged. Fixed here in bfc35528: the extension check now runs first, so fileSize is only consulted for a file that would otherwise have been read as config.

if ext := filepath.Ext(f); ext != ".yml" && ext != ".yaml" {
    continue
}

if fileSize(f) > 1.0 {
    if !nologs {
        fmt.Printf("warning: skipping %s. too big (>1MB)\n", f)
    }
    continue
}

Concretely, on a live ISO the old order named rootfs.squashfs in a "too big (>1MB)" warning; it is now silent, while an oversized .yaml still gets reported.

New spec covers it, and it does fail against the previous order:

[FAILED] Expected
    warning: skipping /tmp/config_big.../rootfs.squashfs. too big (>1MB)
    <string>: rootfs.squashfs

go test ./sdk/collector/... green (54 specs).

@ci-robbot

Copy link
Copy Markdown
Collaborator Author

Picking this up for QA.

@ci-robbot

Copy link
Copy Markdown
Collaborator Author

QA Result: PASS

Check Result
Environment PR head bfc35528. ISO built from the tree with scripts/build-iso.sh: kairos-hadron-v0.5.1-standard-amd64-generic-v4.3.0-rc7-9-gbfc355281-k3sv1.36.4+k3s1.iso, booted live in QEMU/KVM (BIOS, 4 vCPU, 4G) with a cidata datasource. In-guest kairos-agent version -> kairos v4.3.0-rc7-9-gbfc355281
Reproduction Re-QA of the reordering @jimmykarily asked for. A/B on the booted live ISO: ran the collector scan over the real /run/initramfs/live with three builds, base ca70432b, previous QA head 3e26fc63, and this head bfc35528, plus a controlled directory covering each file class
Expected Non-yaml files never warn, whatever their size. An oversized .yaml still warns. The missing-header warning stays. Valid config still ingested
Actual All four as expected. Warning count on real live media: base 10 -> 3e26fc63 5 -> bfc35528 1, the one remaining line being the actionable header warning

Summary: The residual my previous pass flagged is gone. rootfs.squashfs, kernel, initrd and uefi.img no longer produce "too big (>1MB)" lines, while an oversized .yaml still does. Nothing else in the scan changed.

A. Real /run/initramfs/live on the booted live ISO, as root

base ca70432b (10 warnings):

warning: skipping /run/initramfs/live/EFI/BOOT/grub.cfg (extension).
warning: skipping /run/initramfs/live/boot/boot_hybrid.img (extension).
warning: skipping /run/initramfs/live/boot/eltorito.img (extension).
warning: skipping /run/initramfs/live/boot/grub/grub.cfg (extension).
warning: skipping /run/initramfs/live/boot/grub2/grub.cfg (extension).
warning: skipping /run/initramfs/live/boot/initrd. too big (>1MB)
warning: skipping /run/initramfs/live/boot/kernel. too big (>1MB)
warning: skipping /run/initramfs/live/boot/uefi.img. too big (>1MB)
warning: skipping /run/initramfs/live/config.yaml because it has no valid header
warning: skipping /run/initramfs/live/rootfs.squashfs. too big (>1MB)

previous QA head 3e26fc63 (5 warnings, the state I flagged):

warning: skipping /run/initramfs/live/boot/initrd. too big (>1MB)
warning: skipping /run/initramfs/live/boot/kernel. too big (>1MB)
warning: skipping /run/initramfs/live/boot/uefi.img. too big (>1MB)
warning: skipping /run/initramfs/live/config.yaml because it has no valid header
warning: skipping /run/initramfs/live/rootfs.squashfs. too big (>1MB)

this head bfc35528 (1 warning):

warning: skipping /run/initramfs/live/config.yaml because it has no valid header

The four silenced files are exactly the non-yaml ones the ISO ships: rootfs.squashfs (333 MB), initrd (56 MB), kernel (16 MB), uefi.img (4 MB).

B. One file per class, in-guest

Directory with a valid cloud-config, a header-less yaml, a 3 MB .yaml, a 3 MB .squashfs, and two empty non-yaml files:

file base 3e26fc63 bfc35528
good.yaml (valid) ingested ingested ingested
nohdr.yaml no valid header no valid header no valid header
big.yaml (3 MB) too big too big too big
blob.squashfs (3 MB) too big too big silent
shim.efi, acpi.mod (extension) x2 silent silent

Only blob.squashfs changed, which is the intent. Merged output on this head still resolves good.yaml only:

MERGED=#cloud-config
# Sources:
# - /tmp/bd/good.yaml
qa: ok

C. Regressions checked

  • Live system boots and logs in normally; cloud-config from the datasource still applied (id kairos -> uid=1000(kairos) ... 900(admin)).
  • Valid config still ingested end to end: kairos-agent config get qa_marker -> present.
  • kairos-agent upgrade on live media still refuses with cannot upgrade from live media/unknown boot state.
  • CGO_ENABLED=0 go test ./sdk/collector/... ./agent/pkg/config/... -> ok both.
  • Mutation check on the new spec: putting the size check back before the extension check makes reports the size only for a file it would otherwise have read fail (53 Passed | 1 Failed), so the spec really does pin the ordering.

Notes, neither introduced nor changed here

  1. fileSize divides twice with integer arithmetic (kilobytes := bytes / 1024; megabytes := float64(kilobytes / 1024)), so the > 1.0 guard only trips at 2 MB. A 1.4 MB .yaml is read rather than reported. Pre-existing, unrelated to this diff, mentioned only because it makes the "1MB" in the message inaccurate.
  2. kairos-agent config reaches the collector with logs suppressed, so the warnings above are not visible through that command on a booted system; they surface on the paths that scan with logs enabled. This is why the A/B above drives the collector directly. Also pre-existing.

CI on this head is still authorize / pending, so the hosted suites have not run against bfc35528 yet.

@Itxaka

Itxaka commented Sep 8, 2026

Copy link
Copy Markdown
Member

@ci-robbot rebase based of master

The config collector walks /run/initramfs/live (and the other
GetUserConfigDirs paths) recursively and, for every file whose
extension is not .yml or .yaml, prints a 'warning: skipping <path>
(extension).' line. In practice that means kairos-agent upgrade
prints hundreds of warnings for grub.efi, grub.cfg, *.mod, *.pf2
and every other file that lives next to the cloud-configs.

Silently skip non-yaml extensions instead. The header-missing
warning on a real .yaml file (the one jimmykarily explicitly asked
to keep in kairos-io#2064) is untouched.

Fixes kairos-io#2064

Signed-off-by: ci-robbot <ci-robbot@users.noreply.github.com>
@ci-robbot
ci-robbot force-pushed the fix/2064-drop-extension-warning branch from bfc3552 to 4a77042 Compare September 8, 2026 16:04
@ci-robbot

Copy link
Copy Markdown
Collaborator Author

Rebased onto master (58b5d404), now 4a770420. Two files conflicted and the PR shrank, so flagging what changed.

#4527 landed while this was open and took the ordering fix with it: the extension check now runs before the size check on master already, which is what @jimmykarily asked for here. So I dropped my second commit (bfc35528) entirely, along with its spec, which duplicated the says nothing about oversized files that were never configs spec #4527 added.

What is left is the one thing #2064 actually asks for and master still does not do: skip a non-yaml file silently. Master still prints warning: skipping <path> (extension). per file, which on live media is hundreds of lines for grub modules and EFI binaries. The diff is now 6 lines in collector.go plus one spec.

The header warning on a real .yaml sibling is untouched, and the spec asserts both halves. go test ./sdk/collector/... passes.

@mudler
mudler merged commit a0722a4 into kairos-io:master Sep 10, 2026
1 of 2 checks passed
@github-project-automation github-project-automation Bot moved this from QA to Done βœ… in πŸ§™Issue tracking board Sep 10, 2026

This branch was previously deployed

1 inactive deployment
fork-pr-builds β€” 4a770420 Deployed Sep 8, 2026 by ci-robbot via authorize #364
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Don't scan the whole /run/initramfs/live/ for configs

4 participants