Skip to content

worker: preserve container spec ordering after resolution - #5218

Merged
achilleas-k merged 4 commits into
osbuild:mainfrom
ondrejbudai:fix-container-resolver
Aug 10, 2026
Merged

achilleas-k merged 4 commits into
osbuild:mainfrom
ondrejbudai:fix-container-resolver

Conversation

@ondrejbudai

@ondrejbudai ondrejbudai commented Aug 4, 2026 •

Copy link
Copy Markdown
Member

container.Resolver.Finish() sorts results by digest so it's not guaranteed that it keeps the input order. However, osbuild-composer mistakenly started relying on keeping the order:

For bootc-generic-iso with an iso_payload_reference, the os-tree pipeline carries two containers. If the payload container's digest sorts before the source container's, they end up in reversed positions, causing the payload to be deployed as the OS tree and vice versa.

Fix this by switching to the new-ish Resolve() method of the resolver that does resolving synchronously. This allows us to neatly condense the 2 loops into 1 while making sure that the order is not changed.

Kudos to @lucasgarfield for discovering the bug, and to @supakeen for finding out that the container refs get somewhere swapped.

@ondrejbudai
ondrejbudai requested a review from thozza as a code owner August 4, 2026 17:25
@ondrejbudai
ondrejbudai force-pushed the fix-container-resolver branch from 7cc62a5 to 5671b2d Compare August 4, 2026 17:45
achilleas-k
achilleas-k previously approved these changes Aug 4, 2026

@achilleas-k achilleas-k left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! I see now.

LGTM

@ondrejbudai

Copy link
Copy Markdown
Member Author

Well, the former approach wasn't gonna fly because the resolver normalizes the Name field so we cannot use it to preserve the ordering.

When I started digging deeper, I noticed we now have a handy ResolveAll method that does exactly what we need. So we just need some shuffling between different struct formats and calling this method to keep the order.

ondrejbudai added a commit to ondrejbudai/image-builder that referenced this pull request Aug 5, 2026
As a part of osbuild/osbuild-composer#5218
I wanted to switch to the blocking resolver as we want to drop the async
one going forward. However, it's not possible to set the auth file
using the Resolver interface.

Let's add a SetAuthFilePath method to the interface to fill this gap and
unblock osbuild-composer's migration to the blocking resolver.
achilleas-k
achilleas-k previously approved these changes Aug 5, 2026

@achilleas-k achilleas-k left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good!

@achilleas-k
achilleas-k enabled auto-merge (rebase) August 5, 2026 09:05
supakeen
supakeen previously approved these changes Aug 5, 2026

@supakeen supakeen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks OK, small comment about losing the pipeline name.

However, it would be really nice to have a test that asserts the invariant introduced here so it doesn't get optimized away again in the future. I don't know how much work that would be if it's a lot then let's not bother and address the bug first.

Not a functional change, just preparing ground for changing the resolving logic in next commits.
container.Resolver.Finish() sorts results by digest so it's not
guaranteed that it keeps the input order. However, osbuild-composer
mistakenly started relying on keeping the order:

For bootc-generic-iso with an iso_payload_reference, the os-tree
pipeline carries two containers. If the payload container's digest sorts
before the source container's, they end up in reversed positions,
causing the payload to be deployed as the OS tree and vice versa.

Fix this by switching to the new-ish Resolve() method of the resolver that
does resolving synchronously. This allows us to neatly condense the 2 loops
into 1 while making sure that the order is not changed.

Kudos to @lucasgarfield for discovering the bug, and to @supakeen for
finding out that the container refs get somewhere swapped.
Move resolver creation into the caller and pass it as a
container.Resolver interface parameter. This makes the function
testable with a mock resolver. No behaviour change.
@ondrejbudai
ondrejbudai dismissed stale reviews from supakeen and achilleas-k via 74901f4 August 5, 2026 11:20
@ondrejbudai
ondrejbudai force-pushed the fix-container-resolver branch from ea12c7e to 74901f4 Compare August 5, 2026 11:20
Add a test with a mock resolver that verifies ResolveContainers
preserves the input ordering regardless of digest values. The
os-tree pipeline carries two containers whose digests would sort
in reverse order.

Note that I had to make the ResolveContainers function public because
it's in the main module so the test needs to be main_test.
@ondrejbudai
ondrejbudai force-pushed the fix-container-resolver branch from 74901f4 to 7ec73b6 Compare August 5, 2026 11:23
@ondrejbudai

Copy link
Copy Markdown
Member Author

You are right, @supakeen, ResolveAll() looks tempting but it forces us to drop the pipeline name, and it forces us to do a weird double-loop dance for converting the structs.

I decided to use a simple Resolve instead that allows us to have just a single loop and much more readable code. Moreover, I extracted the resolution code into a separate method so it can be tested using a mock resolver. Now the ordering requirement for the resolving job is actually expressed in the codebase which I really like. :)

achilleas-k pushed a commit to src-up/image-builder that referenced this pull request Aug 6, 2026
As a part of osbuild/osbuild-composer#5218
I wanted to switch to the blocking resolver as we want to drop the async
one going forward. However, it's not possible to set the auth file
using the Resolver interface.

Let's add a SetAuthFilePath method to the interface to fill this gap and
unblock osbuild-composer's migration to the blocking resolver.
@achilleas-k
achilleas-k disabled auto-merge August 10, 2026 11:00
@achilleas-k

Copy link
Copy Markdown
Member

Needs a code owner approval from Tomas, but he's out. Merging anyway.

@achilleas-k
achilleas-k merged commit 425903f into osbuild:main Aug 10, 2026
53 checks passed
@schutzbot schutzbot added the prod This PR has landed in production. label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

prod This PR has landed in production.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants