Skip to content

feat(bundle): add resume support to deploy - #1559

Merged
bradclawsie merged 13 commits into
mainfrom
sebastianlaskawiec/cli-177
Sep 24, 2026
Merged

bradclawsie merged 13 commits into
mainfrom
sebastianlaskawiec/cli-177

Conversation

@slaskawi

Copy link
Copy Markdown
Contributor

Description

Adds --resume (-r) to Next-mode uds bundle deploy and uds bundle dev deploy.

Resume runs after package selection and dependency validation. It skips a package only when Zarf records one unambiguous deployment with:

  • the same Zarf package name and namespace
  • the same non-empty package digest
  • the exact intended component set
  • every component in the Succeeded state

Bundle hooks still run when every package is skipped. Cluster-state or intended-package metadata errors stop deployment before hooks or package deployment.

The implementation uses current Zarf APIs for schema-aware package loading, archive digest calculation, selective archive extraction, digest-pinned OCI resolution, and deployment digest recording.

Values-file and deploy-time configuration changes are not represented in the resume match. This boundary is documented in ADR 0027 and the deployment guide.

Related Issue

Relates to CLI-177

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Other (security config, docs update, etc)

Checklist before merging

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
@slaskawi

Copy link
Copy Markdown
Contributor Author

@codex review

@slaskawi

Copy link
Copy Markdown
Contributor Author

@copilot review

@slaskawi

Copy link
Copy Markdown
Contributor Author

@greptileai review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-16T15:14:18.607297Z 3b7fc03 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@greptile-apps

greptile-apps Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The current PR appears safe to merge; no actionable new issue was established from changes made since the previous review.

Summary

The PR adds conservative resume support to Next-mode bundle deployment and expands deploy-time configuration handling.

  • Adds --resume to artifact and development deployment commands.
  • Matches intended package name, namespace, digest, component set, and successful deployment state before skipping packages.
  • Loads metadata without staging complete packages and pins OCI deployment to the digest used for the resume decision.
  • Adds repeatable --set variable overrides and clearer unsigned-artifact verification guidance.
  • Documents resume semantics and adds unit, integration, and cluster coverage.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Parse and validate bundle] --> B[Select packages and validate dependencies]
    B --> C{Resume enabled?}
    C -- No --> G[Run bundle pre-deploy hook]
    C -- Yes --> D[Read deployed Zarf state once]
    D --> E[Load intended package identity]
    E --> F{Exact successful match?}
    F -- Yes --> H[Skip package]
    F -- No --> I[Keep package in deployment DAG]
    H --> J[Filtered deployment DAG]
    I --> J
    J --> G
    G --> K[Deploy remaining packages]
    K --> L[Run bundle post-deploy hook]
    L --> M[Return deployment result]
Loading

Reviews (7) · Last reviewed commit: "comments addressed"

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 998da8c668

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/zarf/layout.go Resolved
Co-authored-by: slaskawi <8417158+slaskawi@users.noreply.github.com>

Copilot AI commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

@copilot review

Pushed a follow-up in 5868626.

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
This reverts commit 5868626.

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
@slaskawi

Copy link
Copy Markdown
Contributor Author

@greptileai review

@slaskawi

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ffb7da48e0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/zarf/resume.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Address the local archive extraction cost, cover the deployed-state read failure path, and update the documented command with verification-policy flags.

Pull request overview

Adds --resume/-r to Next-mode bundle deployments, matching packages against successful Zarf deployment state before redeploying.

Changes:

  • Adds CLI and deployment-layer resume support.
  • Implements digest-, namespace-, and component-aware matching.
  • Adds tests, documentation, and ADR 0027.
File summaries
File Description
tests/testutil/cluster.go Adds a structured deployment test helper.
tests/cluster/bundle_deploy_test.go Tests resume for directory, artifact, and OCI deployments.
pkg/bundle/zarf.go Adds package-spec and state loader adapters.
pkg/bundle/zarf_test.go Tests adapters and dependency behavior.
pkg/bundle/deploy.go Wires resume options into bundle deployment.
internal/zarf/source.go Handles filtered package metadata.
internal/zarf/resume.go Implements deployment-state matching.
internal/zarf/resume_test.go Tests resume matching and orchestration.
internal/zarf/remote.go Resolves and pins OCI digests.
internal/zarf/remote_test.go Tests remote digest behavior.
internal/zarf/local.go Loads local package metadata and digests.
internal/zarf/local_test.go Tests local package loading.
internal/zarf/layout.go Loads package specs from layouts.
internal/zarf/layout_test.go Tests layout filtering and errors.
internal/zarf/deploy.go Applies resume filtering after selection and validation.
internal/cli/bundle/dev_deploy.go Adds the development deploy resume flag.
internal/cli/bundle/dev_deploy_test.go Tests development deploy propagation.
internal/cli/bundle/deploy.go Adds the deploy resume flag.
internal/cli/bundle/deploy_test.go Tests deploy flags and signatures.
internal/cli/bundle/deploy_runner.go Propagates resume through the runner.
internal/cli/bundle/deploy_runner_test.go Tests runner propagation.
docs/how-to-guides/next/deploy-a-bundle.mdx Documents resume usage and limitations.
docs/how-to-guides/migrate-legacy-to-next.mdx Updates migration guidance.
docs/adr/0027-resume-bundle-deploy.md Defines resume behavior and boundaries.
Review details

Suppressed comments (3)

docs/how-to-guides/next/deploy-a-bundle.mdx:104

  • This new copy-paste command omits the artifact verification policy. --resume does not bypass signature verification, so with the default flags this fails validation before resume is reached unless the artifact embeds a usable policy; please show the same --public-key/keyless or explicit unsigned-artifact flags used for the initial deployment.
   CLI_FEATURES=NextMode=true uds bundle deploy ./my-bundle.tar.zst --resume

internal/zarf/local.go:54

  • packager.PackageDigest is not metadata-only for tarballs: it calls Zarf's layout.LoadFromTar, which decompresses the entire archive into Zarf's global temp directory before computing the digest. As a result, every --resume attempt on a large local .tar.zst materializes all payload layers (and line 63 then extracts metadata again), requiring full-package disk/time even when the package will be skipped; it also ignores this loader's configured s.tmpDir. Please use a digest path that can read archive metadata/checksums without materializing the payload, or otherwise avoid this full extraction and honor the configured workspace.
	digest, err := packager.PackageDigest(ctx, path, packager.PackageDigestOptions{Architecture: s.arch})

internal/zarf/resume_test.go:132

  • This case never reaches the state-read branch: because it omits SpecLoader, filterResumeLevels returns resume requires a package spec loader before invoking DeployedPackagesFn. Consequently the test named state read does not cover the new ErrReadDeployedPackages failure path or prove it aborts before hooks and deployment; give this case a valid spec loader (and keep the failing state callback) so that path is exercised.
		{name: "state read", opts: DeployOptions{DeployedPackagesFn: func(context.Context) ([]state.DeployedPackage, error) { return nil, errors.New("state") }}},
  • Files reviewed: 24/24 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
@slaskawi

Copy link
Copy Markdown
Contributor Author

@codex review

@slaskawi

Copy link
Copy Markdown
Contributor Author

@greptileai review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: e8a15c9c36

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical loader mismatch and moderate local archive I/O issue remain, along with documentation corrections.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

docs/how-to-guides/next/deploy-a-bundle.mdx:104

  • This copy-paste resume command omits the trust policy required for the default artifact signature verification path. Unless the user's config already supplies a public-key or keyless policy, it fails validation before deployment; carry over the same verification input shown in the signed-artifact example or state that it is required.

docs/how-to-guides/next/deploy-a-bundle.mdx:101

  • This matching summary omits the Zarf metadata.name, which is part of the actual identity key. A package with the same digest, namespace, and components but a different Zarf name is intentionally redeployed, so the guide should state that name requirement explicitly.
   Re-run a deployment with `--resume` to skip packages Zarf recorded with the same package digest, namespace override, filtered component names, and successful component status:

internal/zarf/local.go:54

  • Calling packager.PackageDigest here uses Zarf's tarball path, which fully decompresses the archive before the selective zarf.yaml/checksums.txt extraction below. A --resume of a local .tar.zst therefore still stages every payload (and then extracts metadata again), so a package that will be skipped pays the full package I/O and disk cost; use a digest path that does not materialize the archive payload or otherwise avoid the full unpack for resume.
	digest, err := packager.PackageDigest(ctx, path, packager.PackageDigestOptions{Architecture: s.arch})
  • Files reviewed: 26/26 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread pkg/bundle/deploy.go Resolved
Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
@slaskawi

Copy link
Copy Markdown
Contributor Author

@codex review

@slaskawi

Copy link
Copy Markdown
Contributor Author

@greptileai review

@slaskawi
slaskawi requested a lite review from Copilot September 16, 2026 10:52
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: c27668e7e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical remote pinning and moderate filtering/temp-directory issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (4)

Previously missed (1) — in code that hasn't changed since the last review.

docs/how-to-guides/next/deploy-a-bundle.mdx:104

  • This copy-paste resume example omits the trust-policy input used by the signed-artifact examples above. Unless the user has configured an embedded trust policy, rerunning this command on a signed artifact fails verification because the earlier --public-key/keyless options are not persisted; include the required trust flags here or explicitly show the unsigned --skip-signature-verification case.

internal/zarf/layout.go:275

  • The resume-only spec path filters out components targeted at another OS, but LoadPackageLayout deploys with BuildComponentFilter alone (lines 259-263). For a package containing only.localOS components, Zarf records the unfiltered component set while the resume matcher sees the OS-filtered set, so an otherwise identical deployment is never skipped. Align the component filter used for spec loading with the one used for deployment.
	filter := filters.Combine(BuildComponentFilter(pkg.OptionalComponents), filters.ByLocalOS(runtime.GOOS))

internal/zarf/layout.go:182

  • The artifact resume spec applies ByLocalOS, while LoadPackageLayout below still applies only BuildComponentFilter (lines 165-168). A package with foreign-OS components therefore has a different intended component set from the set recorded during deployment and will be redeployed on every --resume run instead of being skipped. Use one shared component filter for both paths.
	filter := filters.Combine(BuildComponentFilter(pkg.OptionalComponents), filters.ByLocalOS(runtime.GOOS))

internal/zarf/local.go:54

  • packager.PackageDigest does not accept the configured Next temp directory and, for local archives, calls Zarf's LoadFromTar, which allocates through the process-global config.CommonOptions.TempDirectory. The Next CLI only carries --tmp-dir in ConfigOptions, so resume of a local .tar.zst can ignore the user's temp location and fail when the system temp is unavailable even though normal staging honors the configured directory. Use a digest/loading path that takes s.tmpDir, or otherwise wire this Zarf temp configuration safely.
	digest, err := packager.PackageDigest(ctx, path, packager.PackageDigestOptions{Architecture: s.arch})
  • Files reviewed: 26/26 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread internal/zarf/remote.go Resolved
Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
@slaskawi

Copy link
Copy Markdown
Contributor Author

@greptileai review

@slaskawi

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3b7fc0393f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/zarf/local.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Address the moderate local archive handling issue and the two remaining review findings.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

internal/zarf/local.go:54

  • packager.PackageDigest loads a tarball through Zarf's LoadFromTar, which extracts the entire archive under config.CommonOptions.TempDirectory, not s.tmpDir. This new resume path therefore ignores the configured --tmp-dir and performs a full extraction before the selective metadata extraction below, so large local archives can use the wrong filesystem and lose the intended metadata-only path. Use a temp-dir-aware digest calculation and avoid the duplicate extraction.

docs/how-to-guides/next/deploy-a-bundle.mdx:104

  • This copy-paste resume command omits the trust policy required by artifact deployment. With no --config containing signature_verification and no --skip-signature-verification, the default configuration has no public key or keyless policy, so validation fails before resume runs. Include the required --public-key/keyless flags or explicitly reference the configuration file used by the example.
   CLI_FEATURES=NextMode=true uds bundle deploy ./my-bundle.tar.zst --resume

internal/zarf/remote.go:19

  • Please keep OCI reference parsing and pinning behind internal/oci (or use the already canonicalized repository reference). The repository's Next OCI boundary centralizes ORAS reference handling there; parsing s.ref directly in internal/zarf duplicates that ownership and can diverge from the canonicalization performed by the Zarf/OCI remote layer.
	"oras.land/oras-go/v2/registry"
  • Files reviewed: 26/26 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
@slaskawi
slaskawi marked this pull request as ready for review September 17, 2026 06:23
@slaskawi
slaskawi requested a review from a team as a code owner September 17, 2026 06:23

@bradclawsie bradclawsie left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice so far! One UX question: the ADR says this can be used in the case of an interrupted or repeated deploy. How would I know when to reach for --resume? Does "interrupted" mean a previous deploy failed cleanly with an error message or actual disaster recovery?

@slaskawi

Copy link
Copy Markdown
Contributor Author

Looks nice so far! One UX question: the ADR says this can be used in the case of an interrupted or repeated deploy. How would I know when to reach for --resume? Does "interrupted" mean a previous deploy failed cleanly with an error message or actual disaster recovery?

Thanks for the review @bradclawsie !

Technically, in order the resume functionality uses Zarf Secret to capture all the Packages scheduled for deploying. The ADR captures this by saying:

Zarf has already recorded as the exact intended deployment

At this point, we don't really care what happened. If the Zarf "state" Secret exists for a given package, we'll try to resume it.

@joelmccoy joelmccoy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty happy with this. Would just request we actually test this feature in a a real e2e test for a resume scenario.

Comment thread tests/cluster/bundle_deploy_test.go Resolved
Comment thread internal/cli/bundle/deploy_runner.go Resolved
…i-177

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

# Conflicts:
#	docs/how-to-guides/next/deploy-a-bundle.mdx
#	internal/cli/bundle/deploy.go
#	internal/cli/bundle/deploy_test.go
#	internal/cli/bundle/dev_deploy.go
Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>
chance-coleman
chance-coleman previously approved these changes Sep 23, 2026

@chance-coleman chance-coleman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all my comments were addressed by others, im happy with this. Would want to get @jasonwashburn approval as well before merging.

@jasonwashburn jasonwashburn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like there's conflicts to resolve, but otherwise lgtm

…i-177

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

# Conflicts:
#	tests/cluster/bundle_deploy_test.go
@bradclawsie
bradclawsie merged commit d13efe4 into main Sep 24, 2026
53 checks passed
@bradclawsie
bradclawsie deleted the sebastianlaskawiec/cli-177 branch September 24, 2026 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants