🏆 Live Soccer TV: defenseunicorns/uds cli/pull/

slaskawi · 2026-09-20T18:08:31Z

Description

Cements Next public Go API contracts and divides the Next test suite into unit, library, and CLI layers. Library tests use only public UDS packages; CLI tests use Cobra, with one minimal process check.

Related Issue

Relates to CLI-302

Coverage

Scope	Before	After	Change
All pyramid layers	82.80%	83.44%	+0.64 points
Public API	75.15%	79.39%	+4.24 points

All pyramid layers  82.80% -> 83.44%
Public API          75.15% -> 79.39%

CI timing

Measured required Next CI critical path: 9m00 -> 7m11 (-1m49, -20.2%). Each run includes checkout, tool setup, tests, and cleanup. Optional smoke tests are excluded.

Division	Before	After
Build UDS CLI	51s	53s
Unit tests	5m32	5m17
Unit tests / Next	—	4m17
Library tests / non-cluster	3m08	3m08
Library tests / package verification	—	3m11
Library tests / keyless signing	—	3m33
Library tests / owned cluster	—	4m11
Library tests / in-cluster	—	5m31
CLI tests / non-cluster	7m11	4m54
CLI tests / owned cluster	—	6m59
CLI tests / in-cluster	8m06	7m11
Lint	8m27	7m17

The complete repository workflow also runs unchanged Legacy jobs. It is not compared here because the pre-change reference was a snapshot workflow with optional smoke work enabled.

Type of change

Other (security config, docs update, etc)

Checklist before merging

Test, docs, adr added or updated as needed
Contributor Guide Steps followed

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

slaskawi · 2026-09-20T18:09:33Z

@codex review

slaskawi · 2026-09-20T18:09:34Z

@greptileai review

chatgpt-codex-connector · 2026-09-20T18:09:52Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-21T10:05:29.247170Z	`59f4aa7`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

greptile-apps · 2026-09-20T18:13:16Z

Confidence Score: 5/5

The PR appears safe to merge; no actionable new issue remains, and all previous review findings are resolved.

Summary

This PR establishes and tests the supported Next public Go API while reorganizing Next validation into unit, public-library, and CLI layers.

Adds public API contract coverage for bundle lifecycle, configuration, signing, streams, policies, and cluster behavior.
Converts Next Cobra handlers to return errors through RunE, leaving process-level reporting and exits to the executable.
Splits CI into independently scheduled unit, library, CLI, credentialed, and cluster-backed divisions.
Adds isolated Zarf-ready cluster provisioning and cleanup for supplied-cluster test jobs.
Updates testing guidance and records the architecture in ADR-0027.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  CI[Next CI] --> Unit[Unit tests]
  CI --> Library[Public library tests]
  CI --> CLI[CLI tests]
  Library --> LocalLib[Non-cluster contracts]
  Library --> Signed[Package verification]
  Library --> Keyless[OIDC keyless signing]
  Library --> ClusterLib[Supplied-cluster contracts]
  CLI --> Cobra[Direct Cobra tests]
  CLI --> Process[Minimal process checks]
  CLI --> ClusterCLI[Owned or supplied cluster tests]
  ClusterLib --> Provision[hack/test-cluster.sh]
  ClusterCLI --> Provision
  Provision --> Zarf[Isolated Zarf-ready cluster]

_{Reviews (7) · Last reviewed commit: "test: cement Next public API contracts"}

chatgpt-codex-connector · 2026-09-20T18:13:59Z

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 7c151c91d4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

slaskawi · 2026-09-20T18:40:06Z

@codex review

slaskawi · 2026-09-20T18:40:08Z

@greptileai review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 056f55914d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

slaskawi · 2026-09-21T04:36:21Z

@codex review

slaskawi · 2026-09-21T04:36:23Z

@greptileai review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb8848a5ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

slaskawi · 2026-09-21T05:00:58Z

@codex review

slaskawi · 2026-09-21T05:00:59Z

@greptileai review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da8dd5fe10

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

slaskawi · 2026-09-21T06:27:14Z

@codex review

slaskawi · 2026-09-21T06:27:15Z

@greptileai review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5c45103f0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

slaskawi · 2026-09-21T09:58:32Z

@codex review

slaskawi · 2026-09-21T09:58:33Z

@greptileai review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59f4aa73b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

Copilot

Copilot review overview

🟡 Changes recommended

Root error-handling was updated to rely on RunE/returned errors, but internal Zarf command wiring still uses util.CheckErr (os.Stderr + os.Exit), which breaks the new contract and can terminate in-process executions unexpectedly.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

Refactor internal commands to return errors instead of exiting · New

What changed in this PR

This PR cements Next-mode public Go API contracts (primarily pkg/bundle, pkg/bundle/spec, pkg/iostreams) and restructures the Next test suite into a unit / library / CLI pyramid, with CI and contributor docs updated to match.

Changes:

Adds/expands tests/library contract coverage for public types, errors, signing/verification, deploy source lifecycle, configuration propagation, and extension points.
Refactors Next CLI command wiring to consistently return errors via RunE, adds a test helper for in-process Cobra execution, and limits true process-level checks to a minimal binary entrypoint suite.
Updates CI workflows, local tasks, and contributor documentation to run the new test divisions and (when needed) provision Zarf-ready clusters outside the suites.

File	Description
tests/testutil/path.go	Removes wrappers that executed the built CLI binary; keeps Cobra-based helpers.
tests/testutil/cluster.go	Switches CLI test helpers from exec’ing a binary to in-process Cobra execution via `ExecuteCLI`.
tests/testutil/cli.go	Adds serialized in-process CLI execution helper (`ExecuteCLI`) that restores global process state.
tests/testutil/cli_test.go	Verifies `executeCLI` restores cwd, slog default, and Helm managed-fields manager after errors.
tests/library/streams_contract_test.go	Establishes public `iostreams` behavior contracts and logger propagation.
tests/library/spec_contract_test.go	Establishes public bundle spec validation + typed error identity contracts.
tests/library/source_lifecycle_test.go	Contracts for `PrepareDeploySource` path resolution, tmpdir behavior, and cleanup ownership.
tests/library/signing_contract_test.go	Contracts for public sign/verify flows for local artifacts and OCI references.
tests/library/remove_artifact_test.go	Refactors removal artifact tests to use library fixtures and public entrypoints.
tests/library/reconfigure_contract_test.go	Adds public reconfigure contract coverage (local + OCI) including signature behavior.
tests/library/policy_contract_test.go	Validates exported signing/policy options and their `Validate()` error identities.
tests/library/keyless_contract_test.go	Adds CI-only keyless signing contracts and keyless package verification coverage.
tests/library/fixtures_test.go	Introduces reusable public-library fixtures (bundle, packages, registry) with public-only deps.
tests/library/extension_contract_test.go	Contracts for custom loader/staging behavior and hook visibility.
tests/library/deploy_subset_test.go	Strengthens dependency-violation assertions via typed error extraction.
tests/library/configuration_contract_test.go	Contracts for config propagation to hooks, defaults merge immutability, and template vars.
tests/library/cluster/lifecycle_test.go	Public API cluster lifecycle smoke (owned cluster) via library deploy.
tests/library/cluster/fixtures_test.go	Cluster fixture generation for public API cluster tests (non-owned).
tests/library/cluster/environment_test.go	In-cluster vs owned-cluster environment contracts and k3d lifecycle helpers.
tests/integration/cmd/zarf/zarf_integration_test.go	Removed legacy Next “integration” tier test file (migrated to `tests/cli`).
tests/integration/cmd/bundle/remove_integration_test.go	Removed legacy Next “integration” tier remove suite (migrated to `tests/cli`).
tests/integration/cmd/bundle/deploy_integration_test.go	Removed legacy Next “integration” tier deploy suite (migrated to `tests/cli`).
tests/integration/cmd/bundle/create_integration_test.go	Removed legacy Next “integration” tier create tests (migrated/layered elsewhere).
tests/integration/cmd/bundle/bundle_assertions_test.go	Removed duplicated archive assertion utilities (replaced by fixtures/artifact helper).
tests/fixtures/artifact/archive.go	Adds neutral artifact inspection utilities for OCI archive assertions without large blobs.
tests/cluster/operator_monitor_test.go	Switches to in-process CLI invocation and uses Zarf APIs for deploy/remove of monitor package.
tests/cluster/lifecycle/lifecycle_test.go	Adds CLI lifecycle contract test for owned-cluster creation/cleanup through Cobra execution.
tests/cluster/bundle_remove_test.go	Migrates cluster remove test to in-process CLI execution; removes redundant tarball/OCI variants.
tests/cluster/bundle_deploy_test.go	Consolidates deploy+remove flows for artifact/OCI using in-process CLI and structured results.
tests/cluster/bootstrap_test.go	Removes redundant bootstrap assertion test (covered via new lifecycle coverage).
tests/cli/cmd/zarf/zarf_integration_test.go	Adds minimal process-level entrypoint coverage for Zarf passthrough and exit behavior.
tests/cli/cmd/version/version_integration_test.go	Retags integration test to `cli` layer.
tests/cli/cmd/bundle/signing_integration_test.go	Retags and refactors signing CLI coverage; removes duplicated tamper/pull tests moved to library.
tests/cli/cmd/bundle/remove_integration_test.go	Adds CLI-layer remove suite using in-process Cobra execution helper.
tests/cli/cmd/bundle/reconfigure_integration_test.go	Retags to `cli` and leans on new shared fixture helpers.
tests/cli/cmd/bundle/helpers_test.go	Refactors helpers to use new artifact fixture reader and in-process ExecuteCLI.
tests/cli/cmd/bundle/deploy_integration_test.go	Adds CLI-layer deploy suite using in-process Cobra execution helper.
tests/cli/cmd/bundle/create_integration_test.go	Adds CLI-layer defaults/create contracts and programmatic artifact creation fixture.
tests/cli/cmd/bundle/bundle_integration_test.go	Retags to `cli` and refactors to use new fixture helpers and reduced duplicate coverage.
tasks/tests.yaml	Replaces `next-integration` tasks with explicit `next-unit`, `next-library-`, `next-cli-` tasks.
pkg/bundle/deploy.go	Ensures `PrepareDeploySource` returns absolute paths and resolves tmpdir defaults safely.
internal/cli/version/version.go	Converts version command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/core/operator_monitor.go	Converts operator monitor to `RunE` error returns instead of `util.CheckErr`.
internal/cli/cmd.go	Silences Cobra’s default error/usage printing and routes logging to provided `IOStreams`.
internal/cli/bundle/verify.go	Converts verify command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/sign.go	Converts sign command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/remove.go	Converts remove command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/reconfigure.go	Converts reconfigure command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/push.go	Converts push command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/pull.go	Converts pull command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/inspect.go	Converts inspect command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/dev_remove.go	Converts dev remove command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/dev_deploy.go	Converts dev deploy command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/deploy.go	Converts deploy command to `RunE` error returns instead of `util.CheckErr`.
internal/cli/bundle/create.go	Converts create command to `RunE` error returns instead of `util.CheckErr`.
hack/test-cluster.sh	Adds external cluster provision/cleanup helper for supplied-cluster CI runs.
docs/adr/0027-public-api-test-pyramid.md	Introduces ADR defining the test pyramid and its constraints for Next.
CONTRIBUTING.md	Updates contributor guidance to new Next test tiers and tasks.
AGENTS.md	Updates agent guidance for Next test pyramid, error handling, and task mapping.
.github/workflows/test.yaml	Splits Next CI into unit/library/cli divisions and adds supplied-cluster provisioning for in-cluster suites.
.github/workflows/test-shim.yaml	Updates shim workflow to mirror the new Next pyramid job names.
.agents/skills/testing/SKILL.md	Updates testing skill to reference ADR-0027 and new Next test structure.
.agents/skills/testing/references/TESTING.md	Updates testing strategy docs to the new pyramid and cluster execution model.
.agents/skills/go-development/references/GO_CODE_STANDARDS.md	Updates Go standards doc to reflect RunE error return and new test locations.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

bradclawsie

An incredible amount of work and if your time-savings claims are true, a big win. With so much code here I think it will take some iterations to let everyone get a crack at it. I liked to see the move from Run to RunE. My points so far are minor, looking forward to what others have to say.

chance-coleman

some initial review comments

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

…i-302 Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com> # Conflicts: # tests/cluster/bundle_deploy_test.go # tests/integration/cmd/bundle/create_integration_test.go # tests/integration/cmd/bundle/deploy_integration_test.go

bradclawsie

I've read this over twice. Overall its too much for someone to comprehend. Read yes, comprehend, no. Maybe in the future make these stacked PRs. Given that the code is test-focused I am not too concerned about blast radius; if this were production code I might demur from review. Approved since I didn't see anything that jumps out at me and the code appears to be good quality. Will be interested to see what others say about the sheer volume.

docs: define the Next public API test pyramid

2e8dd63

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread tasks/tests.yaml Resolved

Comment thread .github/workflows/test.yaml Resolved

slaskawi force-pushed the sebastianlaskawiec/cli-302 branch from 7c151c9 to 056f559 Compare September 20, 2026 18:39

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread .github/workflows/test.yaml Resolved

Comment thread .github/workflows/test.yaml Resolved

chatgpt-codex-connector Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread .github/workflows/test.yaml Resolved

Comment thread .github/workflows/test.yaml Resolved

Comment thread tasks/tests.yaml Resolved

slaskawi force-pushed the sebastianlaskawiec/cli-302 branch from 056f559 to fb8848a Compare September 21, 2026 03:34

chatgpt-codex-connector Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread tasks/tests.yaml Resolved

slaskawi force-pushed the sebastianlaskawiec/cli-302 branch from fb8848a to da8dd5f Compare September 21, 2026 05:00

chatgpt-codex-connector Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread .github/workflows/test.yaml Resolved

Comment thread tasks/tests.yaml Resolved

slaskawi force-pushed the sebastianlaskawiec/cli-302 branch from da8dd5f to d5c4510 Compare September 21, 2026 06:26

chatgpt-codex-connector Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread AGENTS.md Resolved

slaskawi force-pushed the sebastianlaskawiec/cli-302 branch 3 times, most recently from 5acd73b to 59f4aa7 Compare September 21, 2026 09:57

chatgpt-codex-connector Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread tasks/tests.yaml Resolved

test: cement Next public API contracts

6eae82b

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

slaskawi force-pushed the sebastianlaskawiec/cli-302 branch from 59f4aa7 to 6eae82b Compare September 21, 2026 10:13

slaskawi marked this pull request as ready for review September 21, 2026 10:25

Copilot AI lite review requested due to automatic review settings September 21, 2026 10:25

slaskawi requested a review from a team as a code owner September 21, 2026 10:25

Copilot started reviewing on behalf of slaskawi September 21, 2026 10:25

View session

Copilot AI reviewed Sep 21, 2026

View reviewed changes

Comment thread internal/cli/cmd.go Resolved

bradclawsie reviewed Sep 21, 2026

View reviewed changes

Comment thread .agents/skills/testing/references/TESTING.md Resolved

Comment thread .github/workflows/test.yaml Resolved

Comment thread docs/adr/0027-public-api-test-pyramid.md Resolved

chance-coleman reviewed Sep 21, 2026

View reviewed changes

Comment thread tests/testutil/path.go Resolved

Comment thread tests/cli/cmd/bundle/create_integration_test.go Resolved

slaskawi commented Sep 22, 2026

View reviewed changes

Comment thread hack/test-cluster.sh Resolved

slaskawi added 2 commits September 22, 2026 15:38

comments addressed

399c54d

Signed-off-by: Sebastian Łaskawiec <sebastian.laskawiec@defenseunicorns.com>

bradclawsie approved these changes Sep 22, 2026

View reviewed changes

chance-coleman approved these changes Sep 23, 2026

View reviewed changes

chance-coleman merged commit 3ff062d into main Sep 23, 2026
53 checks passed

chance-coleman deleted the sebastianlaskawiec/cli-302 branch September 23, 2026 15:15

Conversation

slaskawi commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Coverage

CI timing

Type of change

Checklist before merging

Uh oh!

slaskawi commented Sep 20, 2026

Uh oh!

slaskawi commented Sep 20, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

greptile-apps Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Diagram

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Sep 20, 2026

Uh oh!

slaskawi commented Sep 20, 2026

Uh oh!

slaskawi commented Sep 20, 2026

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

slaskawi commented Sep 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

slaskawi commented Sep 20, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Sep 20, 2026 •

edited

Loading

greptile-apps Bot commented Sep 20, 2026 •

edited

Loading