Skip to content

[improve] Exclude filesystem offloader NAR from base image - #26085

Merged
merlimat merged 2 commits into
apache:masterfrom
merlimat:mmerli/exclude-filesystem-offloader-from-image
Jun 24, 2026
Merged

merlimat merged 2 commits into
apache:masterfrom
merlimat:mmerli/exclude-filesystem-offloader-from-image

Conversation

@merlimat

@merlimat merlimat commented Jun 24, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

The filesystem (Hadoop) tiered-storage offloader NAR bundles hadoop-common and its transitive dependencies, adding ~50MB to the base Pulsar Docker image. The vast majority of deployments use the jcloud offloader (S3/GCS/Azure) or no offloader at all, so this is dead weight in the default image.

Modifications

  • Remove tiered-storage-file-system-*.nar from the offloader staging stage of both the Alpine (Dockerfile) and Wolfi (Dockerfile.wolfi) base image builds, so it never reaches the final image. The jcloud offloader is unaffected.
  • Overlay the filesystem offloader NAR back onto the pulsar-test-latest-version integration-test image, since the filesystem offload integration tests (TestFileSystemOffload, TestOffloadDeletionFS) require it. The NAR is resolved via a nar-typed configuration, staged into the build context, and copied into /pulsar/offloaders/.

The filesystem offloader is still built and shipped in the apache-pulsar-offloaders-*-bin.tar.gz distribution tarball, so users who rely on local-disk/HDFS offload can add the NAR back into /pulsar/offloaders.

Verifying this change

  • Make sure that the change passes the CI checks.

This is a build/packaging-only change. Verified by inspecting the produced image contents and the filesystem offload integration tests, which run against the test image that overlays the NAR.

### Motivation

The filesystem (Hadoop) tiered-storage offloader NAR bundles hadoop-common and
its transitive dependencies, adding ~50MB to the base Pulsar Docker image. The
vast majority of deployments use the jcloud offloader (S3/GCS/Azure) or no
offloader at all, so this is dead weight in the default image.

### Modifications

Remove `tiered-storage-file-system-*.nar` from the offloader staging stage of
both the Alpine (`Dockerfile`) and Wolfi (`Dockerfile.wolfi`) image builds, so
it never reaches the final image. The jcloud offloader is unaffected.

The filesystem offloader is still built and shipped in the
`apache-pulsar-offloaders-*-bin.tar.gz` distribution tarball, so users who rely
on local-disk/HDFS offload can add the NAR back into `/pulsar/offloaders`.
@merlimat merlimat changed the title [improve][docker] Exclude filesystem offloader NAR from base image [improve] Exclude filesystem offloader NAR from base image Jun 24, 2026
The filesystem offloader NAR is excluded from the base Pulsar image, but the
filesystem offload integration tests (TestFileSystemOffload, TestOffloadDeletionFS)
require it. Layer it back into the pulsar-test-latest-version image: resolve the
`tiered-storage-file-system` NAR via a nar-typed configuration, stage it into the
build context, and COPY it into /pulsar/offloaders/.
@merlimat
merlimat merged commit 29e95d3 into apache:master Jun 24, 2026
81 of 83 checks passed
@lhotari lhotari added this to the 5.0.0-M2 milestone Aug 14, 2026
@merlimat
merlimat deleted the mmerli/exclude-filesystem-offloader-from-image branch October 24, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants