Containers

Kubernetes on Fedora 45 / 44: k3s, kind, minikube or kubeadm

Firewalld on Fedora doesn’t block your Kubernetes NodePorts. It blocks your pods from reaching their own host, which is why a fresh k3s install on Fedora Server leaves metrics-server stuck at 0/1 while every NodePort answers from the LAN. Running Kubernetes on Fedora is mostly a matter of knowing which trap belongs to which tool, and each of the four common choices has its own.

Original content from computingforgeeks.com - post 188019

We ran k3s, kind, minikube, and kubeadm (the last one built from Fedora’s native kubernetes1.36 packages) on Fedora 45 and Fedora 44 Cloud VMs, with SELinux enforcing and firewalld on the FedoraServer zone, and broke each one before fixing it. That covers the Podman and Docker drivers, rootless clusters, CRI-O on the versioned packages and a minor upgrade done by swapping packages. Everything below was run in November 2026 with k3s 1.37.2, kind 0.33.0 (0.31.0 on Fedora 44), minikube 1.39.0 and CRI-O 1.36 and 1.37 from the Fedora repos.

The machine decides the tool

So which one should you run? There is not one right way to run Kubernetes on Fedora, but there is a right one for each machine you own. What decides it is where your cluster lives and how long it has to survive a reboot or a logout:

  • kind on rootless Podman for the laptop you code on
  • minikube with Podman or kvm2 when you need addons or a VM kernel
  • k3s for a Fedora Server box or homelab node
  • kubeadm on the kubernetesX.Y packages when you want upstream behaviour

Every number below comes from the lab runs. Create times include the first image pulls, so the minikube Docker figure carries a 507 MB base image download that your later starts skip.

Tool and driverTested onKubernetesCluster ready inMemory used
k3sFedora 44 and 45v1.37.2+k3s1Ready at 47 s node age736 MiB (whole node)
kind, rootless PodmanFedora 45v1.37.035.5 s549 MB node container
kind, Docker (moby-engine)Fedora 44v1.35.038.6 s438 MiB node container
minikube, rootless PodmanFedora 45v1.37.049.0 s504 MB node container
minikube, kvm2Fedora 45v1.37.048.5 s2.9 GB qemu RSS
minikube, DockerFedora 44v1.37.084.4 snot measured
kubeadm + CRI-O (Fedora packages)Fedora 45 and 44v1.36.6, then v1.37.2Ready at 63 to 64 s node agenot measured

Container drivers all land around half a gigabyte, so you can keep a kind cluster and a minikube cluster up side by side on a 16 GB machine, which we did on the Fedora 45 VM. A kvm2 VM costs five to six times that, and you only want it when you need a separate kernel. Kind on Fedora 44 lags because the 44 repo still ships kind 0.31, whose default node image is Kubernetes 1.35, so you get an older cluster there. Node images are built per kind release, so for a newer Kubernetes on 44 use the upstream kind binary rather than forcing a newer image onto kind 0.31. If you are comparing the lightweight distributions on their own terms, the k0s vs k3s vs MicroK8s benchmark goes deeper on k3s.

Fedora ships every Kubernetes minor at once

Fedora does not have one kubernetes package. It carries a separate set per minor release, five of them on both Fedora 45 and 44 right now, so you choose the minor you run instead of taking whatever your release shipped with. You can list them with dnf.

dnf list --available 'kubernetes*'

Fedora 45 offers kubernetes1.33 through kubernetes1.37, each split into the same four subpackages, and Fedora 44 carries the same five minors at the same point releases:

dnf list showing kubernetes1.33 to kubernetes1.37 packages on Fedora 45

The split matters when you install, because each piece has a different job. In the order a kubeadm node needs them:

  1. kubernetesX.Y holds the kubelet and its unit file
  2. kubernetesX.Y-kubeadm adds kubeadm and requires the same minor’s kubelet
  3. kubernetesX.Y-client is kubectl on its own
  4. kubernetesX.Y-systemd runs the control plane as systemd services

You can leave the last one out, because kubeadm runs the apiserver, scheduler and controller manager as static pods instead. Only one minor can be installed at a time. Every -client package provides kubernetes-client and conflicts with the others, so asking for kubernetes1.36 while kubectl from 1.37 is installed fails outright.

Problem: problem with installed package
  - installed package kubernetes1.37-client-1.37.2-1.fc45.x86_64 conflicts with kubernetes-client provided by kubernetes1.36-client-1.36.4-2.fc45.x86_64 from fedora

Adding --allowerasing lets dnf remove the old minor in the same transaction. You will use the same flag to upgrade a kubeadm node later, and it has a side effect covered in the upgrade section. CRI-O follows the same scheme (cri-o1.33 to cri-o1.37), and kind is packaged too. minikube is not, so you get it from upstream’s RPM. For day-to-day dnf work on these packages, the DNF5 cheatsheet for Fedora covers repoquery and history.

k3s needs firewalld to trust its CIDRs

k3s installs cleanly on Fedora with SELinux enforcing, and the node goes Ready, so you won’t see the firewalld problem until something inside your cluster tries to reach the host. Start by checking which zone your interface is in, because the fix depends on it.

sudo firewall-cmd --get-default-zone

Which zone you see decides how much of this section bites you:

  • FedoraServer on Server installs, allowing only ssh, cockpit and dhcpv6-client
  • FedoraWorkstation on desktops, opening TCP and UDP 1025 to 65535
  • public on a Cloud image once you add firewalld yourself

On a Workstation laptop you will miss most of what follows, because the high ports are already open and only host services below 1025 stay blocked. Our Cloud VMs had no firewalld at all, so we installed it and switched the default to FedoraServer to match a server install.

Most guides pipe the installer script straight to sh. Downloading it first lets you read what it does before it runs as root.

curl -sfL https://get.k3s.io -o k3s-install.sh
sudo sh k3s-install.sh

On Fedora the script adds Rancher’s RPM repo and installs k3s-selinux, the policy module that labels the k3s binaries. You’ll notice it pulls the el8 build, because Rancher publishes no Fedora tree, and it still installed and loaded on both releases.

[INFO]  Using v1.37.2+k3s1 as release
[INFO]  Finding available k3s-selinux versions
 k3s-selinux noarch 0:1.6-1.el8 rancher-k3s-common-stable  94.2 KiB
[3/3] Installing k3s-selinux-0:1.6-1.el 100% |   8.2 KiB/s |  94.6 KiB |  00m12s
Complete!
[INFO]  systemd: Starting k3s

Two minutes later on the FedoraServer zone, metrics-server is still 0/1 and kubectl top has nothing to show you. Why would a pod fail to reach its own node? Its log answers that it can’t connect to the kubelet on the node’s IP, and a throwaway busybox pod gets the same answer for port 6443. “No route to host” is not a routing failure here. Firewalld sends that ICMP reject when a packet from cni0 lands in the default zone and matches no allowed port.

k3s metrics-server failing to scrape the node with no route to host under firewalld on Fedora 44

The two directions of traffic are treated differently, and that is the whole story of firewalld on a Kubernetes node. A NodePort request from the LAN is rewritten to a pod address and forwarded, while a pod calling its own node’s IP directly is local delivery that meets the zone’s input rules:

Diagram of firewalld on a Fedora Kubernetes node: NodePort traffic forwarded to the pod, pod to node IP rejected by the FedoraServer zone, accepted once the pod CIDR is in the trusted zone

The k3s requirements page recommends turning firewalld off, and the same page lists the rules for keeping it on, which is what we did. Put the pod CIDR (10.42.0.0/16) and the service CIDR (10.43.0.0/16) in the trusted zone, and open 6443 for clients and agents.

sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16
sudo firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16
sudo firewall-cmd --reload

Within a minute metrics-server turns 1/1 without a restart. Check both the pod and the Metrics API:

sudo k3s kubectl get pods -n kube-system -l k8s-app=metrics-server
sudo k3s kubectl top node

The pod shows 1/1 and the node reports its CPU and memory use:

firewalld trusted zone holding the k3s pod and service CIDRs and kubectl top node working on Fedora 44

Trusting the CIDRs is broader than opening 10250, and that is deliberate, because any host service your pods need (node-exporter, a local registry, an NFS server) would hit the same reject. If you change --cluster-cidr or --service-cidr, change these sources with it. Zone and rule mechanics are covered in configuring firewalld on Fedora. We reproduced the break on Fedora 44. On Fedora 45 the installer took the same el8 module path, but metrics-server came up 1/1 there only because 10250 was already open from the kubeadm test, so treat the two rules as needed on both. The k3s server ran as container_runtime_t on both, and ausearch found no AVC denials on Fedora 44.

dnf install kind brings Docker unless you stop it

Kind is in the Fedora repos, so the obvious command is sudo dnf install kind. What happens when you run it on a Podman machine? dnf installs Docker. The package requires docker-cli or podman-docker, dnf picks the first, and the weak dependencies drag in moby-engine, containerd and buildx.

dnf history showing kind pulling docker-cli and moby-engine as dependencies on Fedora 45

Name the Podman shim in the same command and dnf satisfies the dependency with it instead. kubectl comes from the versioned client package:

sudo dnf install kind podman-docker
sudo dnf install kubernetes1.37-client

With podman-docker present, kind detects Podman on its own and prints “enabling experimental podman provider” without the KIND_EXPERIMENTAL_PROVIDER variable. Create your cluster as a normal user, not with sudo.

kind create cluster --name dev

It finished in 35.5 seconds on Fedora 45, rootless, with no cgroup delegation changes. Fedora’s systemd already delegates the cpu, memory and pids controllers to user sessions, which is what the kind rootless docs ask for.

enabling experimental podman provider
Creating cluster "dev" ...
 ✓ Ensuring node image (kindest/node:v1.37.0) 🖼️
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-dev"

Then we logged out, and the cluster died with us. A rootless container belongs to your user’s systemd instance, and on a machine you reach over SSH that instance stops with your last session. Your node container exits with 137, and the API port refuses connections the next time you log in. A desktop session that stays logged in won’t hit this, but a server or a VM you SSH into will. Enabling linger keeps your user manager running, and the stopped node starts again with its state intact.

sudo loginctl enable-linger $USER
podman start dev-control-plane

After podman start the node was back up within seconds, and with linger on your user manager no longer stops when you log out. Look at the last command in the capture too. The kubelet and apiserver inside the node run as spc_t, the super privileged container type, so SELinux is not confining your kind node at all.

rootless Podman kind node exited 137 after logout, then Ready after enabling linger on Fedora 45

Running Kubernetes nested in a container costs you that confinement, which is fine for a dev cluster. If you would rather keep Docker, the plain dnf install kind on Fedora 44 gave a working Docker setup once the daemon was enabled and the user was in the docker group. If you prefer upstream’s packages to moby-engine, installing Docker on Fedora covers Docker CE.

sudo dnf install kind
sudo systemctl enable --now docker
sudo usermod -aG docker $USER

Log out and back in for the group to apply, then run kind create cluster as before. Membership in the docker group is root-equivalent on that machine, so only add accounts you’d trust with sudo. Docker puts its bridges in a docker firewalld zone with target ACCEPT, so the Docker drivers need no firewall changes for traffic between the node container and the host.

minikube uses sudo until you say rootless

Fedora does not package minikube, and upstream’s RPM is unsigned. On Fedora 45 that matters, because rpm now verifies every package (%_pkgverify_level is all, where Fedora 44 still says digest). If you reach for rpm -i, it refuses the file with “does not verify: no signature”. Install it through dnf instead, which accepts a local file and only warns that it skipped the OpenPGP check.

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm
sudo dnf install ./minikube-latest.x86_64.rpm

The Podman driver has a second surprise. By default minikube runs Podman through sudo, and our first start said so plainly with “Using Podman driver with root privileges”. Your cluster then lives in root’s Podman storage, invisible to your own podman ps. It only worked unattended because the cloud user had passwordless sudo, which the minikube Podman driver docs list as a requirement for rootful mode. On a workstation, switch to rootless before the first start.

minikube config set rootless true
minikube start --driver=podman

The rootless start reports “Using rootless Podman driver” and picks containerd as the runtime, which is what the default --container-runtime=auto resolved to here. Rootless mode also needs subordinate ID ranges in /etc/subuid and /etc/subgid, and Fedora had already given our user a 65,536-ID range in both. The capture shows the refused rpm, the dnf install and the rootless start:

rpm refusing the unsigned minikube RPM and minikube starting with the rootless Podman driver on Fedora 45

The same linger rule applies here, since this is the same kind of rootless container. That leaves you four minikube drivers that worked on these two releases:

  • Podman through sudo, the default, in root’s storage
  • Podman rootless, once rootless is set to true
  • Docker from moby-engine, tested on Fedora 44
  • kvm2 for a full VM through libvirt

When you need a real kernel inside the node (kernel modules, a CSI driver, anything that pokes at the host kernel), use kvm2. It talks to libvirt’s system instance, so install the modular daemons and add yourself to the libvirt group.

sudo dnf install libvirt-daemon-kvm libvirt-daemon-config-network qemu-kvm
sudo systemctl enable --now virtqemud.socket virtnetworkd.socket virtstoraged.socket virtnodedevd.socket virtinterfaced.socket virtsecretd.socket
sudo usermod -aG libvirt $USER

Enable the sockets rather than libvirtd.socket. On Fedora 45 asking for both failed with “Transaction contains conflicting jobs ‘stop’ and ‘start’ for libvirtd.socket”, because the modular daemons replace the monolithic one. After a fresh login, you can run a second profile beside the Podman one.

minikube start -p vm --driver=kvm2

Our VM came up in 48.5 seconds, Buildroot kernel and all. SELinux handled it the way it handles any libvirt guest, so qemu ran as svirt_t with its own category pair, the disk image under ~/.minikube was relabeled svirt_image_t, and nothing was denied. If KVM itself isn’t set up yet, installing KVM on Fedora covers the host side.

kubeadm on the kubernetes1.36 packages

The native packages give you upstream kubeadm with Fedora-built binaries, but Fedora 45 adds traps the upstream docs don’t mention. We hit four on Fedora 45 and two more on a fresh Fedora 44, then confirmed the clean order below on Fedora 44. Install the minor you want with its matching CRI-O:

sudo dnf install kubernetes1.36 kubernetes1.36-kubeadm kubernetes1.36-client cri-o1.36 containernetworking-plugins

Add --allowerasing if a different minor’s kubectl is already installed. Then prepare the host in order, because each step clears a separate kubeadm or CRI-O failure:

  1. Turn off zram swap, or the kubelet won’t start.
  2. Enable IP forwarding, or preflight stops, and load br_netfilter for bridged pod traffic.
  3. On Fedora 45, copy the containers policy files where CRI-O 1.36 looks.
  4. Move CRI-O’s own CNI configs aside so flannel owns the pod network.
  5. Open 6443 and 10250 in the zone.

Error: “running with swap on is not supported”

Fedora sets up zram swap on every install, including the cloud images (8 GB on our 16 GB VMs). kubeadm only warns about it in preflight. Then the kubelet exits every ten seconds with “failed to run Kubelet: running with swap on is not supported, please disable swap or set –fail-swap-on flag to false”, and init times out. Remove the package that creates the device.

sudo swapoff -a
sudo dnf remove zram-generator-defaults

Run swapon --show afterwards. It should print nothing at all.

Error: “/proc/sys/net/ipv4/ip_forward contents are not set to 1”

A fresh Fedora 44 VM had forwarding off, and preflight stopped on it. The Fedora 45 VM passed only because Podman and libvirt had already turned forwarding on, so set it explicitly. Preflight in kubeadm 1.36 and 1.37 checks only ip_forward, but the Fedora 44 VM also had no br_netfilter module loaded, and flannel’s bridge needs it so pod traffic crossing cni0 passes through kube-proxy’s rules. Set both to load at boot:

echo br_netfilter | sudo tee /etc/modules-load.d/k8s.conf
sudo modprobe br_netfilter
echo "net.ipv4.ip_forward = 1" | sudo tee /etc/sysctl.d/99-kubernetes.conf
sudo sysctl --system

Afterwards both net.ipv4.ip_forward and net.bridge.bridge-nf-call-iptables read 1 in sysctl, and the preflight error is gone.

Error: “no policy.json file found at any of the following”

This one is new in Fedora 45. containers-common moved its defaults out of /etc/containers into /usr/share/containers, the same move dnf5 made with repo files. Podman reads the new location, but CRI-O 1.36 only looks in /etc, so every image pull during kubeadm init failed:

kubeadm init image pull failing with no policy.json file found because Fedora 45 moved it to /usr/share/containers

Copying policy.json alone isn’t enough. The next pod failed with “short-name “nginx:1.29-alpine” did not resolve to an alias and no containers-registries.conf(5) was found”, because the search registries and short-name aliases moved too, into registries.conf.d. Copy all three:

sudo cp /usr/share/containers/policy.json /etc/containers/policy.json
sudo cp /usr/share/containers/registries.conf /etc/containers/registries.conf
sudo cp /usr/share/containers/registries.conf.d/*.conf /etc/containers/registries.conf.d/

Fedora 44 still has these files in /etc, so skip this step there. CRI-O 1.37 on Fedora 45 pulled a new image with /etc/containers/policy.json moved away, so it reads the new location, and only the 1.36 package needs the copy.

Error: “cni0” already has an IP address different from 10.244.0.1/24

Fedora’s cri-o packages install their own CNI configs into /etc/cni/net.d. CoreDNS starts before flannel exists, so CRI-O hands it an address from its bridge config, 10.85.0.0/16 on cni0. When flannel arrives and wants cni0 as 10.244.0.1, every new pod fails to get a sandbox:

flannel failing because cni0 already has 10.85.0.1 from the CRI-O bridge config on Fedora 45

Moving only the bridge file aside isn’t enough either. On Fedora 44 we tried that, and CRI-O fell back to 200-loopback.conflist: CoreDNS came up as 127.0.0.1 with “dial tcp 10.96.0.1:443: connect: network is unreachable” in its log and never went Ready. Move both before init, so CRI-O has no network until flannel writes one and CoreDNS waits in Pending instead:

sudo mv /etc/cni/net.d/100-crio-bridge.conflist /etc/cni/net.d/100-crio-bridge.conflist.disabled
sudo mv /etc/cni/net.d/200-loopback.conflist /etc/cni/net.d/200-loopback.conflist.disabled

With both moved, /etc/cni/net.d holds only .disabled files until flannel writes its own config.

Initialize the control plane

Open the two ports kubeadm warns about, start CRI-O and the kubelet, and run init with flannel’s pod CIDR:

sudo firewall-cmd --permanent --add-port={6443,10250}/tcp
sudo firewall-cmd --reload
sudo systemctl enable --now crio kubelet
sudo kubeadm init --pod-network-cidr=10.244.0.0/16

Preflight still prints the firewalld warning even with both ports open, because it only checks whether firewalld is running. Copy the admin kubeconfig, apply flannel, and remove the control-plane taint if this is a single node:

mkdir -p ~/.kube
sudo cp /etc/kubernetes/admin.conf ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/config
curl -sLO https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
kubectl apply -f kube-flannel.yml
kubectl taint nodes $(hostname) node-role.kubernetes.io/control-plane-

A test deployment proves the short-name and CNI fixes together, since it pulls by short name and needs a flannel address:

kubectl create deployment web --image=nginx:1.29-alpine
kubectl rollout status deploy/web
kubectl get pods -A -o wide

On Fedora 44 the node showed Ready at an age of 64 seconds, and every pod outside the host network sat in 10.244.0.0/24 (output trimmed to the namespace, name, ready, status and IP columns):

NAMESPACE     NAME                             READY  STATUS   IP
default       web-7fbc579fd4-zxzf6             1/1    Running  10.244.0.5
kube-flannel  kube-flannel-ds-chc68            1/1    Running  10.128.0.31
kube-system   coredns-589f44dc88-4669x         1/1    Running  10.244.0.4
kube-system   coredns-589f44dc88-n2jv6         1/1    Running  10.244.0.3
kube-system   etcd-probe44                     1/1    Running  10.128.0.31
kube-system   kube-apiserver-probe44           1/1    Running  10.128.0.31
kube-system   kube-controller-manager-probe44  1/1    Running  10.128.0.31
kube-system   kube-proxy-tzr7z                 1/1    Running  10.128.0.31
kube-system   kube-scheduler-probe44           1/1    Running  10.128.0.31

The nginx pod ran as container_t with its own MCS categories, CRI-O as container_runtime_t and the kubelet as kubelet_t, so this is the one setup in the article where SELinux confines the workloads themselves.

A package swap stops the kubelet

Upgrading from 1.36 to 1.37 means trading one set of packages for the next, and the obvious first move breaks the node. Upstream says to upgrade kubeadm first, but kubernetes1.37-kubeadm requires the 1.37 kubelet, so dnf replaces both at once. Removing the old kubelet package also stops and disables kubelet.service, and the next kubeadm upgrade plan fails because the node is NotReady:

dnf swapping kubernetes1.36 for kubernetes1.37 stops and disables the kubelet on Fedora 45

The order that works is to run the new kubeadm from the RPM without installing it, upgrade the control plane, and only then swap packages. Pull the binary out of the package:

mkdir -p ~/k137 && cd ~/k137
dnf download kubernetes1.37-kubeadm
rpm2cpio kubernetes1.37-kubeadm-*.rpm | cpio -idm ./usr/bin/kubeadm
sudo ./usr/bin/kubeadm upgrade plan

The plan lists the target for every control plane component, and etcd moves a minor too:

COMPONENT                 NODE      CURRENT   TARGET
kube-apiserver            probe45   v1.36.6   v1.37.2
kube-controller-manager   probe45   v1.36.6   v1.37.2
kube-scheduler            probe45   v1.36.6   v1.37.2
kube-proxy                          1.36.6    v1.37.2
etcd                      probe45   3.6.8-0   3.7.0-0

Apply it, then drain the node and swap every package, CRI-O included, before bringing the kubelet back:

sudo ./usr/bin/kubeadm upgrade apply v1.37.2 -y #https://kubernetes.io/releases/
kubectl drain $(hostname) --ignore-daemonsets --delete-emptydir-data
sudo dnf install --allowerasing kubernetes1.37 kubernetes1.37-kubeadm kubernetes1.37-client cri-o1.37
sudo systemctl enable --now crio kubelet

The cri-o1.37 install put 100-crio-bridge.conflist back into /etc/cni/net.d, next to the disabled copy. Flannel’s 10-flannel.conflist sorts first, so running pods kept their addresses, but delete the restored file so a flannel restart can’t land pods on the wrong bridge, then let the node take work. Our Fedora 45 upgrade node had never moved 200-loopback.conflist; if you followed the clean order, check whether the new package restored that one too and delete it the same way:

ls /etc/cni/net.d/
sudo rm /etc/cni/net.d/100-crio-bridge.conflist
kubectl uncordon $(hostname)

The node reported the new kubelet version with the matching CRI-O, and the web deployment and CoreDNS came back on 10.244 addresses. Treat the package swap as the kubelet upgrade step, and you get the upstream sequence back.

SELinux confines each tool differently

All four tools ran with SELinux enforcing, and none of them needed it switched off. What differs is how much of the cluster SELinux actually confines, and you can read it from ps -eZ on the host:

SetupProcessSELinux type seenWhat it means
k3sk3s-server, containerdcontainer_runtime_tLabeled by the k3s-selinux module
kind (Podman or Docker), minikube (rootless Podman)kube-apiserver, kubelet in the nodespc_tPrivileged node container, not confined
minikube kvm2qemu-system-x86_64svirt_t (disk svirt_image_t)Isolated like any libvirt guest
kubeadm + CRI-Ocrio / kubelet / podcontainer_runtime_t / kubelet_t / container_tWorkloads confined per pod with MCS labels

If you need a cluster that behaves like production under SELinux (hostPath volumes, privileged pods getting denied), test on kubeadm or k3s, not on kind. A policy that passes in kind proves nothing about labels. Before any of this, a Fedora 45 machine is easier to work with once the basics in things to do after installing Fedora 45 are done.

NodePorts slip past the FedoraServer zone

The belief this article opened with is easy to test from a second machine. The FedoraServer zone on the kubeadm node listed ssh, cockpit, dhcpv6-client, 6443 and 10250, with nothing in the 30000 to 32767 range and no sources in the zone. A curl from the other VM to the web service’s NodePort still got a 200:

curl -s -m 5 -o /dev/null -w "nodeport %{http_code}\n" http://10.128.0.30:31463/

The response came back immediately, with no port opened for it:

nodeport 200

kube-proxy rewrites the destination to a pod IP, so the packet is forwarded rather than delivered to the host, and the zone’s port list never applies. Firewalld doesn’t wave it through blindly, though. Its forward chain accepts every connection that was DNAT’d before it looks at any zone, and the same rule sits in its input chain. You can see it on your own node:

sudo nft list chain inet firewalld filter_FORWARD

The ct status dnat accept line comes right after the established and related rule, ahead of the jump into the zone dispatch:

table inet firewalld {
	chain filter_FORWARD {
		type filter hook forward priority filter + 10; policy accept;
		ct state { established, related } accept
		ct status dnat accept
		iifname "lo" accept
		ct state invalid drop
		ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } reject with icmpv6 addr-unreachable
		jump filter_FORWARD_dispatch
		reject with icmpx admin-prohibited
	}
}

That rule exists because StrictForwardPorts=no is the shipped default in /etc/firewalld/firewalld.conf, and it is also why CoreDNS kept resolving during the k3s break: its calls to the API go through the kubernetes ClusterIP, which is DNAT’d too. Setting StrictForwardPorts=yes removes the generic accept, but it also drops the pod to service traffic every node depends on, so don’t use it to hide NodePorts. Only a pod hitting a host port on the node’s own IP, with no service in between, is plain local delivery: a pod probing port 9100 on its node got “No route to host” in the same lab, while 10250 and 22 were open. So on a default Fedora Server zone the rule is:

  • Pod to a host port on the node IP: firewalld filters it, so trust the pod CIDR or open the port.
  • LAN to NodePort services, and pods to ClusterIPs: DNAT’d, so firewalld accepts them whatever the port list says.
  • Clients to the API server on 6443: plain host traffic, so open it in the zone.

If a NodePort must not be reachable from the LAN, adding or removing ports in the zone won’t stop it. Put the restriction in the network firewall in front of the node, or keep the service as ClusterIP and expose it through an ingress you control.

Keep reading

How to Upgrade Fedora 44 to Fedora 45 (GUI, CLI and Rollback) Fedora How to Upgrade Fedora 44 to Fedora 45 (GUI, CLI and Rollback) Things to Do After Installing Fedora 45 Workstation Fedora Things to Do After Installing Fedora 45 Workstation Enable RPM Fusion and Multimedia Codecs on Fedora 45 / 44 / 43 Fedora Enable RPM Fusion and Multimedia Codecs on Fedora 45 / 44 / 43 OpenCloud vs ownCloud vs Nextcloud: Tested on Identical VMs Cloud OpenCloud vs ownCloud vs Nextcloud: Tested on Identical VMs Kiri Container Runtime on Linux: Install and Benchmark vs Docker Containers Kiri Container Runtime on Linux: Install and Benchmark vs Docker Install Kubernetes Dashboard with NodePort (v7) Kubernetes Install Kubernetes Dashboard with NodePort (v7)

Leave a Comment

Press ESC to close