Half of what the CKS checks lives where kubectl cannot see it. A profile loaded into the node kernel, a syscall filter on disk under the kubelet, an alert in a systemd journal and a line in an audit log are all on that list. That is why CKS practice questions that only show a YAML answer leave you short. You also need to know which machine to SSH into, which file to edit there, and what the failure looks like when you get it wrong.
This set has seven tasks, one for each tool the security specialist outline leans on hardest: NetworkPolicy, Pod Security Admission, AppArmor, seccomp, Trivy image scanning, Falco runtime alerts and API server audit logging. Each task reads the way a CKS task does, so you can try it yourself first. Our solution follows with the exact manifest, the real output and the failed first attempts. I ran every task on Kubernetes 1.37 in November 2026 on a fresh two node kubeadm cluster, and the errors quoted here are the ones it gave us.
What a CKS task expects from you
The Certified Kubernetes Security Specialist is a performance test, the same format as the CKA. You get a terminal, several clusters, and a list of tasks, and a script grades the state you leave behind. The Linux Foundation CKS instructions page sets the ground rules, and these are the ones that shape how you should practise:
- 2 hours for 15 to 20 tasks
- Pass mark of 67 percent, one point higher than the CKA
- Kubernetes v1.37 in the environment today
- A passed CKA required before you can book it
- Docs allowed: kubernetes.io docs and blog, Falco, etcd, Cilium, Istio, bom and ingress-nginx docs
Read that last line carefully, because the Trivy and AppArmor docs are not on the allowed list. The Falco docs are, so you can look up a field name for an output format. For Trivy, plan on trivy image --help unless a task links a page in its own quick reference box, which the allowed resources page also permits. That is why the Trivy step below uses only flags that the help text shows. The outline itself splits into six domains, and the table maps each one to the step in this guide that drills it.
| Domain | Weight | Drilled here |
|---|---|---|
| Cluster Setup | 15% | Step 1 (NetworkPolicy) |
| Cluster Hardening | 15% | Step 7 (audit logs touch API access) |
| System Hardening | 10% | Steps 3 and 4 (AppArmor, seccomp) |
| Minimize Microservice Vulnerabilities | 20% | Step 2 (Pod Security Admission) |
| Supply Chain Security | 20% | Step 5 (Trivy) |
| Monitoring, Logging and Runtime Security | 20% | Steps 6 and 7 (Falco, audit logs) |
Every CKS task also starts on a node you have to reach first, usually with ssh to a hostname the task names. Half of the tasks below finish on the worker node, not on the machine where kubectl runs, so the habit to build is checking the prompt before you type. If you have not done the CKA labs yet, our CKA practice questions cover the cluster administration side that the CKS assumes you already know, and our CKAD practice questions cover the application side.
The lab cluster behind these answers
We built two Ubuntu 24.04 VMs in Google Cloud, both e2-standard-2 with 8 GB of RAM, and joined them with kubeadm. The control plane is cks-cp and the worker is cks-node1. Ubuntu was a deliberate pick, because it ships AppArmor enabled and a recent kernel, which the AppArmor task and Falco’s modern eBPF probe both need. The pieces we used:
- Kubernetes 1.37.2 from pkgs.k8s.io, held with apt-mark
- containerd 2.2.1 with the systemd cgroup driver
- Calico 3.33.0, pod CIDR 192.168.0.0/16
- Falco 0.45.2 and Trivy 0.76.2 for steps 5 and 6
Our diagram shows where each control sits. The API server admits pods and writes the audit log on the control plane, while AppArmor, seccomp, Falco and Calico all do their work on the worker, next to the containers they police.

Our first snag came before any task. With no NetworkPolicy in place at all, a pod still could not resolve DNS, and the message was the same one a deny policy produces, which made it very confusing:
;; connection timed out; no servers could be reached
command terminated with exit code 1
Blame the cloud firewall here, not Kubernetes. The default Calico manifest wraps pod traffic between nodes in IPIP, which is IP protocol 4, and the Google Cloud default-allow-internal rule only passes TCP, UDP and ICMP. Pods on the same node talked fine, pods across nodes did not. One firewall rule fixed it:
gcloud compute firewall-rules create cks-lab-ipip --network=default --allow=4 --source-ranges=10.128.0.0/9
Prove that DNS works on a clean cluster before you write a single policy, otherwise you will blame your YAML for a fault in the network underneath. The same trap exists on AWS security groups and on any homelab firewall that filters by protocol. One more cloud detail: Google Cloud VMs report a long internal FQDN as their hostname, and kubeadm registered our nodes under it on the first try. We reset and ran kubeadm init and kubeadm join again with --node-name, so that nodeName: cks-node1 in the later manifests matches a real node.
Step 1: Default deny a namespace, then allow one client
You are asked to block all ingress and egress traffic in namespace payments by default. Pods in payments must still resolve DNS, and only pods labelled role=gateway in namespace edge may reach pods labelled app=api, on TCP port 80. Our setup had an api nginx pod with a Service, a client pod in payments, a gateway pod in edge and an unlabelled intruder pod in a third namespace called other. To build the same playground, save this as setup.yaml:
vim setup.yaml
It holds the three namespaces, four pods and the Service in one file:
apiVersion: v1
kind: Namespace
metadata:
name: payments
---
apiVersion: v1
kind: Namespace
metadata:
name: edge
---
apiVersion: v1
kind: Namespace
metadata:
name: other
---
apiVersion: v1
kind: Pod
metadata:
name: api
namespace: payments
labels:
app: api
spec:
containers:
- name: nginx
image: nginx:1.29-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: api
namespace: payments
spec:
selector:
app: api
ports:
- port: 80
targetPort: 80
---
apiVersion: v1
kind: Pod
metadata:
name: gateway
namespace: edge
labels:
role: gateway
spec:
containers:
- name: busybox
image: busybox:1.37
command: ["sleep", "3600"]
---
apiVersion: v1
kind: Pod
metadata:
name: intruder
namespace: other
spec:
containers:
- name: busybox
image: busybox:1.37
command: ["sleep", "3600"]
---
apiVersion: v1
kind: Pod
metadata:
name: client
namespace: payments
labels:
app: client
spec:
containers:
- name: busybox
image: busybox:1.37
command: ["sleep", "3600"]
Apply it with kubectl apply -f setup.yaml and wait until all four pods are Running before you add any policy.
A default deny is one policy with an empty pod selector and both policy types listed, so create that file first.
vim default-deny.yaml
Add the policy. Listing Egress under policyTypes with no egress rules is what blocks outbound traffic:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: payments
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
Apply it, then test name resolution from the client pod:
kubectl apply -f default-deny.yaml
kubectl -n payments exec client -- nslookup kubernetes.default.svc.cluster.local
DNS dies with the rest of the traffic, because CoreDNS sits in kube-system and egress to it is now blocked too:
;; connection timed out; no servers could be reached
command terminated with exit code 1
People skip this part of the task when they read too fast. The fix is a second egress policy for every pod in the namespace that allows port 53 on both UDP and TCP to the kube-dns pods. Our file was allow-dns.yaml:
vim allow-dns.yaml
Both selectors sit in the same list item, so a packet must match the kube-system namespace and the kube-dns label together.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns
namespace: payments
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
After applying it the same nslookup returned Address: 10.96.0.1 for the kubernetes Service. One small trap from our run: busybox nslookup ignores the search list in resolv.conf, so the short name kubernetes.default returns NXDOMAIN even when DNS is fine. Test with the full name ending in svc.cluster.local. Now the ingress rule, in allow-gateway.yaml:
vim allow-gateway.yaml
Again the two selectors share one from item. Splitting them into two items (two dashes) would turn the rule into an OR and let every pod in edge through.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-gateway-to-api
namespace: payments
spec:
podSelector:
matchLabels:
app: api
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: edge
podSelector:
matchLabels:
role: gateway
ports:
- protocol: TCP
port: 80
Apply it and test from all three clients. A grader checks the deny paths as well as the allow path, so you should too:
kubectl apply -f allow-gateway.yaml
kubectl -n edge exec gateway -- wget -qO- -T 3 http://api.payments.svc.cluster.local
kubectl -n other exec intruder -- wget -qO- -T 3 http://api.payments.svc.cluster.local
kubectl -n payments exec client -- wget -qO- -T 3 http://api.payments.svc.cluster.local
Only the gateway gets the nginx welcome page, while the intruder and the client both time out.

Why is the client blocked when it sits in the same namespace as the API pod? Because the default deny covers it too, and the allow rule names only the edge gateway. A timeout rather than a connection refused is the sign that the policy dropped the packet. kubectl describe networkpolicy prints the rule in plain words (“Allowing ingress traffic … From NamespaceSelector … PodSelector”) and is the fastest way to check you did not split the selectors by accident.
Step 2: Enforce the restricted Pod Security Standard
In this task, namespace team-a must reject any pod that does not meet the restricted Pod Security Standard, and users must see a warning as well. Pod Security Admission is built into the API server, so the whole job is namespace labels. The labels follow one pattern, pod-security.kubernetes.io/<mode>, where the mode is enforce, audit or warn, as the Pod Security Admission page documents.
Before labelling a namespace that already runs workloads, do a server side dry run. To have something for the dry run to report, we created the namespace with kubectl create namespace team-a and left a privileged nginx pod called legacy in it on purpose:
vim legacy.yaml
Nothing restricts the namespace yet, so this pod is admitted and starts normally after kubectl apply -f legacy.yaml.
apiVersion: v1
kind: Pod
metadata:
name: legacy
namespace: team-a
spec:
containers:
- name: nginx
image: nginx:1.29-alpine
securityContext:
privileged: true
Now run the label as a server side dry run.
kubectl label --dry-run=server --overwrite ns team-a pod-security.kubernetes.io/enforce=restricted
Every existing pod that would violate the level is listed with its reasons, and nothing changes yet.
Warning: existing pods in namespace "team-a" violate the new PodSecurity enforce level "restricted:latest"
Warning: legacy: privileged, allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
namespace/team-a labeled (server dry run)
Apply the real labels next. We set the enforce version to latest and added the warn mode the task asked for:
kubectl label --overwrite ns team-a \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/enforce-version=latest \
pod-security.kubernetes.io/warn=restricted
Then test it the way the grader will, with a plain pod that sets nothing:
kubectl -n team-a run bad --image=nginx:1.29-alpine
Every field you have to set is named in the rejection, which makes it the best checklist you have for writing the compliant pod.

One thing the capture also shows: the legacy pod kept running after the label went on. Enforcement only applies to pods created from then on, so if a task says “make sure no pod in the namespace violates the policy”, you also have to delete or fix the old pods yourself. The compliant manifest below uses the unprivileged nginx image, which listens on 8080 and runs as UID 101:
vim good-pod.yaml
Put runAsNonRoot and the seccomp profile at pod level, and the capability drop and privilege escalation flag on the container:
apiVersion: v1
kind: Pod
metadata:
name: good
namespace: team-a
spec:
securityContext:
runAsNonRoot: true
runAsUser: 101
seccompProfile:
type: RuntimeDefault
containers:
- name: nginx
image: nginxinc/nginx-unprivileged:1.29-alpine
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
Our good pod was admitted and reached Running in 5 seconds. We also tried the same manifest with the stock nginx:1.29-alpine image. It was admitted, then went into CrashLoopBackOff, because its startup needs root to create cache directories, and the log said so plainly: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied). Pick an image built to run as non-root before you fight the manifest.
Step 3: Confine a pod with an AppArmor profile
You are told that a profile called k8s-deny-write exists on node cks-node1, and you must load it, run a pod on that node that uses it, and prove the container cannot write files. AppArmor profiles live in the node kernel, so the first half of this task happens over SSH on the worker. The profile file we used is the deny write example from the Kubernetes AppArmor tutorial, saved under the task’s name:
sudo vim /etc/apparmor.d/k8s-deny-write
Whatever follows the word profile is the name your pod refers to, and it does not have to match the file name.
#include <tunables/global>
profile k8s-deny-write flags=(attach_disconnected) {
#include <abstractions/base>
file,
# Deny all file writes.
deny /** w,
}
Here we hit the failure first, on purpose, because it is the one people meet in practice. We created the pod with the profile name misspelled as k8s-deny-writes, and the pod sat in CreateContainerError. Its events gave the reason clearly:
Warning Failed 6s (x3 over 8s) kubelet spec.containers{busybox}: Error: failed to get container spec opts: failed to generate apparmor spec opts: apparmor profile not found k8s-deny-writes
You get the same message when the name is right but the profile was never loaded, which is the more common case. On the worker, load it and confirm the kernel knows it:
sudo apparmor_parser -q /etc/apparmor.d/k8s-deny-write
sudo aa-status | grep k8s
Back on the control plane, the pod sets the profile through securityContext.appArmorProfile, the field that went stable in Kubernetes 1.31 and replaced the old annotation. Pin the pod to the node that has the profile:
vim aa-pod.yaml
We used nodeName here because the profile exists on one node only. A nodeSelector on a label works just as well if you prefer it:
apiVersion: v1
kind: Pod
metadata:
name: aa-deny-write
spec:
nodeName: cks-node1
securityContext:
appArmorProfile:
type: Localhost
localhostProfile: k8s-deny-write
containers:
- name: busybox
image: busybox:1.37
command: ["sleep", "3600"]
Apply it, then try a write and read the profile that the first process in the container runs under:
kubectl apply -f aa-pod.yaml
kubectl exec aa-deny-write -- touch /tmp/test
kubectl exec aa-deny-write -- cat /proc/1/attr/current
Writing fails with Permission denied, and the process reports the profile in enforce mode.

Both lines matter. Permission denied on its own could come from file ownership, while k8s-deny-write (enforce) proves the denial came from AppArmor. A profile loaded with apparmor_parser does not survive a reboot unless the file stays in /etc/apparmor.d, so leave it there. Our AppArmor cheat sheet covers complain mode and aa-logprof for building profiles of your own.
Step 4: Block a syscall with a seccomp profile
Your next task is a seccomp profile on cks-node1 that blocks the mkdir syscall, plus a pod that runs with it. Seccomp profiles are JSON files that the kubelet reads from /var/lib/kubelet/seccomp, and the path you give in the pod is relative to that directory. On the worker:
sudo mkdir -p /var/lib/kubelet/seccomp/profiles
sudo vim /var/lib/kubelet/seccomp/profiles/no-mkdir.json
Everything is allowed by default, and the two syscalls that create directories return an error. List both names, since a C library can implement the mkdir command with either mkdir or mkdirat.
{
"defaultAction": "SCMP_ACT_ALLOW",
"syscalls": [
{
"names": ["mkdir", "mkdirat"],
"action": "SCMP_ACT_ERRNO"
}
]
}
In your pod, the profile is profiles/no-mkdir.json, relative to the seccomp directory.
vim seccomp-pod.yaml
Paste the pod spec. Putting the profile at pod level applies it to every container in the pod:
apiVersion: v1
kind: Pod
metadata:
name: sec-no-mkdir
spec:
nodeName: cks-node1
securityContext:
seccompProfile:
type: Localhost
localhostProfile: profiles/no-mkdir.json
containers:
- name: busybox
image: busybox:1.37
command: ["sleep", "3600"]
Get the path wrong and the pod lands in CreateContainerError, the same status as the AppArmor mistake. Our test pod pointed at profiles/not-there.json, and the event spelled out the full path the kubelet tried, which tells you at once whether the file is missing or the relative path is wrong:
Error: failed to create containerd container: cannot load seccomp profile "/var/lib/kubelet/seccomp/profiles/not-there.json": open /var/lib/kubelet/seccomp/profiles/not-there.json: no such file or directory
With the correct path, test the block, test that other writes still work, and read the seccomp mode from the process status:
kubectl apply -f seccomp-pod.yaml
kubectl exec sec-no-mkdir -- mkdir /tmp/x
kubectl exec sec-no-mkdir -- touch /tmp/y
kubectl exec sec-no-mkdir -- grep Seccomp /proc/1/status
We ran the same checks against a pod with RuntimeDefault and one with no profile at all, so the three modes sit side by side:

Seccomp mode 2 means a filter is attached, and you can see it on both the custom profile and RuntimeDefault. The unconfined pod shows 0, which is what every pod gets on a default kubeadm cluster unless you set a profile, which the restricted Pod Security level from step 2 forces you to do. (A kubelet started with seccompDefault: true applies RuntimeDefault to every pod instead.) The error text is Operation not permitted, not Permission denied, and that difference tells you seccomp blocked the call rather than AppArmor or file permissions.
Step 5: Find and remove images with critical CVEs
Namespace scan runs three Deployments, and you must scan every image they use with Trivy, then scale any Deployment whose image has a CRITICAL vulnerability down to zero. If your control plane lacks Trivy, add the Aqua Security apt repository and install it together with jq, which the scan loop below uses:
curl -fsSL https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo gpg --dearmor --yes -o /usr/share/keyrings/trivy.gpg && echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb generic main" | sudo tee /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install -y trivy jq
Ours installed Trivy 0.76.2. Our three Deployments came from one file, scan.yaml:
vim scan.yaml
One old image, one current image and a busybox tool pod give the scan three different results to sort:
apiVersion: v1
kind: Namespace
metadata:
name: scan
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: old-web
namespace: scan
spec:
replicas: 1
selector:
matchLabels: {app: old-web}
template:
metadata:
labels: {app: old-web}
spec:
containers:
- name: nginx
image: nginx:1.19
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: new-web
namespace: scan
spec:
replicas: 1
selector:
matchLabels: {app: new-web}
template:
metadata:
labels: {app: new-web}
spec:
containers:
- name: nginx
image: nginx:1.29-alpine
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tools
namespace: scan
spec:
replicas: 1
selector:
matchLabels: {app: tools}
template:
metadata:
labels: {app: tools}
spec:
containers:
- name: busybox
image: busybox:1.36
command: ["sleep", "3600"]
After kubectl apply -f scan.yaml, list the images per Deployment first, because a pod list shows the same image once per replica:
kubectl -n scan get deploy -o jsonpath="{range .items[*]}{.metadata.name}{\"\t\"}{.spec.template.spec.containers[*].image}{\"\n\"}{end}"
Three Deployments, three images, one of them clearly old:
new-web nginx:1.29-alpine
old-web nginx:1.19
tools busybox:1.36
A full table for nginx:1.19 runs to hundreds of lines, so filter by severity and count. Our first scan took 53 seconds because Trivy downloads its vulnerability database on first use, and later scans finished in a few seconds. Budget for that first download when the clock is running. The loop below also needs jq, which we installed alongside Trivy, so run which jq first on any host you did not build yourself. This loop scans each image, prints the CRITICAL count, and scales the offending Deployment in one pass:
kubectl -n scan get deploy -o jsonpath="{range .items[*]}{.metadata.name}{\"\t\"}{.spec.template.spec.containers[*].image}{\"\n\"}{end}" | while read d i; do cnt=$(trivy image -q --severity CRITICAL -f json $i | jq "[.Results[]?.Vulnerabilities // [] | length] | add // 0"); echo "$d $i $cnt"; [ "$cnt" -gt 0 ] && kubectl -n scan scale deploy/$d --replicas=0; done
Two Deployments went to zero, and the counts explain why:

nginx:1.19 carried 42 CRITICAL findings on its Debian 10 base, which nobody would argue with. The surprise was nginx:1.29-alpine with one: a brotli library fix that was out but not yet in that image build. A task that says “critical” means any critical, so it goes. Then read the busybox line closely. Its count is 0, but the full Trivy report for busybox:1.36 shows a dash under Vulnerabilities, which the legend defines as “Not scanned”. Trivy found no package database in that image, so the zero means it could not look, not that the image is clean. Our older Trivy scanning guide covers filesystem and repository scans too.
Step 6: Catch a shell in a container with Falco
Falco already runs on cks-node1 in this task. You need to change the “Terminal shell in container” rule so its alert prints in the format time,user,process,container id, trigger it, and find the pod behind the alert. We installed Falco 0.45.2 from the falcosecurity apt repository with FALCO_DRIVER_CHOICE=modern_ebpf, which needs no kernel headers, and the package enabled falco-modern-bpf.service by itself. The Falco package docs list the other unit names for the kmod driver. To build the same setup on your worker, these were our install commands:
curl -fsSL https://falco.org/repo/falcosecurity-packages.asc | sudo gpg --dearmor --yes -o /usr/share/keyrings/falco-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/falco-archive-keyring.gpg] https://download.falco.org/packages/deb stable main" | sudo tee /etc/apt/sources.list.d/falcosecurity.list
sudo apt-get update
sudo FALCO_FRONTEND=noninteractive FALCO_DRIVER_CHOICE=modern_ebpf apt-get install -y falco
The package starts the service as soon as it installs, so the rule work below can begin right away.
Never edit /etc/falco/falco_rules.yaml, because a package upgrade overwrites it. Overrides go in falco_rules.local.yaml, which Falco loads after the stock rules:
sudo vim /etc/falco/falco_rules.local.yaml
An override block tells Falco which keys of the existing rule to replace. We also raised the priority to WARNING so the alert stands out in the journal:
- rule: Terminal shell in container
output: "%evt.time,%user.name,%proc.name,%container.id,%k8s.ns.name/%k8s.pod.name"
priority: WARNING
override:
output: replace
priority: replace
We got there on the third try, and both failures took the Falco service down into a restart loop. The first version had no override block, so Falco read it as a brand new rule with the same name and refused it because a new rule needs a condition. The second had a typo, %user.nam, and failed to compile. Validate before you restart, every time:
sudo falco -V /etc/falco/falco_rules.yaml -V /etc/falco/falco_rules.local.yaml
sudo systemctl restart falco-modern-bpf.service
For reference, this is how the missing override looked in the journal, with the restart counter climbing beneath it:
LOAD_ERR_YAML_VALIDATE (Error validating internal structure of YAML file): Item has no mapping for key 'condition'
With valid rules loaded, you need a target pod on the worker. Ours was a plain nginx pod called shelly, pinned to the node where Falco runs:
vim shelly.yaml
Falco only sees syscalls on its own node, so nodeName matters here:
apiVersion: v1
kind: Pod
metadata:
name: shelly
namespace: default
spec:
nodeName: cks-node1
containers:
- name: nginx
image: nginx:1.29-alpine
Apply it, then open an interactive shell from the control plane with kubectl exec -it shelly -- sh and exit again. The rule only fires on a shell with a terminal attached, so a shell without a terminal, such as our earlier kubectl exec shelly -- sh -c "id; hostname", did not trigger it. On the worker, read the alert, then map the container ID it prints to a pod. Use the ID from your own alert, since ours below will not exist on your cluster:
sudo journalctl -u falco-modern-bpf.service --since "-1min" --no-pager -o cat | grep -E "Warning"
sudo crictl ps --id 5aeae8d32363
Our four comma separated fields lead the alert, followed by the namespace and pod, then the container fields Falco adds on its own.

Falco 0.45 appends its own container_id=... k8s_pod_name=... fields after the custom output, so your fields sit right after Falco’s own timestamp and priority prefix, ahead of everything Falco adds. We also added the namespace and pod to the format for our own convenience; if a task names its fields exactly, write exactly those and nothing more. Our crictl was missing on the worker, because the 1.37 kubeadm package no longer pulls in cri-tools, so we installed it with apt. Cluster nodes you are handed normally have it, but a homelab built from the kubeadm packages alone may not. For a wider look at what an eBPF sensor records next to auditd, our test of what a Linux EDR agent sees runs Falco on the same kind of host.
Step 7: Turn on API server audit logging
Your last task is to enable audit logging on the cluster. Log Secret access at Metadata level so the secret values never reach the log, log every request on Pods in namespace payments at RequestResponse level, and nothing else. Keep 2 backups, 7 days, 50 MB per file. The API server is a static pod, so this is an edit to its manifest on the control plane, and a wrong edit takes the API down. Copy the manifest outside the manifests directory first:
sudo cp /etc/kubernetes/manifests/kube-apiserver.yaml /root/kube-apiserver.yaml.bak
sudo mkdir -p /etc/kubernetes/audit /var/log/kubernetes/audit
sudo vim /etc/kubernetes/audit/policy.yaml
Rules match top to bottom and the first match wins, so the specific rules go first and the catch all None goes last. Note that resources: ["pods"] does not cover subresources such as pods/exec or pods/log; a task that asks you to audit exec calls needs those listed too.
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
- RequestReceived
rules:
- level: RequestResponse
namespaces: ["payments"]
resources:
- group: ""
resources: ["pods"]
- level: Metadata
resources:
- group: ""
resources: ["secrets"]
- level: None
Now edit the API server manifest. It needs four things: the flags, a volume mount for the policy, a volume mount for the log directory, and the matching hostPath volumes:
sudo vim /etc/kubernetes/manifests/kube-apiserver.yaml
Our first edit added only the flags, and it is worth seeing what that does. The API server container exited and kubectl answered The connection to the server 10.132.0.8:6443 was refused. So how do you read an error when kubectl itself is gone? Run sudo crictl ps -a --name kube-apiserver to find the exited container, then crictl logs on its ID, which printed this (install cri-tools first if your control plane lacks crictl, as our worker did in step 6).
E1111 11:43:23.759469 1 run.go:72] "command failed" err="loading audit policy file: failed to read file path \"/etc/kubernetes/audit/policy.yaml\": open /etc/kubernetes/audit/policy.yaml: no such file or directory"
Our policy file existed on the host, but the container could not see it without a mount. Compare your edit with the backup using sudo diff -u /root/kube-apiserver.yaml.bak /etc/kubernetes/manifests/kube-apiserver.yaml, which printed the complete working change.
@@ -12,6 +12,11 @@
containers:
- command:
- kube-apiserver
+ - --audit-policy-file=/etc/kubernetes/audit/policy.yaml
+ - --audit-log-path=/var/log/kubernetes/audit/audit.log
+ - --audit-log-maxage=7
+ - --audit-log-maxbackup=2
+ - --audit-log-maxsize=50
- --advertise-address=10.132.0.8
- --allow-privileged=true
- --authorization-mode=Node,RBAC
@@ -79,6 +84,11 @@
periodSeconds: 10
timeoutSeconds: 15
volumeMounts:
+ - mountPath: /etc/kubernetes/audit/policy.yaml
+ name: audit-policy
+ readOnly: true
+ - mountPath: /var/log/kubernetes/audit
+ name: audit-log
- mountPath: /etc/ssl/certs
name: ca-certs
readOnly: true
@@ -102,6 +112,14 @@
type: RuntimeDefault
volumes:
- hostPath:
+ path: /etc/kubernetes/audit/policy.yaml
+ type: File
+ name: audit-policy
+ - hostPath:
+ path: /var/log/kubernetes/audit
+ type: DirectoryOrCreate
+ name: audit-log
+ - hostPath:
path: /etc/ssl/certs
type: DirectoryOrCreate
name: ca-certs
Once saved, the manifest was picked up by the kubelet and the API server answered again in under a minute. To check the policy, create a secret in payments, read it back, run a pod there and delete the secret, so the log has events of each kind:
kubectl -n payments create secret generic db-pass --from-literal=password=S3cr3t-lab
kubectl -n payments get secret db-pass -o jsonpath="{.data.password}" | base64 -d; echo
kubectl -n payments run audit-demo --image=busybox:1.37 --restart=Never -- sleep 60
kubectl -n payments delete secret db-pass
Now query the log with jq. This filter keeps only secret events and prints who did what at which level:
sudo jq -c 'select(.objectRef.resource=="secrets") | {verb, user: .user.username, ns: .objectRef.namespace, name: .objectRef.name, level}' /var/log/kubernetes/audit/audit.log | tail
Each secret action is there at Metadata level, and the pod create in payments is the only RequestResponse event:

The capture ends with two counts, the name of the secret and its value, taken straight from the log file:
sudo grep -c db-pass /var/log/kubernetes/audit/audit.log; sudo grep -c S3cr3t /var/log/kubernetes/audit/audit.log
Of all these checks, that grep matters most for a real cluster. The secret name appears 4 times in the log, and the secret value appears 0 times, because Metadata level records who touched what without the request or response body. Write the same policy with Request on secrets and the base64 value lands in the log and in every rotated backup, readable by anyone with root on the control plane. The whole log held 6 lines and 8.7 KB after our test, which shows how much the closing None rule cuts out. RBAC decides who may read those secrets in the first place, and our guide to Kubernetes RBAC roles and ServiceAccounts covers that half.
A hardening checklist to run after the drills
All seven tasks map onto checks you can run against any cluster you look after, not only a practice one. After our run we turned them into a short audit list:
- Every app namespace has a default deny NetworkPolicy plus a DNS egress rule
- Namespaces carry an enforce label, restricted where the workload allows it
- No running pod shows Seccomp mode 0 in its process status
- Custom AppArmor profiles sit in /etc/apparmor.d so they load at boot
- Image scans count “Not scanned” results as failures, not passes
- Falco rule overrides pass falco -V before any restart
- The audit policy logs secrets at Metadata and nothing at Request or above
Items 3 and 5 are the ones that fail quietly. A pod without a seccomp profile and an image Trivy could not read both look fine until you check, so they are worth scripting. Admission policy goes further than Pod Security Admission can, and our comparison of OPA Gatekeeper and Kyverno shows the two engines that cover registry allow lists and image rules. For isolation beyond seccomp and AppArmor, the gVisor vs Kata Containers benchmark measures what a sandboxed RuntimeClass costs. The same gVisor RuntimeClass is the base layer for sandboxing untrusted AI agent workloads with gVisor, where each agent runs in its own locked-down pod. Once these seven tasks come without the page open, run them again on a fresh cluster with a timer, aiming for 6 to 8 minutes each, which is the pace 15 to 20 tasks in 2 hours demands.