Kubernetes System Hardening on GKE — the host that is not the cluster

Cluster Hardening is the API. System Hardening is the machine the kubelet sits on. COS is that machine.

<!-- hal:authoritative:yaml -->

Cluster Hardening is the API. System Hardening is the machine the kubelet sits on. COS is that machine. A second cluster is a later year.

§I — Frame

Friday 08-28 named a leftover second EKS cluster next to a Deployment that already had maxSurge. Tuesday 08-25 named leftover API doors on AKS. Sunday 08-30, on the TF track, named a GCP subnet latch: Private Google Access, VMs without a public IP. Today is a K8s day. The leftover is not a subnet. The leftover is the host.

k8s_day_counter reads 13. Odd is CKS-emphasis. 08-25 closed Cluster Hardening 15% on Q04: Node,RBAC, anonymous-auth=false, NodeRestriction, delete the anonymous binding. The runbook leftover is System Hardening 15%. AppArmor. seccomp. Host OS footprint. Do not open the apiserver static Pod again. Do not open authorized IP ranges again.

The last three cloud overlays in the room are 08-30 GCP PGA, 08-28 EKS, 08-25 AKS. All three vendors sat down. GKE last visited this track on 08-22, and that visit was a Persistent Disk. 08-15 used GKE as a Workload Identity binder. 08-13 used GKE as a Sandbox. Today GKE is a node image. Leave the volume. Leave the GSA. Leave runsc.

The ACE/PCA Compute table names GKE as managed Kubernetes, AWS equivalent EKS (gcp-ace-pca.md). The next row names GKE Autopilot: fully managed K8s, no node management. That second row is the trap. If you pick Autopilot because "managed" sounds like "hardened," you just deleted the host you came to inspect. Standard GKE still gives you a node pool. The pool still has an image. The image is the lesson.

GCP-PCA-Notes put one line under Compute Engine: SHIELDED VM IS BEST PRACTICE IN GCP. The notes do not write "Shielded GKE Nodes." The cheatsheets do not write COS_CONTAINERD. Same class as 08-28: aws-core-services named EKS and stopped before maxSurge. Name the service that exists. Name the field the google plugin publishes. Do not invent a cheatsheet row.

Coin it: the host that is not the cluster.

A cluster is an API. A host is a kernel, an image, a Secure Boot bit, and a profile loaded into that kernel. You can close the API and still boot a fat Ubuntu worker that listens on extra ports and runs with Unconfined. You can open AppArmor on a node whose control plane still answers the world. 08-25 was the second case. Today is the first.

§II — Foundations: four facts about the host

Fact one. System Hardening is not Cluster Hardening.

Official CKS order lists Cluster Setup, then Cluster Hardening, then System Hardening. 08-01 spent Setup (kube-bench, Ingress TLS, metadata). 08-25 spent Hardening (the API). The leftover domain is the node OS.

kubestronaut-hedron files the domain as four bullets (cks/3 System Hardening.md). Minimize host OS footprint: list units, stop what you do not need, purge packages. Least-privilege IAM on the box: users, groups, chmod, not an IAM Role in GCP. Minimize external network access: ss, ufw, 6443 if you must. Kernel tools: AppArmor and seccomp. The exam still asks you to SSH to a worker and load a profile. The fleet on GKE Standard lets you pick the image that already did most of that. The fleet on Autopilot does not let you SSH at all.

If/then: if the stem names kube-apiserver.yaml, you are in 08-25. If the stem names /etc/apparmor.d or /var/lib/kubelet/seccomp, you are here. If the stem names RuntimeClass runsc, you are in 08-13. Three doors. One word (hardening) that people keep using for all three.

Fact two. COS is the host image. Ubuntu is the exam image.

GKE Standard node pools take image_type. COS_CONTAINERD is the production default: Container-Optimized OS, containerd, small footprint, automatic updates, AppArmor in the kernel. Ubuntu with containerd exists when you need extra packages or extra kernel modules. The exam worker is almost always Ubuntu, because apparmor_parser and a file under /etc/apparmor.d are the skill. COS already loaded a profile. Ubuntu has not.

resource "google_container_node_pool" "hardened" {
  name     = "hardened"
  cluster  = google_container_cluster.app.name
  location = "us-central1-a"

  node_config {
    machine_type = "e2-standard-4"
    image_type   = "COS_CONTAINERD"
    shielded_instance_config {
      enable_secure_boot          = true
      enable_integrity_monitoring = true
    }
  }
}

image_type is the OS. shielded_instance_config is the VM wrap the PCA notes mean when they say Shielded VM is best practice: Secure Boot, integrity monitoring, vTPM. The two nests are cousins in the file. They are not the same door. A COS node with Shielded off still has a small image. An Ubuntu node with Shielded on still has a fat userspace. The census has to print both.

The CLI Evan types when the join is one pool and one node:

gcloud container node-pools describe hardened --cluster=app --zone=us-central1-a --format='yaml(config.imageType,config.shieldedInstanceConfig)'
kubectl describe node

K8sUR3 already showed the second command (Ch. 14 adjacent; node conditions in the describe output). One of the conditions on a healthy Linux node is AppArmor enabled. That line is the host talking. It is not a Pod. It is not an API flag. If describe says AppArmor enabled and every Pod still runs Unconfined, the kernel is ready and the workload never asked.

Autopilot: gcp-ace-pca says no node ops. There is no node-pool image_type for you to set. There is no SSH. There is no apparmor_parser. If the task is Q01, Autopilot is the wrong product. If the task is "I never want to SSH," Autopilot is the product and this lesson's host fields do not exist. Pick Standard when you came to inspect the host.

Fact three. AppArmor is a kernel profile. A Pod annotation is not a load.

Rice, Ch. 8, printed p. 97: AppArmor is a Linux security module. A profile binds to an executable and decides capabilities and file access. Look in /sys/module/apparmor/parameters/enabled. If you find y, the LSM is on. Mandatory access control: the administrator sets it, the user cannot override it. Complain mode logs. Enforce mode denies. Install the profile under /etc/apparmor and run apparmor_parser. See loaded profiles in /sys/kernel/security/apparmor/profiles.

Rice is blunt about Kubernetes (printed p. 98): there is a default Docker AppArmor profile, and Kubernetes does not use it by default. You add annotations to use any AppArmor profile on a container in a Kubernetes pod. K8sUR3 Ch. 14 writes the modern field and the old annotation together. securityContext.appArmorProfile.type Localhost plus localhostProfile. Or metadata.annotations container.apparmor.security.beta.kubernetes.io/CONTAINER: localhost/PROFILE. Q01 Verify.bash accepts either. A YAML that names the profile in a comment loads nothing.

seccomp is the sibling (Rice printed pp. 95-97). First it meant four syscalls: sigreturn, exit, read, write on already-open fds. Too small to work. seccomp-bpf (2012) filters by opcode and arguments. Docker's default profile blocks clock_settime, create_module, keyctl, the calls a container has no business making. Kubernetes does not apply that default unless you set seccompProfile.type RuntimeDefault. K8sUR3's amicontained Pod without a SecurityContext printed Seccomp: disabled and AppArmor Profile: docker-default (enforce) on one runtime, which is exactly the split Rice warned you about: one tool defaults, the other does not.

Fact four. Privileged is the host by another name.

K8sUR3 Ch. 14: privileged: true elevates the container to the same permissions as the host. allowPrivilegeEscalation defaults true, and it becomes true if privileged is true. readOnlyRootFilesystem is the cheap write-cut. runAsNonRoot fails the container that still thinks it is uid 0. capabilities drop NET_BIND_SERVICE, add only what you can name. Those fields live on the Pod. They are still System Hardening, because they decide whether the process can touch the host kernel.

hostNetwork, hostPID, hostIPC, hostPath: the Pod is no longer in its own namespaces. kubestronaut's "minimize host OS footprint" bullet is the node side of that. A COS image with docker.sock mounted from the host is a small OS with a large hole. 08-13's gVisor RuntimeClass changes the runtime. Today's fields do not. If you reach for RuntimeClass because "isolation" sounds like AppArmor, you are in last month's file.

§III — Mechanism: the node pool, then the profile, then the Pod

Order of operations on the fleet.

1. Create a Standard GKE cluster. Not Autopilot. The ACE/PCA tree says "Need Kubernetes? GKE. Need managed K8s with no node ops? Autopilot." You need node ops today.
2. Set the node pool image_type to COS_CONTAINERD. Confirm on describe. An omitted image_type often still lands COS. Print it anyway. An inherited default is not a census.
3. Set shielded_instance_config both bits true. Secure Boot refuses unsigned boot loaders. Integrity monitoring attests the firmware and kernel. The PCA line about Shielded VM is this nest when the VM is a GKE node.
4. Confirm AppArmor enabled on kubectl describe node. COS should already show it. If it does not, you are not on the image you thought.
5. Decide whether any Pod needs a Localhost profile. RuntimeDefault / runtime/default is the cheap win K8sUR3 Ch. 14 demonstrates with amicontained: Seccomp: filtering, dozens of blocked syscalls, AppArmor still docker-default. A custom profile is Q01 and Q33. Those files live on the node. COS will not love a random file you SSH'd into /etc/apparmor.d. That is why the exam is Ubuntu and the fleet is COS plus RuntimeDefault plus a PSA restricted namespace.

Order of operations on the exam (Cert file drills this; name it so Ops does not pretend GKE is kubeadm):

1. SSH to the worker, not the control plane. Q01 LabSetUp writes /etc/apparmor.d/nginx_apparmor on the worker.
2. apparmor_parser -q the file. aa-status | grep the profile name. Parser without status is a hope.
3. Edit the Pod. securityContext.appArmorProfile Localhost nginx-profile-2, or the beta annotation. Apply.
4. kubectl exec touch /etc/test. Permission denied is the pass. A Running Pod with the profile missing is a Running Pod.

If/then: if you loaded the profile on the control plane, the worker kubelet never saw it. If you loaded it on node01 and the Pod landed on node02, Unconfined. Q01 pins nodeName. A Deployment does not. DaemonSet the profile loader, or pin, or use RuntimeDefault that the runtime already shipped.

PGA from 08-30 is a subnet flag plus a missing access_config. Today's Shielded bit is a node-config nest plus a missing Ubuntu. Both are GCP. They do not share a field. A census that prints private_ip_google_access has not described the host.

§IV — Worked example: one pool, one Unconfined Deployment

Cluster app in us-central1. Node pool hardened as above. A Deployment web with no securityContext. image nginx. replicas 2.

Describe the pool. imageType COS_CONTAINERD. enableSecureBoot true. enableIntegrityMonitoring true. Describe a node. AppArmor enabled. Get the Pod. No appArmorProfile. No seccompProfile. privileged false only because it defaulted. allowPrivilegeEscalation unset, which means true.

The honest row:

pool=hardened image=COS_CONTAINERD secure_boot=true integrity=true
node=gke-app-hardened-xxxx apparmor_kernel=enabled
deploy=web ns=default seccomp=absent apparmor=absent priv=false ape=unset
why=unconfined_on_a_ready_host

The host is hardened. The workload is not. A dashboard that only greens the node pool will call this cluster done. It is 08-25 inverted: the API is whoever GKE already locked, the Pod still has the host's syscalls.

Fix on the fleet, PSA restricted on the namespace (K8sUR3 Ch. 14, the warn/enforce labels that demand RuntimeDefault and runAsNonRoot). The Pod that cannot set those fields fails admission. That is the point. Do not SSH into COS to plant nginx-profile-2 unless you have a node image pipeline that owns /etc/apparmor.d. Custom Localhost profiles on GKE are a node-image problem, not a kubectl apply problem.

Fix that is a mistake: image_type = "UBUNTU_CONTAINERD" so you can apparmor_parser like the exam. You just grew the host OS footprint to make a drill easier. kubestronaut's first bullet was minimize that footprint. COS was the minimize. Ubuntu is the exam. Keep them separate.

Fix that is last month: add a RuntimeClass gvisor and label the namespace. Rice Ch. 8 puts gVisor after AppArmor for a reason (printed p. 100). 08-13 already spent it. Today's Unconfined nginx does not need a guest kernel. It needs RuntimeDefault and priv false.

§V — Connection to prior lessons

08-25 closed the API on AKS: authorized IP ranges, disable local accounts, Azure RBAC. Poulton already told you a hosted apiserver hides the static Pod. GKE hides it too. System Hardening still has a surface on GKE: the node pool. Cluster Hardening's flags do not. Do not hunt anonymous-auth in this file.

08-28 closed a rolling update on EKS. Two ReplicaSets, one cluster. A surge does not change the node image. maxSurge 100% is not Secure Boot. Leave the Deployment strategy alone.

08-22 closed GKE PD CSI Retain. The volume that survived the claim is a disk. COS is an image. Both are GKE. They do not share a resource. Do not open PersistentVolume.

08-19 closed EKS encryptionConfig. etcd ciphertext is Cluster Hardening adjacent / Minimize Microservice store. Not a host profile.

08-16 closed AKS scheduling. Taints are not AppArmor.

08-15 closed GKE Workload Identity. That is who the Pod is. Today's question is what the Pod's process may ask the kernel. Identity is a later shirt. Syscalls are this one.

08-13 closed GKE Sandbox / gVisor / RuntimeClass. Rice's next section after AppArmor. Host kernel boundary by a guest kernel. Today stays on LSM and seccomp. If you reach for runsc, you are rereading 08-13.

08-30 closed PGA. Subnet flag. No access_config. Same cloud, different product. Compute Engine without a public IP is not a GKE node. Do not print privateIpGoogleAccess and call the node hardened.

§VI — Connection to today's Dev and Cert

Dev is the Python census. CoreV1Api.list_pod_for_all_namespaces. Print privileged, allow_privilege_escalation, seccomp_profile, app_armor_profile, the beta annotation. Tag unconfined, privileged_host, ape_open. Do not patch. 08-28 already taught that a leftover checker that patches a Deployment is a rollout. A leftover checker that patches securityContext is a host change on every Pod the selector hits. Inventory is a timer that prints.

Cert is Q01 and Q33 by hand. SSH to the worker. apparmor_parser. Localhost profile on the Pod. Custom seccomp JSON under /var/lib/kubelet/seccomp/profiles with SCMP_ACT_ERRNO default and four allowed syscalls. The exam is kubeadm Ubuntu. The fleet is GKE COS. Learn both. Grade the exam on aa-status and the JSON path. Grade the fleet on image_type and RuntimeDefault.

§VII — Close

The API is 08-25. The guest runtime is 08-13. The host is this file. COS is the small image. Shielded is the VM wrap. AppArmor enabled on describe node is the kernel. Unconfined on the Pod is the leftover.

Read the node pool first. Then describe node. Then get the Pod securityContext. Decide whether you are on the exam or on the fleet. The exam wants apparmor_parser. The fleet wants COS_CONTAINERD plus both Shielded bits plus RuntimeDefault. Neither wants Autopilot when the stem says SSH. Neither wants a second cluster because a profile was missing.

Examine well. The host is not the cluster.

Related

• Prior arc: EKS rolling update (2026-08-28)
• Pair hub: Cross-References/domains/01-Earth-DevOps
• Grounding tome: Rice Ch.8, Seccomp and AppArmor, pp. 95-99