Kubernetes Cluster Hardening on AKS — the API that still answers
The control plane is hidden. The API still answers. Close the range before you hunt a flag.
<!-- hal:authoritative:yaml -->
The control plane is hidden. The API still answers. Close the range before you hunt a flag.
§I — Frame
Saturday 08-22 named the leftover volume. Wednesday 08-19 named the leftover Secret. Saturday 08-16 named the Pending that is not storage. Today the leftover is the API itself.
k8s_day_counter reads 11. Odd is CKS-emphasis. 08-19 closed Minimize Microservice Vulnerabilities on the store half and reserved Cluster Hardening Q04 for the next CKS fire. That fire is this one. The leftover domains were Cluster Hardening 15% and System Hardening 15%. Official order puts Cluster Hardening first. Trust the runbook. Q04 is the exam form. AKS is the cloud form.
The last two K8s Ops visits sat on GKE (08-22) and EKS (08-19). AKS is the rebalance. 08-16 already used AKS, and it used AKS as a scheduler. Today AKS is a door. Do not open the system-pool taint file again.
Poulton walks a request through the API server in one picture: TLS, then authentication, then authorization, then admission (Ch. 13, API security big picture, pp. 176-177). He then writes the sentence this overlay lives on: "You won't be able to interrogate the API server like this on a hosted Kubernetes cluster. This is because critical control plane features like this are hidden from you" (Ch. 13, p. 183). Q04 edits /etc/kubernetes/manifests/kube-apiserver.yaml. AKS does not give you that file. The flags still matter. The knobs move.
Coin it: the API that still answers.
A public AKS API with an empty authorized-IP list answers from any address that can route to it. A cluster that still has local accounts answers a kubeconfig user who is not in Entra. A leftover ClusterRoleBinding that grants cluster-admin to system:anonymous answers a caller who never authenticated. Three doors. One failure name.
§II — Foundations: three doors on one request
Poulton's airport picture is the right order (Ch. 13, p. 177). Passport first. Ticket second. Cabin policy third. Skip a door and the later doors never see the passenger.
Door 1 is reach. The request has to arrive on the secure port. Poulton assumes that for everything that follows, and then warns that some clusters also expose an unsecured port on every control plane node. That port bypasses TLS, authentication, and authorization. It does not bypass admission. "Even though the unsecured port is only accessible if you can log on to a control plane node, you should probably disable it on most clusters" (Ch. 13, pp. 176-177). On AKS the analog is not a localhost flag you grep. It is a public API frontend whose allow-list is empty. Empty means the world.
Door 2 is identity. Authentication proves the caller is who the request claims. Fail and the API returns HTTP 401 (Ch. 13, p. 177). Kubernetes has no built-in identity database. It will not store a user for you. Client certificates work out of the box. Production wants the corporate or cloud directory (Ch. 13, pp. 177-178). On AKS that directory is Microsoft Entra ID. 08-23 already taught Entra as AuthN versus AuthZ. Today Entra is the source the API is supposed to ask. Local accounts are the source it still asks if you never turned them off.
Door 3 is permission. Authorization runs immediately after a successful 401-avoidance. Modules are pluggable. The first module that authorizes wins. RBAC is the common module, stable since 1.8, least-privilege, deny-by-default (Ch. 13, pp. 178-179). No rule means no. cluster-admin is the opposite rule: all verbs on all resources in all namespaces (Ch. 13, p. 185). Poulton's Docker Desktop walk finds a pre-created ClusterRoleBinding that hands cluster-admin to system:serviceaccounts in kube-system. "This might be OK for development environments… but it's not appropriate for production" (Ch. 13, p. 184). Q04 LabSetUp is the exam version of that sentence: a binding named system:anonymous bound to cluster-admin, plus --authorization-mode=AlwaysAllow, plus --anonymous-auth deleted so it defaults on.
Admission is the cabin check. Mutating controllers run first. Validating controllers cannot edit. Both skip read-only requests. Poulton's Docker Desktop apiserver enables NodeRestriction (Ch. 13, p. 185). Q04 requires that plugin by name. NodeRestriction stops a kubelet from writing objects it does not own. It is not a NetworkPolicy. 08-01 already spent the perimeter. Leave it there.
Four CKS Cluster Hardening bullets sit on these doors.
- Restrict access to the Kubernetes API. Door 1, plus
--anonymous-auth=false. - Use RBAC to minimize exposure. Door 3. Delete the anonymous
cluster-adminbinding. Do not leaveAlwaysAllow. - Exercise caution in using service accounts. Disable default automount. Bind a Role that lists pods and cannot list secrets. That is Q03 and Q37. Cert carries them. Ops names them so the trio stays honest.
- Update Kubernetes frequently. Q15 is the drain /
kubeadm upgrade node/ uncordon procedure. AKS names the same bullet as an auto-upgrade channel. Today that is a one-line leftover, not the spine.
§III — Mechanism: AKS, because the static Pod is gone
Poulton greps --authorization-mode=Node,RBAC off a Docker Desktop apiserver Pod (Ch. 13, p. 183). Then he tells you the hosted cluster will not let you do that. AKS is that hosted cluster. Microsoft runs kube-apiserver. You never SSH to it. You never edit the manifest. You never wait thirty seconds for a static Pod to recycle. The exam still grades the flags. Production grades the Azure objects that stand in for them.
Three AKS fields do the work.
Authorized IP ranges. apiServerAccessProfile.authorizedIpRanges is an allow-list in front of the public API. Set it and the frontend refuses source addresses that are not on the list. Leave it null or empty and the API still answers from any public path that can route. That is Door 1 with a Microsoft name. A private cluster (enablePrivateCluster: true) removes the public frontend entirely. It is a stronger door, and it is a different lesson: Private Link, a jump box, DNS in the VNet. Today the public cluster is the honest case. Most fleets start there. Most fleets forget the list.
Disable local accounts. AKS can mint cluster-local users that live in kubeconfig and do not live in Entra. Those users survive a directory offboarding. They are Door 2's leak. --disable-local-accounts turns them off. After that, the only human identity the API will accept is an Entra identity (or a workload identity the cluster already federated). 08-04 taught the GKE form of that federation. Do not rebuild Workload Identity here. The human back door is the new fact.
Azure RBAC for Kubernetes Authorization. --enable-azure-rbac (with AKS-managed Entra) puts Azure RBAC in the authorization path. A Contributor on the ARM resource is no longer a silent cluster-admin on the API. You grant Azure Kubernetes Service RBAC Reader or RBAC Admin or a custom role at the cluster or namespace scope. Door 3 moves partly into ARM. Kubernetes RBAC objects still exist. A leftover ClusterRoleBinding to system:anonymous still exists if someone applied it. Azure RBAC does not vacuum the API. It adds a second authorizer. The leftover binding is still a finding.
az aks show is the read. Ask for the four fields that name the doors.
az aks show -g rg-prod -n aks-prod --query '{
authorizedIps: apiServerAccessProfile.authorizedIpRanges,
private: apiServerAccessProfile.enablePrivateCluster,
disableLocalAccounts: disableLocalAccounts,
azureRbac: aadProfile.enableAzureRbac
}'Read the output the way 08-22 read kubectl get sc. Four columns. Null authorized IPs on a public cluster is the API that still answers. disableLocalAccounts: false is the kubeconfig user who is not in Entra. azureRbac: false with local accounts on is the cluster-admin kubeconfig from az aks get-credentials --admin. --admin is the break-glass. If local accounts are disabled, --admin stops working. That is the point.
The Bootcamp cheatsheet names AKS as "Managed Kubernetes" and stops. The knobs are not in that file. Poulton's hosted-cluster sentence is the tome that licenses the move. Q04 is the exam that still wants the flags. Both are true on the same Tuesday.
Q04 LabSetUp also sets --authorization-mode=AlwaysAllow and swaps NodeRestriction for AlwaysAdmit. AlwaysAllow is the authorizer that never says no. AlwaysAdmit is the admission plugin that never says no. Poulton's deny-by-default sentence is the opposite of both (Ch. 13, p. 179). AKS will not let you set AlwaysAllow. A human can still apply a binding that has the same effect for one subject. That is why the leftover census exists on a managed control plane: the flag you cannot see is not the only way the API still answers.
§IV — Worked example: close the range, then look for the leftover binding
Start from a public AKS cluster that was created with the portal defaults. authorizedIpRanges is empty. Local accounts are on. Azure RBAC is off. az aks get-credentials --admin still writes a user that is not Entra.
Close Door 1 and Door 2 together.
az aks update \
--resource-group rg-prod \
--name aks-prod \
--api-server-authorized-ip-ranges 203.0.113.10/32 \
--disable-local-accountsThe update rewrites the public frontend. From a laptop that is not 203.0.113.10, kubectl get nodes becomes a timeout or a 403 at the edge. That is success. From the jump box that owns the /32, kubectl still works. Add the office egress and the pipeline NAT later. Do not add 0.0.0.0/0 to "keep it working." That list is then theater.
Turn on Azure RBAC if the cluster is already Entra-integrated.
az aks update \
--resource-group rg-prod \
--name aks-prod \
--enable-aad \
--enable-azure-rbacGrant the on-call group Azure Kubernetes Service RBAC Cluster Admin at the managed-cluster scope. Grant the app team Azure Kubernetes Service RBAC Writer at the namespace scope. Do not hand Owner on the resource group and call it done. Owner is ARM. The API is a different plane. 08-23 already separated AuthN from AuthZ. This is that split with a cluster as the resource.
Now the leftover. Q04 LabSetUp creates this object on a kubeadm cluster.
kubectl create clusterrolebinding system:anonymous \
--clusterrole=cluster-admin \
--user=system:anonymousAKS did not create that binding. A human can. A broken Helm chart can. A "temporary" debug session can. Authorized IP ranges do not delete it. Disable-local-accounts does not delete it. Azure RBAC does not delete it. kubectl get clusterrolebinding system:anonymous still returns an object. A caller who reaches the API as system:anonymous still holds cluster-admin if --anonymous-auth is on.
On AKS you cannot flip --anonymous-auth. Microsoft's control plane already sets the flag the exam wants. You can still have the binding. The day's Dev lesson is the census that prints that binding and will not delete it. The day's Cert lesson is the kubeadm edit that sets the three flags and then deletes the binding. Ops stops at the Azure door plus the leftover object you can still see with kubectl.
Confirm the four fields again. Then list the dangerous subjects.
kubectl get clusterrolebinding -o json | python3 -c '
import json, sys
doc = json.load(sys.stdin)
danger = {"system:anonymous", "system:unauthenticated"}
for b in doc.get("items", []):
role = (b.get("roleRef") or {}).get("name")
for s in b.get("subjects") or []:
name = s.get("name")
if name in danger or (role == "cluster-admin" and name == "system:authenticated"):
print(b["metadata"]["name"], role, s.get("kind"), name)
'An empty print is a good Tuesday. A row is the API that still answers after you closed the range. Treat the row as an incident, not as a cleanup script. The Dev lesson explains why the loop must not delete.
One more AKS-specific leftover: --admin credentials that were written to a laptop last quarter. Disabling local accounts invalidates them on the next handshake. It does not shred the file. Rotate the humans. Collect the kubeconfigs. The file on disk is a Door 2 object the API no longer honors and a secret the laptop still holds.
Auto-upgrade is the fourth Cluster Hardening bullet in one line. autoUpgradeProfile.upgradeChannel on the AKS object is patch, stable, rapid, or none. none is the postponed upgrade. Q15 is the kubeadm form. Do not spend this fire on a drain. Name the channel. Leave the procedure to Cert.
§V — Connection to prior lessons
08-04 taught RBAC as CKA Cluster Architecture: Role, ClusterRole, RoleBinding, ClusterRoleBinding, CSR, kubectl auth can-i. That lesson built the objects. Today uses them as a leftover. The question is no longer "can you mint a Role." The question is "which binding still grants the world cluster-admin after you closed the Azure door."
08-01 taught NetworkPolicy on EKS. That is Cluster Setup, the perimeter around pods. Today's perimeter is the API frontend. A default-deny ingress does not stop kubectl from the public internet. A closed authorized-IP list does not stop east-west pod traffic. Spend the right lesson on the right door.
08-16 used AKS as a scheduler: system-pool taint, user-pool capacity, the Pending that is not storage. The cluster in that file can still have an empty authorized-IP list. Scheduling success is not hardening.
08-19 reserved Q04 in its pairing section. This fire spends the reservation. Encryption-at-rest stays in etcd. Today never opens etcd.
08-07 used AKS as a supply-chain seat: ACR, digest pinning, admission. The cargo is not the API. Do not reopen ImagePolicyWebhook.
08-22 leftover was a claimRef. Today's leftover is a subject. Same discipline: read, name, do not "fix" in a loop.
§VI — Connection to today's Dev and Cert
Dev is the Python census. RbacAuthorizationV1Api.list_cluster_role_binding. Print bindings whose subjects are system:anonymous or system:unauthenticated, or whose role is cluster-admin bound to system:authenticated. Do not call delete. 08-22 already taught that a leftover checker that patches rebinds the wrong disk. A leftover checker that deletes ClusterRoleBindings deletes the binding a human is still investigating.
Cert is Q04 by hand. Edit the static Pod. --authorization-mode=Node,RBAC. --anonymous-auth=false. --enable-admission-plugins=NodeRestriction. Delete clusterrolebinding system:anonymous. Wait for the apiserver. Use /etc/kubernetes/admin.conf. That file does not exist on AKS. The flags still have to live in your head, because the exam is kubeadm and the fleet is AKS.
Q03 and Q37 sit beside Q04 in the same 15%. Automount off on default. A Role that lists pods and cannot list secrets. Ops named them. Cert drills them. Dev does not impersonate; 08-04 already did SelfSubjectAccessReview.
§VII — Close
Three doors. Reach, identity, permission. AKS hides the static Pod and leaves you the Azure fields. An empty authorized-IP list on a public cluster is the API that still answers. Local accounts are the user who is not in Entra. A leftover anonymous cluster-admin binding is the permission the doors did not cancel.
Read az aks show first. Then list ClusterRoleBindings. Then decide whether you are on the exam or on the fleet. The exam wants the flags. The fleet wants the range.
Examine well. The control plane is hidden. The binding is not.
Related
- Prior arc: EKS secrets encryption (2026-08-19)
- Pair hub: Cross-References/domains/01-Earth-DevOps
- Grounding tome: Poulton Ch.13, API security and RBAC, pp. 176-186