CKS Encryption at Rest — the secret that is still plaintext

The file is on the control plane. The first provider writes. etcdctl is the only view that counts.

<!-- hal:authoritative:yaml -->

The file is on the control plane. The first provider writes. etcdctl is the only view that counts.

§I — Frame

k8s_day_counter is 9, odd, so this slot is CKS. Four CKS domains are already closed on this arc. Cluster Setup 10% closed 08-01. Minimize Microservice Vulnerabilities 20% closed 07-26 on the restricted profile. Supply Chain Security 20% closed 08-07. Monitoring, Logging and Runtime Security 20% closed 08-13. Blueprint on the Cert slot sits at 70%. The leftovers are Cluster Hardening 15% and System Hardening 15%.

Today is not those leftovers. The day's Ops overlay is EKS encryptionConfig and a customer CMK. The failure that overlay names is the secret that is still plaintext. Enabling encryption later does not rewrite old objects. The exam form of that sentence is Q08. Bootcamp files Q08 under Minimize Microservice Vulnerabilities (20%). That is the honest domain tag, even though 07-26 already spent the domain on PSS. This fire is the store half.

The exam cluster is kubeadm. You have SSH to the control plane. You can write /etc/kubernetes/enc/enc.yaml. You can edit the static Pod. You can run etcdctl. EKS hid all three. The physics did not change.

Poulton's short answer is the stem (Ch. 11, pp. 148-150): Secrets are not secure. Kubernetes stores them as base64. Anyone who can read etcd, or who can kubectl get secret -o yaml, can decode them. EncryptionConfiguration is how you change the store. The Secret the API returns is still plaintext so the app can read it. Q27 and Q40 are that last sentence as a graded task. They are not a store audit. Leave them adjacent.

Call the failure the secret that is still plaintext. Hold it through five steps. A green kubectl get after step 3 is the trap. Step 5 is the rewrite. Without it, test-unencrypted is the same bytes it was when you created it.

§II — The file and the order

enc.yaml is a static-Pod input

Q08 SolutionNotes is five steps. Steps 1 and 2 are the file.

head -c 32 /dev/urandom | base64

Thirty-two bytes. That is the AES-CBC key. Paste it once. Back it up. Lose it and you cannot unwrap. Rotate it later by adding a new key at the top of the list and rewriting. Do not delete the old key until every object has been rewritten.

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: <paste-base64-key-here>
      - identity: {}

Path on disk: /etc/kubernetes/enc/enc.yaml. Verify.bash greps that path. A file in /tmp is a file the apiserver will never see.

Provider order is the whole control. The first provider encrypts new writes. Later providers are readers. identity: {} last means: write with aescbc, still read the old clear objects during migration. That fallback is how you survive the window between "flag is on" and "every Secret has been replaced."

Swap the order and you have chosen the failure.

providers:
  - identity: {}
  - aescbc:
      keys:
        - name: key1
          secret: <key>

First provider is identity. Every new write is plaintext. aescbc is only a reader of old ciphertext. The console, if you had one, would still say encryption is configured. The store is the secret that is still plaintext, now as a default write path you chose. EKS will not let you paste that file. A kubeadm cluster will. The exam will.

SolutionNotes names the other providers so you do not wander. aescbc is the recommended production provider. aesgcm is faster and weaker on key management. secretbox is newer (1.27+), XSalsa20+Poly1305. Verify.bash greps aescbc. Use aescbc unless the stem names something else.

The static Pod must mount the file

Step 3 is /etc/kubernetes/manifests/kube-apiserver.yaml. Three edits.

- --encryption-provider-config=/etc/kubernetes/enc/enc.yaml

Then a volumeMount:

- name: enc
  mountPath: /etc/kubernetes/enc
  readOnly: true

Then a volume:

- name: enc
  hostPath:
    path: /etc/kubernetes/enc
    type: DirectoryOrCreate

The flag without the mount is the 08-13 audit-policy miss in a new coat. The apiserver starts, cannot open the file, and exits. kubectl get nodes returns connection refused. Wait. Check crictl ps -a and the apiserver container log. Do not edit the manifest a second time while the first restart is in flight.

Wait about thirty seconds. Confirm the static Pod is Running. Verify.bash checks the flag, the file, aescbc, identity, the mount, and the Running pod. It does not open etcd. Passing Verify.bash is not proof the store is encrypted.

§III — Worked example

The stem created test-unencrypted in default before you arrived. That object is the secret that is still plaintext. Your job is to make the store encrypt Secrets, then make that object join the new write path.

After the apiserver is Running with the flag, step 4 is the only store view you have.

ETCDCTL_API=3 etcdctl \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  get /registry/secrets/default/test-unencrypted | hexdump -C

Before the rewrite you will see plaintext key and value bytes. After a successful write through aescbc you will see the prefix k8s:enc:aescbc:v1:key1 plus binary. That prefix is the proof. kubectl get secret test-unencrypted -o yaml will print data: both times. Poulton's echo UGFzc3dvcmQxMjM= | base64 -d is that paste (Ch. 11, p. 150). The API decoded on the way out. It is doing its job. The job is not a store audit.

Step 5 is the rewrite.

kubectl get secrets --all-namespaces -o json | kubectl replace -f -

Replace reads each object through the API (plaintext) and writes it back. The write hits the first provider. The bytes in etcd change. The string a client sees does not. Without this line, old secrets remain in plaintext in etcd. SolutionNotes wrote that sentence so the exam would not forget it. The day's Ops lesson is the same sentence on EKS: associate, then annotate or replace. eksctl will skip the rewrite if you pass --encrypt-existing-secrets=false. On the exam there is no eksctl. There is this pipe.

Run the hexdump again. The prefix must be there. If it is not, the first provider is identity, or the replace never ran, or you hexdumped a different object.

A create after the flag is on writes through aescbc on the first try. The trap is the object that already existed. Grade yourself on test-unencrypted, not on a Secret you created to feel better.

Poulton's threat model (Ch. 15, p. 204) is why the key is a 32-byte file on the control plane and not a story. Kubernetes 1.7 added encryption of Secrets and left it off. The DEK lives on the same node as the Secret. Access to that node bypasses the wrap. Version 1.11 let the KEK live outside the cluster. The exam does not have KMS. It has aescbc and a key you generated. EKS is the outside-key form. The exam is this file.

§IV — Failure mode: the secret that is still plaintext

Flag on, no rewrite. The apiserver is Running. Verify.bash is green. test-unencrypted is the 2025 bytes. The hexdump has no prefix. This is the failure the lesson is named for. Step 5 is not optional.

Identity first. You reversed the providers so "migration would be safer." New writes are plaintext. Old ciphertext, if any, still decrypts. The hexdump of a new Secret has no prefix. Put aescbc first. Leave identity: {} last.

**Trusting kubectl get.** Q27 and Q40 train the muscle: jsonpath plus base64 -d, then a volume mount whose files are already decoded. That muscle is correct for those stems. It is the wrong proof for Q08. The API always returns plaintext for a Secret it can decrypt, including a Secret it never encrypted.

Flag without mount. The static Pod crash-loops. You spend the clock on a second edit. Same as the 08-13 audit-policy file that never got a volume. Make all three edits, then wait.

A key you cannot find later. Rotation is a new key at the top of the list plus a rewrite, then a waiting period, then removal of the old key. Deleting the only key is how you make the cluster permanently unable to unwrap. Ops named the CMK version of that sentence. The exam version is a file you forgot to copy off the node.

Talking to the API from Python and calling it etcd. That is the day's Dev lesson. The checker can see encryptionConfig and creationTimestamp. It cannot see /registry/secrets. If the task is the hexdump prefix, you are in this file. Stay here.

§V — Pairing

Ops is the EKS association and the rewrite you cannot confirm with kubectl get. Dev is the two reads that see the association and cannot see etcd. This file is Q08 by hand: the YAML, the order, the mount, the hexdump, the replace.

Q27 and Q40 stay adjacent. They are the decode-and-mount half of the same domain. Q37 is Cluster Hardening: a Role that can list pods and cannot list secrets. Do not spend this fire on it. Q04 is Cluster Hardening leftover: --anonymous-auth=false, Node,RBAC, NodeRestriction, remove the system:anonymous ClusterRoleBinding. Adjacent only. Next K8s day 2026-08-22 reads an even counter after this advance and returns to CKA. The next CKS-emphasis fire can take Q04.

08-13 remains Falco, audit policy, immutable containers, and RuntimeClass. 08-07 remains trivy and ImagePolicyWebhook. 08-01 remains kube-bench. 07-26 remains the restricted profile. None of them open etcd.

§VI — Exam drill

Related

• CKS Runtime Security (2026-08-13)
• CKS Supply Chain Security (2026-08-07)
• CKS PSS / restricted profile (2026-07-26)
• Cross-References/Certifications-Roadmap
• Poulton Ch.11 pp.148-150; Ch.15 p.204 referenced

🫡 ⚖️ 📜 Leo.Syri — Praetor Consulate, Imperium Luminaura Filed 2026-08-19 · Fajr · K8s day 28 · tenth visit · trio #94 · k8s_day_counter 9 (CKS-emphasis)