Python and the EKS Encryption Config — a checker that cannot see etcd
The client lists Secrets. The API already decoded them. Your loop is not etcdctl.
<!-- hal:authoritative:yaml -->
The client lists Secrets. The API already decoded them. Your loop is not etcdctl.
§I — Frame
The day's Ops lesson puts a customer CMK on cluster.encryptionConfig and names the failure: the secret that is still plaintext. Enabling encryption on an existing EKS cluster does not rewrite old objects. kubectl get still prints data:. The store, which you cannot SSH onto, may still hold the 2025 bytes.
This slot is Python touching K8s. k8s_day_counter is odd, so Cert is CKS, and Dev stays on the client. 08-10 already watched events. 08-13 already mutated a RuntimeClass with kopf. 08-16 already predicted a scheduler no and refused to bind. 08-04 already asked SelfSubjectAccessReview. None of those tools answer the question Ops just asked: does this cluster have a CMK association, and which Secrets were created before that association existed?
Call the failure the checker that cannot see etcd. A first draft will list_secret_for_all_namespaces, look at data, and print "plaintext" for every object. That is the API doing its job. It is not a store audit. Q08 sees the store with etcdctl | hexdump and the prefix k8s:enc:aescbc:v1:key1. Your process is not etcdctl. On EKS you will never be etcdctl.
Today's tool does two reads. boto3 eks.describe_cluster for encryptionConfig (resources, provider.keyArn). The Kubernetes client for Secret metadata (name, namespace, creationTimestamp). It flags a cluster with no association, or a Secret whose timestamp is earlier than an operator-supplied encryption_enabled_at. It does not bind. It does not create. It does not open a watch. It does not call etcd.
§II — The two reads
The cluster object, as boto3 sees it
from typing import Any
def encryption_config(eks, name: str) -> list[dict[str, Any]]:
cluster = eks.describe_cluster(name=name)["cluster"]
return list(cluster.get("encryptionConfig") or [])
describe_cluster is a regional EKS API call. It is not a Kubernetes API call. It returns the association: a list of objects with resources (the only supported value is secrets) and provider.keyArn. Empty list or a missing key means this cluster has no customer CMK on Secrets. That is a real finding. It is the finding Lensix-style scanners print when encryptionConfig is absent.
The response does not include the time the association appeared. It does not include a census of etcd. It does not include ciphertext. If you need "created before encryption existed," you bring that instant in from outside: a ticket, a CloudTrail AssociateEncryptionConfig event, the createdAt on the describe-update you saved when you ran the enable. The function above cannot invent it.
def first_secrets_key_arn(cfg: list[dict[str, Any]]) -> str | None:
for item in cfg:
resources = item.get("resources") or []
if "secrets" not in resources:
continue
provider = item.get("provider") or {}
arn = provider.get("keyArn")
if arn:
return arn
return None
A 1.28 cluster may have AWS-managed envelope encryption and still return no keyArn. Ops already named that split. This checker reports the CMK association. It does not claim to see platform wrap.
Secret metadata, as the client sees it
from kubernetes import client
def secret_meta(core: client.CoreV1Api) -> list[dict[str, str]]:
rows = []
for item in core.list_secret_for_all_namespaces().items:
ts = item.metadata.creation_timestamp
rows.append(
{
"namespace": item.metadata.namespace or "",
"name": item.metadata.name or "",
"created": ts.isoformat() if ts else "",
}
)
return rows
Name, namespace, creationTimestamp. Not data. Not stringData. Not a hex dump. Poulton's kubectl get secret creds -o yaml is the trap this function refuses (Ch. 11, pp. 148-150): the data map is base64, it decodes without a key, and it looks the same whether etcd holds ciphertext or not. Listing data would only prove you have get on Secrets.
list_secret_for_all_namespaces is a privileged read. The tool should say so. It is still not etcd. Service-account token Secrets, helm release Secrets, and the default-token objects Q27 and Q40 teach you to mount will all appear. Filter in the report if you must. Do not filter in a way that hides a pre-encryption Opaque Secret in payments.
The list call paginates. list_secret_for_all_namespaces returns a _continue token when the cluster has more objects than the page size. A one-shot .items walk that ignores _continue is a silent undercount. The payments Secret that is still plaintext may live on page two. Loop until the token is empty. Do not treat a short first page as a census.
def secret_meta(core: client.CoreV1Api) -> list[dict[str, str]]:
rows: list[dict[str, str]] = []
_continue = None
while True:
page = core.list_secret_for_all_namespaces(_continue=_continue)
for item in page.items:
ts = item.metadata.creation_timestamp
rows.append(
{
"namespace": item.metadata.namespace or "",
"name": item.metadata.name or "",
"created": ts.isoformat() if ts else "",
}
)
_continue = page.metadata._continue
if not _continue:
break
return rows
A 403 is not "no Secrets." It is "this ServiceAccount cannot list cluster-wide." Catch ApiException with status 403 and print list_forbidden. Do not print cluster_clean. The same 403 on a single-namespace list_namespaced_secret is a narrower grant. It is still not etcd. Either way the checker has no store view.
Do not add create_namespaced_pod_binding. That is 08-16. Do not add a kopf mutate. That is 08-13. Do not add SelfSubjectAccessReview. That is 08-04. Today's client reads two objects and prints findings.
§III — Worked example
The operator recorded that atlas-prod associated a CMK at 2026-03-01T14:22:00+00:00 (the createdAt on the AssociateEncryptionConfig update). The checker loads kubeconfig, builds an EKS client in us-east-1, and runs both reads.
from datetime import datetime, timezone
from kubernetes import client, config
import boto3
def parse_ts(value: str) -> datetime | None:
if not value:
return None
return datetime.fromisoformat(value.replace("Z", "+00:00"))
def findings(
cfg: list,
rows: list[dict[str, str]],
enabled_at: datetime | None,
) -> list[dict[str, str]]:
out = []
arn = first_secrets_key_arn(cfg)
if arn is None:
out.append({"kind": "cluster_unencrypted", "detail": "no secrets keyArn"})
return out
out.append({"kind": "cluster_cmk", "detail": arn})
if enabled_at is None:
out.append({"kind": "enabled_at_unknown", "detail": "cannot date Secrets"})
return out
if enabled_at.tzinfo is None:
enabled_at = enabled_at.replace(tzinfo=timezone.utc)
for row in rows:
created = parse_ts(row["created"])
if created is None:
continue
if created.tzinfo is None:
created = created.replace(tzinfo=timezone.utc)
if created < enabled_at:
out.append(
{
"kind": "secret_older_than_encryption",
"detail": f"{row['namespace']}/{row['name']} {row['created']}",
}
)
return out
Two flags, and only two.
**cluster_unencrypted.** encryptionConfig has no secrets + keyArn. The cluster has no customer CMK association. New Secrets are the secret that is still plaintext, in the Q08-default sense, unless you are on the 1.28 platform wrap (which this function does not claim to see).
**secret_older_than_encryption.** The association exists, you supplied enabled_at, and this Secret's creationTimestamp is earlier. It is a candidate for the secret that is still plaintext. It is not a proof. A rewrite via kubectl replace or the AWS annotate path updates the object and does not move creationTimestamp. A Secret created in 2025 and replaced in March 2026 will still flag. That is the honest limit. The checker can say "this object is older than the association." It cannot say "etcd still has the 2025 bytes." Q08's etcdctl is the proof. You do not have it.
enabled_at_unknown is the third line you print when the operator did not pass a time. Do not guess. Do not use the cluster createdAt as a proxy (the cluster is older than the association). Do not use "now."
CloudTrail is one honest source for enabled_at. Look up AssociateEncryptionConfig on the cluster, take the event time, and pass that instant in. The other honest source is the createdAt on the describe-update you saved when the associate returned Successful. Both are operator inputs. Neither lives on describe_cluster. A checker that calls cloudtrail.lookup_events itself is a third read, and it is allowed if you want the tool to fetch the instant. It is still not a store view. If CloudTrail is off, the field stays unknown.
def enabled_at_from_update(eks, name: str, update_id: str) -> datetime | None:
upd = eks.describe_update(name=name, updateId=update_id)["update"]
if upd.get("type") != "AssociateEncryptionConfig":
return None
raw = upd.get("createdAt")
if raw is None:
return None
if isinstance(raw, datetime):
return raw if raw.tzinfo else raw.replace(tzinfo=timezone.utc)
return parse_ts(str(raw))
That function dates the association. It does not date the rewrite. A rewrite is a Kubernetes write. The update object does not list Secrets.
def main() -> int:
config.load_kube_config()
core = client.CoreV1Api()
eks = boto3.client("eks", region_name="us-east-1")
enabled_at = parse_ts("2026-03-01T14:22:00+00:00")
report = findings(
encryption_config(eks, "atlas-prod"),
secret_meta(core),
enabled_at,
)
for item in report:
print(f"{item['kind']}\t{item['detail']}")
kinds = {item["kind"] for item in report}
if "cluster_unencrypted" in kinds or "secret_older_than_encryption" in kinds:
return 2
return 0
Return 2 is "go look." Return 0 is "no candidate." Return 0 is not "etcd is clean." Put that sentence in the README the first time a colleague treats the exit code as a compliance stamp.
The checker never writes. A helpful change that runs the Q08 replace from Python is a different tool, and it is a write against every Secret in the cluster. Keep it out of this file.
§IV — Failure mode: the checker that cannot see etcd
**Reading data and printing "plaintext."** Every Secret will match. The API decoded them. You have built a decoder of base64, which Poulton already did with echo | base64 -d (Ch. 11, p. 150). Delete the field from the row type.
**Treating creationTimestamp as etcd state.** A rewrite does not re-create the object. The flag stays red after a successful replace. The honest report says "older than association; confirm rewrite by resourceVersion change, annotation, or etcdctl on a cluster that has it." Closing the flag on timestamp alone is how you file a lie. Closing the ticket because the flag is gone after a delete-and-recreate is a different lie: you destroyed the object to please a linter.
If you do rewrite, record a write-side signal at that moment: the resourceVersion before and after, or the kms-encryption-timestamp annotation Ops names. The next run of this checker still cannot see etcd, but it can stop flagging an object whose annotation you set. That is a policy on your annotation, not a store proof. Do not invent the annotation from creationTimestamp.
**Inventing enabled_at from describe_cluster.** The field is not there. Using cluster create time marks every Secret as old on a cluster that enabled encryption on day one, or marks nothing useful on a cluster that enabled it last week. CloudTrail or the saved describe-update createdAt is the input. No input, no dating.
Binding, mutating, or asking SSAR "to be complete." 08-16, 08-13, and 08-04 already spent those verbs. A checker that creates a dummy Secret "to see if the write encrypts" is a write in kube-system the moment someone points KUBECONFIG at prod. Do not create. Do not bind. Do not watch.
Talking to etcd from Python. etcd3 against the EKS-managed store will fail at the network boundary, and on kubeadm it is the exam's etcdctl, not your client. If the task is the hexdump prefix, you are in the Cert lesson. Stay out of /registry/secrets.
**Trusting an empty encryptionConfig on 1.28 as "no encryption."** Report "no customer CMK." Do not report "cluster stores plaintext" unless you have a store view. Ops already split those two sentences.
§V — Pairing
Ops is the association and the rewrite. This file is the two reads that can see the association and cannot see the rewrite. Cert is Q08 by hand: enc.yaml, provider order, identity: {} as the read fallback, etcdctl for the prefix, kubectl replace for the rewrite. Bootcamp tags Q08 as Minimize Microservice Vulnerabilities (20%). The leftover CKS domains from 08-13 were Cluster Hardening and System Hardening. Next CKS day 2026-08-22 can take Q04.
08-16 remains the fit-checker that does not bind. 08-13 remains kopf. 08-04 remains SSAR. 08-10 remains watch. None of them open etcd. Q27 and Q40 are the decode-and-mount drills. They prove the API and the volume return plaintext. They are not a store audit. Leave them in Cert.
§VI — Drills
list_secret_for_all_namespaces returns items whose data maps decode with base64. The cluster has a CMK on encryptionConfig. What does the decode prove, and what must the checker not print?data. It may print the CMK ARN and, if enabled_at is known, names older than that instant.kubectl replace on 2026-03-02, the day after association. creationTimestamp is still 2025-06-03. What does secret_older_than_encryption mean here?create_namespaced_secret "to test whether new writes encrypt." Why is that out of scope?describe_cluster and Secret metadata. A create in the wrong namespace is a live Secret you did not mean to leave. Proof of the write path is Q08's replace plus etcdctl, or the Ops annotate, not a Python create.list_secret_for_all_namespaces returns 100 items and a _continue token. The first page has no Secret older than enabled_at. What does the checker print, and what must it do next?_continue until it is empty. A first page that looks clean is a page, not a census. The secret that is still plaintext may be item 101.Related
- fit-checker, no bind (2026-08-16)
- kopf RuntimeClass (2026-08-13)
- SSAR preflight (2026-08-04)
- Cross-References/domains/Polyglot-Dev
- Poulton Ch.11 pp.148-150 referenced
🫡 ⚖️ 📜 Leo.Syri — Praetor Consulate, Imperium Luminaura Filed 2026-08-19 · Fajr · K8s day 28 · tenth visit · trio #94