Hedronite · Cert Lesson · CKA Cluster Architecture · Sun 2026-10-18

Kustomize for the CKA bases, overlays, and the ConfigMap hash that triggers a rollout

Edit a ConfigMap in place and nothing restarts. Change it through a generator and the new name rolls the Pods.

Lesson Class: Cert (CKA · Cluster Architecture 25% · Workloads 15%)

Tested with: kubectl v1.37.1 · Kustomize v5.8.1 · render only

Lab: lab/ base + staging + prod overlays · run.sh

Source: Kubernetes docs · Kustomize reference · CNCF CKA curriculum v1.37

One Base, Many Overlays

The base never knows who extends it.

The Hash Is the Restart

New content, new name, new Pod template.

Point -k at a Directory

Render first; apply what you read.

The ConfigMap said debug. The Pods had never heard of it.

The problem

A web application reads its logging verbosity from an environment variable named LOG_LEVEL. The value lives in a ConfigMap, a Kubernetes object that stores non-secret configuration as key-value pairs. A Deployment, the object that manages a set of identical Pods, pulls every key from that ConfigMap with envFrom. A Pod is the smallest runnable unit in Kubernetes: one or more containers that run together.

During an incident, an admin edits the ConfigMap in place and sets LOG_LEVEL to debug. Nothing changes, and the Pods keep logging at the old level.

Two documented facts explain this. The Kubernetes docs state that "ConfigMaps consumed as environment variables are not updated automatically and require a pod restart", because a container's environment is fixed when the container starts. The Deployment docs also say a rollout, meaning the gradual replacement of old Pods with new ones, is triggered "if and only if the Deployment's Pod template (that is, .spec.template) is changed". Editing a separate ConfigMap never touches that template. The cluster ends up with a correct ConfigMap, stale Pods, and nothing that prompts a restart.

Meanwhile the team keeps one copy of the YAML for staging and another for prod. YAML is the indentation-based text format Kubernetes resources are written in. Someone bumps an image tag in one copy and forgets the other, and the two slowly drift apart until nobody can say which file matches the cluster.

The idea, in plain words

Kustomize is a standalone tool that customizes Kubernetes objects through a kustomization file, and since version 1.14 kubectl can use it directly. You keep one shared directory called a base, then add a small overlay directory per environment that names the base and states only what differs. Kustomize merges them and prints finished YAML. For configuration it generates the ConfigMap itself, appends a hash of the content to the name, and rewrites every reference to match. A changed value therefore gives a new name, a new name changes the Deployment's Pod template, and a changed template replaces the Pods. The Cloud Native Computing Foundation (CNCF) Certified Kubernetes Administrator (CKA) curriculum v1.37 tests this under "Use Helm and Kustomize to install cluster components" (Cluster Architecture, Installation and Configuration, 25%) and, for the ConfigMap side, "Use ConfigMaps and Secrets to configure applications" (Workloads and Scheduling, 15%). Helm, the other tool in that first objective, is out of scope here.

Kustomization files, bases and overlays

A kustomization is a directory that contains a file named kustomization.yaml. That file lists the resources to include and the changes to apply to them. A base is a directory with a kustomization.yaml holding a set of resources and their customization. An overlay is a directory with a kustomization.yaml that refers to other kustomization directories as its bases. A base has no knowledge of any overlay, so one base can serve many overlays. That one-way dependency is what stops drift: shared content exists once, and an overlay can only add differences on top of it. Modern files list the base under resources:.

Here is the lab's base kustomization.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml
configMapGenerator:
- name: web-config
  literals:
  - LOG_LEVEL=info
  - FEATURE_SEARCH=off

The resources list names two ordinary files, a Deployment and a Service. The configMapGenerator entry tells Kustomize to build a ConfigMap called web-config from two literal key=value pairs. No ConfigMap file exists in the directory, so the generator is the only place the data is defined. That keeps a single source of truth for the settings, which is the property the later sections depend on.

The base Deployment, in base/deployment.yaml, is a normal one:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: web
        image: nginx:1.27
        envFrom:
        - configMapRef:
            name: web-config

The Deployment asks for one replica of an nginx:1.27 container. Its envFrom block with a configMapRef imports every key of web-config as an environment variable. The name written there is the generator's name, not a final name. The Kubernetes docs say to reference a generated ConfigMap by the name of its generator, and Kustomize then replaces that name with the generated one. The base also holds service.yaml, a Service named web that selects Pods labeled app: web and forwards port 80 to target port 80.

Now the staging overlay:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: staging
namePrefix: staging-
resources:
- ../../base
configMapGenerator:
- name: web-config
  behavior: merge
  literals:
  - LOG_LEVEL=debug

The namespace and namePrefix lines are cross-cutting fields, which means Kustomize applies them to every resource in the build. The docs list namespace, namePrefix, nameSuffix, labels and commonAnnotations in this group. The resources entry points at the base directory. The generator uses behavior: merge to update one value in the base's ConfigMap.

You can see the result without a cluster. kubectl kustomize overlays/staging renders the final YAML to your terminal, and kubectl apply -k overlays/staging sends the same result to the cluster. The render was:

$ kubectl kustomize overlays/staging
apiVersion: v1
data:
  FEATURE_SEARCH: "off"
  LOG_LEVEL: debug
kind: ConfigMap
metadata:
  name: staging-web-config-tc964fmdf5
  namespace: staging
---
apiVersion: v1
kind: Service
metadata:
  name: staging-web
  namespace: staging
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: staging-web
  namespace: staging
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - envFrom:
        - configMapRef:
            name: staging-web-config-tc964fmdf5
        image: nginx:1.27
        name: web

All three resources gained namespace: staging, and the Deployment and Service names gained the staging- prefix. The ConfigMap kept FEATURE_SEARCH: "off" from the base and took LOG_LEVEL: debug from the overlay. The Service selector is still app: web, so it matches the unchanged Pod labels. Rendering first is a good habit because you read exactly what the cluster will receive.

Symptom: kubectl apply -k overlays/staging/kustomization.yaml does not work. Cause: the -k flag must point to a kustomization directory, not to a file inside it. Fix: pass the directory, as in kubectl apply -k overlays/staging.

configMapGenerator and the content-hash suffix

A generator builds a ConfigMap from literals or files rather than from a hand-written resource. Kustomize appends a content hash to the generated name. The docs explain why: this ensures a new ConfigMap is generated when the contents change. Kustomize also rewrites every reference to the generator's name. Without both behaviors, you would have to rename the ConfigMap and edit the Deployment by hand on every change. Here is the base render:

$ kubectl kustomize base
apiVersion: v1
data:
  FEATURE_SEARCH: "off"
  LOG_LEVEL: info
kind: ConfigMap
metadata:
  name: web-config-8h4d7b7td6
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - envFrom:
        - configMapRef:
            name: web-config-8h4d7b7td6
        image: nginx:1.27
        name: web

The ConfigMap is named web-config-8h4d7b7td6, and the Deployment's configMapRef inside the Pod template carries that same string. You wrote web-config in the source, and Kustomize substituted the hashed name. Because the reference sits inside .spec.template, the hashed name becomes part of the Pod template itself.

Now change one value in the base, LOG_LEVEL=info to warn, render the prod overlay before and after, and diff the two files:

### change one value in the base: LOG_LEVEL=info -> warn
$ diff prod-before.yaml prod-after.yaml
4c4
<   LOG_LEVEL: info
---
>   LOG_LEVEL: warn
7c7
<   name: prod-web-config-7tb4c84c84
---
>   name: prod-web-config-h7hgb6tg66
40c40
<             name: prod-web-config-7tb4c84c84
---
>             name: prod-web-config-h7hgb6tg66

Three lines changed. The data line changed as expected, the ConfigMap's name changed from suffix 7tb4c84c84 to h7hgb6tg66, and the Deployment's configMapRef changed in step with it. That third change alters .spec.template, so applying it starts a rollout, and the new Pods read the new ConfigMap at container start. Otherwise the environment variables would stay frozen, as the opening problem showed. Because the change produces a new ConfigMap instead of editing the old one, Pods from the old template keep the values they started with while the replacements come up, so no Pod ever sees a half-edited configuration.

Volume mounts behave differently. Per the docs, projected keys from a ConfigMap volume are eventually updated, but a container using a ConfigMap as a subPath mount never receives updates. The hash rename therefore matters most for environment variables and subPath mounts.

You can turn the suffix off with generatorOptions and disableNameSuffixHash: true. Here is the same one-value change with it on:

### same change with disableNameSuffixHash: true
$ diff nohash-before.yaml nohash-after.yaml
4c4
<   LOG_LEVEL: info
---
>   LOG_LEVEL: warn

Only the data line differs. The name is stable, so the Deployment renders identically, .spec.template is unchanged, and no rollout happens. Applying this reproduces the opening problem: a ConfigMap that says warn and Pods still running with info. If you disable the suffix, something else has to restart the Pods.

Generators in overlays need one more rule. The Kustomize reference says an overlay generator does not cause suffixing of an existing ConfigMap's name. To get suffixing, put the generator in the base, and the overlay generator will correctly update the suffix based on the new content. In this lab the base holds the generator, so the merged staging ConfigMap received a fresh suffix, staging-web-config-tc964fmdf5, different from the base's web-config-8h4d7b7td6. The reference states the result directly: changes to the contents change the suffix, which creates a new ConfigMap that Kustomize points Workloads to.

Symptom: a Deployment still points at an old ConfigMap after the data changed. Cause: the Deployment names a suffixed ConfigMap by hand. In the lab, a hand-written web-config-abc123 was left untouched while the generated name was web-config-8h4d7b7td6:

7:  name: web-config-8h4d7b7td6
37:            name: web-config-abc123

Fix: write the generator's name, web-config, in the Deployment and let Kustomize substitute the hashed name.

Patches, images and merge generators

Overlays change things in three main ways. A patch edits part of a resource, and Kustomize supports StrategicMerge and Json6902 patches through the patches field. A strategic merge patch is a partial resource that Kubernetes-aware logic merges into the original. The images field changes a container image without any patch. A generator with behavior: merge adds or updates values in a ConfigMap that comes from the base. The docs recommend small patches that each do one thing, so an overlay stays short enough to review.

The prod overlay uses all three:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: prod
namePrefix: prod-
resources:
- ../../base
patches:
- path: replicas.yaml
images:
- name: nginx
  newTag: "1.27.3"
configMapGenerator:
- name: web-config
  behavior: merge
  literals:
  - FEATURE_SEARCH=on

The patches entry loads replicas.yaml, shown next. The images entry finds containers using the image nginx and sets the tag to 1.27.3. The generator overrides FEATURE_SEARCH and leaves LOG_LEVEL alone. Cross-cutting fields set the namespace to prod and the prefix to prod-.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3

Kustomize matches the target of a patch using the group, version, kind and name in the patch file. Here that is apps/v1, Deployment, web, so the patch names the base Deployment, not the prefixed one. The render shows where it landed:

$ kubectl kustomize overlays/prod
apiVersion: v1
data:
  FEATURE_SEARCH: "on"
  LOG_LEVEL: info
kind: ConfigMap
metadata:
  name: prod-web-config-7tb4c84c84
  namespace: prod
---
apiVersion: v1
kind: Service
metadata:
  name: prod-web
  namespace: prod
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prod-web
  namespace: prod
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - envFrom:
        - configMapRef:
            name: prod-web-config-7tb4c84c84
        image: nginx:1.27.3
        name: web

The Deployment is prod-web with replicas: 3, the image is nginx:1.27.3, and the ConfigMap holds FEATURE_SEARCH: "on" next to the inherited LOG_LEVEL: info. Notice that the three renders produced three different ConfigMap names, web-config-8h4d7b7td6, staging-web-config-tc964fmdf5 and prod-web-config-7tb4c84c84, because the content differs in each environment. The two kinds of overlay change also behave differently once applied: per the Deployment docs, a later edit that only raises replicas scales without a rollout, while a new image tag changes .spec.template and replaces the Pods.

Symptom: a generator in an overlay fails with a name conflict, or a merge or replace does not hit the intended ConfigMap. Cause: behavior defaults to create, which reports a name conflict if the ConfigMap exists. With merge or replace, the name and namespace must match the target exactly, because configMapGenerator runs before the namespace transformer. Fix: set behavior: merge (or replace) and use the base's generator name, web-config, as the prod overlay does.

Putting it together

An exam-style task might read: "Configure prod so the app runs three replicas with a new setting, and apply it." Work in this order:

Render with kubectl kustomize overlays/prod and read the output for the names, namespace and image you expect.
Preview the change against the cluster with kubectl diff -k overlays/prod.
Apply with kubectl apply -k overlays/prod.
Verify with kubectl get -k overlays/prod or kubectl describe -k overlays/prod.
Remove everything the kustomization created with kubectl delete -k overlays/prod.

Each of these commands takes a directory, so a path to a single file will not work. Rendering first also costs nothing, because a render needs no cluster and changes nothing.

Tested with: kubectl v1.37.1 (Kustomize v5.8.1), render only. This lab rendered the YAML and compared the output. It applied nothing to a cluster, so the rollout behavior described above follows the Kubernetes documentation and the rendered Pod templates rather than an observed Pod restart.

Check yourself

**An admin changes LOG_LEVEL in a plain ConfigMap that a Deployment consumes through envFrom. Why do the running Pods keep the old value?** Environment variables from a ConfigMap are not updated automatically and need a Pod restart. The Deployment's .spec.template did not change, so no rollout replaced the Pods.

After a value changes in the base, which two places in the prod render change besides the data line? The ConfigMap's metadata.name gets a new hash suffix, and the Deployment's configMapRef name changes to match. The second change alters the Pod template, so a rollout starts.

**What happens to the Deployment if disableNameSuffixHash: true is set and a value changes?** The ConfigMap keeps its name and the Deployment renders identically. The Pod template does not change, so no rollout occurs and running Pods keep the old environment.

**kubectl apply -k overlays/prod/kustomization.yaml does not work. What is wrong, and what do you type instead?** -k must point to a kustomization directory, not a file. Use kubectl apply -k overlays/prod.

Sources

Kubernetes docs, "Declarative Management of Kubernetes Objects Using Kustomize": https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/
Kubernetes docs, ConfigMaps: https://kubernetes.io/docs/concepts/configuration/configmap/
Kubernetes docs, Deployments: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
Kustomize reference, configMapGenerator: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/configmapgenerator/
CNCF Certified Kubernetes Administrator (CKA) Curriculum v1.37 (CKA_Curriculum_v1.37.pdf): https://github.com/cncf/curriculum