CKA: CustomResourceDefinitions and operators from install order to schema validation, pruning and deletion

Install the type, wait for it, read its schema, and remember that deleting it deletes everything of that kind.

The problem

The Certified Kubernetes Administrator (CKA) exam, run by the Cloud Native Computing Foundation (CNCF), lists "Understand CRDs, install and configure operators" under the domain "Cluster Architecture, Installation and Configuration". That domain carries 25% of the v1.37 curriculum. The competency names a CustomResourceDefinition (CRD), the object that teaches the Kubernetes API server a new type of object. Instances of that type are called custom resources.

The failures it covers are easy to meet. An administrator installs an operator from a vendor's YAML (YAML Ain't Markup Language) file and applies a custom resource before the definition exists. The lab did exactly that, and kubectl v1.37.1 printed one of two messages, depending on what it remembered. On a machine where kubectl had never talked to this server, it said:

error: resource mapping not found for name: "nightly" namespace: "" from "cr/nightly.yaml": no matches for kind "Backup" in version "ops.example.com/v1"
ensure CRDs are installed first

On the recorded run, kubectl had cached the server's list of types from an earlier run, so it sent the request and the server answered:

Error from server (NotFound): error when creating "cr/nightly.yaml": the server could not find the requested resource (post backups.ops.example.com)

Both mean the type does not exist yet. Later a field silently disappears from a stored object. A cleanup then deletes a CRD, and every backup object goes with it.

The exam expects you to read a CRD, create and inspect custom resources, and know what the API server checks. The lab used a throwaway local kube-apiserver v1.37.1 with etcd (the key-value store where the API server keeps its objects), no nodes and no controller. Nothing reconciled the objects.

The idea, in plain words

An API group is a named family of related resource types, such as ops.example.com here. A resource is one endpoint in that API, such as backups, and a kind is the type name written inside each object, such as Backup. A CRD teaches the API server a new resource: a name, a group, a scope (namespaced or cluster-wide), one or more versions and a schema. Once the CRD reports the Established condition, the API server stores, validates, defaults and prunes objects of that type like built-in ones. A controller is a program that runs a control loop, watching objects and acting to make reality match them. An operator is a custom resource plus a controller that acts on it. Installing one means installing the CRDs first and the controller second, because the controller has nothing to watch until the types exist.

Anatomy of a CRD and the install order

The lab's definition describes a Backup type. It is shown in full, because the exam may ask you to read one quickly.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: backups.ops.example.com
spec:
  group: ops.example.com
  scope: Namespaced
  names:
    plural: backups
    singular: backup
    kind: Backup
    shortNames: [bk]
  versions:
  - name: v1
    served: true
    storage: true
    subresources:
      status: {}
    additionalPrinterColumns:
    - name: Database
      type: string
      jsonPath: .spec.database
    - name: Schedule
      type: string
      jsonPath: .spec.schedule
    - name: Retention
      type: integer
      jsonPath: .spec.retentionDays
    - name: Last-Run
      type: string
      jsonPath: .status.lastRun
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            required: [database, schedule]
            x-kubernetes-validations:
            - rule: "self.keepLast <= self.retentionDays"
              message: "keepLast cannot be larger than retentionDays"
            properties:
              database:
                type: string
                minLength: 1
              schedule:
                type: string
                pattern: '^(@(hourly|daily|weekly)|(\S+ ){4}\S+)$'
              retentionDays:
                type: integer
                description: Days to keep each backup.
                minimum: 1
                maximum: 90
                default: 7
              keepLast:
                type: integer
                minimum: 1
                default: 3
          status:
            type: object
            properties:
              lastRun:
                type: string

Start at the top. The metadata.name must match the spec fields and take the form <plural>.<group>, so here backups plus ops.example.com. The scope: Namespaced line means each Backup lives inside a namespace. The book's example CRD, books.nigelpoulton.com, used Cluster and annotated the field "Can be "Namespaced" or "Cluster"". Under names, plural is the resource name used on the command line and in the API, and kind is the type name inside objects. The shortNames: [bk] entry is a shorthand that kubectl accepts, matching the book's "Short name used by kubectl". Each entry under versions has served (the API answers requests for it) and storage (objects are saved in etcd in that version). The reason storage: true appears once is the rule that "One and only one version must be marked as the storage version." The subresources and additionalPrinterColumns blocks give a separate status endpoint and extra columns for kubectl get. The schema block holds the validation rules, which the next section covers.

Step 2 installed the file and waited for the Established condition, which signals that the API server has started serving the new endpoint:

customresourcedefinition.apiextensions.k8s.io/backups.ops.example.com created
customresourcedefinition.apiextensions.k8s.io/backups.ops.example.com condition met

Reading the conditions back showed two lines, and kubectl api-resources listed the type:

NamesAccepted=True
Established=True

NAME      SHORTNAMES   APIVERSION           NAMESPACED   KIND
backups   bk           ops.example.com/v1   true         Backup

The CRD task page in the Kubernetes docs warns that "It might take a few seconds for the endpoint to be created." The lab saw this. Immediately after Established, kubectl explain backup.spec.retentionDays failed:

error: couldn't find resource for "ops.example.com/v1, Resource=backups"

A few seconds later the same command worked:

GROUP:      ops.example.com
KIND:       Backup
VERSION:    v1

FIELD: retentionDays <integer>


DESCRIPTION:
    Days to keep each backup.

The second run shows that explain reads the schema the CRD supplied, including the description text. The first run shows that "Established" and "every client can see the type" are not the same instant. Wait for the condition, then retry explain once if it complains.

Symptom: applying a CRD fails with metadata.name: Invalid value: "backup.ops.example.com": must be spec.names.plural+"."+spec.group.

Cause: the name used the singular, backup, instead of the plural, backups.

Fix: name the object <plural>.<group>, here backups.ops.example.com.

The schema: structure, defaults and validation rules

An OpenAPI v3 schema is a machine-readable description of which fields an object may have and what types and limits they carry. In apiextensions.k8s.io/v1 the schema must be structural. The CRD task page defines it as "an OpenAPI v3.0 validation schema which: specifies a non-empty type (via type in OpenAPI) for the root, for each specified field of an object node (via properties or additionalProperties in OpenAPI) and for each item in an array node". The requirement exists so that the server can reliably default and prune fields, because it can only do that when every field has a declared type. The lab's non-structural.yaml left type off the spec property, and the server refused it:

The CustomResourceDefinition "restores.ops.example.com" is invalid: spec.validation.openAPIV3Schema.properties[spec].type: Required value: must not be empty for specified object fields

Defaulting fills in missing fields from the schema's default values. The same page says defaulting happens "on the object in the request to the API server using the request version defaults, when reading from etcd using the storage version defaults", and that "Defaults applied when reading data from etcd are not automatically written back". The file nightly.yaml set only database and schedule. Reading it back showed the defaults filled in:

{"database":"notes","keepLast":3,"retentionDays":7,"schedule":"0 3 * * *"}

NAME      DATABASE   SCHEDULE    RETENTION   LAST-RUN
nightly   notes      0 3 * * *   7           

The table is the printer columns at work. Each column pulls one value by its JSON path, which saves an -o yaml call when scanning many objects.

Validation then rejects bad values. The file bad-values.yaml broke two keyword rules at once, and both were reported in one response, in the format the docs show for spec.replicas in body should be less than or equal to 10:

The Backup "bad-values" is invalid: 
* spec.retentionDays: Invalid value: 365: spec.retentionDays in body should be less than or equal to 90
* spec.schedule: Invalid value: "every night": spec.schedule in body should match '^(@(hourly|daily|weekly)|(\S+ ){4}\S+)$'

The third check is a rule written in CEL, the Common Expression Language. The docs explain that "Validation rules use the Common Expression Language (CEL) to validate custom resource values. Validation rules are included in CustomResourceDefinition schemas using the x-kubernetes-validations extension." Our rule self.keepLast <= self.retentionDays sits on the spec object, so self means that object. The file keep-too-many.yaml set retentionDays: 2 and keepLast: 5:

The Backup "keep-too-many" is invalid: spec: Invalid value: keepLast cannot be larger than retentionDays

That message is the message field from the CRD. Rules only see the object being written, since "All validation rules are scoped to the current object: no cross-object or stateful validation rules are supported." Because this test set both fields explicitly, the lab did not test a rule against a defaulted value. It showed only that an explicit violation is rejected.

Symptom: an exam task says a custom resource must not be created when one field exceeds another, and a pattern or maximum cannot express it.

Cause: OpenAPI keywords check one field at a time against fixed limits.

Fix: add an x-kubernetes-validations rule with a message on the parent object, comparing the two fields through self.

Unknown fields: rejected, warned or pruned

Pruning is the removal of fields that the schema does not declare. The CRD task page states that an "unknown field is pruned (removed) before being persisted." That is the point where a typo becomes a lost setting. The client adds a first line of defense, because "kubectl also checks for unknown fields and rejects those objects well before they would be sent to the API server." The --validate flag on kubectl apply controls that check: strict is the default, warn and ignore are the alternatives, and true and false are accepted aliases for strict and ignore, per the kubectl apply reference. The file unknown-field.yaml adds compress: true, which the schema never declared. With the default, the lab got:

$ kubectl apply -f cr/unknown-field.yaml
Error from server (BadRequest): error when creating "cr/unknown-field.yaml": Backup in version "v1" cannot be handled as a Backup: strict decoding error: unknown field "spec.compress"
[exit 1]

With --validate=warn, the reference says kubectl "will warn about unknown or duplicate fields without blocking the request":

$ kubectl apply --validate=warn -f cr/unknown-field.yaml
Warning: unknown field "spec.compress"
backup.ops.example.com/weekly created
[exit 0]

The stored spec had no compress:

{"database":"notes","keepLast":3,"retentionDays":30,"schedule":"@weekly"}

After deleting weekly, the lab applied again with --validate=false, which the reference describes as performing no schema validation and "silently dropping any unknown or duplicate fields". The object was created with no message at all, and the stored spec was the same line as above, again without compress.

So in both successful cases the field was dropped, and the only difference was whether a warning appeared. A misspelled field name therefore loses that setting, and only strict validation tells you before the object is stored.

Symptom: a setting "does nothing", and kubectl get -o yaml does not show the field you applied.

Cause: the field is not in the schema, often a typo or a field from a newer CRD version, and it was pruned because validation was warn or ignore.

Fix: apply with the default strict validation, check the field with kubectl explain, and upgrade the CRD before using new fields.

Status, deletion and what an operator adds

A subresource is a separate API endpoint attached to a resource, so it can be written independently. The CRD above enabled status: {}. The CRD task page says "PUT / POST / PATCH requests to the custom resource ignore changes to the status stanza." The lab tested this by applying with-status.yaml, which carries a status.lastRun value. The object was created, but reading .status back printed an empty line, so the server discarded the supplied status. Then the lab patched the status subresource directly:

$ kubectl patch backup hourly --subresource=status --type=merge -p {"status":{"lastRun":"2026-10-27T03:00:00Z"}}
backup.ops.example.com/hourly patched

NAME      DATABASE   SCHEDULE    RETENTION   LAST-RUN
hourly    notes      @hourly     7           2026-10-27T03:00:00Z
nightly   notes      0 3 * * *   7           
weekly    notes      @weekly     30          

The split exists so that access can be fine-grained: the Custom Resources page describes it as "user writes the spec section and the controller writes the status section." Users declare what they want, and the controller reports what happened, without either overwriting the other. In the lab no controller existed, so status changed only because the lab patched it by hand.

Deletion is the sharpest edge. Step 8 listed three Backup objects (hourly, nightly and weekly), then deleted the CRD:

customresourcedefinition.apiextensions.k8s.io "backups.ops.example.com" deleted

Error from server (NotFound): Unable to list "ops.example.com/v1, Resource=backups": the server could not find the requested resource (get backups.ops.example.com)

The CRD task page explains why: "When you delete a CustomResourceDefinition, the server will uninstall the RESTful API endpoint and delete all custom objects stored in it."

An operator adds the acting part. The Operator pattern page says "Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components. Operators follow Kubernetes principles, notably the control loop." It also says "The most common way to deploy an operator is to add the Custom Resource Definition and its associated Controller to your cluster. The Controller will normally run outside of the control plane, much as you would run any containerized application." The Custom Resources page puts the other half plainly: "On their own, custom resources let you store and retrieve structured data." The lab's Backup objects were exactly that: stored data, with nothing acting on them.

Symptom: after uninstalling an operator, all of its custom resources are gone.

Cause: the uninstall deleted the CRD, which deletes every object of that type.

Fix: back up the objects first with kubectl get <plural> -A -o yaml. To stop reconciliation only, remove the controller and leave the CRD. That advice follows from the docs above, because the controller runs separately from the definition. The lab did not test an operator uninstall.

Putting it together

A checklist for the exam:

• Install the CRDs first, then the controller, because the controller needs the types.
• Wait for the Established condition before creating custom resources, and retry explain once if it lags.
• Read the schema with kubectl explain rather than guessing field names.
• Keep validation strict, so unknown fields are rejected instead of pruned.
• Write status through the status subresource, since ordinary writes ignore it.
• Remember that deleting a CRD deletes every object of that type.

The lab verified kube-apiserver and kubectl v1.37.1 with etcd 3.6.15, one namespace, no nodes and no controller. It did not test a running operator, Helm installs, conversion between versions, validation ratcheting, webhooks or cluster-scoped CRDs. Treat any claim about those as outside what this lesson demonstrated.

Check yourself

Sources

• CNCF Certified Kubernetes Administrator (CKA) Curriculum v1.37
• Kubernetes docs: Extend the Kubernetes API with CustomResourceDefinitions
• Kubernetes docs: Custom Resources
• Kubernetes docs: Operator pattern
• Kubernetes docs: kubectl apply reference
• Nigel Poulton, The Kubernetes Book, chapter 14 (The Kubernetes API)