Extending the API
Custom Resources (CRDs)
1.What is a CRD ?
A CustomResourceDefinition teaches kube-apiserver a new kind, with its schema, validation, defaults and versions. The API server serves and stores the objects itself.
- The CRD is cluster-scoped (
apiextensions.). Its metadata.name must bek8s. io/ v1 <plural>.<group>. - spec.scope (Namespaced or Cluster) is for the custom objects, not the CRD.
- A CRD only stores data. A controller acts on it (Operators).
Use the kind once the CRD is Established.
CRD or aggregated API ?CRD : kube-apiserver serves it and stores it in etcd, with no server to run and validation from the schema and CEL. Aggregated API : an APIService routes /apis/<group>/<version> to your own server (like metrics.k8s.io from metrics-server), which picks its own storage, verbs and subresources. Start with a CRD.
2.A CRD in YAML
apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: crontabs.stable.example.com # plural.group spec: group: stable.example.com scope: Namespaced # of the CronTabs names: kind: CronTab plural: crontabs # URL segment shortNames: ["ct"] categories: ["all"] # kubectl get all
# then, an object of the new kind
apiVersion: stable.example.com/v1
kind: CronTab
metadata: {name: nightly}
spec: {cronSpec: "0 2 * * *", image: "backup:1.4"}
versions:
- name: v1
served: true
storage: true # exactly one
subresources:
status: {}
scale:
specReplicasPath: .spec.replicas
statusReplicasPath: .status.replicas
additionalPrinterColumns:
- name: Image
type: string
jsonPath: .spec.image
priority: 1 # only in -o wide
schema:
openAPIV3Schema: {...} # structural
conversion:
strategy: None # the default
3.Field by field
| Field | Rule / default | Purpose |
|---|---|---|
| names.singular, names.listKind | default lowercased kind, <Kind>List | CLI alias, kind of list responses |
| names.shortNames, names.categories | optional, lowercase | kubectl get ct, kubectl get all |
| versions[].served, storage | required, exactly one storage: true | serve over REST, version written to etcd |
| versions[].schema.openAPIV3Schema | structural | validation, pruning, defaulting |
| versions[].subresources | status: {} and/or scale | /status for controllers, /scale for kubectl scale and HPA (labelSelectorPath) |
| versions[].additionalPrinterColumns | none = NAME, AGE | kubectl get columns |
| versions[].selectableFields | max 8 | --field-selector on custom fields |
| versions[].deprecated | default false | warning header on requests |
| spec.conversion.strategy | None (default) or Webhook | how objects convert between versions |
| status.storedVersions | API server adds, migration trims | every version ever stored |
In short : A CRD adds a kind (names, scope, versions, schema) that kube-apiserver serves.
1.Structural schema and pruning
- Structural (a must in v1) : a
typeon every node, no type or default inside allOf, anyOf, oneOf, not. - Pruning drops unknown fields (API warns, kubectl rejects).
x-kubernetes-preserve-unknown-fieldsopts a subtree out. - default: on request, on read and after a webhook patch.
nullable: truekeeps a null. x-kubernetes-list-type(atomic by default, set, map) andx-kubernetes-list-map-keystell server-side apply how to merge a list.
Dashed : admission. On read : storage-version defaults, then conversion.
2.CEL validation rules
image: type: string maxLength: 256 # bounds the CEL cost x-kubernetes-validations: - rule: "self == oldSelf" # transition message: "image is immutable" replicas: {type: integer, default: 1}
A rule also takes messageExpression, reason, fieldPath, optionalOldSelf.
x-kubernetes-validationsholds CEL rules (Stable 1.29).selfis the node the rule sits on.- Uses
oldSelf? It is a transition rule, run only on updates where old and new both exist. optionalOldSelf: true(Stable 1.33) runs it on create too.- Ratcheting (Stable 1.33, on) passes an update if the invalid part did not change.
3.Versions and conversion
- Exactly one version has
storage: true. Each has its own schema, subresources and columns. - None (default) only rewrites apiVersion. A renamed or split field needs a Webhook.
- status.storedVersions lists every version ever stored. Drop a version only once it has left the list.
- v1.37 : StorageVersionMigration is GA and on by default. It rewrites every object at the storage version.
Changing storage: true never rewrites old objects. Real example : Gateway API CRDs (channels, v1beta1 to v1, storedVersions).
4.Commands
| Command | Does |
|---|---|
kubectl wait --for=condition=Established crd/<name> | Block until the new endpoint is served |
kubectl api-resources --api-group=stable.example.com | Kinds, shortNames and scope the CRD added |
kubectl explain crontabs.spec --api-version=stable.example.com/v1 | Field docs from the published OpenAPI v3 schema |
kubectl apply --dry-run=server -f my-crontab.yaml | Full server check (schema, CEL, admission), nothing saved |
kubectl get ct -o wide | Printer columns, plus those with priority > 0 |
kubectl get crd <name> -o jsonpath='{.status.storedVersions}' | Versions that may still have objects in etcd |
Gotchas
- Deleting a CRD deletes every object of that kind.
- With
statuson, a normal update ignores .status. Use--subresource=status. - A failing conversion webhook can block reads, updates, even deletes.
Good to know
- Most RBAC roles miss new kinds. Add Roles.
- *.k8s.io and *.kubernetes.io groups need the
api-approved.annotation.kubernetes. io - CEL over budget rejects the CRD. Bound it with maxItems, maxProperties, maxLength.
In short : A structural schema lets the API server prune, default and check every write with CEL. Versions share one storage version, and a StorageVersionMigration moves old objects to it.