API machinery

Admission Control

1.What is admission control ?

The API server's last gate before etcd. Admission plugins can change a write, then reject it.

  • Runs in kube-apiserver after authn and authz. Writes only : create, update, delete, connect.
  • Mutating plugins may change the object. Validating plugins only say yes or no.
  • One rejection, in either phase, fails the request.
  • Only new writes are checked. Existing objects are left alone.
Client kubectl Authentication fail = 401 Authorization fail = 403 Admission, writes only Mutating, then validating (see 2.) may change it, then yes or no saved only if nothing rejected etcd get / list / watch skip it

Any reject in either phase ends the request with an error.

2.Inside admission, in order (v1.37)

  • Built-in mutators first : ServiceAccount, DefaultStorageClass, LimitRanger ...
  • Then each bound MutatingAdmissionPolicy (CEL).
  • Then mutating webhooks, serial, any order.
  • Object changed ? Built-ins run again, and reinvocationPolicy: IfNeeded hooks may too.
  • Schema check, built-in validators (PodSecurity ...), ValidatingAdmissionPolicies, validating webhooks (parallel), ResourceQuota last. One deny is enough.
Mutating phase Built-in mutators e.g. ServiceAccount Mutating policies CEL, in-process Mutating webhooks serial, one at a time changed ? loop again Object validation the final object Validating phase Validating policies after built-in validators Validating webhooks all in parallel Response any deny = rejected Webhook order is not guaranteed, so every mutator must be idempotent.

3.Built-in plugins

Compiled into kube-apiserver, 25 on by default in v1.37. Change the set with --enable-admission-plugins / --disable-admission-plugins (order does not matter).

PluginTypeWhat it does
NamespaceLifecycleMRejects objects in missing or terminating namespaces. default, kube-system, kube-public cannot be deleted
LimitRangerM + VApplies LimitRange defaults, enforces the namespace min / max
ResourceQuotaVRejects a write that would push the namespace past its quota
ServiceAccountM + VDefaults the SA to default, rejects a missing SA, adds the API token volume
PodSecurityVPod Security Standards from namespace labels (stable v1.25), see Pod Security
DefaultStorageClassMNew PVC with no class gets the default one. Create only, newest default wins

M = mutating, V = validating.

4.Picking a mechanism

MechanismRuns in, logic inGood for
ValidatingAdmissionPolicykube-apiserver, CEL (GA v1.30)Deny / warn / audit rules, no server to run
MutatingAdmissionPolicykube-apiserver, CEL (GA v1.36)Simple defaults : labels, sidecars, PSA labels
Validating webhookyour HTTPS server, any languageComplex rules, external data. Called in parallel
Mutating webhookyour HTTPS server, any languageComplex mutation, external APIs. Called serially
Kyverno (v1.19)webhooks, YAML + CELPolicy library, generate / cleanup, image checks
OPA Gatekeeperwebhooks, RegoConstraintTemplate + Constraint, audits old objects
Rule of thumb : a CEL policy whenever it can express the rule, a webhook for complex logic or outside data. For CRDs, use their built-in validation rules and defaulting, not a webhook (see CRDs).
In short : Admission is the last gate before etcd : mutating plugins may change a write, validating plugins may reject it, and your own rules go in CEL policies or webhooks.

1.Webhooks : how a call works

  • Register in a Mutating or ValidatingWebhookConfiguration (v1).
  • The API server POSTs an AdmissionReview. The reply echoes request.uid and sets allowed.
  • Changes return as a base64 JSONPatch. warnings[] reach the client as Warning headers (code 299).
  • clientConfig : a Service (port 443, path / by default) or an https URL.
  • caBundle : CA for the serving cert, valid for <svc>.<ns>.svc. Filled by cert-manager's cainjector, see cert-manager.

2.Webhook settings

FieldValuesDefaultNotes
failurePolicyFail, IgnoreFailErrors and timeouts only. allowed: false always denies
timeoutSeconds1 to 3010On timeout, failurePolicy decides
sideEffectsNone, NoneOnDryRunrequiredLegacy v1beta1 Some / Unknown fail dry-run
matchPolicyExact, EquivalentEquivalentAlso catches other group / versions of the resource
namespaceSelectorlabel selectorallLeave out kube-system and the webhook's own namespace
objectSelectorlabel selectorallOpt-in hooks only : whoever writes the labels can skip it
matchConditionsup to 64 CELnoneCalled only if all are true (GA v1.30)
reinvocationPolicyNever, IfNeededNeverMutating only. IfNeeded needs an idempotent hook

Matching : rules (operations, apiGroups, apiVersions, resources, scope), then the two selectors and matchConditions. Since v1.37 (beta, on by default), TokenReview, SubjectAccessReview and other virtual auth resources skip webhooks.

3.ValidatingAdmissionPolicy : CEL, no server

# demo : at most 5 replicas per Deployment
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: max-replicas.example.com
spec:
  matchConstraints:
    resourceRules:
    - apiGroups:   ["apps"]
      apiVersions: ["v1"]
      operations:  ["CREATE", "UPDATE"]
      resources:   ["deployments"]
  validations:
  - expression: "object.spec.replicas <= 5"
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: max-replicas-demo.example.com
spec:
  policyName: max-replicas.example.com
  validationActions: [Deny]  # [Warn, Audit] to trial
  matchResources:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: demo
  • GA since v1.30. Does nothing until a binding names it in policyName. failurePolicy defaults to Fail.
  • validationActions : Deny, Warn, Audit. Never Deny and Warn together.
  • paramKind + paramRef pass a ConfigMap or any CR to CEL as params. parameterNotFoundAction (Allow / Deny) is required.
  • CEL sees object, oldObject (null on CREATE), request, namespaceObject, variables, authorizer. Type errors : status.typeChecking.
MutatingAdmissionPolicy (GA v1.36, on by default) : CEL emits an ApplyConfiguration (merged like server-side apply) or JSONPatch ops. reinvocationPolicy is required. Binds the same way.

Manifest-based admission (beta v1.37) : staticManifestsDir loads webhooks and policies from disk, enforced from API server start, even with etcd down. Names end in .static.k8s.io.

4.When a webhook goes down

Node fails replicas ran there Webhook Pods gone ReplicaSet controller new Pod kube-apiserver Webhook Svc no endpoints error or 10s timeout Pod CREATE rejected failurePolicy: Fail never comes back Configs never go to webhooks : the delete works.
Avoid the deadlock
  • namespaceSelector : skip kube-system, own ns.
  • 2+ replicas behind the Service.
  • Mutating hooks fail open, a validator checks.
  • Small timeoutSeconds : mutating hooks are serial.
  • Break glass : kubectl delete the Validating / MutatingWebhookConfiguration
In short : A webhook puts your HTTPS server on the write path, so scope it and keep it up. A CEL policy runs inside the API server and acts only once a binding points at it.