Supply chain security

Enforcing Image Policy

1.Where a check can run

A policy covers only the requests and fields it sees. Webhooks and VAP syntax : Admission Control.

  • CI checks manifests but only advises : anyone can kubectl apply around it.
  • Admission runs once per API write, before any pull.
  • The runtime checks every real pull, never cached images.

2.The gap between admit and pull

  • Admission verifies app:1.0 at digest A. The tag moves to B. A later pull gets B, never verified.
  • Fix : pin digests in Git, or let the verifier append one. All three keep the tag : podinfo:6.9.2@sha256:cab90e...
CI : kyverno apply, policy-tester advice only kubectl / GitOps mutating : tag -> tag@sha256 validating : VAP, verifiers API server, once per write etcd, then ReplicaSet controller Pod CREATE : admission again Pod-only rules bite here kubelet : image cached ? yes no no pull no node check runtime pull CRI-O policy.json / verifiers CRI-O re-checks at create if a policy path is set containerd : transfer service pulls only

3.Every field that holds an image

A Pod-only rule admits the Deployment, then denies its Pods as ReplicaSet FailedCreate, as in Pod Security. kubectl debug --copy-to makes a new Pod, so pods rules see it.

FieldReached byTrap
containers, initContainersPod CREATEExamples often check containers only
ephemeralContainerskubectl debug : UPDATEpods, * miss it : list pods/ephemeralcontainers
volumes[].image.referencePod CREATESkipped by every admission verifier by default
spec.template.specDeployment, StatefulSet, DaemonSet, JobUnchecked unless listed
spec.jobTemplate.spec.template.specCronJobOne more path for your CEL

4.What built-in CEL can do

variables:  # Pod form
- name: images
  expression: >-
    object.spec.?containers.orValue([]).map(c, c.image) +
    object.spec.?initContainers.orValue([]).map(c, c.image) +
    object.spec.?ephemeralContainers.orValue([]).map(c, c.image) +
    object.spec.?volumes.orValue([]).filter(v, has(v.image))
      .map(v, v.image.?reference.orValue(''))
validations:
- expression: variables.images.all(i, i.matches('@sha256:[a-f0-9]{64}$'))
  reason: Forbidden  # 403, not 422
  • Can : startsWith, matches, all / exists.
  • Cannot : parse a reference, normalize nginx, call a registry, resolve a tag.
  • End prefixes in / : blocks registry.example.com.evil.io.
  • Forbid :latest, even next to a digest.

5.Built-ins outside a webhook

  • ImagePolicyWebhook (off by default) : Pods and debug containers only. defaultAllow: false fails closed.
  • Static image VAP (manifest-based, beta v1.37) : no paramKind, so a hard-coded allow-list. Not deletable through the API. A broken file keeps the old one.
  • AlwaysPullImages : see Images, Registries and Pulls.

6.Commands

CommandShows
kubectl debug -it <pod> --image=<ref>@sha256:<d> --target=<c>A debug container : a separate request on the subresource
kubectl describe rs <rs>Late FailedCreate from a Pod-only rule
kubectl get validatingadmissionpolicy <name> -o jsonpath='{.status.typeChecking}'CEL type errors
kubectl get pod <p> -o jsonpath='{.spec.containers[*].image}'Whether digest pinning happened
In short : Place each check, cover every image field, and keep CEL to string hygiene (no signatures).

1.Kyverno ImageValidatingPolicy

policies.kyverno.io/v1, GA since 1.17 (v1 storage planned for 1.20). The Namespaced kind acts in its own namespace.

spec fieldWhat it does
matchConstraintsWebhook rules, copied verbatim : list pods/ephemeralcontainers yourself
matchImageReferencesA scope on the raw string : unmatched images are skipped, not denied
imagesExtra extractors (CEL). None built in for image volumes
attestors, attestationscosign (key, keyless, trustedRoot) or notary. Claims by intoto.type or referrer.type
validationsCEL : verifyImageSignatures, verifyAttestationSignatures (count > 0)
validationConfigurationsmutateDigest, verifyDigest, required : all default true

2.One Pod through Kyverno

  • Autogen covers controllers only when resources is exactly ["pods"]. Adding pods/ephemeralcontainers turns it off : use two policies.
API server ivpol.mutate mutateDigest ivpol.validate (validating webhook) extract built-ins + images match others skipped verifyDigest no digest : deny verify cache 60m required all kept images verified Deny / Warn / Audit Audit PolicyReport reports ctrl 1h, no cache API server ivpol.mutate mutateDigest ivpol.validate extract built-ins + images match others skipped verifyDigest no digest : deny verify cache 60m required all verified Deny / Warn / Audit Audit PolicyReport reports ctrl 1h, no cache

3.An IVPOL that pins a release workflow

matchImageReferences: [{glob: "ghcr.io/kyverno/*"}]
attestors:
- name: github
  cosign:
    keyless:
      identities:
      - issuer: https://token.actions.githubusercontent.com
        subjectRegExp: '^https://github\.com/kyverno/kyverno/\.github/workflows/.+@refs/tags/v.+$'
    source: {repository: ghcr.io/kyverno/signatures}
validations:
- expression: (images.containers + images.initContainers)
    .all(i, verifyImageSignatures(i, [attestors.github]) > 0)
  • Anchor subjectRegExp. Kyverno keeps its signatures in another repo.
  • cosign v3 bundles are auto-detected. cosign attestors check only in-toto.
  • trustedRoot sits next to keyless, not inside it (docs nest it).
  • IVPOL never reads the Pod's imagePullSecrets. spec.credentials (Kyverno namespace Secrets) replaces the helpers, no merge.

4.Kyverno version traps

  • 1.19.0 made all three real : upgrades can deny.
  • 1.19.1 fixes GHSA-5cjf-wwfg-pj4c : a one-image exception exempted the whole resource.
  • verifyImages (ClusterPolicy) : deprecated 1.19, removed in 1.20 (late 2026).
  • 1.19 is tested only on Kubernetes 1.33 to 1.35.

5.Sigstore policy-controller

  • Opt-in namespaces only. no-match-policy defaults to deny, so does a typo.
  • CIP : every matching one must pass, one authority in each is enough. ctlog = Rekor.
  • v0.15.1 cannot verify cosign v3 bundle image signatures : bundle is attestations only.
  • Chart 0.10.8 runs v0.13.1. No cache.

6.Commands

CommandShows
kubectl get ivpol <name> -o jsonpath='{.status.autogen}'Which controllers got variants
kubectl get ivpol <name> -o jsonpath='{.spec.validationConfigurations}'Stored defaults (all three true)
kubectl get policyreports -n <ns>Audit results
kubectl label namespace payments policy.sigstore.dev/include=trueOpts a namespace in
kubectl patch configmap/config-policy-controller -n cosign-system --type merge -p '{"data":{"no-match-policy":"warn"}}'Warn on unmatched images
In short : IVPOL verifies and pins matches. policy-controller cannot verify cosign v3 bundles.

1.Ratify behind Gatekeeper

  • External data : a Gatekeeper Provider, called from Rego with external_data. Responses cached 3m.
  • Ratify v1.4.6 is the stable line (config.ratify.deislabs.io/v1beta1).
  • v2.0.0-beta.2 (Executor) says "do not use in production" and renamed its API between betas.
  • The Gatekeeper chart fails open (Ignore, 3s). The sample constraint covers only Pods in default.
API server Gatekeeper mutating Assign : tag -> digest Gatekeeper validating RatifyVerification (Rego) external_data Ratify store -> notation, cosign -> threshold allow / deny timeout admitted Ignore (chart)

2.Checks on the node

  • CRI-O : policy.json per Pod namespace (/etc/crio/policies/<ns>.json) with sigstoreSigned.
  • containerd 2.x : bindir verifiers (exit 0 allows), only on transfer-service pulls, the CRI default since 2.1.
  • Legacy registry.mirrors / configs / auths or discard_unpacked_layers = true silently fall back to local pulls with no verifiers (kind's default did). Check : journalctl -u containerd | grep -i 'Falling back to local image pull'
  • ctr image pull --local skips verifiers too : test with crictl pull.
CRI-Ocontainerd
Keyless identityExact email SAN only : no GitHub workflow URIsWhatever your binary does
cosign v3 bundlesNo (container-libs issue 388)Your binary
Cached imagesRe-checked at create when a policy path is setNever checked
Pod showsSignatureValidationFailedErrImagePull, then ImagePullBackOff
Warn modeNone : test on a canary poolNone

3.Failure and caching

  • Fail : a registry or Rekor outage blocks admission.
  • Kyverno with Ignore still denied on registry errors (observed once). [Audit] still mutates.
  • Caches : Kyverno 60m, Gatekeeper 3m, policy-ctrl none.
  • Private Sigstore : Kyverno trustedRoot / tuf, policy-controller TrustRoot, Ratify verifier parameters.

4.Digest mutation and GitOps drift

  • Kyverno (autogen), policy-controller and Ratify rewrite templates to tag@sha256 : Argo CD and Flux see drift.
  • Pin digests in Git, so there is nothing to rewrite : the digest strategy of Argo CD Image Updater, or a Flux ImagePolicy with digestReflectionPolicy (Flux).
  • Or hide the diff : ignoreDifferences in Argo CD.

5.Check in CI too, as advice

  • kyverno apply ivpol.yaml --resource rendered.yaml runs only the validating path : tag-only images fail verifyDigest. A policy file without .yaml is skipped with exit 0.
  • policy-tester -policy cip.yaml -image REF checks one image against one CIP.

6.Which engine does what

VAPKyverno 1.19policy-ctrl 0.15.1Ratify + GKCRI-Ocontainerd
Signaturesnocosign, notarycosign .sig onlynotation, cosignsigstoreyour binary
Tag to digestnomutateDigestalwaysAssignn/an/a
kubectl debugif listedif listeddigest onlysent by defaultat pullat pull
Image volumesif CEL readsextractornonoyesyes
Pod templatesyour rulesautogenyesmutate onlyn/an/a
Unmatchedyour CELskippeddenyby scopesdefaultallowed
Trial mode[Warn, Audit][Audit] or [Warn]mode, no-match : warndryrun, then warnnonenone
On failureFailFailFailIgnorepull failspull fails
In short : Pin digests, cover every field, trial in warn mode, and back admission with node checks.