Supply chain security

Image Signing: Sigstore and Notation

1.What a signature proves

  • Binds an identity to one digest, never a tag.
  • Both tools resolve a tag and warn : use @sha256:.
  • Sigstore : cosign, an OIDC identity, a public log.
  • Notary Project : Notation, your own X.509 PKI.
Digest, not tag

2.cosign with a key

  • cosign generate-key-pair : encrypted key file, or a KMS.
  • Sign and verify IMAGE@sha256:... with --key.
  • Key mode still logs to public Rekor by default.
  • Private : signing config without rekorTlogUrls, verify with --insecure-ignore-tlog.
  • v3.0.6+ : --key plus identity flags is an error.
Key URIHolds
cosign.keyfile, plus cosign.pub
awskms:///alias/xAWS KMS, by alias
gcpkms://projects/P/...Google Cloud KMS
hashivault://, openbao://Vault transit (OpenBao v3.0.6)
k8s://ns/namea Secret : key, password, pub

3.Keyless, step by step

  • Fulcio : OIDC token to a 10-minute cert. Identity in the SAN, issuer in OID 1.3.6.1.4.1.57264.1.8.
  • Signed : a DSSE in-toto Statement, predicate type https://sigstore.dev/cosign/sign/v1.
CI job or a laptop TUF CDN tuf-repo-cdn.sigstore.dev 1 OIDC token (aud=sigstore) trusted_root.json, signing config cosign Fulcio CA, checks the token 2 token + public key 3 10-min cert + SCT precert CT log embeds the SCT cosign 4 signs DSSE(in-toto Statement{subject: digest}) 7 push 5 timestamp 6 log entry registry 7 bundle v0.3, a referrer TSA RFC 3161 timestamp Rekor v1 SET + inclusion proof

Bundle v0.3 = leaf cert + tlog entry + timestamp + DSSE envelope. The identity stays in public logs for good.

4.Reading the identity

  • A reusable workflow puts its own path in the SAN. cosign pins the caller only by deprecated OID .5 (--certificate-github-workflow-repository).
  • The caller sits in OIDs .12 (source repo) and .18 (build config) : that needs a policy engine.
Signer--certificate-oidc-issuerSAN (--certificate-identity)
GitHub Actionshttps://token.actions.githubusercontent.comhttps://github.com/ORG/REPO/.github/workflows/F.yml@refs/tags/T
GitLab CIhttps://gitlab.comhttps://gitlab.com/PROJECT//.gitlab-ci.yml@REF
Google SAhttps://accounts.google.comkrel-trust@k8s-releng-prod.iam.gserviceaccount.com
Kubernetes SAyour cluster issuerhttps://kubernetes.io/namespaces/NS/serviceaccounts/SA (RBAC)
SPIFFEyour configured issuerspiffe://... from sub (SPIFFE, SPIRE)

5.Trust roots, Rekor v1 and v2

  • Trust : TUF ships trusted_root.json. Its timestamp.json lives a week, so air-gapped verifiers pass --trusted-root.
  • Public default : Rekor v1, for the foreseeable future. v2 is opt-in (signing-config create --with-default-rekor-v2).
  • Rekor v2 has no SET, so a TSA is mandatory. Shards rotate : never hardcode log2025-1.
  • No revocation : short certs plus log monitoring. Private instance : cosign trusted-root create.
In short : Sign the digest, not the tag. Keyless signing turns a CI identity into a 10-minute certificate and a public log entry, so the identity you pin when verifying is the whole security story.

1.Where signatures live

  • cosign v3 : a bundle as an OCI 1.1 referrer, type ...sigstore.bundle.v0.3+json.
  • Legacy cosign (v2 default) : tag sha256-<hex>.sig, no subject, so the Referrers API never lists it.
  • .sig tag and fallback index sha256-<hex> differ. The index is a racy read-modify-write.
  • --registry-referrers-mode=oci-1-1 is not the v3 switch : experimental, legacy-format writes only.
FormatTagsubjectListedWritten by
Legacy cosignsha256-<hex>.signonov2 default; v3 with 2 legacy flags
cosign v3 bundlenoneyesyesv3 default; v2.6 with a flag
Notationnoneyesyesnotation
Fallback indexsha256-<hex>n/an/aboth, without the API; notation v1.x always

2.Verifying with cosign

  • Keyless verify requires an identity and an issuer flag (or their -regexp forms).
  • The regexps are unanchored : write ^...$. A .* accepted a stranger's bundle in the lab : any Sigstore user passes.
  • One identity per run. Always verify IMAGE@sha256:....
Anchored GitHub Actions verify
$ cosign verify ghcr.io/ORG/REPO@sha256:<d> \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity-regexp '^https://github\.com/ORG/REPO/\.github/workflows/release\.yml@refs/tags/v[0-9.]+$'

3.What "verified" means

  • cosign runs eight checks, in the order on the right.
  • Key mode swaps cert, SCT and identity for the key, and needs no timestamp.
  • The predicate type is not checked. An SBOM attestation from the pinned identity passes cosign verify (see SBOMs, Provenance and Attestations).
  • --insecure-ignore-tlog still prints "verified offline". Never scrape stderr.
1 trusted root (TUF or file) 2 tlog inclusion proof 3 signing time (SET or TSA) 4 cert chains to Fulcio 5 SCT from the CT log 6 signature over payload 7 identity + issuer 8 subject digest = image not checked : predicateType --insecure-ignore-tlog skips key mode : no timestamp --insecure-ignore-sct skips key mode : 4, 5, 7 usethe key instead

4.Old and new verifiers

  • Old verifiers say Error: no signatures found (exit 10) on a v3 signature.
  • If any bundle exists, v3 ignores legacy .sig signatures.
  • Floor : cosign v3.1.3+ or v2.6.5+. GHSA-fx35-mq7g-6g98 has no CVE ID, so CVE scanners miss it.
  • CI lags : cosign-installer v4.1.2 defaults to v3.0.6, Alpine 3.20 ships 2.2.1. Pin it and print cosign version.
Signed with< v2.6.0v2.6.0 to v2.6.2v2.6.3 to v2.6.5v3.x
Legacy .sigyesyesyesyes, if no bundles
v3 bundle referrernoonly with --new-bundle-formatyes (auto-detect)yes
Both on one image.sig only.sig onlybundles onlybundles only

5.Commands

CommandWhat it shows
cosign sign --yes IMAGE@sha256:<d>Keyless sign, bundle pushed as a referrer
cosign sign --key cosign.key IMAGE@sha256:<d>
cosign verify --key cosign.pub IMAGE@sha256:<d>
Key mode, sign and verify
cosign tree IMAGE.sig and .att tags, plus referrers
cosign bundle inspect b.sigstore.jsonCert extensions, SCTs, tlog, timestamps (no crypto checks)
cosign versionClient version : check it in every CI image
In short : cosign v3 stores a bundle as a referrer of the digest, old verifiers cannot see it, and a verify is only as strong as an anchored identity plus issuer.

1.Notation : your own PKI

  • Keys via plugins : azure-kv v1.2.1, AWS Signer.
  • notation sign : JWS (default) or COSE envelope, optional RFC 3161 countersignature.
  • v1.3.2 writes a referrer with config.mediaType ...notary.signature (no artifactType) and the sha256-<hex> index.
signer verifier key in a KMS plugin + CA chain notation sign target descriptor JWS / COSE + optional TSA registry referrer subject = digest notation verify trustpolicy.json registryScopes trust store roots integrity, authenticity, expiry, timestamp, revocation (OCSP, CRL)

2.trustpolicy.json, trust stores

  • Stores hold root certs only, in ~/.config/notation/truststore/x509/.
  • Policy pick : exact repo, then one "*", else fail.
  • Identities need C, ST, O. No serverAuth EKU, unlike cert-manager TLS certs.
  • notation policy import trustpolicy.json, then notation verify -v by digest.
One strict statement
{"version": "1.0", "trustPolicies": [{
  "name": "app",
  "registryScopes": ["example.com/app"],
  "signatureVerification": {"level": "strict"},
  "trustStores": ["ca:acme", "tsa:my-tsa"],
  "trustedIdentities": [
    "x509.subject: C=US, ST=WA, O=acme.io"]
}]}

3.Verification levels

  • audit passes a wrong signer with exit 0.
  • skip with "*" : v1.3.2 accepts it, the spec forbids it.
  • No TSA : a signature dies with its leaf cert.
  • Unknown revocation fails strict (OCSP 2s, CRL 5s).

E enforce (fails), L log and go on, - skipped.

Checkstrictpermis.auditskip
IntegrityEEE-
AuthenticityEEL-
TimestampELL-
ExpiryELL-
RevocationELL-

4.cosign or Notation ?

Notation v1.3.2 (2025-04-27) is current (v2 : alpha). Flux checks both on OCI sources. Cluster : Image Policy.

Aspectcosign v3.1Notation v1.3
Root of trustTUF trusted_root.json or --trusted-rootRoot certs in trust stores
Identity checkSAN + issuer (OID .8), or a raw keytrustedIdentities x509 subject, or a plugin identity
Signing timeRekor v1 SET, or RFC 3161 (must with Rekor v2)Optional RFC 3161 countersignature
TransparencyRekor + CT log by defaultNone built in
RevocationNone : short certs, log monitoringOCSP first, CRL fallback

5.Copy, then verify again

  • oras cp -r (preview) misses legacy .sig. cosign copy (deprecated v3.0.5) misses v3 bundles.
  • regctl image copy --referrers --digest-tags SRC DST carries both. Keep the digest unchanged.
  • Re-verify at the destination. Notation also needs the new repository in registryScopes.

6.Worked example : Kubernetes v1.37.1

  • Images : krel-trust@k8s-releng-prod.iam.gserviceaccount.com, issuer Google, legacy .sig.
  • That .sig holds krel-staging and krel-trust signatures : either passes. Pin the digest (Deployments).
  • Binaries : krel-staging, with flags deprecated and due to go in v4. No .sigstore.json yet.
$ cosign verify registry.k8s.io/kubectl@sha256:b7cab618... --certificate-identity krel-trust@... --certificate-oidc-issuer https://accounts.google.com
$ for f in kubectl kubectl.sig kubectl.cert; do curl -sSfLO "https://dl.k8s.io/release/v1.37.1/bin/linux/amd64/$f"; done
$ cosign verify-blob kubectl --signature kubectl.sig --certificate kubectl.cert --certificate-identity krel-staging@... --certificate-oidc-issuer https://accounts.google.com
In short : Notation trades the public log for your own CA and a trust policy. Either way, copy signatures with the image, verify again, and check Kubernetes against the krel identities by digest.