Workload identity

SPIRE

1.Why SPIRE ?

Pods talk to each other, but how does Pod A prove it is Pod A ? IPs change, so an IP is a bad identity.

  • Static secrets (API keys, long-lived certs) get leaked, copied and never rotated
  • Service Account tokens prove identity only to the k8s API, not across clusters or clouds
  • SPIRE checks facts (node, namespace, SA, image) then issues a short-lived SVID
  • Result : zero-trust mTLS with no secret to store

SPIFFE = the standard. SPIRE = the software. CNCF Graduated (TOC vote 2022-08-23, announced 2022-09-20).

Static secret Long-lived key leaks, copied,never rotated With SPIRE Workload SVID (short) auto-rotated, used for mTLS SVID lifetime (SPIRE defaults) X.509 SVID : 1h, agent renews at half-life JWT SVID : 5 min, minted per audience hardened Helm chart : X.509 4h, JWT 1h

2.Components

The hardened chart's recommended layout puts the server in spire-server and the agents in spire-system. One server, one agent per node.

  • Node attestor proves the node (options in 3.)
  • Workload attestor inspects the caller (k8s via kubelet, unix via uid/gid)
  • SVID = X.509 cert (mTLS) or JWT (bearer calls)
  • Trust domain = the security boundary, its trust bundle holds the CA certs
Pieces
  • SPIRE ServerSigning authority. Holds entries, issues SVIDs. Usually a StatefulSet.
  • SPIRE AgentDaemonSet. Attests to the server, then attests local workloads.
  • Workload APIUnix socket where a workload asks "who am I ?".
  • controller-managerTurns CRDs into registration entries.
  • SPIFFE CSI driverMounts the agent socket into Pods.

3.Attestation and SVID Flow

Two steps : the agent proves the node, then identifies the process.

Server Agent Pod 1 node attest 2 give me SVID kubelet 3 which pod ? 4 request SVID 5 signed SVID 6 SVID + bundle dashed = done ahead of time, cached 7 renewed at half-life, before expiry
Node attestorProves withCatch
k8s_psatProjected SA tokenDefault on k8s
aws_iidEC2 identity docAWS only
gcp_iitGCE identity tokenGCP only
azure_msiManaged identityAzure only
join_tokenOne-time tokenManual, tests
x509popExisting node certNeeds your PKI

one per agent, the server must run the same plugin

In short : SPIRE attests the node, then the workload, and hands out short-lived SVIDs for mTLS. No static secret to leak.

1.Registration Entries

Entry = SPIFFE ID + parent ID + selectors. The parent ID points at the agent allowed to hand it out.

ID looks like spiffe://trust-domain/path

Kind (cluster scope)Use
ClusterSPIFFEIDWhich pods get which ID
ClusterFederatedTrustDomainFederate with another trust domain
ClusterStaticEntryHand-written entry for non-pod things

all spire.spiffe.io/v1alpha1

Server Agent (node entry) webk8s:ns:defaultk8s:sa:web dbk8s:ns:datak8s:sa:db web + db parent ID = the agent's ID spiffe://td/spire/agent/k8s_psat/<cluster>/<node>

2.k8s Automation

spire-controller-manager is a reconcile loop, like a Deployment controller, but the output is SPIRE entries.

Controller manager flow
ClusterSPIFFEIDYou apply the CRD.
controllerWatches Pods and the CRD.
Server APIEntries created, updated, deleted.
Pod deletedIts entry is removed.
kind: ClusterSPIFFEID
metadata: {name: web}
spec:
  className: spire-mgmt-spire
  spiffeIDTemplate: "spiffe://{{ .TrustDomain }}\
    /ns/{{ .PodMeta.Namespace }}\
    /sa/{{ .PodSpec.ServiceAccountName }}"
  podSelector:
    matchLabels: {app: web}
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: default
Reading it apiVersion: spire.spiffe.io/v1alpha1. Pods labelled app: web in default get an ID built from their Namespace and ServiceAccount. className names the chart's controller manager (release namespace, then release): without it, the CR is ignored. A workload fetches it over the Workload API socket, no credentials needed.
Resulting IDServiceAccount web in default gets
spiffe://<trust-domain>/ns/default/sa/web
no Pod IP
In short : Entries map selectors to SPIFFE IDs under a parent agent. ClusterSPIFFEID CRDs let the controller manager write them for you.

1.Install and Inspect

Two Helm charts from the hardened repo, CRDs first, then server + agent + controller manager, both released into spire-mgmt.

spire-crdsCRDs chart, release in spire-mgmt
spireServer, agent, controller manager
get podsIn spire-server and spire-system
entry showEntries exist
CommandDescription
helm upgrade --install --create-namespace -n spire-mgmt spire-crds spire-crds --repo https://spiffe.github.io/helm-charts-hardened/Install the CRDs first (release namespace: spire-mgmt)
helm upgrade --install -n spire-mgmt spire spire --repo https://spiffe.github.io/helm-charts-hardened/ -f values.yamlServer, agent, controller manager. The values set the trust domain, the cluster name and global.spire.recommendations.enabled
kubectl get pods -n spire-serverServer and controller manager (agents and CSI driver: -n spire-system)
kubectl exec -n spire-server spire-server-0 -c spire-server -- spire-server entry showList registration entries
spire-server entry create -spiffeID <id> -parentID <id> -selector <sel>Create an entry by hand, e.g. -selector k8s:ns:default
spire-server entry delete -entryID <id>Delete an entry
kubectl get clusterspiffeidList ClusterSPIFFEID objects
kubectl logs -n spire-system -l app.kubernetes.io/name=agentAgent logs when attestation fails

2.No SVID ? Debug Order

agent logsAttest errors ?
entry showEntry exists ?
selectorsMatch ns / SA ?
parent IDNode's agent ?
first "no"
= the bug

3.Gotchas

Setup
  • Install the CRDs before the main chart (separate spire-crds chart)
  • Server needs persistent storage (SQLite by default) or entries and CA state are lost, prod usually wants external SQL
  • Charts are beta and the controller API is v1alpha1, use released versions not git main
  • Recheck versions before quoting (latest seen : v1.15.5, chart spire 0.30.4)
Runtime
  • A typo in a selector like k8s:ns means silently no SVID
  • Parent ID must match the agent that really runs the workload
  • SVIDs are short-lived, apps must reload certs via the Workload API or a library
  • Not a secret store, that is Vault or External Secrets
In short : Install spire-crds, then spire. Check entries with entry show, and when no SVID shows up, look at selectors, parent IDs and the agent logs.