Certificates and secrets

External Secrets Operator

1.What is ESO ?

ESO reads secrets from a vault or cloud secret manager (AWS, GCP, Azure, Vault...) and keeps a normal Kubernetes Secret in sync with them.

Git holds only pointers to secrets, never the values. Pods stay unchanged, they still mount a normal Secret.

Latest : v2.12.0 (CRDs use external-secrets.io/v1). CNCF Sandbox project.

In Simple Words
ESO fetches secrets from outside the cluster and turns them into Kubernetes Secrets, then keeps them fresh.

Vault -> ESO -> Secret -> Pod

2.Why it exists ?

  • Secrets in K8s are only base64. Committing them to Git leaks credentials.
  • The real source of truth lives in AWS, GCP, Azure or Vault. Copying values by hand goes stale.
  • Rotating a secret means someone must update every cluster.

Fix : declare WHERE the secret lives in Git, and a controller fetches and refreshes it.

Key Points
  • Declarative, no values in Git
  • Re-fetches every refreshInterval
  • Many backends, one API
  • Output is a plain Secret
  • Templating reshapes values
  • PushSecret goes the other way
  • Generators create values
  • Works with Flux/Argo (just YAML)

3.Parts of ESO

  • ControllerMain operator. Reconciles ExternalSecret and PushSecret, calls providers, writes Secrets.
  • WebhookValidates the ESO CRDs (own deployment).
  • Cert-controllerManages TLS certs for the webhook (own deployment).
  • GeneratorsMake values: Password, UUID, SSHKey, ECR/GCR/ACR tokens and more.
Install Order
  1. Install ESO with Helm
  2. Create a store with auth
  3. Create an ExternalSecret
  4. Controller writes the Secret
  5. Pod mounts the Secret
  6. Refresh loop keeps it current

4.How the Sync Works

watch uses ExternalSecret(Git / kubectl) ESO controller SecretStore Provider(AWS SM, Vault) auth + fetch values K8s Secret create / update plain Secret,Pod unchanged Pod env / mount ExternalSecret(Git / kubectl) watch ESO controller SecretStore fetch values writes Provider(AWS SM, Vault) K8s Secret env / mount Pod plain Secret,Pod unchanged
Reconcile Loop
  1. Watch ExternalSecret
  2. Resolve store
  3. Fetch remote data
  4. Apply template
  5. Write Secret
  6. Wait refreshInterval, repeat from 3

Ready=True on success, SecretSyncedError on failure.

In short : ESO pulls secrets from an external provider and keeps a normal Kubernetes Secret fresh, so Git never holds a value.

1.ESO Kinds + Store Scoping

A SecretStore only works in its own namespace. A ClusterSecretStore is shared, so restrict it with conditions. Tip : one SecretStore per namespace, or one shared ClusterSecretStore.

KindScopePurpose
SecretStoreNamespacedProvider config and auth
ClusterSecretStoreClusterShared provider config
ExternalSecretNamespacedFetch remote data into a Secret
ClusterExternalSecretClusterExternalSecrets in many namespaces
PushSecretNamespacedPush Secret to provider
ClusterPushSecretClusterPushSecret across namespaces
ClusterSecretStore shared ExternalSecretNamespace A ExternalSecretNamespace B SecretStore own store,only in A

2.Example ExternalSecret + Gotchas

kind: ExternalSecret
metadata: {name: db-creds}
spec:
  refreshInterval: 1h
  secretStoreRef: {name: vault, kind: SecretStore}
  target: {name: db-creds, creationPolicy: Owner}
  data:
  - secretKey: password
    remoteRef: {key: prod/db, property: password}
Gotchas
  • Store must be Ready first
  • Env vars need a reloader, files refresh
  • Prefer workload identity (IRSA)
  • Low refreshInterval hits rate limits
  • Owner policy : delete ES, Secret goes too
  • Migrate v1beta1 / v1alpha1 to v1

3.ESO vs Plain Secret vs CSI Driver

Source of truthCreates K8s SecretRotationPod changesHow it works
Plain SecretIn clusterManualManualNoneStatic
ESOExternal providerYes, automaticEvery refreshIntervalNoneController + CRDs
Secrets Store CSIExternal providerOptional (sync)Mount rotationCSI volume in PodNode driver

4.ESO Commands

CommandDescriptionExample
helm install external-secrets external-secrets/external-secrets -n external-secrets --create-namespaceInstall operator + CRDs--set installCRDs=false--set installCRDs=false
kubectl get externalsecret -AList, with sync statuskubectl get es -n appkubectl get es -n app
kubectl describe externalsecret <name>Conditions + eventskubectl describe es db-credskubectl describe es db-creds
kubectl get secretstore,clustersecretstore -ACheck stores are Readykubectl get csskubectl get css
kubectl -n external-secrets logs deploy/external-secretsController logsadd -f to followadd -f to follow
kubectl annotate es <name> force-sync=$(date +%s) --overwriteForce immediate refreshes = externalsecretes = externalsecret
In short : Pick a namespaced SecretStore or a shared ClusterSecretStore, point an ExternalSecret at it, and check stores are Ready first. Unlike the CSI driver, Pods need no changes.