GitOps and delivery
Argo CD
1.What is Argo CD ?
A GitOps continuous delivery tool for Kubernetes. The desired state lives in Git (or a Helm or OCI repo). A controller keeps comparing it with the live cluster, and can sync the cluster back to it.
- Pull model : Argo CD pulls from Git, so CI needs no cluster credentials.
- Renders Kustomize, Helm, Jsonnet, plain YAML, or a plugin.
- Extras : web UI, CLI, SSO and its own RBAC on top of Kubernetes.
- Lives in
argocd, drives one or many clusters.
2.Words to know
| Word | Meaning |
|---|---|
| Target state | What Git says should run |
| Live state | What really runs in the cluster |
| Sync status | Synced, OutOfSync or Unknown |
| Sync | Make live match target (apply) |
| Refresh | Re-read Git and diff, no apply. Hard refresh also drops the cache |
| Health | Is it running and serving ? A separate answer : Synced can still be Degraded |
Argo project, CNCF graduated 2022-12-06. Latest v3.5.4 (2026-10-06), tested on Kubernetes v1.33 to v1.36. The three newest minors get patches. 3.6 is a release candidate.
3.The parts
A few workloads; Argo's own share one image (quay.). Real state lives in CRs, ConfigMaps and Secrets.
- repo-server clones and renders. No Kubernetes rights, no stored credentials.
- application-controller is a StatefulSet and a classic controller. Scale it to shard clusters.
- notifications-controller sends alerts. dex is optional with direct OIDC. redis is safe to lose.
4.Detection vs correction
Argo CD always notices drift. Fixing it is opt-in, one switch at a time.
- Every 120s (plus up to 60s jitter), or at once on a Git webhook, the controller refreshes each app.
- repo-server renders the manifests, the controller diffs them with the live objects, and marks the app Synced or OutOfSync. That is detection.
- Correction is a sync, and it is manual by default : press Sync, or run
argocd app sync.
| syncPolicy | New commit in Git | File removed from Git | Someone runs kubectl edit |
|---|---|---|---|
| none (default) | OutOfSync, waits for you | OutOfSync, waits for you | OutOfSync, waits for you |
| automated | Synced by itself | Keeps running, stays OutOfSync | Stays OutOfSync, not reverted |
| + prune: true | Synced by itself | Deleted from the cluster | Stays OutOfSync, not reverted |
| + selfHeal: true | Synced by itself | Deleted only if prune is on too | Reverted to Git |
OpenGitOps asks for "continuously reconciled". Out of the box Argo CD only observes continuously. Flux fixes drift on every interval.
5.Turning it on
spec:
syncPolicy:
automated:
enabled: true # 3.1+, unset = on
prune: true # default false
selfHeal: true # default false
CLI : argocd app set APP --sync-policy automated --auto-prune --self-heal. enabled: false pauses auto-sync but keeps the flags.
- Auto-sync tries once per commit SHA (plus parameters). A SHA that failed is not tried again.
- Self-heal waits a backoff : 2s, times 3, max 300s. Docs saying "5 seconds" are stale.
- No
retryblock : auto-sync retries 5 times (5s doubling, max 3m). Aretryblock withoutlimit: 0 retries. - No rollback while auto-sync is on.
1.The three CRDs
All are argoproj. only (no beta, no v1), namespaced, and live in argocd by default. See CRDs.
| Kind (short) | Job | Key fields |
|---|---|---|
| Application (app) | Source(s) to one destination | project, source, destination, syncPolicy |
| ApplicationSet (appset) | Template that stamps out many Applications | generators, template, strategy |
| AppProject (appproj) | Guard rails : what may go where | sourceRepos, destinations, roles, syncWindows |
2.An Application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: guestbook
namespace: argocd
finalizers: [resources-finalizer.argocd.argoproj.io]
spec:
project: default
source:
repoURL: https://github.com/my-org/apps.git
path: guestbook # targetRevision unset = HEAD
destination:
server: https://kubernetes.default.svc
namespace: guestbook
- targetRevision : a branch, a tag, a semver range (matches tags only) or a commit SHA.
- sources : a list instead of
source, e.g. a Helm chart plus values from your repo. - destination :
serverURL or clustername, never both. - finalizer : without it, deleting the Application leaves every resource running. See finalizers.
- project : required. The
defaultproject allows anything. A project that may deploy into argocd is as good as admin.
3.Sync phases and waves
- Order : phase, wave (default 0, negatives allowed), kind (Namespaces first, custom resources last), name.
- Apply the first wave that is not yet synced and healthy.
- Wait for Healthy, pause 2s, next wave. Prunes go in reverse wave order.
4.Hooks
kind: Job # batch/v1 metadata: name: db-migrate annotations: argocd.argoproj.io/hook: PreSync argocd.argoproj.io/hook-delete-policy: HookSucceeded argocd.argoproj.io/sync-wave: "-1" spec: ... # a normal Job
- Kinds : PreSync, Sync, PostSync, SyncFail, Skip. On app deletion : PreDelete (3.3+), PostDelete.
- No delete policy means
BeforeHookCreation. Hooks skip a selective sync. - Helm : Argo CD runs
helm template, sohelm lsshows nothing. pre-install / pre-upgrade become PreSync, post-install / post-upgrade become PostSync.
5.Health
| Kind | Healthy when |
|---|---|
| Workloads | updated replicas = desired (rollouts) |
| LB Service, Ingress | LB hostname or IP in status |
| PVC | phase is Bound |
| Job, CronJob | Complete. A failed Job, or a CronJob whose last run failed (3.2+), is Degraded |
| Custom resources | 238 bundled Lua checks, or yours |
| Application | no check (since 1.8) |
- App health = worst direct child. Best to worst : Healthy, Suspended, Progressing, Missing, Degraded, Unknown.
- Own check : Lua in
argocd-cm, keyresource..customizations. health.<group>_<kind> - App of apps : add one for
argoproj., or waves never wait for a child app.io_ Application
1.ApplicationSets
Generators make parameter sets. The template is rendered once per set into an Application.
| Generator | Params from | Re-reads |
|---|---|---|
| list | fixed values | never |
| clusters | clusters, by label | Secret events |
| git | dirs or files in a repo | every 3 min |
| matrix, merge | child generators combined | fastest child |
| pullRequest | open PRs | every 30 min |
Also scmProvider, clusterDecisionResource, plugin. 3.6 (RC) adds oci. RollingSync (Beta) syncs the apps step by step and turns their auto-sync off.
- Bootstrap : since 3.4 the docs recommend it, with the cluster generator, over app of apps (Cluster API).
- Admin only : it can create Applications in any project. Put a template variable in
projectonly if admins own every generator source.
2.Install : server-side since 3.3
$ kubectl create namespace argocd $ kubectl apply -n argocd --server-side --force-conflicts \ -f https://raw.githubusercontent.com/argoproj/argo-cd/v3.5.4/manifests/install.yaml
kubectl apply fails. See SSA.| Manifest | For |
|---|---|
| install.yaml | trying it, not production |
| ha/install.yaml | production, 3+ nodes |
| namespace-install | one namespace, no CRDs |
| core-install.yaml | no API server, UI or SSO |
--force-conflictsoverwrites live edits. Customize with Kustomize patches.- Self-managed Argo CD needs
ServerSideApply=true.
3.Commands
| Command | Does |
|---|---|
argocd admin initial-password -n argocd | First admin password. Change it, then delete its Secret |
kubectl port-forward svc/argocd-server -n argocd 8080:443 | Reach the API unexposed, then argocd login localhost:8080 |
argocd app get my-app --hard-refresh | Status. Hard refresh also drops the manifest cache |
argocd app diff my-app | Git vs live. Exit 0 same, 1 different, 2 error |
argocd app sync my-app --prune --dry-run | Preview a sync that also deletes extras. Drop --dry-run to run it |
argocd app history my-app, argocd app rollback my-app ID | Roll back to a past sync. Not while auto-sync is on |
argocd appset generate appset.yaml | Show the Applications an ApplicationSet would make |
argocd cluster add CONTEXT | Register a cluster (adds an admin-level argocd-manager SA) |
4.Argo CD vs Flux
| Topic | Argo CD | Flux |
|---|---|---|
| UI | Built in | In Flux Operator |
| Users | Own SSO + RBAC | Kubernetes RBAC |
| Apply | Client-side, SSA opt-in | Server-side apply |
| Drift | Seen always, fixed if selfHeal | Fixed every interval |
| Helm | helm template, no release | Real Helm releases |
Both CNCF graduated in 2022. New image tags into Git : Argo CD Image Updater, a separate argoproj-labs controller.
3.0 tightened RBAC defaults. 3.5 renders with Helm 4 only. 3.6 (RC) diffs ServerSideApply apps server-side.
5.Gotchas
ignoreDifferencesonly hides the diff. Sync still writes those fields unlessRespectIgnoreDifferences=true.- An HPA or KEDA owns replicas ? Leave
replicasout of Git. - Secrets injected while rendering are cached in plain text in Redis. Keep them in the cluster with External Secrets.
- Since 3.0 Argo CD tracks resources with an annotation, not a label. Sync every app once after upgrading.