CI and builds

Actions Runner Controller

1.What is ARC ?

  • GitHub's Kubernetes operator for self-hosted Actions runners, and the reference implementation of its scale set APIs. This sheet is ARC 0.15.0 (gha-runner-scale-set charts), pre-1.0, CRDs v1alpha1.
  • Two Helm charts published only as OCI packages, plus one controller image with no latest tag : always pass --version. Runner Pods are ephemeral : one job, then deleted.
  • Scale set mode is the only one GitHub supports. Legacy ARC (actions.summerwind.dev) is community-only, with no controller release since v0.27.6. Moving over is a rebuild, not an upgrade.

2.The pieces

KindOne perDoes
AutoscalingRunnerSetscale set (Helm release)Wanted state : URL, Secret, min and max, Pod template, mode
AutoscalingListenerscale set, controller namespaceThe listener Pod, its ServiceAccount and config Secret. Role and RoleBinding sit in the runner namespace
EphemeralRunnerSetscale set, same nameThe replica count the listener patches
EphemeralRunnerrunnerOne bare Pod, its JIT config, its phase

All four are namespaced, actions.github.com/v1alpha1. Runner Pods are not Kubernetes Jobs (Jobs).

3.From queued job to runner Pod

GitHub Actions service jobs for the scale set 1. long poll (about 50s) 2. assigned jobs = N listener Pod controller namespace 3. patch replicas EphemeralRunnerSet replicas = N + minRunners 4. JIT config EphemeralRunner + Pod bare Pod, deleted after the job 5. register, run ONE job
  1. The listener long-polls over HTTPS : back at once if a message waits, else after about 50s.
  2. The message carries the assigned jobs count.
  3. The listener patches replicas with its own ServiceAccount.
  4. The controller asks for a JIT config per runner. The Pod gets only ACTIONS_RUNNER_INPUT_JITCONFIG, ARC never injects the PAT or App key.
  5. The runner registers with it, runs one job, then the Pod is deleted at once (0.15.0). Runners are created up to 8 at a time.

A job no runner takes is requeued up to 3 times, and GitHub gives up after 24 hours. No webhook, no REST polling budget.

4.The scaling rule

desired = min(minRunners + jobs, maxRunners)

Example, min 2 and max 10 : 5 assigned jobs give 7 runners, 12 give 10. No schedule is built in : patch minRunners from a CronJob (Jobs).

minRunners is idle runners kept warm, not a floor on total Pods. Unset maxRunners means no cap. Both at 0 is a drain : no runners at all.

5.Targeting a scale set

  • runs-on: <scale set name> : the Helm release name or runnerScaleSetName, at most 45 characters.
  • Extra labels (0.14.0+) use the chart key scaleSetLabels, not the docs' runnerScaleSetLabels, and reach GitHub only at creation.
  • self-hosted is never added for you.
jobs:
  build:
    runs-on: example-set
In short : ARC keeps one ephemeral runner Pod per assigned GitHub job, plus the warm ones you ask for. A listener per scale set is told the job count, so target the set by its name.

1.Install

CommandDoes
helm install arc -n arc-systems --create-namespace --version 0.15.0 oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controllerController and CRDs, own namespace
helm install arc-runner-set -n arc-runners --create-namespace --version 0.15.0 -f values.yaml oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-setOne scale set, another namespace. Name = runs-on target
kubectl create secret generic gh-app -n arc-runners --from-literal=github_app_id=... --from-literal=github_app_installation_id=... --from-file=github_app_private_key=key.pemApp Secret for githubConfigSecret

2.Auth to GitHub

CredentialLevelNeeds
GitHub App (recommended)repo or orgrepo : Administration read and write, Metadata read. org : Self-hosted runners read and write
Classic PATrepo, org, enterpriserepo, admin:org, manage_runners:enterprise. The only option for enterprise sets
Fine-grained PATrepo or orgrepo : Administration read and write. org : Administration read, Self-hosted runners read and write
  • Create the Secret yourself in the scale set namespace and reference it by name. IDs are quoted strings, never --set. A copy also lands in the controller namespace (listener config). A node clock ahead of GitHub gives 401.

3.Container modes

containerMode.typePod shapePrivilegedRunner's Kubernetes rights
unsetrunner container onlynonone
dindrunner + dind sidecaryesnone
kubernetesrunner + -workflow Pod, PVCnoPods, exec, logs, Jobs, Secrets
kubernetes-novolume (0.13.0+)as above, no PVCnosame, no batch/jobs
dind mode : one Pod dind sidecar privileged socket runner uses DOCKER_HOST kubernetes mode : two Pods per job runner Pod kube-mode Role hooks create workflow Pod job + services work PVC (RWO, one per runner) novolume : no PVC, files are copied
  • dind is privileged : it needs the privileged level and is node root (Image Builds). A native sidecar.
  • kubernetes needs kubernetesModeWorkVolumeClaim, or the render has spec: null and the API server rejects it.
  • Every job needs a job container (ACTIONS_RUNNER_REQUIRE_JOB_CONTAINER). In kubernetes mode the workflow Pod is pinned to the runner's node.

4.The runner image

  • ghcr.io/actions/actions-runner : Ubuntu 24.04, UID 1001, docker group GID 123, passwordless sudo, Docker CLI and Buildx. The daemon binaries are there, but nothing starts them.
  • ARC turns runner self-update off, so the image tag is the runner version. GitHub stops sending jobs to a runner more than 30 days behind : pin and bump, or float latest with imagePullPolicy: Always (pulls).

5.Rendering for GitOps

  • controllerServiceAccount : set it, or helm template fails ("No gha-rs-controller deployment found").
  • CRDs (0.6 to 1.3 MB each) : client-side apply fails, may not be more than 262144 bytes. Use server-side apply (API machinery).
  • Argo CD : flags.excludeLabelPropagationPrefixes: ["argocd.argoproj.io/instance"], or it fights ARC's children.
  • Delete order : scale sets first, controller last, or finalizers strand objects (finalizers). In Flux, dependsOn the controller.
In short : Install the controller and each scale set as OCI charts in two namespaces, with a GitHub App Secret. dind is privileged. kubernetes mode adds a Pod per job.

1.Security model

  • Never on public repositories : anyone who can open a pull request runs code. Ephemeral runners stop persistence, not a job reading its secrets.
  • dind is node root (Image Builds). In the kubernetes modes the runner's Role can read Secrets in its namespace, the App key included : keep ACTIONS_RUNNER_REQUIRE_JOB_CONTAINER=true, one scale set per namespace.
  • Workflow Pods run as the namespace's default ServiceAccount : set serviceAccountName in a hook template and bind it narrowly (RBAC).
  • Write an egress policy : DNS, 443 out, block 169.254.0.0/16. Workflow Pods carry another label, so they need their own (NetworkPolicy).

2.Upgrades and the version guard

controller 0.15.x reads each set's version label same minor 0.15.y : keep, update other, or no label DELETE the set

A minor upgrade is a reinstall :

  1. Uninstall every scale set, wait for cleanup.
  2. Uninstall the controller.
  3. Delete the four actions.github.com CRDs.
  4. Reinstall both charts at the same version.

Applying both new charts at once lets the old controller delete the new sets.

  • Patch upgrades update in place since 0.15.0. Every minor since 0.11.0 changed all four CRDs, and Helm never upgrades CRDs. Flux does only with upgrade.crds: CreateReplace (CRDs, Flux deep dive).
  • An outdated runner exits with code 7. 0.15.0 parks the whole set Outdated until the runner template changes (a new image does, maxRunners does not).
Seen in practicePitfall : re-rendering with a few --set flags instead of the full values. The chart has no values schema, so a missing work volume claim renders null and helm template raises no error. Do better : keep every value in one file, render both charts from it, diff the full set spec, grep for null.

3.Watching it

  • Since 0.15.0 kubectl get shows only min, max and phase : counts live in metrics, which the chart turns off (no Service either). Set the metrics: block and scrape listeners by Pod label (Prometheus).
  • Gauges : gha_assigned_jobs, gha_desired_runners, gha_busy_runners, gha_controller_failed_ephemeral_runners, and new gha_controller_outdated_ephemeral_runners. Opt-in job_workflow_ref labels add a series per pull request.

4.ARC, KEDA or a fixed pool

ARC scale setsKEDA github-runnerGitea Runner
Learns about jobslistener long poll, told the countREST API every pollingIntervalrunner polls Gitea
Scales0 to N, one Pod per jobScaledJob or Deploymentfixed pool you size
RegistrationJIT per runner, by the controlleryour runner image registerstoken, then .runner
Rate limitsnone on the scaling path5,000/h PAT, 15,000/h Appn/a

KEDA adds self-hosted, linux, x64 labels by default, ARC never adds self-hosted. Drains and OOM kills end jobs : karpenter.sh/do-not-disrupt helps with autoscalers, not kubectl drain (Karpenter, limits).

5.Commands

CommandDoes
kubectl get autoscalingrunnersets -n arc-runnersMin, max and phase per set
kubectl get pods -n arc-systems -l app.kubernetes.io/component=runner-scale-set-listenerOne listener per set
kubectl logs -n arc-systems -l app.kubernetes.io/name=gha-rs-controllerController logs (default names)
kubectl patch autoscalingrunnerset <name> -n arc-runners --type=merge -p '{"spec":{"minRunners":2}}'Warm runners, from a CronJob
In short : ARC keeps one ephemeral Pod per assigned GitHub job. Reinstall for a minor, and treat dind runners as node root and kube-mode runners as Secret readers.