Networking and traffic

MetalLB

1.Why MetalLB ?

On bare metal a LoadBalancer Service stays EXTERNAL-IP <pending> forever: no cloud controller hands out IPs.

  • NodePort is ugly (high ports, clients must know node IPs)
  • Cloud LBs (ELB, GCP LB) are not available on-prem, in homelabs or at the edge
  • MetalLB = IP allocation from your pools + announcement to the network (L2 or BGP)

Fills in type: LoadBalancer, the Service type after NodePort.

Without MetalLB Service LB EXTERNAL-IP<pending> With MetalLB Service LB 192.168.10.5 announced to LAN or routers

2.Components

Installs into metallb-system, idle until you create a pool and an advertisement.

  • Two jobs : assign IPs, then advertise them
  • Works with IPv4, IPv6 and dual stack
  • Config is Custom Resources only
  • controllerDeployment. Watches Services, picks an IP from an IPAddressPool, writes it to the Service status.
  • speakerDaemonSet. Announces the IP via ARP/NDP or BGP. Uses memberlist (7946 TCP/UDP) to spot dead nodes.
  • frr-k8sDefault BGP backend. Wrapper around FRR, adds BFD and IPv6.

3.L2 Mode Flow

Controller allocates, speaker announces. Only the leader node answers ARP/NDP.

Allocation and announcement (L2)
kubectl applyService type=LoadBalancer
API serverSends a watch event.
controllerPicks a free IP from the pool, sets status.
speaker (leader)Answers ARP/NDP for the IP.
Then traffic :
ClientsHit the IP.
LeaderGets all traffic for that IP.
One leader per IP, chosen via memberlist. If it dies a new one wins, brief gap.

4.BGP Mode

Every node peers with your routers and announces the Service IP. Traffic is spread across nodes (ECMP).

  • Needs a BGP-capable router
  • Backends : FRR-K8s (default), native, legacy FRR (deprecated)
Node 1 Node 2 Node 3 BGP session Router Client flows spread (ECMP)
AspectL2 (ARP/NDP)BGP
NeedsSame LANBGP router
Load balancingNo, one node per IPYes (ECMP)
FailoverNew leader, brief gapRoute withdrawn, BFD speeds it up
SetupVery lowHigher (ASNs, peers)
UseHomelab, small LANDatacenter, larger scale
L2 = failover only, one node takes it all.
BGP = real load spreading across nodes.
In short : Controller allocates an IP from your pool, speaker announces it via L2 (one leader, failover) or BGP (ECMP, needs routers).

1.Config Objects (CRDs)

Before v0.13.0 config was a ConfigMap. Now it is CRDs in metallb-system. Convert old config with quay.io/metallb/configmaptocrs.

KindapiVersionPurpose
IPAddressPoolv1beta1IP ranges to allocate
L2Advertisementv1beta1Announce via ARP/NDP
BGPPeerv1beta2Router to peer with
BGPAdvertisementv1beta1Announce via BGP
BFDProfilev1beta1BFD timers
L2Advertisement BGPAdvertisement ipAddressPools selects the pool IPAddressPool Service picks a pool peers (opt.) BFDProfile BGPPeer bfdProfile (opt.)
kind: IPAddressPool
metadata: {name: first-pool, namespace: metallb-system}
spec:
  addresses: [192.168.10.0/24]
---
kind: L2Advertisement
metadata: {name: example, namespace: metallb-system}
spec:
  ipAddressPools: [first-pool]
Pool and advert
  • Both use apiVersion: metallb.io/v1beta1
  • A Service can pick a pool with the annotation metallb.io/address-pool
  • A pool with no advertisement is unreachable
  • An L2Advertisement with no ipAddressPools announces every pool

2.Commands

CommandDescription
kubectl apply -f metallb-frr-k8s.yamlInstall, FRR-K8s backend (recommended). metallb-native.yaml is L2 only
helm install metallb metallb/metallb -n metallb-system --create-namespaceInstall with Helm, after helm repo add
kubectl get pods -n metallb-systemCheck controller and speaker
kubectl get ipaddresspools -n metallb-systemList the pools
kubectl logs -n metallb-system ds/speakerDebug announcements and leader election
kubectl describe svc <name>Events show IP and announcing node

3.Gotchas

  • kube-proxy in IPVS mode needs ipvs.strictARP: true or L2 misbehaves (ipvs is deprecated since v1.35)
  • Install alone does nothing: no pool, no advertisement, no IP
  • Pool IPs must be unused on your LAN (outside the DHCP range)
  • Port 7946 (TCP/UDP) must be open between nodes for memberlist
  • L2 is failover, not load balancing. Legacy FRR mode is deprecated
In short : A pool plus an advertisement is all MetalLB needs. Check pods, pools and speaker logs when an IP stays pending.