Networking internals

Network Policy in Depth

1.A request, not a firewall

  • No status : the API server defaults, validates, stores. Alpha status left in v1.28.
  • Not in conformance : the 49 enforcement e2e tests are feature-tagged.
  • No enforcer (plain flannel) : every policy is stored and silently ignored.
  • Agents : kube-router (k3s), Calico Felix, cilium-agent, antrea-agent, kube-network-policies.
kubectl apply kube-apiserver defaults, validates etcd no status written back watch policy agent on every node per-node rules iptables / nftables eBPF / OVS programs Pod A Service DNAT ClusterIP:80 -> Pod B:8080 egress check A's node rewritten before any check ingress check B's node wire Pod B policy sees Pod B's IP and port 8080, never the ClusterIP

2.Defaulting traps

  • Egress-only, no policyTypes : all ingress is cut too.
  • kubectl describe netpol : "isolated for ingress".
  • No podSelector (optional, v1.34+) : every Pod.

v1.37. Basics : Namespaces and Ingress, Architecture.

You wroteStored (seen on v1.37.0)
no policyTypes, no rules[Ingress]
egress rules only[Ingress, Egress]
no podSelectorpodSelector: {}
a port, no protocolprotocol: TCP

3.After DNAT : Pod ports

  • Calico, Cilium, Antrea, kube-network-policies check after Service DNAT. Docs are silent.
  • Name the Pod's targetPort or port name, and the backend Pods.
  • Named ports resolve per destination Pod. endPort needs a numeric port.
  • Only TCP, UDP, SCTP. ICMP is rejected.
# Service api : port 80 to containerPort "http" (8080)
ports: [{name: http, port: 80, targetPort: http}]
# NetworkPolicy api-from-frontend
ingress:
- from: [{podSelector: {matchLabels: {app: frontend}}}]
  ports: [{protocol: TCP, port: http}]
# or 8080. 'port: 80' never matches

4.Where NAT and IP rules blur

  • ipBlock is for external CIDRs. Pod IPs in it : e2e expects a match, as kind does. Cilium ignores them by default.
  • Strict CIDRs since v1.36 (Beta) : 10.0.0.5/24, leading zeros, IPv4-mapped IPv6 fail on create.
  • Stored values ratchet with a warning, so a fresh copy fails. Test with kubectl apply --dry-run=server --validate=strict.
TrafficThe policy sees
NodePort, LB with eTP=Clustera node IP (SNAT). Use eTP=Local or allow node IPs
probes, local host processesthe local node : ingress allowed (kind : root's sockets only)
a hostNetwork Podthe node IP : undefined
NodeLocal DNSCachehostNetwork : only an ipBlock matches

See Service Dataplane, Cluster DNS, Pod Security.

5.The edges the API leaves open

EdgeThe API saysImplementations
ICMPundefinedkube-router accepts echo, unreachable, time-exceeded. Calico can name ICMP. Cilium blocks unrelated ICMP under L4 rules
Pod to own nodegodoc : ingress from it. Docs : "to and from"kube-network-policies enforces Pod-to-host. Felix default DROP, calico.yaml ACCEPT
Pod to itself via its Servicecannot block itselfthe upstream e2e test expects it blocked
  • Undefined too : open connections after a change, new Pods before programming (windows). Not expressible : explicit deny, logging, Service or FQDN peers, L7, a cluster-wide default.
In short : A NetworkPolicy is stored; a per-node agent enforces it after DNAT. Name Pods and Pod ports, mind policyTypes defaulting, and treat ICMP and node traffic as undefined.

1.ClusterNetworkPolicy (v1alpha2)

  • network-policy-api v0.2.0 (2026-04-21) : one cluster-scoped CRD, cnp.
  • Replaces ANP, BANP. Allow is now Accept, ports now protocols. v1beta1 PR #411 open.
  • tier Admin or Baseline. priority 0 to 1000, lower wins, ties ambiguous.
  • tcp: {} is rejected. Any port : range: {start: 1, end: 65535}.
PeerIngressEgressChan.
namespacesyesyesstd
podsyesyesstd
networksnoyes, hits Pods toostd
nodes, domain­Namesnoyesexp
  • Subject : namespaces or pods, not hostNetwork. 25 rules max.
  • Install from the v0.2.0 tag : main changed the schema.
  • kubectl get cnp : Tier, Priority. (channels)

2.Evaluation order

new connection Admin CNPs priority, then rule order Accept : allow, final Deny : drop, final Pass or no match NetworkPolicy selects the Pod for this direction ? its rules decide yes no Baseline CNPs Accept / Deny decide no match : allow
  • Accept, Deny are final. Pass skips the rest of Admin.
  • Accept covers replies and related ICMP. An egress Accept keeps the far end's ingress rules.
  • DNS guarantee : an Admin Accept to k8s-app: kube-dns Pods, port 53. Upstream user-story-2 uses app: kube-dns, a wrong label.

3.Who implements what (checked 2026-10-07)

ImplementationCNP v1alpha2Notes
kube-network-policies v1.1.2yes, v1.0.0+reference. nftables + NFQUEUE, fails open. kind v0.33.0 embeds an older copy : no CNP
Calico v3.33.0yes, v3.32+tiers kube-admin, kube-baseline. ANP, BANP dropped. A bad peer fails the rule
Cilium 1.20.2opt-ink8sClusterNetworkPolicy.enabled. Conformance needs [pods, nodes] CIDR match
Antrea v2.7.0alpha gate, offgate ClusterNetworkPolicy. v1alpha1 deprecated
OVN-K8s, Kube-OVNOVN-K8s v1alpha1Kube-OVN : pre-v0.2.0 build, v0.2.0 on master

The upstream list (21 April 2026) omits Cilium 1.20 and Antrea 2.7. No conformance reports from them or Calico.

4.k3s : kube-router, one packet

  • k3s v1.37.1+k3s1 : fork v2.6.3-k3s1, iptables + ipset, no CNP.
  • --disable-network-policy leaves its rules behind.
  • Unmarked : NFLOG group 100, then REJECT. Ping works.
  • New Pods fail open : no --netpol-default-deny.
  • Chains : iptables -t filter -S | grep KUBE-.
FORWARD KUBE-ROUTER-FORWARD RETURN ClusterIP, NodePort (pre-DNAT) KUBE-POD-FW-<hash> accept node-local sources KUBE-NWPLCY-COMMON drop INVALID, accept ESTABLISHED, ICMP KUBE-NWPLCY-<hash> : ipset match, mark 0x10000 mark 0x20000 : ACCEPT no mark : NFLOG 100, REJECT

5.Calico and Cilium

Calico
  • Tiers by order. NetworkPolicies sit in default as knp.default.<name>.
  • Allow, Deny, Pass, Log. Follow Log with Allow, or the tier's end drop wins.
  • cali-pi- / cali-po- chains, or a tc hook in eBPF mode.
Cilium
  • Labels become identities. Per endpoint : a BPF map of (identity, port, proto), 16k.
  • ingressDeny / egressDeny beat every allow. toFQDNs needs a rules.dns allow.
  • Dump : cilium-dbg bpf policy get <ep-id>.

Cilium 1.20 may balance NodePort at the client (Service Dataplane). Antrea has two CNP kinds.

In short : CNP adds Admin and Baseline tiers around NetworkPolicy. Each dataplane enforces it its own way : an iptables REJECT on k3s, a missing eBPF map entry on Cilium.

1.Stored is not enforced

  • Only a probe seen to fail proves a deny.
  • Counts lie : KUBE-POD-FW can read 0 on a node without iptables; nft showed 70.
  • An untested deny is a guess : probe it from a Pod that should fail.

The official task expects wget: download timed out; k3s refuses at once.

DataplaneClient seesagnhost
kube-router, k3srefused at onceREFUSED
Calico, kube-network-policiestimeoutTIMEOUT
Cilium L3/L4timeout (opt-in ICMP on egress)TIMEOUT
Cilium L7HTTP 403, DNS REFUSEDconnects

2.A per-node probe for your cluster

  • Server Pods on each node. Default-deny ingress, only label allowed passes.
  • A client DaemonSet probes from every node : all must fail.
  • Control 1 : one allowed client must pass, or the network is just broken.
  • Control 2 : agent off on one node, FAIL there only.
  • No ping. Self-health on 127.0.0.1 : own-Service calls may be blocked.
client, node 1 client, node 2 client, node 3 server Service server Pods one per node must fail control 1 : allowed label must pass on every node control 2 : agent off on node 2 FAIL there, and only there
# a sketch, not run. Namespace np-probe : Service server, client DaemonSet (agnhost)
for p in $(kubectl -n np-probe get pods -l app=client -o name); do
  n=$(kubectl -n np-probe get "$p" -o jsonpath='{.spec.nodeName}')
  if kubectl -n np-probe exec "$p" -- /agnhost connect server.np-probe.svc.cluster.local:80 \
       --timeout=3s --protocol=tcp 2>err.txt
  then echo "FAIL $n: denied connection worked"; else echo "ok $n: $(cat err.txt)"; fi
done   # ok lines read TIMEOUT or REFUSED

Upstream e2e diffs an expected x/y/z truth table against agnhost results. No expected X, no proof.

3.Watching verdicts

DataplaneWhereRead it with
k3s kube-routerNFLOG 100. Empty podSelector not logged (k3s #8008)tcpdump -ni nflog:100
CiliumHubble (relay off). 133 no allow, 181 deny rulehubble observe --verdict DROPPED
CalicoLog rules : calico-packet. Whisker previewjournalctl -k | grep calico-packet

Antrea : np.log. kube-network-policies : -v=2 (not on kind : no flags). policy-assistant simulates from YAML (v1, v1alpha1 only).

4.Start-up and change windows

ImplementationNew Pod before programmingOpen connection after a new deny
kube-router, k3sfails open (no --netpol-default-deny)survives
Calicoopt-in hold, linuxPolicySetupTimeoutSeconds (default 0)survives until reconnect
Ciliumreserved:init, allowed unless selectednot documented
kube-network-policiesNRI learns Pod IPs earlycut (--strict-mode, default)
kind (its built-in copy)open until its node lists it (0.7 s)survives (no strict mode)

The docs suggest an init container that waits for its dependencies, and advise against changing policies under live connections.

5.Troubleshooting checklist

  1. Anything enforcing ? flannel alone is not.
  2. Pod selected ? kubectl describe netpol.
  3. Both ends allow ? DNS (Cluster DNS) ?
  1. Pod port named ? Source NATed ?
  2. Higher tier ? kubectl get cnp, Calico tiers.
  3. Read the verdict, then probe every node.
In short : Prove every deny with a probe that must fail on every node, read the verdict where your dataplane logs it, and expect start-up and change windows to differ by plugin.