Networking internals
Network Policy in Depth
1.A request, not a firewall
- No status : the API server defaults, validates, stores. Alpha
statusleft in v1.28. - Not in conformance : the 49 enforcement e2e tests are feature-tagged.
- No enforcer (plain flannel) : every policy is stored and silently ignored.
- Agents : kube-router (k3s), Calico Felix, cilium-agent, antrea-agent, kube-network-policies.
2.Defaulting traps
- Egress-only, no
policyTypes: all ingress is cut too. kubectl describe netpol: "isolated for ingress".- No
podSelector(optional, v1.34+) : every Pod.
v1.37. Basics : Namespaces and Ingress, Architecture.
| You wrote | Stored (seen on v1.37.0) |
|---|---|
no policyTypes, no rules | [Ingress] |
| egress rules only | [Ingress, Egress] |
no podSelector | podSelector: {} |
a port, no protocol | protocol: TCP |
3.After DNAT : Pod ports
- Calico, Cilium, Antrea, kube-network-policies check after Service DNAT. Docs are silent.
- Name the Pod's
targetPortor port name, and the backend Pods. - Named ports resolve per destination Pod.
endPortneeds a numericport. - Only TCP, UDP, SCTP.
ICMPis rejected.
# Service api : port 80 to containerPort "http" (8080) ports: [{name: http, port: 80, targetPort: http}] # NetworkPolicy api-from-frontend ingress: - from: [{podSelector: {matchLabels: {app: frontend}}}] ports: [{protocol: TCP, port: http}] # or 8080. 'port: 80' never matches
4.Where NAT and IP rules blur
ipBlockis for external CIDRs. Pod IPs in it : e2e expects a match, as kind does. Cilium ignores them by default.- Strict CIDRs since v1.36 (Beta) :
10.0.0.5/24, leading zeros, IPv4-mapped IPv6 fail on create. - Stored values ratchet with a warning, so a fresh copy fails. Test with
kubectl apply --dry-run=server --validate=strict.
| Traffic | The policy sees |
|---|---|
| NodePort, LB with eTP=Cluster | a node IP (SNAT). Use eTP=Local or allow node IPs |
| probes, local host processes | the local node : ingress allowed (kind : root's sockets only) |
| a hostNetwork Pod | the node IP : undefined |
| NodeLocal DNSCache | hostNetwork : only an ipBlock matches |
5.The edges the API leaves open
| Edge | The API says | Implementations |
|---|---|---|
| ICMP | undefined | kube-router accepts echo, unreachable, time-exceeded. Calico can name ICMP. Cilium blocks unrelated ICMP under L4 rules |
| Pod to own node | godoc : ingress from it. Docs : "to and from" | kube-network-policies enforces Pod-to-host. Felix default DROP, calico.yaml ACCEPT |
| Pod to itself via its Service | cannot block itself | the upstream e2e test expects it blocked |
- Undefined too : open connections after a change, new Pods before programming (windows). Not expressible : explicit deny, logging, Service or FQDN peers, L7, a cluster-wide default.
In short : A NetworkPolicy is stored; a per-node agent enforces it after DNAT. Name Pods and Pod ports, mind
policyTypes defaulting, and treat ICMP and node traffic as undefined.1.ClusterNetworkPolicy (v1alpha2)
- network-policy-api v0.2.0 (2026-04-21) : one cluster-scoped CRD,
cnp. - Replaces ANP, BANP.
Allowis nowAccept,portsnowprotocols. v1beta1 PR #411 open. - tier Admin or Baseline. priority 0 to 1000, lower wins, ties ambiguous.
tcp: {}is rejected. Any port :range: {start: 1, end: 65535}.
| Peer | Ingress | Egress | Chan. |
|---|---|---|---|
namespaces | yes | yes | std |
pods | yes | yes | std |
networks | no | yes, hits Pods too | std |
nodes, domainNames | no | yes | exp |
- Subject : namespaces or pods, not hostNetwork. 25 rules max.
- Install from the
v0.2.0tag :mainchanged the schema. kubectl get cnp: Tier, Priority. (channels)
2.Evaluation order
Accept,Denyare final.Passskips the rest of Admin.Acceptcovers replies and related ICMP. An egressAcceptkeeps the far end's ingress rules.- DNS guarantee : an Admin
Accepttok8s-app: kube-dnsPods, port 53. Upstream user-story-2 usesapp: kube-dns, a wrong label.
3.Who implements what (checked 2026-10-07)
| Implementation | CNP v1alpha2 | Notes |
|---|---|---|
| kube-network-policies v1.1.2 | yes, v1.0.0+ | reference. nftables + NFQUEUE, fails open. kind v0.33.0 embeds an older copy : no CNP |
| Calico v3.33.0 | yes, v3.32+ | tiers kube-admin, kube-baseline. ANP, BANP dropped. A bad peer fails the rule |
| Cilium 1.20.2 | opt-in | k8sClusterNetworkPolicy.. Conformance needs [pods, nodes] CIDR match |
| Antrea v2.7.0 | alpha gate, off | gate ClusterNetworkPolicy. v1alpha1 deprecated |
| OVN-K8s, Kube-OVN | OVN-K8s v1alpha1 | Kube-OVN : pre-v0.2.0 build, v0.2.0 on master |
The upstream list (21 April 2026) omits Cilium 1.20 and Antrea 2.7. No conformance reports from them or Calico.
4.k3s : kube-router, one packet
- k3s v1.37.1+k3s1 : fork v2.6.3-k3s1, iptables + ipset, no CNP.
--disable-network-policyleaves its rules behind.- Unmarked : NFLOG group 100, then REJECT. Ping works.
- New Pods fail open : no
--netpol-default-deny. - Chains :
iptables -t filter -S | grep KUBE-.
5.Calico and Cilium
Calico
- Tiers by
order. NetworkPolicies sit indefaultasknp..default.<name> - Allow, Deny, Pass, Log. Follow Log with Allow, or the tier's end drop wins.
cali-pi-/cali-po-chains, or a tc hook in eBPF mode.
Cilium
- Labels become identities. Per endpoint : a BPF map of (identity, port, proto), 16k.
ingressDeny/egressDenybeat every allow.toFQDNsneeds arules.dnsallow.- Dump :
cilium-dbg bpf policy get <ep-id>.
Cilium 1.20 may balance NodePort at the client (Service Dataplane). Antrea has two CNP kinds.
In short : CNP adds Admin and Baseline tiers around NetworkPolicy. Each dataplane enforces it its own way : an iptables REJECT on k3s, a missing eBPF map entry on Cilium.
1.Stored is not enforced
- Only a probe seen to fail proves a deny.
- Counts lie :
KUBE-POD-FWcan read 0 on a node withoutiptables;nftshowed 70. - An untested deny is a guess : probe it from a Pod that should fail.
The official task expects wget: download timed out; k3s refuses at once.
| Dataplane | Client sees | agnhost |
|---|---|---|
| kube-router, k3s | refused at once | REFUSED |
| Calico, kube-network-policies | timeout | TIMEOUT |
| Cilium L3/L4 | timeout (opt-in ICMP on egress) | TIMEOUT |
| Cilium L7 | HTTP 403, DNS REFUSED | connects |
2.A per-node probe for your cluster
- Server Pods on each node. Default-deny ingress, only label
allowedpasses. - A client DaemonSet probes from every node : all must fail.
- Control 1 : one allowed client must pass, or the network is just broken.
- Control 2 : agent off on one node, FAIL there only.
- No ping. Self-health on
127.0.0.1: own-Service calls may be blocked.
# a sketch, not run. Namespace np-probe : Service server, client DaemonSet (agnhost) for p in $(kubectl -n np-probe get pods -l app=client -o name); do n=$(kubectl -n np-probe get "$p" -o jsonpath='{.spec.nodeName}') if kubectl -n np-probe exec "$p" -- /agnhost connect server.np-probe.svc.cluster.local:80 \ --timeout=3s --protocol=tcp 2>err.txt then echo "FAIL $n: denied connection worked"; else echo "ok $n: $(cat err.txt)"; fi done # ok lines read TIMEOUT or REFUSED
Upstream e2e diffs an expected x/y/z truth table against agnhost results. No expected X, no proof.
3.Watching verdicts
| Dataplane | Where | Read it with |
|---|---|---|
| k3s kube-router | NFLOG 100. Empty podSelector not logged (k3s #8008) | tcpdump -ni nflog:100 |
| Cilium | Hubble (relay off). 133 no allow, 181 deny rule | hubble observe --verdict DROPPED |
| Calico | Log rules : calico-packet. Whisker preview | journalctl -k | grep calico-packet |
Antrea : np.log. kube-network-policies : -v=2 (not on kind : no flags). policy-assistant simulates from YAML (v1, v1alpha1 only).
4.Start-up and change windows
| Implementation | New Pod before programming | Open connection after a new deny |
|---|---|---|
| kube-router, k3s | fails open (no --netpol-default-deny) | survives |
| Calico | opt-in hold, linux (default 0) | survives until reconnect |
| Cilium | reserved:init, allowed unless selected | not documented |
| kube-network-policies | NRI learns Pod IPs early | cut (--strict-mode, default) |
| kind (its built-in copy) | open until its node lists it (0.7 s) | survives (no strict mode) |
The docs suggest an init container that waits for its dependencies, and advise against changing policies under live connections.
5.Troubleshooting checklist
- Anything enforcing ? flannel alone is not.
- Pod selected ?
kubectl describe netpol. - Both ends allow ? DNS (Cluster DNS) ?
- Pod port named ? Source NATed ?
- Higher tier ?
kubectl get cnp, Calico tiers. - Read the verdict, then probe every node.
In short : Prove every deny with a probe that must fail on every node, read the verdict where your dataplane logs it, and expect start-up and change windows to differ by plugin.