Skip to content

fix(net): track gateway transit flows from its routed CIDRs - #930

Open
Semen Kolesnikov (Lotarcc) wants to merge 1 commit into
Azure:mainfrom
Lotarcc:fix/net-gateway-notrack-local-source
Open

Semen Kolesnikov (Lotarcc) wants to merge 1 commit into
Azure:mainfrom
Lotarcc:fix/net-gateway-notrack-local-source

Conversation

@Lotarcc

@Lotarcc Semen Kolesnikov (Lotarcc) commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

On a gateway, UNBOUNDED-NOTRACK keeps conntrack for packets to the local Site
(-d <returnCIDR> -j RETURN) but not for packets from it. When a pod on a non-gateway node in the
gateway's Site reaches a remote Site through the gateway, the request arrives on the tunnel with a
remote destination and is NOTRACKed, while the reply arrives with a local destination and is
tracked. Conntrack marks the reply INVALID and KUBE-FORWARD drops it.

The chain now also returns -s <cidr> for the routedCidrs of the gateway's own pools, so a gateway
that carries both the request and the reply tracks both. Other local sources, such as node IPs, stay
untracked: their replies can bypass the gateway, and tracking only the request would make the rest of
that flow INVALID instead. NAT is unchanged.

With two or more gateways in a pool, this fixes only the flows whose reply comes back through the
same gateway. The sending node and the remote node each pick a gateway per flow from their own flow
hash (hrw_select in bpf/unbounded_encap.c), so a reply can reach a gateway that never saw the
request, and that gateway still drops it. Untracking the reply there is not an option: remote-initiated
flows to the same local CIDRs are masqueraded on that gateway and need conntrack, and the raw table
cannot tell the two apart. Making both ends pick the same gateway would cover that case; it is a
routing change, so it is not part of this PR.

reconcileFamily and ensureChain now take the package's iptablesClient interface so the chain
layout can be tested with the existing fake.

Reproduced on AKS (Azure CNI pod subnets, gateway mode) and in a kind lab (unmanaged Calico Site with
an Internal gateway pool, remote managed Site over GENEVE). 30 TCP connections per path, 1s apart,
fresh cluster per run:

Path 1 gateway, main 1 gateway, this PR 2 gateways, main 2 gateways, this PR
member pod -> remote pod 0/30 30/30 0/30 22/30
remote pod -> member pod 30/30 30/30 30/30 30/30
remote pod -> gateway pod 30/30 30/30 30/30 30/30

On main, the member node's host network fails the same way when it sends from an address inside the
routedCidrs (in the lab, the unbounded0 address from its podCIDR), and passes with this change.
Other paths are unchanged.

Part of #921.

Testing: new TestReconcileFamilyTracksRoutedSources (fails without the source rules),
go test ./internal/net/netlink/ ./cmd/unbounded-net-node/ on Linux, make lint, kind lab runs.

@Lotarcc

Copy link
Copy Markdown
Contributor Author

I've corrected the description. With two or more gateways in a pool, a reply that comes back through
a different gateway than its request is still dropped (22/30 in a two-gateway lab run, 0/30 on main).
Fixing that means making both ends pick the same gateway, which is a routing change rather than a
conntrack one, so I've kept it out of this PR. Happy to open an issue for it if that helps.

@Lotarcc

Copy link
Copy Markdown
Contributor Author

Patrick W. Healy (@phealy) Cameron Childress (@cchildress) I checked this against #915-#918: it still merges cleanly, and since #917 the new source rules cover only the gateway's assigned pool. Tests pass on current main. Could one of you review it when you have time? #990 and #993 are rebased too and are waiting for a workflow run approval.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant