Support multiple device-pinned listens sharing a single port

Nginx's config-level duplicate-listen check rejected the
documented pattern of `listen 80 device=X ipng_source_tag=A;
listen 80 device=Y ipng_source_tag=B;` with "a duplicate listen
0.0.0.0:80", and even when the dedup was bypassed the kernel
refused the second bind() because the first socket was already
holding the port without SO_BINDTODEVICE.

The listen wrapper now detects same-sockaddr duplicates before
the core handler sees them and records them with `needs_clone=1`.
In init_module, phase 1 clones an ngx_listening_t for each such
duplicate, phase 3 closes every inherited naked fd, and phase 4
rebinds every target with SO_REUSEADDR + SO_REUSEPORT +
SO_BINDTODEVICE set before bind(). SO_REUSEPORT keeps
`nginx -s reload` from colliding with the still-bound sockets
held by old workers during graceful drain; IPV6_V6ONLY matches
nginx's default so the IPv6 listen doesn't claim the IPv4
wildcard and collide with sibling IPv4-specific listens.

Restructure 01-module to cover the pattern end-to-end: four
device-pinned listens on port 8080 (eth1 shares tag `tag1`
across v4 and v6; eth2 splits into `tag2-v4` / `tag2-v6`),
clients and server both get IPv6 addresses, and a new
"Per-(device, family) request count accuracy" case proves that
10 requests on each of the four combinations yields tag1=20,
tag2-v4=10, tag2-v6=10. Mgmt/direct traffic moves to port 9180
so it no longer clashes with the shared-port wildcards.

Document the constraint in docs/user-guide.md: all listens on
a given port must carry `device=`, and direct traffic belongs
on a separate port.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-18 11:45:40 +02:00
parent fdef2a552b
commit df05bae8a3
7 changed files with 400 additions and 153 deletions

View File

@@ -3,12 +3,13 @@
#
# Three nodes:
# server — nginx with the module, a slow Python backend, two data-plane interfaces
# client1 — sends traffic via eth1 (attributed to source_tag=cl1)
# client2 — sends traffic via eth2 (attributed to source_tag=cl2)
# client1 — sends traffic via eth1 (attributed to source_tag=tag1 for both v4 and v6)
# client2 — sends traffic via eth2 (attributed to source_tag=tag2-v4 for v4,
# tag2-v6 for v6 — demonstrates per-(device, family) attribution)
#
# Links:
# server:eth1 ←→ client1:eth1 (10.0.1.0/24)
# server:eth2 ←→ client2:eth1 (10.0.2.0/24)
# Links (each carries a /24 and a /64):
# server:eth1 ←→ client1:eth1 (10.0.1.0/24 + 2001:db8:1::/64)
# server:eth2 ←→ client2:eth1 (10.0.2.0/24 + 2001:db8:2::/64)
name: ipng-stats-test
@@ -38,6 +39,7 @@ topology:
cmd: bash /start.sh
env:
MY_IP: 10.0.1.2/24
MY_IP6: 2001:db8:1::2/64
client2:
kind: linux
@@ -48,6 +50,7 @@ topology:
cmd: bash /start.sh
env:
MY_IP: 10.0.2.2/24
MY_IP6: 2001:db8:2::2/64
links:
- endpoints: ["server:eth1", "client1:eth1"]