Support multiple device-pinned listens sharing a single port

Nginx's config-level duplicate-listen check rejected the
documented pattern of `listen 80 device=X ipng_source_tag=A;
listen 80 device=Y ipng_source_tag=B;` with "a duplicate listen
0.0.0.0:80", and even when the dedup was bypassed the kernel
refused the second bind() because the first socket was already
holding the port without SO_BINDTODEVICE.

The listen wrapper now detects same-sockaddr duplicates before
the core handler sees them and records them with `needs_clone=1`.
In init_module, phase 1 clones an ngx_listening_t for each such
duplicate, phase 3 closes every inherited naked fd, and phase 4
rebinds every target with SO_REUSEADDR + SO_REUSEPORT +
SO_BINDTODEVICE set before bind(). SO_REUSEPORT keeps
`nginx -s reload` from colliding with the still-bound sockets
held by old workers during graceful drain; IPV6_V6ONLY matches
nginx's default so the IPv6 listen doesn't claim the IPv4
wildcard and collide with sibling IPv4-specific listens.

Restructure 01-module to cover the pattern end-to-end: four
device-pinned listens on port 8080 (eth1 shares tag `tag1`
across v4 and v6; eth2 splits into `tag2-v4` / `tag2-v6`),
clients and server both get IPv6 addresses, and a new
"Per-(device, family) request count accuracy" case proves that
10 requests on each of the four combinations yields tag1=20,
tag2-v4=10, tag2-v6=10. Mgmt/direct traffic moves to port 9180
so it no longer clashes with the shared-port wildcards.

Document the constraint in docs/user-guide.md: all listens on
a given port must carry `device=`, and direct traffic belongs
on a separate port.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-18 11:45:40 +02:00
parent fdef2a552b
commit df05bae8a3
7 changed files with 400 additions and 153 deletions

View File

@@ -1,5 +1,17 @@
# SPDX-License-Identifier: Apache-2.0
# Test nginx configuration for the ipng_stats module.
#
# Data plane (port 8080) uses four wildcard listens — two address
# families × two devices — to exercise per-(device, family)
# attribution. eth1 uses the same tag (`tag1`) for IPv4 and IPv6,
# while eth2 splits them (`tag2-v4` / `tag2-v6`) so the e2e suite
# can verify that the module can either combine or distinguish
# families per device.
#
# Mgmt/direct traffic hits a separate server block on port 9180.
# Mixing a naked `listen 8080;` or a specific-address `listen
# 172.20.40.2:8080;` with device-tagged wildcards on the same port
# is not supported — see docs/user-guide.md.
load_module /usr/lib/nginx/modules/ngx_http_ipng_stats_module.so;
@@ -30,12 +42,13 @@ http {
ipng_stats_logtail ipng_stats_logtail udp://127.0.0.1:9514 buffer=4k flush=500ms if=$logtail_enabled;
server {
# Mgmt-only listener for direct traffic (tagged "direct").
listen 172.20.40.2:8080;
# Per-interface listeners for attributed traffic.
listen 10.0.1.1:8080 device=eth1 ipng_source_tag=cl1;
listen 10.0.2.1:8080 device=eth2 ipng_source_tag=cl2;
# Per-device wildcard listens. All four share port 8080; the
# kernel's SO_BINDTODEVICE filtering routes each incoming packet
# to the socket pinned to the interface it arrived on.
listen 8080 device=eth1 ipng_source_tag=tag1;
listen [::]:8080 device=eth1 ipng_source_tag=tag1;
listen 8080 device=eth2 ipng_source_tag=tag2-v4;
listen [::]:8080 device=eth2 ipng_source_tag=tag2-v6;
server_name _;
@@ -52,6 +65,19 @@ http {
}
}
server {
# Direct (mgmt) traffic: no device binding on the listen,
# `ipng_stats_default_source direct;` therefore tags it "direct".
# Separate port so it doesn't collide with the device-tagged
# wildcards above.
listen 172.20.40.2:9180;
server_name _;
location / {
return 200 "ok direct\n";
}
}
server {
listen 172.20.40.2:9113;