Allow plain and device-tagged listens to share a sockaddr (v0.7.1)

The previous wrapper skipped nginx's duplicate-listen check only
for listens that carried device=, so a `listen 80;` next to a
`listen 80 device=eth0 ...;` in the same server block was
rejected at config time. Under SO_BINDTODEVICE that restriction
tracked a real kernel constraint (device-tagged listens created
separate sockets, a bare listen alongside them was genuinely
ambiguous). Under the IP_PKTINFO model introduced in 450391a
the constraint no longer exists — all same-sockaddr listens
collapse to one wildcard kernel socket and attribution is a
per-connection cmsg readback — but the wrapper kept enforcing
the old rule by accident.

Extend the (cscf, sockaddr) dedup in the listen wrapper to
cover plain listens too: the first occurrence at a given
(server, sockaddr) pair calls nginx's handler and registers the
kernel socket, and every subsequent sibling — plain or
device-tagged — is accepted without tripping nginx's
duplicate-listen check. Device-tagged siblings additionally
push a binding into the attribution table as before; plain
siblings contribute only the seen-list entry. No code path
exercised by the existing 22 e2e tests changes behavior.

Update FR-1.5, the user-guide "shared port" section, the
module's top-of-function comments, and the test nginx.conf
comment to describe the relaxed rule. Bump VERSION and add a
debian/changelog entry for 0.7.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-19 09:17:44 +02:00
parent 8e0b1cdde9
commit badb684431
6 changed files with 77 additions and 51 deletions

View File

@@ -23,7 +23,7 @@ BUILD_DIR := $(CURDIR)/build
# the package version from there directly. The C code picks up VERSION # the package version from there directly. The C code picks up VERSION
# via the generated src/version.h (written by the version-header target # via the generated src/version.h (written by the version-header target
# below and depended on by the module build). # below and depended on by the module build).
VERSION := 0.7.0 VERSION := 0.7.1
NGINX_SRC ?= NGINX_SRC ?=

13
debian/changelog vendored
View File

@@ -1,3 +1,16 @@
nginx-ipng-stats-plugin (0.7.1-1) unstable; urgency=medium
* Pre-release v0.7.1.
- Listen wrapper now deduplicates on (server block, sockaddr)
across both plain and device-tagged listens. A plain
`listen 80;` and `listen 80 device=eth0 ...;` in the same
server block now coexist, matching the runtime reality under
the IP_PKTINFO attribution model (all same-sockaddr listens
collapse to one wildcard kernel socket). Docs updated to
drop the "must be device-tagged" restriction.
-- Pim van Pelt <pim@ipng.ch> Sun, 19 Apr 2026 10:00:00 +0200
nginx-ipng-stats-plugin (0.7.0-1) unstable; urgency=medium nginx-ipng-stats-plugin (0.7.0-1) unstable; urgency=medium
* Pre-release v0.7.0. * Pre-release v0.7.0.

View File

@@ -84,10 +84,12 @@ Each requirement carries a unique identifier (`FR-X.Y` or `NFR-X.Y`) so that lat
- **FR-1.3** A listening socket with neither `device=` nor `ipng_source_tag=` MUST be tagged with the configured default source string (see - **FR-1.3** A listening socket with neither `device=` nor `ipng_source_tag=` MUST be tagged with the configured default source string (see
`ipng_stats_default_source`, FR-5.3). The default default is the literal string `direct`. `ipng_stats_default_source`, FR-5.3). The default default is the literal string `direct`.
- **FR-1.4** A listening socket with `device=X` but no `ipng_source_tag=` MUST be tagged with the interface name `X`. - **FR-1.4** A listening socket with `device=X` but no `ipng_source_tag=` MUST be tagged with the interface name `X`.
- **FR-1.5** Two `listen` directives that share `address:port` but differ in `device=` MUST coexist. Since no `SO_BINDTODEVICE` - **FR-1.5** Two or more `listen` directives sharing `address:port` MUST coexist regardless of whether each carries `device=`.
is applied, the kernel delivers all matching SYNs to a single wildcard listening socket and the module distinguishes them by Since no `SO_BINDTODEVICE` is applied, the kernel delivers all matching SYNs to a single wildcard listening socket and the
reading `ifindex` from the per-connection cmsg — so "multiple device-tagged listens at the same port" at the config level module distinguishes them by reading `ifindex` from the per-connection cmsg. The listen wrapper therefore deduplicates on
collapses to one kernel socket at runtime without any userspace contortions. `(server block, sockaddr)` across both plain and device-tagged listens: the first occurrence registers the kernel socket,
and subsequent same-sockaddr siblings (plain or device-tagged) are accepted without tripping nginx's duplicate-listen check.
Device-tagged siblings additionally register an entry in the attribution table.
- **FR-1.6** A `listen` directive that uses a wildcard address (`80`, `[::]:80`) together with `device=<ifname>` MUST attribute - **FR-1.6** A `listen` directive that uses a wildcard address (`80`, `[::]:80`) together with `device=<ifname>` MUST attribute
every connection whose ingress interface is `<ifname>` — regardless of which local address the client addressed — to that every connection whose ingress interface is `<ifname>` — regardless of which local address the client addressed — to that
listen's source tag. Traffic on other interfaces MUST fall back to the configured default source (see FR-1.3). listen's source tag. Traffic on other interfaces MUST fall back to the configured default source (see FR-1.3).

View File

@@ -112,8 +112,9 @@ http {
ipng_stats_flush_interval 1s; ipng_stats_flush_interval 1s;
ipng_stats_default_source direct; ipng_stats_default_source direct;
# Attributed vhost. Every listen on this port must be device-tagged — # Attributed vhost. Wildcard listens below register one binding
# see "All listens on a shared port must be device-tagged" below. # per (device, family); all collapse to a single kernel socket
# under the IP_PKTINFO attribution model.
server { server {
include /etc/nginx/ipng-stats/listens.conf; include /etc/nginx/ipng-stats/listens.conf;
@@ -176,22 +177,14 @@ You do not need to enumerate VIPs in `listen`. A wildcard `listen 80 device=gre-
served through the `gre-mg1` interface, and nginx routes per-request to the right vhost by `server_name` / `Host:` header. Adding a new served through the `gre-mg1` interface, and nginx routes per-request to the right vhost by `server_name` / `Host:` header. Adding a new
VIP is a `server_name` change; adding a new interface is an append to `listens.conf`. VIP is a `server_name` change; adding a new interface is an append to `listens.conf`.
### All listens on a shared port must be device-tagged
If you use multiple `listen` directives on the same port (e.g. port 80), **every one of them must carry `device=<ifname>`**. Mixing
a device-tagged listen with a plain `listen 80;` or with an address-specific `listen 192.0.2.1:80;` on the same port is **not
supported** — nginx's config-level dedup rejects same-sockaddr listens within a server block, and the module's wrapper only
exempts directives that carry `device=`.
For "direct" traffic — clients hitting the host on a non-attributed interface — use a **separate port** on the direct interface
(e.g. `listen 198.51.100.1:8081;`). That listen then has no `device=`, so it falls back to the tag set by
`ipng_stats_default_source` (`direct` by default).
### Sharing a single port across address families and devices ### Sharing a single port across address families and devices
Within the device-tagged set, you're free to share port numbers freely across devices and address families: as long as each listen Under the `IP_PKTINFO` attribution model, all listens at a given sockaddr collapse to a single wildcard kernel socket at runtime —
has a distinct `device=`, the kernel keeps them apart, and within one device you can either reuse a single tag or split by family. the kernel stamps every accepted connection with its ingress ifindex, and the module looks that up in the table of `device=`
For example: bindings registered by the listen wrapper. Multiple device-tagged wildcard listens on port 80 are therefore not "multiple
sockets"; they're one wildcard socket plus N entries in the attribution table.
A device can reuse one tag across address families or split into per-family tags — whichever reads better in the scrape output:
```nginx ```nginx
listen 80 device=gre-mg1 ipng_source_tag=mg1; listen 80 device=gre-mg1 ipng_source_tag=mg1;
@@ -200,6 +193,12 @@ listen 80 device=gre-mg2 ipng_source_tag=mg2-v4;
listen [::]:80 device=gre-mg2 ipng_source_tag=mg2-v6; # per-family tags listen [::]:80 device=gre-mg2 ipng_source_tag=mg2-v6; # per-family tags
``` ```
A plain `listen 80;` can sit alongside device-tagged listens in the same server block; the wrapper treats the first occurrence
at a given `(server, sockaddr)` pair as the one that registers the kernel socket and lets subsequent device-tagged siblings
register bindings without tripping nginx's duplicate-listen check. Traffic arriving on an interface that has no binding falls
back to `ipng_stats_default_source` (`direct` by default). Keeping "direct" traffic on its own port — e.g.
`listen 198.51.100.1:8081;` — remains a fine pattern when you want a hard split, but it's no longer required.
## 4. Verify with curl ## 4. Verify with curl
Generate some traffic (or wait for real traffic), then scrape the endpoint locally: Generate some traffic (or wait for real traffic), then scrape the endpoint locally:

View File

@@ -214,11 +214,16 @@ typedef struct {
/* Per-(cscf, sockaddr) tracking used only during config parse. The /* Per-(cscf, sockaddr) tracking used only during config parse. The
* listen wrapper uses it to avoid invoking the core `listen` handler * listen wrapper uses it to avoid invoking the core `listen` handler
* twice for the same (server block, sockaddr) pair — a valid config * twice for the same (server block, sockaddr) pair — a valid config
* with N device-tagged listens on the same port in the same server * with N listens on the same port in the same server block generates
* block generates exactly one core-handler call and one cscf * exactly one core-handler call and one cscf attachment, regardless
* attachment, with the remaining listens handled by init_module * of whether those listens are plain or device-tagged. Under the
* cloning. Across server blocks the same sockaddr legitimately recurs, * IP_PKTINFO attribution model all listens at a given sockaddr
* and each distinct cscf gets its own core-handler call. */ * collapse to a single wildcard kernel socket at runtime, so mixing
* a plain `listen 80;` with `listen 80 device=eth0 ...;` in one
* server block is legitimate and the wrapper must not let nginx's
* own same-sockaddr dedup reject it. Across server blocks the same
* sockaddr legitimately recurs, and each distinct cscf gets its own
* core-handler call. */
typedef struct { typedef struct {
void *cscf; /* ngx_http_core_srv_conf_t * */ void *cscf; /* ngx_http_core_srv_conf_t * */
ngx_sockaddr_t sockaddr; ngx_sockaddr_t sockaddr;
@@ -544,15 +549,10 @@ ngx_http_ipng_stats_listen_wrapper(ngx_conf_t *cf, ngx_command_t *cmd,
i++; i++;
} }
if (device.len == 0 && source.len == 0) { /* Parse the listen address so we can dedup per (cscf, sockaddr)
/* Plain listen with no module-specific parameters — let nginx * across both plain and device-tagged listens. If the address
* handle it end-to-end. */ * doesn't parse, fall back to nginx's handler and let it report
return ngx_http_core_listen_orig(cf, cmd, conf); * the error. */
}
/* Parse the listen address ourselves so we can dedup per (cscf,
* sockaddr) — nginx's core handler rejects the same sockaddr
* appearing twice in the same server block. */
ngx_memzero(&u, sizeof(ngx_url_t)); ngx_memzero(&u, sizeof(ngx_url_t));
u.url = value[1]; u.url = value[1];
u.listen = 1; u.listen = 1;
@@ -564,13 +564,6 @@ ngx_http_ipng_stats_listen_wrapper(ngx_conf_t *cf, ngx_command_t *cmd,
imcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_ipng_stats_module); imcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_ipng_stats_module);
if (imcf->bindings == NULL) {
imcf->bindings = ngx_array_create(cf->pool, 8,
sizeof(ngx_http_ipng_stats_binding_t));
if (imcf->bindings == NULL) {
return NGX_CONF_ERROR;
}
}
if (imcf->listens_seen == NULL) { if (imcf->listens_seen == NULL) {
imcf->listens_seen = ngx_array_create(cf->pool, 16, imcf->listens_seen = ngx_array_create(cf->pool, 16,
sizeof(ngx_http_ipng_stats_seen_t)); sizeof(ngx_http_ipng_stats_seen_t));
@@ -580,11 +573,14 @@ ngx_http_ipng_stats_listen_wrapper(ngx_conf_t *cf, ngx_command_t *cmd,
} }
/* Skip the core handler when this (cscf, sockaddr) pair was /* Skip the core handler when this (cscf, sockaddr) pair was
* already processed — matches nginx's own "duplicate listen" * already processed — all listens at a given sockaddr (plain or
* check and lets a server block carry multiple device-tagged * device-tagged) collapse to one kernel socket under the
* listens at the same port. Across different server blocks the * IP_PKTINFO model, so a plain `listen 80;` and `listen 80
* same sockaddr re-appears and nginx merges the cscf via * device=eth0 ...;` in the same server block are compatible:
* ngx_http_add_server. */ * nginx's handler runs once (creating the socket), subsequent
* same-sockaddr listens just register additional device
* bindings. Across different server blocks the same sockaddr
* re-appears and nginx merges the cscf via ngx_http_add_server. */
void *cscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_core_module); void *cscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_core_module);
ngx_http_ipng_stats_seen_t *seen = imcf->listens_seen->elts; ngx_http_ipng_stats_seen_t *seen = imcf->listens_seen->elts;
ngx_uint_t same_cscf_sockaddr = 0; ngx_uint_t same_cscf_sockaddr = 0;
@@ -614,6 +610,22 @@ ngx_http_ipng_stats_listen_wrapper(ngx_conf_t *cf, ngx_command_t *cmd,
ngx_memcpy(&s->sockaddr, u.addrs[0].sockaddr, u.addrs[0].socklen); ngx_memcpy(&s->sockaddr, u.addrs[0].sockaddr, u.addrs[0].socklen);
} }
/* Plain listens contribute no attribution binding — they only
* populate listens_seen so later device-tagged siblings on the
* same sockaddr aren't rejected by nginx's duplicate-listen
* check. */
if (device.len == 0 && source.len == 0) {
return NGX_CONF_OK;
}
if (imcf->bindings == NULL) {
imcf->bindings = ngx_array_create(cf->pool, 8,
sizeof(ngx_http_ipng_stats_binding_t));
if (imcf->bindings == NULL) {
return NGX_CONF_ERROR;
}
}
/* Dedup bindings on (device, family) pair. Same device appearing /* Dedup bindings on (device, family) pair. Same device appearing
* under multiple server blocks (because they share a listen * under multiple server blocks (because they share a listen
* include) collapses to a single binding per family. The same * include) collapses to a single binding per family. The same

View File

@@ -8,10 +8,10 @@
# can verify that the module can either combine or distinguish # can verify that the module can either combine or distinguish
# families per device. # families per device.
# #
# Mgmt/direct traffic hits a separate server block on port 9180. # Mgmt/direct traffic hits a separate server block on port 9180
# Mixing a naked `listen 8080;` or a specific-address `listen # a clean port split rather than a technical requirement; plain and
# 172.20.40.2:8080;` with device-tagged wildcards on the same port # device-tagged listens may share a port under the IP_PKTINFO model
# is not supported — see docs/user-guide.md. # (see docs/user-guide.md).
load_module /usr/lib/nginx/modules/ngx_http_ipng_stats_module.so; load_module /usr/lib/nginx/modules/ngx_http_ipng_stats_module.so;