ipng/nginx-ipng-stats-plugin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix shared-listen-include pattern across multiple server blocks

Browse Source 
v0.3.0's listen wrapper treated every listen beyond the first at a
given sockaddr as a skip-core "duplicate", which was correct for
two `listen 80 device=X/Y;` lines in one server block but broke on
the real deployment pattern where every server-*.conf pulls in the
same `include listens.conf;`. Symptoms:

  * every server block after the first ended up with no listen
    directive processed, so nginx assigned them the default
    `*:80`, producing a flood of "conflicting server name"
    warnings and attaching every server block to an unrelated
    wildcard bind;
  * the bindings list grew linearly with the number of server
    blocks, so init_module tried to create (server_blocks) ×
    (devices × families) listening sockets and hit EMFILE.

Replace the single dedup with two independent checks:

  * listens_seen is a (cscf, sockaddr) ledger. The core listen
    handler is invoked at most once per (server block, sockaddr),
    matching nginx's own duplicate check so server-block N just
    attaches its cscf to the existing address via
    ngx_http_add_server.

  * `bind` is added only for the first global occurrence of each
    sockaddr; subsequent cscfs inherit opt.set/opt.bind from the
    first, which is what keeps nginx's "duplicate listen options"
    check happy across server blocks.

  * bindings dedup on (sockaddr, device) globally, so init_module
    creates one socket per unique pair regardless of how many
    server blocks reference it.

Add a regression test at tests/01-module/ that wires three server
blocks to the same ipng-listens.inc and asserts that nginx -t is
clean and exactly four sockets are bound on port 8080.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pim van Pelt
2026-04-18 13:15:04 +02:00
parent ef821e577b
commit 1f144f4c19
5 changed files with 169 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
tests/01-module/01-e2e.robot
View File

@@ -29,6 +29,23 @@ Module loads
${output} = Docker Exec ${SERVER} nginx -t 2>&1
Should Contain ${output} syntax is ok
Shared-listen-include across multiple server blocks
[Documentation] Three server blocks all pull in the same
... ipng-listens.inc (see docs/user-guide.md). nginx
... must start without "conflicting server name" or
... "duplicate listen options" warnings, and the
... module must end up with exactly one listening
... socket per (device, family) pair — not one per
... (server block × device × family), which would
... exhaust the fd table on a real host.
${output} = Docker Exec ${SERVER} nginx -t 2>&1
Should Not Contain ${output} conflicting server name
Should Not Contain ${output} duplicate listen
${listens} = Docker Exec ${SERVER} ss -tlnH
${count} = Get Regexp Matches ${listens} :8080\\s
Length Should Be ${count} 4
... Expected 4 listening sockets on port 8080 (v4+v6 × eth1+eth2); got ${count}
Prometheus scrape
[Documentation] Scrape returns HELP/TYPE preamble.
${output} = Scrape Prometheus

1
tests/01-module/lab/ipng-stats.clab.yml
View File

@@ -26,6 +26,7 @@ topology:
binds:
- ../../../build:/opt/build:ro
- ./server/nginx.conf:/opt/config/nginx.conf:ro
- ./server/ipng-listens.inc:/opt/config/ipng-listens.inc:ro
- ./server/slow-backend.py:/opt/config/slow-backend.py:ro
- ./server/start.sh:/start.sh:ro
cmd: bash /start.sh

12
tests/01-module/lab/server/ipng-listens.inc Normal file
View File

@@ -0,0 +1,12 @@
# SPDX-License-Identifier: Apache-2.0
# Shared listen include — matches the deployment pattern from
# docs/user-guide.md, where every server block pulls in the same set of
# device-tagged wildcard listens. The 01-module suite includes this
# file from multiple server blocks to exercise the wrapper's dedup
# logic: a naive implementation would either error with "duplicate
# listen options" or create N * (devices × families) sockets.
listen 8080 device=eth1 ipng_source_tag=tag1;
listen [::]:8080 device=eth1 ipng_source_tag=tag1;
listen 8080 device=eth2 ipng_source_tag=tag2-v4;
listen [::]:8080 device=eth2 ipng_source_tag=tag2-v6;

35
tests/01-module/lab/server/nginx.conf
View File

@@ -41,30 +41,45 @@ http {
'$ipng_source_tag\t$server_addr\t$scheme';
ipng_stats_logtail ipng_stats_logtail udp://127.0.0.1:9514 buffer=4k flush=500ms if=$logtail_enabled;
# Three server blocks that all pull in the same listen include —
# mirrors the real-world pattern where every site-*.conf has the
# same `include listens.conf;`. The wrapper must:
# * invoke nginx's listen handler exactly once per (server, addr)
# pair, so each server block gets its own cscf attached but no
# server block triggers nginx's "duplicate listen options"
# check;
# * dedup bindings globally on (sockaddr, device), so init_module
# creates exactly four sockets here (two families × two
# devices) rather than 3 × 4 = 12.
# The default server owns the locations used by the traffic tests;
# the two extras exist only to exercise the shared-include pattern.
server {
# Per-device wildcard listens. All four share port 8080; the
# kernel's SO_BINDTODEVICE filtering routes each incoming packet
# to the socket pinned to the interface it arrived on.
listen 8080 device=eth1 ipng_source_tag=tag1;
listen [::]:8080 device=eth1 ipng_source_tag=tag1;
listen 8080 device=eth2 ipng_source_tag=tag2-v4;
listen [::]:8080 device=eth2 ipng_source_tag=tag2-v6;
include /opt/config/ipng-listens.inc;
server_name _;
location / {
return 200 "ok $server_addr\n";
}
location /notfound {
return 404 "nope\n";
}
location /slow {
proxy_pass http://127.0.0.1:29080/;
}
}
server {
include /opt/config/ipng-listens.inc;
server_name extra-a.test;
location / { return 200 "a\n"; }
}
server {
include /opt/config/ipng-listens.inc;
server_name extra-b.test;
location / { return 200 "b\n"; }
}
server {
# Direct (mgmt) traffic: no device binding on the listen,
# `ipng_stats_default_source direct;` therefore tags it "direct".