ipng/nginx-ipng-stats-plugin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Strip socket options on cross-cscf repeat listens (v0.7.2)

Browse Source 
Make the shared-listen-include pattern work with `reuseport` and the
other socket-level listen options. Nginx core enforces at-most-once
per sockaddr on options that set lsopt.set=1 (reuseport, bind,
backlog=, rcvbuf=, sndbuf=, setfib=, fastopen=, accept_filter=,
deferred, ipv6only=, so_keepalive=) and emits "duplicate listen
options for <addr>" otherwise. That rule collides with a single
listens.conf included from every vhost — each vhost's include
re-submits the same options.

The listen wrapper now detects the cross-cscf case, strips those
options from cf->args before delegating to the core handler, and
logs one notice per stripped listen. The first cscf owns the
options on the kernel socket; later cscfs merge cleanly via
ngx_http_add_server. Protocol-level flags (ssl, http2, quic,
proxy_protocol) pass through untouched since nginx OR-merges those
across cscfs.

This unblocks `reuseport` for deployments that want better
new-connection spread across workers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pim van Pelt
2026-04-19 16:23:58 +02:00
parent badb684431
commit 7ed77f5b22
8 changed files with 168 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
src/ngx_http_ipng_stats_module.c
View File

@@ -499,6 +499,50 @@ ngx_http_ipng_stats_preconfig(ngx_conf_t *cf)
}
/* Return 1 if `a` is a listen parameter that makes nginx set
* `lsopt.set = 1` (i.e. a socket-level option that nginx enforces
* at-most-once per sockaddr via its "duplicate listen options" check
* in ngx_http_add_addresses). Keep in sync with nginx core's
* ngx_http_core_listen — currently: bind, setfib=, fastopen=,
* backlog=, rcvbuf=, sndbuf=, accept_filter=, deferred, ipv6only=,
* reuseport, so_keepalive=. */
static ngx_uint_t
ngx_http_ipng_stats_arg_is_set_triggering(ngx_str_t *a)
{
static const struct { const char *s; size_t n; ngx_uint_t exact; } tbl[] = {
{ "bind", 4, 1 },
{ "deferred", 8, 1 },
{ "reuseport", 9, 1 },
{ "setfib=", 7, 0 },
{ "fastopen=", 9, 0 },
{ "backlog=", 8, 0 },
{ "rcvbuf=", 7, 0 },
{ "sndbuf=", 7, 0 },
{ "accept_filter=",14, 0 },
{ "ipv6only=", 9, 0 },
{ "so_keepalive=", 13, 0 },
};
ngx_uint_t k;
for (k = 0; k < sizeof(tbl) / sizeof(tbl[0]); k++) {
if (tbl[k].exact) {
if (a->len == tbl[k].n
&& ngx_strncmp(a->data, tbl[k].s, tbl[k].n) == 0)
{
return 1;
}
} else {
if (a->len > tbl[k].n
&& ngx_strncmp(a->data, tbl[k].s, tbl[k].n) == 0)
{
return 1;
}
}
}
return 0;
}
/* The wrapper extracts device= and ipng_source_tag= from cf->args, compacting
* the array in place, then calls the original ngx_http_core_module
* listen handler. After a successful call it records a binding in
@@ -580,24 +624,59 @@ ngx_http_ipng_stats_listen_wrapper(ngx_conf_t *cf, ngx_command_t *cmd,
* nginx's handler runs once (creating the socket), subsequent
* same-sockaddr listens just register additional device
* bindings. Across different server blocks the same sockaddr
* re-appears and nginx merges the cscf via ngx_http_add_server. */
* re-appears and nginx merges the cscf via ngx_http_add_server.
*
* The `sockaddr_seen_elsewhere` flag catches the
* shared-listen-include pattern where the SAME sockaddr appears
* under multiple server blocks: nginx rejects a second call
* carrying socket-level options (reuseport, bind, backlog=, ...)
* with "duplicate listen options". Those options belong to the
* single kernel socket that backs the sockaddr — the first cscf
* has already applied them; stripping them off subsequent
* same-sockaddr listens lets the include stay symmetrical. */
void *cscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_core_module);
ngx_http_ipng_stats_seen_t *seen = imcf->listens_seen->elts;
ngx_uint_t same_cscf_sockaddr = 0;
ngx_uint_t sockaddr_seen_elsewhere = 0;
for (i = 0; i < imcf->listens_seen->nelts; i++) {
if (seen[i].cscf != cscf) continue;
if (seen[i].socklen != u.addrs[0].socklen) continue;
if (ngx_cmp_sockaddr((struct sockaddr *) &seen[i].sockaddr,
seen[i].socklen,
u.addrs[0].sockaddr,
u.addrs[0].socklen, 1) == NGX_OK)
u.addrs[0].socklen, 1) != NGX_OK)
{
continue;
}
if (seen[i].cscf == cscf) {
same_cscf_sockaddr = 1;
break;
}
sockaddr_seen_elsewhere = 1;
}
if (!same_cscf_sockaddr) {
if (sockaddr_seen_elsewhere) {
ngx_uint_t stripped = 0;
i = 1;
while (i < cf->args->nelts) {
if (ngx_http_ipng_stats_arg_is_set_triggering(&value[i])) {
for (j = i; j + 1 < cf->args->nelts; j++) {
value[j] = value[j + 1];
}
cf->args->nelts--;
stripped = 1;
continue;
}
i++;
}
if (stripped) {
ngx_conf_log_error(NGX_LOG_NOTICE, cf, 0,
"ipng_stats: stripped socket options from "
"duplicate listen on %V — already applied by "
"an earlier server block", &value[1]);
}
}
rv = ngx_http_core_listen_orig(cf, cmd, conf);
if (rv != NGX_CONF_OK) {
return rv;