Count discarded POST bodies against bytes_in

Counter coverage for nginx_ipng_bytes_in_total was missing most of
the wire for rate-limited POST workloads. Observed on a CT-log-style
box: the plugin was reporting ~300 KB/s inbound while btop showed
6+ MB/s on the NIC, a ~20x gap. Root cause is in nginx itself: when
a request is rejected before a handler reads the body (limit_req
403, auth_request denial, 413 on fixed Content-Length, early
`return 4xx;`), nginx routes the body through
ngx_http_discard_request_body / HTTP/2 skip_data. Both paths drop
the bytes without ever incrementing r->request_length, so
$request_length (and therefore the plugin's bytes_in counter)
reflects only the request line and headers.

Log handler now adds r->headers_in.content_length_n to bin_sz when
r->discard_body is set and the client advertised a Content-Length.
That's a tight upper bound on the body bytes actually received —
abusive clients like the CT-log hammer case typically send the
full declared body before nginx's RST propagates. Normal 200 POSTs
are unchanged (they go through the buffered body path, which does
update r->request_length, and r->discard_body stays 0).

docs/nginx-issues.md catalogs the full analysis — which nginx paths
update r->request_length and which don't, how 413 differs between
fixed-CL and chunked and HTTP/2, why r->connection->sent is safe to
accumulate (it's reset per-request on keepalive and per-stream on
HTTP/2), and what the residual gap looks like after this fix
(HTTP/2 framing overhead, TLS/TCP, chunked requests with no
Content-Length header). Option B (per-connection recv wrapping) is
sketched there for later if the residual matters.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-22 09:57:45 +02:00
parent 96f77baacd
commit 05d405aba5
2 changed files with 133 additions and 0 deletions

View File

@@ -2065,6 +2065,17 @@ ngx_http_ipng_stats_log_handler(ngx_http_request_t *r)
bin_sz = r->request_length > 0 ? (uint64_t) r->request_length : 0;
bout_sz = (uint64_t) r->connection->sent;
/* When nginx rejects a request before any handler reads the body
* (rate-limit 403, auth_request denial, 413 on fixed Content-Length,
* early `return 4xx;` on POST, ...), the body goes through
* ngx_http_discard_request_body / HTTP/2 skip_data, which do NOT
* update r->request_length. If the client advertised a
* Content-Length, fall back to that as a tight upper bound on the
* bytes we actually received. See docs/nginx-issues.md. */
if (r->discard_body && r->headers_in.content_length_n > 0) {
bin_sz += (uint64_t) r->headers_in.content_length_n;
}
slot->requests += 1;
slot->bytes_in += bin_sz;
slot->bytes_out += bout_sz;