PRE-RELEASE v0.8.2
This commit is contained in:
2
Makefile
2
Makefile
@@ -23,7 +23,7 @@ BUILD_DIR := $(CURDIR)/build
|
|||||||
# the package version from there directly. The C code picks up VERSION
|
# the package version from there directly. The C code picks up VERSION
|
||||||
# via the generated src/version.h (written by the version-header target
|
# via the generated src/version.h (written by the version-header target
|
||||||
# below and depended on by the module build).
|
# below and depended on by the module build).
|
||||||
VERSION := 0.7.2
|
VERSION := 0.8.2
|
||||||
|
|
||||||
NGINX_SRC ?=
|
NGINX_SRC ?=
|
||||||
|
|
||||||
|
|||||||
@@ -7,10 +7,10 @@ nginx on Debian Trixie.
|
|||||||
The module attributes every HTTP request to the interface it arrived on, reading the ingress `ifindex` per connection from the
|
The module attributes every HTTP request to the interface it arrived on, reading the ingress `ifindex` per connection from the
|
||||||
kernel's `IP_PKTINFO` / `IPV6_PKTINFO` cmsg. Listening sockets stay plain wildcards, so outgoing packets follow the normal
|
kernel's `IP_PKTINFO` / `IPV6_PKTINFO` cmsg. Listening sockets stay plain wildcards, so outgoing packets follow the normal
|
||||||
routing table — which is what makes this safe for DSR / maglev deployments where the SYN arrives via a GRE tunnel and the
|
routing table — which is what makes this safe for DSR / maglev deployments where the SYN arrives via a GRE tunnel and the
|
||||||
SYN-ACK must leave via the default route. Counters — requests, status codes, bytes, latency histograms — are exposed as
|
SYN-ACK must leave via the default route. Counters — requests, status codes, bytes, latency histograms — plus point-in-time
|
||||||
Prometheus text or JSON from a single HTTP scrape endpoint, filtered per-source. This is useful for any deployment where
|
gauges of requests currently in flight (`active`, `reading`, `writing`) are exposed as Prometheus text or JSON from a single
|
||||||
traffic arrives on distinct interfaces — GRE tunnels, VLANs, bonded links, or plain ethernet — and per-interface observability
|
HTTP scrape endpoint, filtered per-source. This is useful for any deployment where traffic arrives on distinct interfaces —
|
||||||
is needed.
|
GRE tunnels, VLANs, bonded links, or plain ethernet — and per-interface observability is needed.
|
||||||
|
|
||||||
Without any `device=`/`ipng_source_tag=` parameters, the module still counts and exposes per-VIP traffic under the configurable
|
Without any `device=`/`ipng_source_tag=` parameters, the module still counts and exposes per-VIP traffic under the configurable
|
||||||
default source tag (`direct`), which makes it a useful plain observability module for any nginx host.
|
default source tag (`direct`), which makes it a useful plain observability module for any nginx host.
|
||||||
|
|||||||
19
debian/changelog
vendored
19
debian/changelog
vendored
@@ -1,3 +1,22 @@
|
|||||||
|
nginx-ipng-stats-plugin (0.8.2-1) unstable; urgency=medium
|
||||||
|
|
||||||
|
* Pre-release v0.8.2.
|
||||||
|
- New per (source_tag, vip) in-flight gauges:
|
||||||
|
nginx_ipng_active, nginx_ipng_reading, nginx_ipng_writing.
|
||||||
|
Lifecycle: POST_READ handler increments active+reading
|
||||||
|
on each main request, a header filter transitions
|
||||||
|
reading->writing when headers are sent, and a pool
|
||||||
|
cleanup decrements on request finalization. Gauges live
|
||||||
|
in a dedicated rbtree in the shared zone; the slab mutex
|
||||||
|
is taken only on first insert per (source, vip) pair —
|
||||||
|
subsequent transitions are lock-free atomic inc/dec on
|
||||||
|
the cached node. Subrequests and internal redirects do
|
||||||
|
not double-count. Gauges are emitted in both Prometheus
|
||||||
|
(`gauge` typed) and JSON (`"gauges":{...}` per record)
|
||||||
|
scrape outputs. Invariant: reading + writing = active.
|
||||||
|
|
||||||
|
-- Pim van Pelt <pim@ipng.ch> Thu, 23 Apr 2026 12:00:00 +0200
|
||||||
|
|
||||||
nginx-ipng-stats-plugin (0.7.2-1) unstable; urgency=medium
|
nginx-ipng-stats-plugin (0.7.2-1) unstable; urgency=medium
|
||||||
|
|
||||||
* Pre-release v0.7.2.
|
* Pre-release v0.7.2.
|
||||||
|
|||||||
@@ -128,6 +128,12 @@ Each requirement carries a unique identifier (`FR-X.Y` or `NFR-X.Y`) so that lat
|
|||||||
codes are observed. Operators who need a full per-code breakdown SHOULD enable `ipng_stats_logtail` (FR-8) and derive the per-code
|
codes are observed. Operators who need a full per-code breakdown SHOULD enable `ipng_stats_logtail` (FR-8) and derive the per-code
|
||||||
view from the access-log stream off the hot path; the stats zone intentionally trades that resolution for a much smaller scrape
|
view from the access-log stream off the hot path; the stats zone intentionally trades that resolution for a much smaller scrape
|
||||||
response.
|
response.
|
||||||
|
- **FR-2.7** The module MUST additionally maintain, per `(source, vip)` pair, three point-in-time gauges of requests currently in
|
||||||
|
flight: `active` (observed at `POST_READ` but not yet finalized), `reading` (in the pre-response phases — rewrite/access/content),
|
||||||
|
and `writing` (past the header-send transition). The invariant `reading + writing = active` MUST hold at any instant. Subrequests
|
||||||
|
and internal redirects MUST NOT double-count the parent request. Updates to the gauges are atomic increments/decrements on the
|
||||||
|
request lifecycle hooks — no slab lock after the first time a `(source, vip)` pair is seen — so the hot-path rule in FR-4.1 still
|
||||||
|
holds for ordinary counter updates while gauges are maintained lock-free.
|
||||||
|
|
||||||
**FR-3 Scrape endpoint**
|
**FR-3 Scrape endpoint**
|
||||||
|
|
||||||
@@ -147,6 +153,8 @@ Each requirement carries a unique identifier (`FR-X.Y` or `NFR-X.Y`) so that lat
|
|||||||
`source_tag` and `vip`. Counter metrics (`nginx_ipng_requests_total`, `nginx_ipng_bytes_{in,out}_total`, `nginx_ipng_latency_total`)
|
`source_tag` and `vip`. Counter metrics (`nginx_ipng_requests_total`, `nginx_ipng_bytes_{in,out}_total`, `nginx_ipng_latency_total`)
|
||||||
additionally carry a `code` label with a class value (`1xx`..`5xx`/`unknown`). Histogram series (duration, upstream response,
|
additionally carry a `code` label with a class value (`1xx`..`5xx`/`unknown`). Histogram series (duration, upstream response,
|
||||||
request/response byte size) MUST NOT carry a `code` label — they aggregate across all classes for a given `(source, vip)` pair.
|
request/response byte size) MUST NOT carry a `code` label — they aggregate across all classes for a given `(source, vip)` pair.
|
||||||
|
Gauge series (`nginx_ipng_active`, `nginx_ipng_reading`, `nginx_ipng_writing`) MUST be labelled with `source_tag` and `vip` only
|
||||||
|
(no `code`) and MUST be typed as `gauge` in the exposition preamble.
|
||||||
|
|
||||||
**FR-4 Hot path and flush**
|
**FR-4 Hot path and flush**
|
||||||
|
|
||||||
@@ -407,6 +415,12 @@ Histogram lanes are kept per `(source, vip, class)` in storage, then summed acro
|
|||||||
A parallel table keyed by `(source_id, vip_id)` — one row per VIP — holds the EWMAs for instantaneous rate. EWMAs are floats but updated
|
A parallel table keyed by `(source_id, vip_id)` — one row per VIP — holds the EWMAs for instantaneous rate. EWMAs are floats but updated
|
||||||
only from the flush tick, so there is no float contention on the request path.
|
only from the flush tick, so there is no float contention on the request path.
|
||||||
|
|
||||||
|
A second, smaller rbtree lives alongside the counter tree — one node per `(source_id, vip_id)` pair — holding three atomic gauge
|
||||||
|
lanes (`active`, `reading`, `writing`; FR-2.7). Unlike the counter path, gauges are updated from request lifecycle hooks
|
||||||
|
(`POST_READ`, header filter, pool cleanup) with atomic inc/dec directly on the shared node. The slab mutex is taken only the first
|
||||||
|
time a `(source, vip)` pair is seen; subsequent transitions on that pair are lock-free. Gauge nodes are never evicted — their
|
||||||
|
cardinality equals the number of distinct `(source, vip)` pairs and is small in practice.
|
||||||
|
|
||||||
The module also keeps a small string interning table for source and VIP strings, keyed by the integer IDs above, so that the scrape
|
The module also keeps a small string interning table for source and VIP strings, keyed by the integer IDs above, so that the scrape
|
||||||
endpoint can recover the original strings without re-parsing configuration.
|
endpoint can recover the original strings without re-parsing configuration.
|
||||||
|
|
||||||
|
|||||||
@@ -249,6 +249,14 @@ nginx_ipng_bytes_in_total{source_tag="mg1",vip="192.0.2.10",code="2xx"} 9876543
|
|||||||
# Histogram series (request_duration, upstream_response, bytes_in, bytes_out)
|
# Histogram series (request_duration, upstream_response, bytes_in, bytes_out)
|
||||||
# do NOT carry a `code` label — they aggregate across classes per (source, vip).
|
# do NOT carry a `code` label — they aggregate across classes per (source, vip).
|
||||||
nginx_ipng_request_duration_seconds_bucket{source_tag="mg1",vip="192.0.2.10",le="0.050"} 11200
|
nginx_ipng_request_duration_seconds_bucket{source_tag="mg1",vip="192.0.2.10",le="0.050"} 11200
|
||||||
|
|
||||||
|
# In-flight gauges per (source, vip). These are point-in-time request counts,
|
||||||
|
# not rates: `active` = requests observed at POST_READ that haven't finalized
|
||||||
|
# yet; `reading` = in pre-response phases (rewrite/access/content); `writing`
|
||||||
|
# = past header send. reading + writing = active at any instant.
|
||||||
|
nginx_ipng_active{source_tag="mg1",vip="192.0.2.10"} 3
|
||||||
|
nginx_ipng_reading{source_tag="mg1",vip="192.0.2.10"} 1
|
||||||
|
nginx_ipng_writing{source_tag="mg1",vip="192.0.2.10"} 2
|
||||||
```
|
```
|
||||||
|
|
||||||
For JSON output instead, set the `Accept` header:
|
For JSON output instead, set the `Accept` header:
|
||||||
@@ -300,6 +308,11 @@ sum by (vip) (rate(nginx_ipng_requests_total[5m]))
|
|||||||
# p95 request duration per (source_tag, vip):
|
# p95 request duration per (source_tag, vip):
|
||||||
histogram_quantile(0.95,
|
histogram_quantile(0.95,
|
||||||
sum by (source_tag, vip, le) (rate(nginx_ipng_request_duration_seconds_bucket[5m])))
|
sum by (source_tag, vip, le) (rate(nginx_ipng_request_duration_seconds_bucket[5m])))
|
||||||
|
|
||||||
|
# In-flight concurrency per (source_tag, vip). Gauges are exported as-is;
|
||||||
|
# use max_over_time for load-shedding alerts or avg_over_time for capacity
|
||||||
|
# planning:
|
||||||
|
max_over_time(nginx_ipng_active[5m])
|
||||||
```
|
```
|
||||||
|
|
||||||
## 6. Set up a global logtail access log
|
## 6. Set up a global logtail access log
|
||||||
|
|||||||
@@ -159,6 +159,34 @@ typedef struct ngx_http_ipng_stats_slot_s {
|
|||||||
} ngx_http_ipng_stats_slot_t;
|
} ngx_http_ipng_stats_slot_t;
|
||||||
|
|
||||||
|
|
||||||
|
/* In-flight gauge node: one per (source_id, vip_id) pair, tracking the
|
||||||
|
* count of requests currently in each lifecycle phase. Inserted under
|
||||||
|
* the slab mutex the first time a pair is observed, then cached in the
|
||||||
|
* per-request ctx so transitions touch only the atomic lanes — no
|
||||||
|
* locking on the hot path after the first hit for a given pair.
|
||||||
|
*
|
||||||
|
* Never evicted: one node per distinct (source, vip) pair is small in
|
||||||
|
* practice (tens) and the node is cheap. */
|
||||||
|
typedef struct {
|
||||||
|
ngx_rbtree_node_t rbnode; /* key = hash of (source_id, vip_id) */
|
||||||
|
ngx_queue_t lru; /* for iteration at render time */
|
||||||
|
ngx_uint_t source_id;
|
||||||
|
ngx_uint_t vip_id;
|
||||||
|
ngx_atomic_uint_t active;
|
||||||
|
ngx_atomic_uint_t reading;
|
||||||
|
ngx_atomic_uint_t writing;
|
||||||
|
} ngx_http_ipng_stats_gauge_t;
|
||||||
|
|
||||||
|
|
||||||
|
/* Per-request state carried through the lifecycle hooks. Allocated at
|
||||||
|
* POST_READ, transitioned by the header filter, decremented by the
|
||||||
|
* pool cleanup. `state` is 1 = reading, 2 = writing, 0 = not tracked. */
|
||||||
|
typedef struct {
|
||||||
|
ngx_http_ipng_stats_gauge_t *gauge;
|
||||||
|
ngx_uint_t state;
|
||||||
|
} ngx_http_ipng_stats_ctx_t;
|
||||||
|
|
||||||
|
|
||||||
/* String interning tables live at the head of the shared-memory zone.
|
/* String interning tables live at the head of the shared-memory zone.
|
||||||
* They're flat arrays of ngx_str_t whose data pointers reference memory
|
* They're flat arrays of ngx_str_t whose data pointers reference memory
|
||||||
* allocated from the zone's slab pool. Workers look up strings by
|
* allocated from the zone's slab pool. Workers look up strings by
|
||||||
@@ -179,6 +207,13 @@ typedef struct {
|
|||||||
ngx_http_ipng_stats_intern_t sources;
|
ngx_http_ipng_stats_intern_t sources;
|
||||||
ngx_http_ipng_stats_intern_t vips;
|
ngx_http_ipng_stats_intern_t vips;
|
||||||
|
|
||||||
|
/* In-flight gauges, keyed by (source_id, vip_id). Atomically
|
||||||
|
* updated from the request lifecycle hooks; snapshotted by the
|
||||||
|
* scrape handler under the slab mutex. */
|
||||||
|
ngx_rbtree_t gauge_rbtree;
|
||||||
|
ngx_rbtree_node_t gauge_sentinel;
|
||||||
|
ngx_queue_t gauge_lru;
|
||||||
|
|
||||||
/* Meta-counters for the plugin itself (FR-6 observability of
|
/* Meta-counters for the plugin itself (FR-6 observability of
|
||||||
* the plugin in the design doc). */
|
* the plugin in the design doc). */
|
||||||
ngx_atomic_uint_t zone_full_events;
|
ngx_atomic_uint_t zone_full_events;
|
||||||
@@ -328,6 +363,12 @@ static void ngx_http_ipng_stats_exit_worker(ngx_cycle_t *cycle);
|
|||||||
static void ngx_http_ipng_stats_rescan_timer(ngx_event_t *ev);
|
static void ngx_http_ipng_stats_rescan_timer(ngx_event_t *ev);
|
||||||
|
|
||||||
static ngx_int_t ngx_http_ipng_stats_log_handler(ngx_http_request_t *r);
|
static ngx_int_t ngx_http_ipng_stats_log_handler(ngx_http_request_t *r);
|
||||||
|
static ngx_int_t ngx_http_ipng_stats_post_read_handler(ngx_http_request_t *r);
|
||||||
|
static ngx_int_t ngx_http_ipng_stats_header_filter(ngx_http_request_t *r);
|
||||||
|
static void ngx_http_ipng_stats_ctx_cleanup(void *data);
|
||||||
|
static ngx_http_ipng_stats_gauge_t *ngx_http_ipng_stats_gauge_get(
|
||||||
|
ngx_http_ipng_stats_shctx_t *sh, ngx_slab_pool_t *slab,
|
||||||
|
ngx_uint_t source_id, ngx_uint_t vip_id);
|
||||||
static ngx_int_t ngx_http_ipng_stats_content_handler(ngx_http_request_t *r);
|
static ngx_int_t ngx_http_ipng_stats_content_handler(ngx_http_request_t *r);
|
||||||
|
|
||||||
static void ngx_http_ipng_stats_flush_timer(ngx_event_t *ev);
|
static void ngx_http_ipng_stats_flush_timer(ngx_event_t *ev);
|
||||||
@@ -365,6 +406,8 @@ static char *(*ngx_http_core_listen_orig)(ngx_conf_t *cf,
|
|||||||
|
|
||||||
static ngx_http_ipng_stats_worker_t ngx_http_ipng_stats_worker;
|
static ngx_http_ipng_stats_worker_t ngx_http_ipng_stats_worker;
|
||||||
|
|
||||||
|
static ngx_http_output_header_filter_pt ngx_http_ipng_stats_next_header_filter;
|
||||||
|
|
||||||
extern ngx_module_t ngx_http_core_module;
|
extern ngx_module_t ngx_http_core_module;
|
||||||
|
|
||||||
|
|
||||||
@@ -1200,7 +1243,7 @@ ngx_http_ipng_stats_logtail(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
|
|||||||
|
|
||||||
|
|
||||||
/* ----------------------------------------------------------------- */
|
/* ----------------------------------------------------------------- */
|
||||||
/* Postconfig: install log-phase handler */
|
/* Postconfig: install phase handlers and header filter */
|
||||||
/* ----------------------------------------------------------------- */
|
/* ----------------------------------------------------------------- */
|
||||||
|
|
||||||
static ngx_int_t
|
static ngx_int_t
|
||||||
@@ -1240,6 +1283,22 @@ ngx_http_ipng_stats_postconfig(ngx_conf_t *cf)
|
|||||||
}
|
}
|
||||||
*h = ngx_http_ipng_stats_log_handler;
|
*h = ngx_http_ipng_stats_log_handler;
|
||||||
|
|
||||||
|
/* POST_READ is the earliest phase at which the request is parsed
|
||||||
|
* enough to resolve source and vip; we register the in-flight
|
||||||
|
* gauge there so `reading` covers rewrite/access/content. */
|
||||||
|
h = ngx_array_push(&cmcf->phases[NGX_HTTP_POST_READ_PHASE].handlers);
|
||||||
|
if (h == NULL) {
|
||||||
|
return NGX_ERROR;
|
||||||
|
}
|
||||||
|
*h = ngx_http_ipng_stats_post_read_handler;
|
||||||
|
|
||||||
|
/* Header filter: transitions reading -> writing when nginx starts
|
||||||
|
* sending the response. Inserted at the top of the chain so the
|
||||||
|
* transition is observed before any downstream filter mutates the
|
||||||
|
* response. */
|
||||||
|
ngx_http_ipng_stats_next_header_filter = ngx_http_top_header_filter;
|
||||||
|
ngx_http_top_header_filter = ngx_http_ipng_stats_header_filter;
|
||||||
|
|
||||||
return NGX_OK;
|
return NGX_OK;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1313,6 +1372,10 @@ ngx_http_ipng_stats_init_zone(ngx_shm_zone_t *shm_zone, void *data)
|
|||||||
ngx_http_ipng_stats_rbtree_insert);
|
ngx_http_ipng_stats_rbtree_insert);
|
||||||
ngx_queue_init(&sh->lru);
|
ngx_queue_init(&sh->lru);
|
||||||
|
|
||||||
|
ngx_rbtree_init(&sh->gauge_rbtree, &sh->gauge_sentinel,
|
||||||
|
ngx_http_ipng_stats_rbtree_insert);
|
||||||
|
ngx_queue_init(&sh->gauge_lru);
|
||||||
|
|
||||||
sh->sources.nalloc = 16;
|
sh->sources.nalloc = 16;
|
||||||
sh->sources.entries = ngx_slab_alloc(slab,
|
sh->sources.entries = ngx_slab_alloc(slab,
|
||||||
sh->sources.nalloc * sizeof(ngx_str_t));
|
sh->sources.nalloc * sizeof(ngx_str_t));
|
||||||
@@ -2116,6 +2179,141 @@ ngx_http_ipng_stats_log_handler(ngx_http_request_t *r)
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/* ----------------------------------------------------------------- */
|
||||||
|
/* In-flight gauge lifecycle hooks */
|
||||||
|
/* ----------------------------------------------------------------- */
|
||||||
|
|
||||||
|
/* POST_READ phase handler: first point at which the request is fully
|
||||||
|
* parsed enough to resolve source and vip. Finds or creates the
|
||||||
|
* gauge node, increments active + reading, and registers a pool
|
||||||
|
* cleanup that decrements on request finalization. Skips subrequests
|
||||||
|
* and internal redirects (the gauge is already ticking on r->main). */
|
||||||
|
static ngx_int_t
|
||||||
|
ngx_http_ipng_stats_post_read_handler(ngx_http_request_t *r)
|
||||||
|
{
|
||||||
|
ngx_http_ipng_stats_main_conf_t *imcf;
|
||||||
|
ngx_http_ipng_stats_loc_conf_t *ilcf;
|
||||||
|
ngx_http_ipng_stats_shctx_t *sh;
|
||||||
|
ngx_slab_pool_t *slab;
|
||||||
|
ngx_http_ipng_stats_ctx_t *ctx;
|
||||||
|
ngx_http_ipng_stats_gauge_t *g;
|
||||||
|
ngx_pool_cleanup_t *cln;
|
||||||
|
ngx_str_t source, vip;
|
||||||
|
u_char vipbuf[NGX_SOCKADDR_STRLEN];
|
||||||
|
ngx_uint_t source_id, vip_id;
|
||||||
|
|
||||||
|
if (r != r->main || r->internal) {
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
imcf = ngx_http_get_module_main_conf(r, ngx_http_ipng_stats_module);
|
||||||
|
if (imcf == NULL || imcf->shm_zone == NULL || !imcf->enabled) {
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
ilcf = ngx_http_get_module_loc_conf(r, ngx_http_ipng_stats_module);
|
||||||
|
if (ilcf == NULL || !ilcf->enabled) {
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ngx_http_ipng_stats_resolve_source(r, imcf, &source) != NGX_OK) {
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ngx_connection_local_sockaddr(r->connection, NULL, 0) != NGX_OK) {
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ngx_http_ipng_stats_canonical_vip(r, vipbuf, sizeof(vipbuf), &vip)
|
||||||
|
!= NGX_OK)
|
||||||
|
{
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
slab = (ngx_slab_pool_t *) imcf->shm_zone->shm.addr;
|
||||||
|
sh = imcf->shm_zone->data;
|
||||||
|
|
||||||
|
ngx_shmtx_lock(&slab->mutex);
|
||||||
|
if (ngx_http_ipng_stats_intern_shared(sh, slab, &sh->sources, &source,
|
||||||
|
&source_id) != NGX_OK
|
||||||
|
|| ngx_http_ipng_stats_intern_shared(sh, slab, &sh->vips, &vip,
|
||||||
|
&vip_id) != NGX_OK)
|
||||||
|
{
|
||||||
|
ngx_shmtx_unlock(&slab->mutex);
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
ngx_shmtx_unlock(&slab->mutex);
|
||||||
|
|
||||||
|
g = ngx_http_ipng_stats_gauge_get(sh, slab, source_id, vip_id);
|
||||||
|
if (g == NULL) {
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx = ngx_pcalloc(r->pool, sizeof(*ctx));
|
||||||
|
cln = ngx_pool_cleanup_add(r->pool, 0);
|
||||||
|
if (ctx == NULL || cln == NULL) {
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
ctx->gauge = g;
|
||||||
|
ctx->state = 1; /* reading */
|
||||||
|
cln->handler = ngx_http_ipng_stats_ctx_cleanup;
|
||||||
|
cln->data = ctx;
|
||||||
|
|
||||||
|
ngx_http_set_ctx(r, ctx, ngx_http_ipng_stats_module);
|
||||||
|
|
||||||
|
(void) ngx_atomic_fetch_add(&g->active, 1);
|
||||||
|
(void) ngx_atomic_fetch_add(&g->reading, 1);
|
||||||
|
|
||||||
|
return NGX_DECLINED;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/* Header filter: first call for the main request transitions reading
|
||||||
|
* -> writing. Subrequests have their own header-filter invocations;
|
||||||
|
* we ignore those so a subrequest doesn't prematurely flip the main
|
||||||
|
* request's gauge state. Internal redirects re-enter the filter
|
||||||
|
* chain; the state check prevents double transitions. */
|
||||||
|
static ngx_int_t
|
||||||
|
ngx_http_ipng_stats_header_filter(ngx_http_request_t *r)
|
||||||
|
{
|
||||||
|
ngx_http_ipng_stats_ctx_t *ctx;
|
||||||
|
|
||||||
|
if (r == r->main) {
|
||||||
|
ctx = ngx_http_get_module_ctx(r, ngx_http_ipng_stats_module);
|
||||||
|
if (ctx != NULL && ctx->gauge != NULL && ctx->state == 1) {
|
||||||
|
(void) ngx_atomic_fetch_add(&ctx->gauge->reading,
|
||||||
|
(ngx_atomic_uint_t) -1);
|
||||||
|
(void) ngx_atomic_fetch_add(&ctx->gauge->writing, 1);
|
||||||
|
ctx->state = 2;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return ngx_http_ipng_stats_next_header_filter(r);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
static void
|
||||||
|
ngx_http_ipng_stats_ctx_cleanup(void *data)
|
||||||
|
{
|
||||||
|
ngx_http_ipng_stats_ctx_t *ctx = data;
|
||||||
|
|
||||||
|
if (ctx->gauge == NULL) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (ctx->state == 1) {
|
||||||
|
(void) ngx_atomic_fetch_add(&ctx->gauge->reading,
|
||||||
|
(ngx_atomic_uint_t) -1);
|
||||||
|
} else if (ctx->state == 2) {
|
||||||
|
(void) ngx_atomic_fetch_add(&ctx->gauge->writing,
|
||||||
|
(ngx_atomic_uint_t) -1);
|
||||||
|
}
|
||||||
|
(void) ngx_atomic_fetch_add(&ctx->gauge->active,
|
||||||
|
(ngx_atomic_uint_t) -1);
|
||||||
|
ctx->gauge = NULL;
|
||||||
|
ctx->state = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/* ----------------------------------------------------------------- */
|
/* ----------------------------------------------------------------- */
|
||||||
/* String interning (called under slab mutex) */
|
/* String interning (called under slab mutex) */
|
||||||
/* ----------------------------------------------------------------- */
|
/* ----------------------------------------------------------------- */
|
||||||
@@ -2170,6 +2368,66 @@ ngx_http_ipng_stats_intern_shared(ngx_http_ipng_stats_shctx_t *sh,
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/* ----------------------------------------------------------------- */
|
||||||
|
/* In-flight gauges */
|
||||||
|
/* ----------------------------------------------------------------- */
|
||||||
|
|
||||||
|
/* Hash of (source_id, vip_id). Separate constants from the counter
|
||||||
|
* rbtree's hash so collisions don't line up between the two trees. */
|
||||||
|
static ngx_inline ngx_uint_t
|
||||||
|
ngx_http_ipng_stats_gauge_hash(ngx_uint_t source_id, ngx_uint_t vip_id)
|
||||||
|
{
|
||||||
|
return (source_id * 2246822519u) ^ (vip_id * 3266489917u);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/* Find or create the gauge node for (source_id, vip_id). Takes the
|
||||||
|
* slab mutex only when a new node must be inserted; the caller holds
|
||||||
|
* the returned pointer for the rest of the request and does lock-free
|
||||||
|
* atomic inc/dec on the lanes. Returns NULL on slab exhaustion. */
|
||||||
|
static ngx_http_ipng_stats_gauge_t *
|
||||||
|
ngx_http_ipng_stats_gauge_get(ngx_http_ipng_stats_shctx_t *sh,
|
||||||
|
ngx_slab_pool_t *slab, ngx_uint_t source_id, ngx_uint_t vip_id)
|
||||||
|
{
|
||||||
|
ngx_uint_t hash;
|
||||||
|
ngx_rbtree_node_t *rb;
|
||||||
|
ngx_http_ipng_stats_gauge_t *g = NULL;
|
||||||
|
|
||||||
|
hash = ngx_http_ipng_stats_gauge_hash(source_id, vip_id);
|
||||||
|
|
||||||
|
ngx_shmtx_lock(&slab->mutex);
|
||||||
|
|
||||||
|
rb = sh->gauge_rbtree.root;
|
||||||
|
while (rb != &sh->gauge_sentinel) {
|
||||||
|
if (hash < rb->key) { rb = rb->left; continue; }
|
||||||
|
if (hash > rb->key) { rb = rb->right; continue; }
|
||||||
|
g = (ngx_http_ipng_stats_gauge_t *) rb;
|
||||||
|
if (g->source_id == source_id && g->vip_id == vip_id) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
rb = rb->right;
|
||||||
|
g = NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (g == NULL) {
|
||||||
|
g = ngx_slab_calloc_locked(slab, sizeof(*g));
|
||||||
|
if (g == NULL) {
|
||||||
|
(void) ngx_atomic_fetch_add(&sh->zone_full_events, 1);
|
||||||
|
ngx_shmtx_unlock(&slab->mutex);
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
g->rbnode.key = hash;
|
||||||
|
g->source_id = source_id;
|
||||||
|
g->vip_id = vip_id;
|
||||||
|
ngx_rbtree_insert(&sh->gauge_rbtree, &g->rbnode);
|
||||||
|
ngx_queue_insert_tail(&sh->gauge_lru, &g->lru);
|
||||||
|
}
|
||||||
|
|
||||||
|
ngx_shmtx_unlock(&slab->mutex);
|
||||||
|
return g;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/* ----------------------------------------------------------------- */
|
/* ----------------------------------------------------------------- */
|
||||||
/* Global logtail: write + flush */
|
/* Global logtail: write + flush */
|
||||||
/* ----------------------------------------------------------------- */
|
/* ----------------------------------------------------------------- */
|
||||||
@@ -2461,6 +2719,9 @@ typedef struct {
|
|||||||
uint64_t bytes_out_sum;
|
uint64_t bytes_out_sum;
|
||||||
uint64_t req_total; /* total requests for this (source, vip) */
|
uint64_t req_total; /* total requests for this (source, vip) */
|
||||||
uint64_t up_total; /* upstream observations */
|
uint64_t up_total; /* upstream observations */
|
||||||
|
uint64_t active; /* in-flight gauges (request lifecycle) */
|
||||||
|
uint64_t reading;
|
||||||
|
uint64_t writing;
|
||||||
uint64_t *dhist; /* nbuckets+1 */
|
uint64_t *dhist; /* nbuckets+1 */
|
||||||
uint64_t *uhist;
|
uint64_t *uhist;
|
||||||
uint64_t *bin_hist; /* nbytebuckets+1 */
|
uint64_t *bin_hist; /* nbytebuckets+1 */
|
||||||
@@ -2737,6 +2998,60 @@ ngx_http_ipng_stats_snapshot_nodes(ngx_http_ipng_stats_shctx_t *sh,
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/* Walk the gauge rbtree under the slab mutex and fold in-flight gauge
|
||||||
|
* values into the aggregation table. Creates new agg entries for
|
||||||
|
* (source, vip) pairs that have in-flight requests but no completed
|
||||||
|
* ones yet, so scrapes reflect load even before the first log-phase
|
||||||
|
* update for a pair. Respects the same filter semantics as the
|
||||||
|
* counter walk. */
|
||||||
|
static void
|
||||||
|
ngx_http_ipng_stats_snapshot_gauges(ngx_http_ipng_stats_shctx_t *sh,
|
||||||
|
ngx_str_t *filter_src, ngx_str_t *filter_vip,
|
||||||
|
ngx_str_t *src_tbl, ngx_uint_t n_src,
|
||||||
|
ngx_str_t *vip_tbl, ngx_uint_t n_vip,
|
||||||
|
ngx_http_ipng_stats_agg_t *aggs, ngx_uint_t naggs_alloc,
|
||||||
|
ngx_uint_t *naggs_io)
|
||||||
|
{
|
||||||
|
ngx_queue_t *q;
|
||||||
|
ngx_http_ipng_stats_gauge_t *g;
|
||||||
|
ngx_str_t *src_entry, *vip_entry;
|
||||||
|
ngx_http_ipng_stats_agg_t *a;
|
||||||
|
|
||||||
|
for (q = ngx_queue_head(&sh->gauge_lru);
|
||||||
|
q != ngx_queue_sentinel(&sh->gauge_lru);
|
||||||
|
q = ngx_queue_next(q))
|
||||||
|
{
|
||||||
|
g = ngx_queue_data(q, ngx_http_ipng_stats_gauge_t, lru);
|
||||||
|
if (g->source_id >= n_src || g->vip_id >= n_vip) continue;
|
||||||
|
|
||||||
|
src_entry = &src_tbl[g->source_id];
|
||||||
|
vip_entry = &vip_tbl[g->vip_id];
|
||||||
|
|
||||||
|
if (filter_src->len > 0
|
||||||
|
&& (src_entry->len != filter_src->len
|
||||||
|
|| ngx_memcmp(src_entry->data, filter_src->data,
|
||||||
|
filter_src->len) != 0))
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (filter_vip->len > 0
|
||||||
|
&& (vip_entry->len != filter_vip->len
|
||||||
|
|| ngx_memcmp(vip_entry->data, filter_vip->data,
|
||||||
|
filter_vip->len) != 0))
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
a = ngx_http_ipng_stats_agg_get(aggs, naggs_io, naggs_alloc,
|
||||||
|
g->source_id, g->vip_id);
|
||||||
|
if (a == NULL) continue;
|
||||||
|
a->active = g->active;
|
||||||
|
a->reading = g->reading;
|
||||||
|
a->writing = g->writing;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/* -- Prometheus ---------------------------------------------------- */
|
/* -- Prometheus ---------------------------------------------------- */
|
||||||
|
|
||||||
|
|
||||||
@@ -2817,6 +3132,13 @@ ngx_http_ipng_stats_render_prom(ngx_http_request_t *r,
|
|||||||
"# TYPE nginx_ipng_bytes_in histogram\n"
|
"# TYPE nginx_ipng_bytes_in histogram\n"
|
||||||
"# HELP nginx_ipng_bytes_out Request size histogram in bytes.\n"
|
"# HELP nginx_ipng_bytes_out Request size histogram in bytes.\n"
|
||||||
"# TYPE nginx_ipng_bytes_out histogram\n"
|
"# TYPE nginx_ipng_bytes_out histogram\n"
|
||||||
|
"# HELP nginx_ipng_active Requests currently in flight.\n"
|
||||||
|
"# TYPE nginx_ipng_active gauge\n"
|
||||||
|
"# HELP nginx_ipng_reading In-flight requests in the pre-response "
|
||||||
|
"phases (rewrite/access/content).\n"
|
||||||
|
"# TYPE nginx_ipng_reading gauge\n"
|
||||||
|
"# HELP nginx_ipng_writing In-flight requests past header send.\n"
|
||||||
|
"# TYPE nginx_ipng_writing gauge\n"
|
||||||
"# HELP nginx_ipng_ifindex_misses_total Connections whose ingress "
|
"# HELP nginx_ipng_ifindex_misses_total Connections whose ingress "
|
||||||
"ifindex did not match any configured device= binding.\n"
|
"ifindex did not match any configured device= binding.\n"
|
||||||
"# TYPE nginx_ipng_ifindex_misses_total counter\n"
|
"# TYPE nginx_ipng_ifindex_misses_total counter\n"
|
||||||
@@ -2891,6 +3213,10 @@ ngx_http_ipng_stats_render_prom(ngx_http_request_t *r,
|
|||||||
snaps, nsnaps_alloc, &nsnaps,
|
snaps, nsnaps_alloc, &nsnaps,
|
||||||
aggs, naggs_alloc, &naggs);
|
aggs, naggs_alloc, &naggs);
|
||||||
|
|
||||||
|
ngx_http_ipng_stats_snapshot_gauges(sh, filter_source, filter_vip,
|
||||||
|
src_tbl, n_src, vip_tbl, n_vip,
|
||||||
|
aggs, naggs_alloc, &naggs);
|
||||||
|
|
||||||
ngx_shmtx_unlock(&slab->mutex);
|
ngx_shmtx_unlock(&slab->mutex);
|
||||||
|
|
||||||
/* Per-node counters. */
|
/* Per-node counters. */
|
||||||
@@ -2915,9 +3241,11 @@ ngx_http_ipng_stats_render_prom(ngx_http_request_t *r,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/* One chain link per (source, vip) for the four aggregated histograms.
|
/* One chain link per (source, vip) for the four aggregated
|
||||||
* Size: per-bucket line ~96B, + sum/count/+Inf per metric ~96B each. */
|
* histograms plus the three in-flight gauges. Size: per-bucket
|
||||||
hist_sz = 256 + 96 * (2 * (nb + 1) + 2 * (nbb + 1)) + 4 * 200;
|
* line ~96B, sum/count/+Inf per metric ~96B each, three gauge
|
||||||
|
* lines ~80B each. */
|
||||||
|
hist_sz = 512 + 96 * (2 * (nb + 1) + 2 * (nbb + 1)) + 4 * 200;
|
||||||
|
|
||||||
for (i = 0; i < naggs; i++) {
|
for (i = 0; i < naggs; i++) {
|
||||||
ngx_http_ipng_stats_agg_t *a = &aggs[i];
|
ngx_http_ipng_stats_agg_t *a = &aggs[i];
|
||||||
@@ -2944,6 +3272,13 @@ ngx_http_ipng_stats_render_prom(ngx_http_request_t *r,
|
|||||||
"nginx_ipng_bytes_out", src, vip,
|
"nginx_ipng_bytes_out", src, vip,
|
||||||
imcf->byte_bucket_bounds, nbb, a->bout_hist,
|
imcf->byte_bucket_bounds, nbb, a->bout_hist,
|
||||||
(double) a->bytes_out_sum, 0);
|
(double) a->bytes_out_sum, 0);
|
||||||
|
p = ngx_sprintf(p,
|
||||||
|
"nginx_ipng_active{source_tag=\"%V\",vip=\"%V\"} %uL\n"
|
||||||
|
"nginx_ipng_reading{source_tag=\"%V\",vip=\"%V\"} %uL\n"
|
||||||
|
"nginx_ipng_writing{source_tag=\"%V\",vip=\"%V\"} %uL\n",
|
||||||
|
src, vip, a->active,
|
||||||
|
src, vip, a->reading,
|
||||||
|
src, vip, a->writing);
|
||||||
cl->buf->last = p;
|
cl->buf->last = p;
|
||||||
if (ngx_http_ipng_stats_append(&last, cl) != NGX_OK) {
|
if (ngx_http_ipng_stats_append(&last, cl) != NGX_OK) {
|
||||||
return NGX_HTTP_INTERNAL_SERVER_ERROR;
|
return NGX_HTTP_INTERNAL_SERVER_ERROR;
|
||||||
@@ -3045,11 +3380,15 @@ ngx_http_ipng_stats_render_json(ngx_http_request_t *r,
|
|||||||
snaps, nsnaps_alloc, &nsnaps,
|
snaps, nsnaps_alloc, &nsnaps,
|
||||||
aggs, naggs_alloc, &naggs);
|
aggs, naggs_alloc, &naggs);
|
||||||
|
|
||||||
|
ngx_http_ipng_stats_snapshot_gauges(sh, filter_source, filter_vip,
|
||||||
|
src_tbl, n_src, vip_tbl, n_vip,
|
||||||
|
aggs, naggs_alloc, &naggs);
|
||||||
|
|
||||||
ngx_shmtx_unlock(&slab->mutex);
|
ngx_shmtx_unlock(&slab->mutex);
|
||||||
|
|
||||||
/* One JSON record per aggregated (source, vip). Size upper-bound
|
/* One JSON record per aggregated (source, vip). Size upper-bound
|
||||||
* accounts for: fixed overhead, up to NCLASSES class entries, 4
|
* accounts for: fixed overhead, up to NCLASSES class entries, 4
|
||||||
* histograms. */
|
* histograms, 3 gauges. */
|
||||||
rec_sz = 512
|
rec_sz = 512
|
||||||
+ 160 * NGX_HTTP_IPNG_STATS_NCLASSES
|
+ 160 * NGX_HTTP_IPNG_STATS_NCLASSES
|
||||||
+ 48 * (2 * (nb + 1) + 2 * (nbb + 1))
|
+ 48 * (2 * (nb + 1) + 2 * (nbb + 1))
|
||||||
@@ -3131,7 +3470,9 @@ ngx_http_ipng_stats_render_json(ngx_http_request_t *r,
|
|||||||
imcf->byte_bucket_bounds[j], a->bout_hist[j]);
|
imcf->byte_bucket_bounds[j], a->bout_hist[j]);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
p = ngx_sprintf(p, "}}}");
|
p = ngx_sprintf(p,
|
||||||
|
"}},\"gauges\":{\"active\":%uL,\"reading\":%uL,\"writing\":%uL}}",
|
||||||
|
a->active, a->reading, a->writing);
|
||||||
|
|
||||||
cl->buf->last = p;
|
cl->buf->last = p;
|
||||||
if (ngx_http_ipng_stats_append(&last, cl) != NGX_OK) {
|
if (ngx_http_ipng_stats_append(&last, cl) != NGX_OK) {
|
||||||
|
|||||||
@@ -49,9 +49,16 @@ Shared-listen-include across multiple server blocks
|
|||||||
Length Should Be ${count} 2
|
Length Should Be ${count} 2
|
||||||
... Expected 2 listening sockets on port 8080 (v4+v6 wildcards); got ${count}
|
... Expected 2 listening sockets on port 8080 (v4+v6 wildcards); got ${count}
|
||||||
# Proves the cross-cscf option-stripping path actually fired for
|
# Proves the cross-cscf option-stripping path actually fired for
|
||||||
# the 2nd and 3rd server blocks. `nginx -t` replays the whole
|
# the 2nd and 3rd server blocks at daemon startup — the wrapper
|
||||||
# config and emits the wrapper's NOTICE each time it strips.
|
# logs a NOTICE per stripped listen via ngx_conf_log_error. We
|
||||||
Should Contain ${output} stripped socket options from duplicate listen
|
# read the container's startup log rather than `nginx -t`'s
|
||||||
|
# output because nginx keeps the config-parse log level at ERR
|
||||||
|
# until the error_log directive has been fully applied, so
|
||||||
|
# `nginx -t` suppresses the NOTICE even though the wrapper is
|
||||||
|
# invoked.
|
||||||
|
${rc} ${startup_log} = Run And Return Rc And Output
|
||||||
|
... docker logs ${SERVER} 2>&1
|
||||||
|
Should Contain ${startup_log} stripped socket options from duplicate listen
|
||||||
|
|
||||||
Prometheus scrape
|
Prometheus scrape
|
||||||
[Documentation] Scrape returns HELP/TYPE preamble.
|
[Documentation] Scrape returns HELP/TYPE preamble.
|
||||||
@@ -139,6 +146,53 @@ Duration histogram
|
|||||||
Should Contain ${json} request_duration_ms
|
Should Contain ${json} request_duration_ms
|
||||||
Should Contain ${json} buckets
|
Should Contain ${json} buckets
|
||||||
|
|
||||||
|
# --- In-flight gauges ---
|
||||||
|
|
||||||
|
Gauge preamble in Prometheus output
|
||||||
|
[Documentation] The scrape preamble advertises the three gauge metric
|
||||||
|
... names and types. Present whether or not any traffic
|
||||||
|
... has ever been observed.
|
||||||
|
${output} = Scrape Prometheus
|
||||||
|
Should Match Regexp ${output} (?m)^# TYPE nginx_ipng_active gauge$
|
||||||
|
Should Match Regexp ${output} (?m)^# TYPE nginx_ipng_reading gauge$
|
||||||
|
Should Match Regexp ${output} (?m)^# TYPE nginx_ipng_writing gauge$
|
||||||
|
|
||||||
|
Gauges zero at rest
|
||||||
|
[Documentation] After the earlier traffic tests have drained and the
|
||||||
|
... flush tick has run, the in-flight gauges for (tag1,
|
||||||
|
... 10.0.1.1) must all report zero — no request is still
|
||||||
|
... resident in the plugin's per-request ctx.
|
||||||
|
Wait For Flush
|
||||||
|
${output} = Scrape With Filter source_tag=tag1&vip=10.0.1.1
|
||||||
|
Should Match Regexp ${output} nginx_ipng_active\\{[^}]*\\}\\s+0\\b
|
||||||
|
Should Match Regexp ${output} nginx_ipng_reading\\{[^}]*\\}\\s+0\\b
|
||||||
|
Should Match Regexp ${output} nginx_ipng_writing\\{[^}]*\\}\\s+0\\b
|
||||||
|
|
||||||
|
Gauges appear in JSON output
|
||||||
|
[Documentation] Each record in the JSON output carries a gauges object
|
||||||
|
... with active/reading/writing keys.
|
||||||
|
${rc} ${output} = Run And Return Rc And Output
|
||||||
|
... curl -sf -H 'Accept: application/json' '${SCRAPE_URL}?source_tag=tag1&vip=10.0.1.1' | python3 -m json.tool
|
||||||
|
Should Be Equal As Integers ${rc} 0
|
||||||
|
Should Contain ${output} "gauges"
|
||||||
|
Should Contain ${output} "active"
|
||||||
|
Should Contain ${output} "reading"
|
||||||
|
Should Contain ${output} "writing"
|
||||||
|
|
||||||
|
Gauges observed non-zero under concurrent load
|
||||||
|
[Documentation] Fire a burst of concurrent requests against the
|
||||||
|
... /slow backend (50 ms each, single-threaded) and
|
||||||
|
... observe at least one of active/reading/writing
|
||||||
|
... going non-zero for (tag1, 10.0.1.1) while the
|
||||||
|
... burst is in flight. After it drains the gauges
|
||||||
|
... return to zero. Uses 20 requests (drain <= ~2s
|
||||||
|
... given the single-threaded backend) to keep the
|
||||||
|
... test fast but still wide enough for the scrape
|
||||||
|
... to catch overlap.
|
||||||
|
Run docker exec -d ${CLIENT1} sh -c 'for i in $(seq 20); do curl -sf http://10.0.1.1:8080/slow >/dev/null & done; wait'
|
||||||
|
Wait Until Keyword Succeeds 5s 50ms Nonzero Gauge Observed tag1 10.0.1.1
|
||||||
|
Wait Until Keyword Succeeds 15s 500ms Gauges Drained tag1 10.0.1.1
|
||||||
|
|
||||||
# --- Scrape filters ---
|
# --- Scrape filters ---
|
||||||
|
|
||||||
Filter by source_tag
|
Filter by source_tag
|
||||||
@@ -347,6 +401,26 @@ Get Request Count
|
|||||||
END
|
END
|
||||||
RETURN ${total}
|
RETURN ${total}
|
||||||
|
|
||||||
|
Nonzero Gauge Observed
|
||||||
|
[Documentation] Fails unless the scrape shows a non-zero value for at
|
||||||
|
... least one of {active, reading, writing} on the given
|
||||||
|
... (source_tag, vip) pair.
|
||||||
|
[Arguments] ${source} ${vip}
|
||||||
|
${output} = Scrape With Filter source_tag=${source}&vip=${vip}
|
||||||
|
Should Match Regexp ${output}
|
||||||
|
... nginx_ipng_(active|reading|writing)\\{[^}]*\\}\\s+[1-9]\\d*
|
||||||
|
|
||||||
|
Gauges Drained
|
||||||
|
[Documentation] Fails unless all three gauges are exactly zero for
|
||||||
|
... the given (source_tag, vip) pair. Use with Wait
|
||||||
|
... Until Keyword Succeeds to poll until a request burst
|
||||||
|
... has fully finalized.
|
||||||
|
[Arguments] ${source} ${vip}
|
||||||
|
${output} = Scrape With Filter source_tag=${source}&vip=${vip}
|
||||||
|
Should Match Regexp ${output} nginx_ipng_active\\{[^}]*\\}\\s+0\\b
|
||||||
|
Should Match Regexp ${output} nginx_ipng_reading\\{[^}]*\\}\\s+0\\b
|
||||||
|
Should Match Regexp ${output} nginx_ipng_writing\\{[^}]*\\}\\s+0\\b
|
||||||
|
|
||||||
# --- Container helpers ---
|
# --- Container helpers ---
|
||||||
|
|
||||||
Docker Exec
|
Docker Exec
|
||||||
|
|||||||
Reference in New Issue
Block a user