vpp-maglev/docs/user-guide.md

User Guide

maglevd

maglevd is the health-checker daemon. It probes backends according to the configuration file, maintains their health state, and exposes a gRPC API for inspection and control.

Flags

Flag	Environment variable	Default	Description
--config	MAGLEV_CONFIG	/etc/vpp-maglev/maglev.yaml	Path to the YAML configuration file.
--grpc-addr	MAGLEV_GRPC_ADDR	:9090	TCP address on which the gRPC server listens.
--metrics-addr	MAGLEV_METRICS_ADDR	:9091	TCP address for the Prometheus /metrics HTTP endpoint. Set to empty to disable.
--vpp-api-addr	MAGLEV_VPP_API_ADDR	/run/vpp/api.sock	VPP binary API socket path. Set to empty to disable VPP integration.
--vpp-stats-addr	MAGLEV_VPP_STATS_ADDR	/run/vpp/stats.sock	VPP stats socket path.
--log-level	MAGLEV_LOG_LEVEL	info	Log verbosity: debug, info, warn, or error.
--check	—	—	Read and validate the config file, then exit. Exits 0 if the config is valid, 1 on YAML parse error, 2 on semantic error.
--reflection	—	true	Enable gRPC server reflection. Allows grpcurl to introspect the API without the .proto file. Set to false to disable.
--version	—	—	Print version, commit hash, and build date, then exit.

Flags take precedence over environment variables. Both are optional; defaults are used for anything not set.

Signals

Signal	Effect
SIGHUP	Reload the configuration file (same code path as config reload in maglevc). The file is checked before applying; if there is a parse or semantic error the reload is aborted and the error is logged (the daemon continues running with its current config). New backends are started, removed backends are stopped, backends whose health-check config is unchanged continue probing without interruption.
SIGTERM / SIGINT	Graceful shutdown. Active gRPC streams are closed, the server drains, then the process exits.

Capabilities

maglevd requires CAP_NET_RAW when any health check uses type: icmp. All other check types (tcp, http) use normal TCP sockets and require no special capabilities.

Logging

All log output is written to stdout as JSON using Go's log/slog. The first line logged after the logger is configured is a starting record that includes version, commit, and date. Every state change emits a backend-transition line at INFO level. Per-mutation VPP LB sync events (vpp-lb-sync-vip-added, vpp-lb-sync-vip-removed, vpp-lb-sync-as-added, vpp-lb-sync-as-removed, vpp-lb-sync-as-weight-updated) are also emitted at INFO so the CLI watch events stream and the web frontend see every dataplane change without raising the log level. Set --log-level debug to see individual probe attempts and every VPP binary-API call (vpp-api-send / vpp-api-recv with full payload) as they happen.

Prometheus metrics

maglevd exposes Prometheus metrics on --metrics-addr (default :9091) at the /metrics path. Metric families:

Health-check and backend state (gauges, on-demand):

Metric	Labels	Description
maglev_backend_state	backend, address, healthcheck, state	1 for the current state row per backend, 0 otherwise.
maglev_backend_health	backend	Current rise/fall counter value.
maglev_backend_enabled	backend	1 if enabled, 0 if disabled.
maglev_frontend_pool_backend_weight	frontend, pool, backend	Configured weight from YAML.

Probe counters and latency (inline):

Metric	Labels	Description
maglev_probe_total	backend, type, result, code	Probes executed. result is success or failure.
maglev_probe_duration_seconds	backend, type	Histogram of probe wall time.
maglev_backend_transitions_total	backend, from, to	State machine transitions.

VPP integration (when enabled):

Metric	Labels	Description
maglev_vpp_connected	—	1 if maglevd currently has a live VPP connection.
maglev_vpp_uptime_seconds	—	Seconds since VPP started (from /sys/boottime).
maglev_vpp_connected_seconds	—	Seconds since maglevd established the current VPP connection.
maglev_vpp_info	version, build_date, pid	Static VPP build metadata; always 1.
maglev_vpp_api_total	msg, direction, result	VPP binary-API calls. direction is send or recv; result is success or failure.
maglev_vpp_lbsync_total	scope, kind	Per-mutation sync counters. scope is all or vip; kind is one of vip_added, vip_removed, as_added, as_removed, as_weight_updated.

gRPC server (standard go-grpc-middleware/prometheus metrics): grpc_server_started_total, grpc_server_handled_total, grpc_server_msg_received_total, grpc_server_msg_sent_total, and grpc_server_handling_seconds — all labelled by grpc_service, grpc_method, grpc_type, and grpc_code. Every method is pre-registered at zero so time series exist on the first scrape.

---

maglevc

maglevc is the interactive control-plane client. It connects to a running maglevd over gRPC and either executes a single command or drops into an interactive shell.

Usage

maglevc [--server host:port] [--color[=bool]] [command...]

Flag	Default	Description
--server	localhost:9090	Address of the maglevd gRPC server.
--color	mode-aware	Colorize static field labels (dark blue ANSI). Defaults to true in the interactive shell and false in one-shot mode, so output piped into scripts stays free of escape codes. Pass --color=true or --color=false explicitly to override either default.

When command arguments are supplied the command is executed and maglevc exits; in this mode ANSI color is off by default so the output is script-safe. When no arguments are given an interactive shell is started, the build version is printed on entry, and color is on by default.

Commands

show version                     Print build version, commit hash, and build date.

show frontends [<name>]          Without name: list all frontend names.
                                 With name: show address, protocol, port, src-ip-sticky,
                                 description, and pools. Each pool lists its backends
                                 with two weight columns:
                                   weight     — configured weight from the YAML
                                   effective  — state-aware weight after pool failover
                                                (what gets programmed into VPP)
                                 Disabled backends are marked with [disabled].

show backends [<name>]           Without name: list all backend names.
                                 With name: show address, current state (with duration),
                                 enabled flag, health check, and recent state transitions
                                 with timestamps and how long ago each occurred.

show healthchecks [<name>]       Without name: list all health-check names.
                                 With name: show full health-check configuration.

show vpp info                    Show VPP version, build date, PID, uptime, and when
                                 maglevd connected. Returns an error if VPP is not
                                 connected.
show vpp lb state                Show the VPP load-balancer plugin state: global
                                 configuration, configured VIPs, and their attached
                                 application servers (address, weight, bucket count).
                                 Returns an error if VPP is not connected.
show vpp lb counters             Show per-VIP and per-backend packet/byte counters
                                 from the VPP stats segment, refreshed roughly every
                                 five seconds by maglevd. Each VIP row reports the LB
                                 plugin counters (next, first, untracked, no-server)
                                 and the FIB packets/bytes at the VIP's host prefix.
                                 Each backend row reports FIB packets/bytes at the
                                 backend's /32 or /128 prefix. Use Prometheus for
                                 live rates; this command shows absolute values.

sync vpp lb state [<name>]       Reconcile the VPP load-balancer dataplane from the
                                 running config. Without a name: runs a full sync —
                                 creates missing VIPs, removes stale VIPs, and adjusts
                                 application-server membership and weights across all
                                 frontends. With a name: only the named frontend's VIP
                                 is reconciled, and no VIPs are removed. A full sync
                                 also runs automatically every
                                 maglev.vpp.lb.sync-interval (default 30s) to catch
                                 drift, and once on startup.

set backend <name> pause         Stop health checking for a backend. Cancels the probe
                                 goroutine so no further traffic is sent, and sets the
                                 state to 'paused'. The backend's transition history is
                                 preserved, so 'show backend <name>' still shows where
                                 it came from.
set backend <name> resume        Resume health checking. A fresh probe goroutine is
                                 started and the backend re-enters unknown state.
set backend <name> disable       Stop probing entirely and remove the backend from
                                 rotation. The backend remains visible (state: disabled)
                                 with its transition history intact and can be re-enabled
                                 without reloading configuration.
set backend <name> enable        Re-enable a disabled backend. A fresh probe goroutine is
                                 started and the backend re-enters unknown state.

set frontend <name> pool <pool> backend <name> weight <0-100> [flush]
                                 Set the weight of a backend within a pool. Weight 0 keeps
                                 the backend in the pool but assigns it no traffic. Takes
                                 effect immediately: maglevd pushes the change into VPP
                                 via a targeted single-VIP reconcile, so there's no need
                                 to wait for the periodic sync tick.

                                 Without `flush`, the new weight is installed in Maglev's
                                 new-bucket mapping but VPP's flow table is left alone.
                                 Existing sessions keep reaching this backend until they
                                 naturally drain — useful for graceful draining where
                                 you want new connections to land elsewhere but don't
                                 want to reset any in-flight traffic.

                                 With `flush`, the corresponding application-server row
                                 is rewritten with `lb_as_set_weight(is_flush=true)`,
                                 which clears VPP's flow table entries for this backend.
                                 Existing sessions are dropped immediately — useful when
                                 the backend is being taken out of service for emergency
                                 reasons and you don't want to wait for flows to drain.

                                 Examples:
                                   set frontend web pool primary backend nginx0 weight 50
                                   set frontend web pool primary backend nginx0 weight 0 flush

watch events                     Stream all events (log, backend transitions, frontend)
  [num <n>]                      Stop after receiving n events.
  [log [level <level>]]          Include log events. level is debug|info|warn|error
                                 (default: info). Omitting log/backend/frontend enables all.
  [backend]                      Include backend transition events.
  [frontend]                     Include frontend events (reserved for future use).

                                 Each event is printed as compact JSON on its own line.
                                 Press any key or Ctrl-C to stop. Examples:

                                   watch events
                                   watch events num 20
                                   watch events log level debug
                                   watch events backend num 100
                                   watch events log level debug backend

config check                     Ask maglevd to read and validate its current config file.
                                 Prints "config ok" on success, or the error (parse or
                                 semantic) returned by the daemon.
config reload                    Check and reload the configuration file. Equivalent to
                                 sending SIGHUP to maglevd. Prints "config reloaded" on
                                 success, or the specific error (parse, semantic, or
                                 reload) that prevented the reload.

quit / exit                      Leave the interactive shell.

Interactive shell

The shell prompt is maglev> . Two completion mechanisms are available:

Tab completion — pressing <Tab> at any point completes the current token. Fixed keywords (commands and subcommands) are completed from the command tree. Backend, frontend, and health-check names are fetched live from the server with a 1-second timeout. If the partial token is unambiguous the word is completed in place; if multiple candidates exist they are listed and the prompt is restored.

Inline help (?) — typing ? at any point prints the available completions for the current position, with a short description next to each keyword. The ? character is not added to the input line.

Commands and keywords support prefix matching: typing sh ba is equivalent to show backends, and sh ba nginx0 is equivalent to show backends nginx0.

---

maglevd-frontend

maglevd-frontend is an optional web dashboard that connects to one or more running maglevd instances over gRPC and renders a live view of frontends, backends, health checks, and VPP load-balancer state. It is a single Go binary with the SolidJS SPA embedded via //go:embed; no runtime file dependencies.

Installed by the Debian package to /usr/sbin/maglevd-frontend but not enabled by default — the operator opts in via:

systemctl enable --now vpp-maglev-frontend

The systemd unit (vpp-maglev-frontend.service) reads its arguments from /etc/default/vpp-maglev via MAGLEV_FRONTEND_ARGS. The same env file is shared with maglevd; all maglevd-frontend-specific variables are prefixed with MAGLEV_FRONTEND_ so there's no overlap.

Flags

Flag	Environment variable	Default	Description
--server	MAGLEV_FRONTEND_SERVERS	(required)	Comma-separated list of host:port maglevd addresses.
--listen	MAGLEV_FRONTEND_LISTEN	:8080	HTTP bind address.
--log-level	MAGLEV_FRONTEND_LOG_LEVEL	info	Structured-log verbosity for maglevd-frontend's own logs.
--version	—	—	Print version, commit hash, and build date, then exit.

In addition to flags, two env-only variables control the admin surface:

Environment variable	Purpose
MAGLEV_FRONTEND_USER	HTTP basic-auth username for /admin/.
MAGLEV_FRONTEND_PASSWORD	HTTP basic-auth password for /admin/.

When both are set and non-empty the admin surface is mounted and the SPA's "admin…" toggle becomes visible. When either is missing or empty the /admin/ route returns 404 and the SPA hides the toggle — /view/ is always reachable read-only.

HTTP surface

• /view/ — static SPA (dashboard). No authentication.
• /view/api/state, /view/api/state/{name} — full JSON snapshot for every maglevd, or one maglevd.
• /view/api/maglevds — configured maglevds and connection status.
• /view/api/version — build info + admin_enabled flag.
• /view/api/events — Server-Sent Events stream; log, backend, frontend, maglevd-status, vpp-status events with Last-Event-ID replay from a 30-second / 2000-event ring buffer.
• /healthz — liveness; returns 200 if the HTTP server is up.
• /admin/ — SPA shell behind basic auth (when configured).
• POST /admin/api/{maglevd}/backend/{name}/{action} — backend lifecycle action. action is pause, resume, enable, or disable. Returns the fresh backend snapshot as JSON.
• POST /admin/api/{maglevd}/frontend/{fe}/pool/{pool}/backend/{name}/weight — weight change. Body: {"weight": 0-100, "flush": bool}. When flush=true, VPP's flow table for the backend is cleared; otherwise only the new-buckets map is updated and existing sessions keep reaching the backend until they finish.

Reverse-proxy requirements (SSE)

Nginx, HAProxy, or any proxy in front of maglevd-frontend must:

• Disable buffering on the events endpoint. X-Accel-Buffering: no is sent by the server; a global proxy_buffering off; in the nginx server block is the more robust answer.
• Raise proxy_read_timeout to at least 300s so the stream isn't torn down between the 15-second : ping heartbeats the server sends.
• Not wrap the events endpoint in any gzip/brotli middleware — response compression buffers until its window fills and destroys the live-stream property.

See maglevd-frontend(8) for the full reference.