vpp-maglev/docs/user-guide.md

# User Guide

## maglevd

`maglevd` is the health-checker daemon. It probes backends according to the
configuration file, maintains their health state, and exposes a gRPC API for
inspection and control.

### Flags

| Flag | Environment variable | Default | Description |
|---|---|---|---|
| `--config` | `MAGLEV_CONFIG` | `/etc/vpp-maglev/maglev.yaml` | Path to the YAML configuration file. |
| `--grpc-addr` | `MAGLEV_GRPC_ADDR` | `:9090` | TCP address on which the gRPC server listens. |
| `--metrics-addr` | `MAGLEV_METRICS_ADDR` | `:9091` | TCP address for the Prometheus `/metrics` HTTP endpoint. Set to empty to disable. |
| `--vpp-api-addr` | `MAGLEV_VPP_API_ADDR` | `/run/vpp/api.sock` | VPP binary API socket path. Set to empty to disable VPP integration. |
| `--vpp-stats-addr` | `MAGLEV_VPP_STATS_ADDR` | `/run/vpp/stats.sock` | VPP stats socket path. |
| `--log-level` | `MAGLEV_LOG_LEVEL` | `info` | Log verbosity: `debug`, `info`, `warn`, or `error`. |
| `--check` | — | — | Read and validate the config file, then exit. Exits 0 if the config is valid, 1 on YAML parse error, 2 on semantic error. |
| `--reflection` | — | `true` | Enable gRPC server reflection. Allows `grpcurl` to introspect the API without the `.proto` file. Set to `false` to disable. |
| `--version` | — | — | Print version, commit hash, and build date, then exit. |

Flags take precedence over environment variables. Both are optional; defaults
are used for anything not set.

### Signals

| Signal | Effect |
|---|---|
| `SIGHUP` | Reload the configuration file (same code path as `config reload` in `maglevc`). The file is checked before applying; if there is a parse or semantic error the reload is aborted and the error is logged (the daemon continues running with its current config). New backends are started, removed backends are stopped, backends whose health-check config is unchanged continue probing without interruption. |
| `SIGTERM` / `SIGINT` | Graceful shutdown. Active gRPC streams are closed, the server drains, then the process exits. |

### Capabilities

`maglevd` requires `CAP_NET_RAW` when any health check uses `type: icmp`.
All other check types (`tcp`, `http`) use normal TCP sockets and require no
special capabilities.

### Logging

All log output is written to stdout as JSON using Go's `log/slog`. The first
line logged after the logger is configured is a `starting` record that includes
`version`, `commit`, and `date`. Every state change emits a `backend-transition`
line at `INFO` level. Set `--log-level debug` to see individual probe attempts,
every VPP binary-API call (`vpp-api-send` / `vpp-api-recv` with full payload),
and the per-VIP sync operations (`vpp-lbsync-vip-add`, `vpp-lbsync-as-weight`,
etc.) as they happen.

### Prometheus metrics

`maglevd` exposes Prometheus metrics on `--metrics-addr` (default `:9091`) at
the `/metrics` path. Metric families:

**Health-check and backend state (gauges, on-demand):**
| Metric | Labels | Description |
|---|---|---|
| `maglev_backend_state` | `backend`, `address`, `healthcheck`, `state` | 1 for the current state row per backend, 0 otherwise. |
| `maglev_backend_health` | `backend` | Current rise/fall counter value. |
| `maglev_backend_enabled` | `backend` | 1 if enabled, 0 if disabled. |
| `maglev_frontend_pool_backend_weight` | `frontend`, `pool`, `backend` | Configured weight from YAML. |

**Probe counters and latency (inline):**
| Metric | Labels | Description |
|---|---|---|
| `maglev_probe_total` | `backend`, `type`, `result`, `code` | Probes executed. `result` is `success` or `failure`. |
| `maglev_probe_duration_seconds` | `backend`, `type` | Histogram of probe wall time. |
| `maglev_backend_transitions_total` | `backend`, `from`, `to` | State machine transitions. |

**VPP integration (when enabled):**
| Metric | Labels | Description |
|---|---|---|
| `maglev_vpp_connected` | — | 1 if maglevd currently has a live VPP connection. |
| `maglev_vpp_uptime_seconds` | — | Seconds since VPP started (from `/sys/boottime`). |
| `maglev_vpp_connected_seconds` | — | Seconds since maglevd established the current VPP connection. |
| `maglev_vpp_info` | `version`, `build_date`, `pid` | Static VPP build metadata; always 1. |
| `maglev_vpp_api_total` | `msg`, `direction`, `result` | VPP binary-API calls. `direction` is `send` or `recv`; `result` is `success` or `failure`. |
| `maglev_vpp_lbsync_total` | `scope`, `kind` | Per-mutation sync counters. `scope` is `all` or `vip`; `kind` is one of `vip_added`, `vip_removed`, `as_added`, `as_removed`, `as_weight_updated`. |

**gRPC server (standard `go-grpc-middleware/prometheus` metrics):**
`grpc_server_started_total`, `grpc_server_handled_total`,
`grpc_server_msg_received_total`, `grpc_server_msg_sent_total`, and
`grpc_server_handling_seconds` — all labelled by `grpc_service`,
`grpc_method`, `grpc_type`, and `grpc_code`. Every method is
pre-registered at zero so time series exist on the first scrape.

---

## maglevc

`maglevc` is the interactive control-plane client. It connects to a running
`maglevd` over gRPC and either executes a single command or drops into an
interactive shell.

### Usage

```sh
maglevc [--server host:port] [--color[=bool]] [command...]
```

| Flag | Default | Description |
|---|---|---|
| `--server` | `localhost:9090` | Address of the `maglevd` gRPC server. |
| `--color` | `true` | Colorize static field labels in output (dark blue ANSI). Pass `--color=false` to disable, e.g. when piping. |

When `command` arguments are supplied the command is executed and `maglevc`
exits. When no arguments are given an interactive shell is started and the
build version is printed on entry.

### Commands

```
show version                     Print build version, commit hash, and build date.

show frontends [<name>]          Without name: list all frontend names.
                                 With name: show address, protocol, port, description,
                                 and pools. Each pool lists its backends with two
                                 weight columns:
                                   weight     — configured weight from the YAML
                                   effective  — state-aware weight after pool failover
                                                (what gets programmed into VPP)
                                 Disabled backends are marked with [disabled].

show backends [<name>]           Without name: list all backend names.
                                 With name: show address, current state (with duration),
                                 enabled flag, health check, and recent state transitions
                                 with timestamps and how long ago each occurred.

show healthchecks [<name>]       Without name: list all health-check names.
                                 With name: show full health-check configuration.

show vpp info                    Show VPP version, build date, PID, uptime, and when
                                 maglevd connected. Returns an error if VPP is not
                                 connected.
show vpp lbstate                 Show the VPP load-balancer plugin state: global
                                 configuration, configured VIPs, and their attached
                                 application servers (address, weight, bucket count).
                                 Returns an error if VPP is not connected.

sync vpp lbstate [<name>]        Reconcile the VPP load-balancer dataplane from the
                                 running config. Without a name: runs a full sync —
                                 creates missing VIPs, removes stale VIPs, and adjusts
                                 application-server membership and weights across all
                                 frontends. With a name: only the named frontend's VIP
                                 is reconciled, and no VIPs are removed. A full sync
                                 also runs automatically every
                                 maglev.vpp.lb.sync-interval (default 30s) to catch
                                 drift, and once on startup.

set backend <name> pause         Stop health checking for a backend. Cancels the probe
                                 goroutine so no further traffic is sent, and sets the
                                 state to 'paused'. The backend's transition history is
                                 preserved, so 'show backend <name>' still shows where
                                 it came from.
set backend <name> resume        Resume health checking. A fresh probe goroutine is
                                 started and the backend re-enters unknown state.
set backend <name> disable       Stop probing entirely and remove the backend from
                                 rotation. The backend remains visible (state: disabled)
                                 with its transition history intact and can be re-enabled
                                 without reloading configuration.
set backend <name> enable        Re-enable a disabled backend. A fresh probe goroutine is
                                 started and the backend re-enters unknown state.

set frontend <name> pool <pool> backend <name> weight <0-100>
                                 Set the weight of a backend within a pool. Weight 0 keeps
                                 the backend in the pool but assigns it no traffic.
                                 Takes effect immediately without reloading configuration.

watch events                     Stream all events (log, backend transitions, frontend)
  [num <n>]                      Stop after receiving n events.
  [log [level <level>]]          Include log events. level is debug|info|warn|error
                                 (default: info). Omitting log/backend/frontend enables all.
  [backend]                      Include backend transition events.
  [frontend]                     Include frontend events (reserved for future use).

                                 Each event is printed as compact JSON on its own line.
                                 Press any key or Ctrl-C to stop. Examples:

                                   watch events
                                   watch events num 20
                                   watch events log level debug
                                   watch events backend num 100
                                   watch events log level debug backend

config check                     Ask maglevd to read and validate its current config file.
                                 Prints "config ok" on success, or the error (parse or
                                 semantic) returned by the daemon.
config reload                    Check and reload the configuration file. Equivalent to
                                 sending SIGHUP to maglevd. Prints "config reloaded" on
                                 success, or the specific error (parse, semantic, or
                                 reload) that prevented the reload.

quit / exit                      Leave the interactive shell.
```

### Interactive shell

The shell prompt is `maglev> `. Two completion mechanisms are available:

**Tab completion** — pressing `<Tab>` at any point completes the current token.
Fixed keywords (commands and subcommands) are completed from the command tree.
Backend, frontend, and health-check names are fetched live from the server with
a 1-second timeout. If the partial token is unambiguous the word is completed
in place; if multiple candidates exist they are listed and the prompt is
restored.

**Inline help (`?`)** — typing `?` at any point prints the available
completions for the current position, with a short description next to each
keyword. The `?` character is not added to the input line.

Commands and keywords support **prefix matching**: typing `sh ba` is equivalent
to `show backends`, and `sh ba nginx0` is equivalent to `show backends nginx0`.