Files
vpp-maglev/Makefile
Pim van Pelt 6d78921edd Restart-neutral VPP LB sync; deterministic AS ordering; maglevt cadence; v0.9.5
Three reliability fixes bundled with docs updates.

Restart-neutral VPP LB sync via a startup warmup window
(internal/vpp/warmup.go). Before this, a maglevd restart would
immediately issue SyncLBStateAll with every backend still in
StateUnknown — mapped through BackendEffectiveWeight to weight
0 — and VPP would black-hole all new flows until the checker's
rise counters caught up, several seconds later. The new warmup
tracker owns a process-wide state machine gated by two config
knobs: vpp.lb.startup-min-delay (default 5s) is an absolute
hands-off window during which neither the periodic sync loop
nor the per-transition reconciler touches VPP; vpp.lb.
startup-max-delay (default 30s) is the watchdog for a per-VIP
release phase that runs between the two, releasing each frontend
as soon as every backend it references reaches a non-Unknown
state. At max-delay a final SyncLBStateAll runs for any stragglers
still in Unknown. Config reload does not reset the clock. Both
delays can be set to 0 to disable the warmup entirely. The
reconciler's suppressed-during-warmup events log at DEBUG so
operators can still see them with --log-level debug. Unit tests
cover the tracker state machine, allBackendsKnown precondition,
and the zero-delay escape hatch.

Deterministic AS iteration in VPP LB sync. reconcileVIP and
recreateVIP now issue their lb_as_add_del / lb_as_set_weight
calls in numeric IP order (IPv4 before IPv6, ascending within
each family) via a new sortedIPKeys helper, instead of Go map
iteration order. VPP's LB plugin breaks per-bucket ties in the
Maglev lookup table by insertion position in its internal AS
vec, so without a stable call order two maglevd instances on
the same config could push identical AS sets into VPP in
different orders and produce divergent new-flow tables. Numeric
sort is used in preference to lexicographic so the sync log
stays human-readable: string order would place 10.0.0.10 before
10.0.0.2, and the same problem in v6. Unit tests cover empty,
single, v4/v6 numeric vs lexicographic, v4-before-v6 grouping,
a 1000-iteration stability loop against Go's randomised map
iteration, insertion-order invariance, and the desiredAS
call-site type.

maglevt interval fix. runProbeLoop used to sleep the full
jittered interval after every probe, so a 100ms --interval
with a 30ms probe actually produced a 130ms period. The sleep
now subtracts result.Duration so cadence matches the flag.
Probes that overrun clamp sleep to zero and fire the next
probe immediately without trying to catch up on missed cycles
— a slow backend doesn't get flooded with back-to-back probes
at the moment it's already struggling.

Docs. config-guide now documents flush-on-down and the new
startup-min-delay / startup-max-delay knobs; user-guide's
maglevd section explains the restart-neutrality property, the
three warmup phases, and the relevant slog lines operators
should watch for during a bounce.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 12:53:42 +02:00

255 lines
11 KiB
Makefile

BINARIES := maglevd maglevc maglevd-frontend maglevt
MODULE := git.ipng.ch/ipng/vpp-maglev
PROTO_DIR := proto
PROTO_FILE := $(PROTO_DIR)/maglev.proto
GEN_FILES := internal/grpcapi/maglev.pb.go internal/grpcapi/maglev_grpc.pb.go
# Web bundle is built by Vite and embedded by the Go binary via //go:embed.
# Any change under cmd/frontend/web/src/ retriggers an npm build; the
# generated cmd/frontend/web/dist/index.html is the sentinel.
FRONTEND_WEB_SRC := $(shell find cmd/frontend/web/src -type f 2>/dev/null) \
cmd/frontend/web/index.html \
cmd/frontend/web/package.json \
cmd/frontend/web/vite.config.ts \
cmd/frontend/web/tsconfig.json
FRONTEND_WEB_DIST := cmd/frontend/web/dist/index.html
NATIVE_ARCH := $(shell go env GOARCH)
VERSION := 0.9.5
COMMIT_HASH := $(shell git rev-parse --short HEAD 2>/dev/null || echo unknown)
DATE := $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
LDFLAGS := -X '$(MODULE)/cmd.version=$(VERSION)' \
-X '$(MODULE)/cmd.commit=$(COMMIT_HASH)' \
-X '$(MODULE)/cmd.date=$(DATE)'
# CGO_ENABLED=0 produces fully static binaries: no libc dependency, so the
# debian package runs on any Linux-amd64/arm64 host regardless of glibc
# version (or musl). The only behavioural change is that Go's net package
# uses the pure-Go resolver instead of libc's getaddrinfo, which skips
# /etc/nsswitch.conf and NSS modules — fine for DNS-only lookups, which is
# all maglevd does. govpp has no cgo (its binapi is hand-marshalled Go
# structs) so this has no effect on multi-VPP-version compatibility.
export CGO_ENABLED := 0
TEST ?= tests/
VPP_API_DIR ?= $(HOME)/src/vpp/build-root/install-vpp_debug-native/vpp/share/vpp/api
# GO_VERSION is what install-deps-go downloads from go.dev when the
# system Go is missing or older than this. Debian Trixie only ships
# golang-go 1.24 (main), and go.mod requires 1.25+, so the `apt install
# golang-go` path isn't sufficient — we fall back to the upstream
# tarball in /usr/local/go. Override on the command line to pull a
# specific patch release: make install-deps GO_VERSION=1.25.5
GO_VERSION ?= 1.25.0
# GOLANGCI_LINT_VERSION is the minimum golangci-lint version that
# install-deps-go-tools accepts. Raised to 1.64.0 because earlier
# releases don't understand Go 1.25 syntax (1.64 is the last v1 line
# and shipped Go 1.25 support; any v2.x release satisfies the floor
# trivially via version sort). install-deps-go-tools always `go
# install`s @latest, then asserts the resulting binary reports a
# version >= this floor as a sanity check. Override on the command
# line if you want to force a specific minimum, e.g.
# make install-deps GOLANGCI_LINT_VERSION=2.0.0
GOLANGCI_LINT_VERSION ?= 1.64.0
.PHONY: all build build-amd64 build-arm64 test proto vpp-binapi lint fixstyle fixstyle-web pkg-deb robot-test clean maglevd-frontend-web install-deps install-deps-apt install-deps-go install-deps-go-tools
all: build
build: $(GEN_FILES) $(FRONTEND_WEB_DIST)
mkdir -p build/$(NATIVE_ARCH)
go build -ldflags "$(LDFLAGS)" -o build/$(NATIVE_ARCH)/maglevd ./cmd/maglevd/
go build -ldflags "$(LDFLAGS)" -o build/$(NATIVE_ARCH)/maglevc ./cmd/maglevc/
go build -ldflags "$(LDFLAGS)" -o build/$(NATIVE_ARCH)/maglevd-frontend ./cmd/frontend/
go build -ldflags "$(LDFLAGS)" -o build/$(NATIVE_ARCH)/maglevt ./cmd/tester/
build-amd64: $(GEN_FILES) $(FRONTEND_WEB_DIST)
mkdir -p build/amd64
GOOS=linux GOARCH=amd64 go build -ldflags "$(LDFLAGS)" -o build/amd64/maglevd ./cmd/maglevd/
GOOS=linux GOARCH=amd64 go build -ldflags "$(LDFLAGS)" -o build/amd64/maglevc ./cmd/maglevc/
GOOS=linux GOARCH=amd64 go build -ldflags "$(LDFLAGS)" -o build/amd64/maglevd-frontend ./cmd/frontend/
GOOS=linux GOARCH=amd64 go build -ldflags "$(LDFLAGS)" -o build/amd64/maglevt ./cmd/tester/
build-arm64: $(GEN_FILES) $(FRONTEND_WEB_DIST)
mkdir -p build/arm64
GOOS=linux GOARCH=arm64 go build -ldflags "$(LDFLAGS)" -o build/arm64/maglevd ./cmd/maglevd/
GOOS=linux GOARCH=arm64 go build -ldflags "$(LDFLAGS)" -o build/arm64/maglevc ./cmd/maglevc/
GOOS=linux GOARCH=arm64 go build -ldflags "$(LDFLAGS)" -o build/arm64/maglevd-frontend ./cmd/frontend/
GOOS=linux GOARCH=arm64 go build -ldflags "$(LDFLAGS)" -o build/arm64/maglevt ./cmd/tester/
# maglevd-frontend-web rebuilds the SolidJS bundle. The Go binary embeds the
# resulting cmd/frontend/web/dist/ via //go:embed, so a `go build` after
# this target picks up any asset changes automatically.
maglevd-frontend-web: $(FRONTEND_WEB_DIST)
$(FRONTEND_WEB_DIST): $(FRONTEND_WEB_SRC)
cd cmd/frontend/web && npm install && npm run build
pkg-deb: build-amd64 build-arm64
debian/build-deb.sh amd64 $(VERSION)
debian/build-deb.sh arm64 $(VERSION)
test: $(GEN_FILES)
go test ./...
proto: $(GEN_FILES)
$(GEN_FILES): $(PROTO_FILE)
protoc \
--go_out=. --go_opt=module=$(MODULE) \
--go-grpc_out=. --go-grpc_opt=module=$(MODULE) \
$(PROTO_FILE)
# vpp-binapi regenerates the Go bindings for VPP API files used by maglevd
# from a local VPP build. The LB plugin ships with upstream VPP; any newer
# messages (e.g. lb_conf_get, lb_as_v2_dump) require a VPP build that has
# them. Override VPP_API_DIR on the command line to point at another tree:
# make vpp-binapi VPP_API_DIR=/path/to/share/vpp/api
vpp-binapi:
@command -v binapi-generator >/dev/null 2>&1 || { \
echo "installing binapi-generator..."; \
go install go.fd.io/govpp/cmd/binapi-generator@v0.12.0; \
}
rm -rf internal/vpp/binapi
mkdir -p internal/vpp/binapi
binapi-generator \
--input=$(VPP_API_DIR) \
--output-dir=internal/vpp/binapi \
--import-prefix=$(MODULE)/internal/vpp/binapi \
--no-source-path-info \
--no-version-info \
lb lb_types
rm -f internal/vpp/binapi/lb/lb_rpc.ba.go
fixstyle: fixstyle-web
gofmt -w .
fixstyle-web:
cd cmd/frontend/web && npx prettier --write .
lint:
golangci-lint run ./...
# install-deps is an opt-in "set up a fresh developer box" target. Tested
# on Debian Trixie; the apt half should also work on Bookworm and recent
# Ubuntu LTS. Splits into three sub-targets so they can be run individually:
#
# install-deps-apt — Debian-packaged build-time deps (nodejs, npm,
# protoc, git, make, dpkg-dev, curl).
# install-deps-go — ensure a Go toolchain >= $(GO_VERSION) is on
# the system. Downloads the upstream tarball
# into /usr/local/go when the system Go is
# missing or older than the go.mod floor.
# install-deps-go-tools — `go install` the helpers this repo needs
# (protoc-gen-go, protoc-gen-go-grpc, golangci-
# lint) and assert golangci-lint is new enough
# to understand Go 1.25 syntax.
#
# Each sub-target is idempotent and safe to re-run.
install-deps: install-deps-apt install-deps-go install-deps-go-tools
@echo ""
@echo "==> All build dependencies installed."
@echo " Make sure these are on PATH:"
@echo " /usr/local/go/bin (Go toolchain)"
@echo " \$$(go env GOPATH)/bin (protoc-gen-go, golangci-lint, ...)"
install-deps-apt:
@set -eu; \
if [ "$$(id -u)" = 0 ]; then SUDO=""; else SUDO="sudo"; fi; \
echo "==> Installing apt packages (nodejs, npm, protoc, git, make, dpkg-dev)"; \
$$SUDO apt-get update; \
$$SUDO apt-get install -y --no-install-recommends \
nodejs npm protobuf-compiler git make dpkg-dev \
ca-certificates curl tar
# install-deps-go short-circuits when go env GOVERSION already reports a
# version >= GO_VERSION. Otherwise it downloads the official upstream
# tarball (https://go.dev/dl/) and extracts it to /usr/local/go, matching
# the layout that go.dev recommends and that most Debian setups use for
# "Go newer than apt provides".
install-deps-go:
@set -eu; \
if [ "$$(id -u)" = 0 ]; then SUDO=""; else SUDO="sudo"; fi; \
echo "==> Checking Go toolchain (required: $(GO_VERSION)+)"; \
if command -v go >/dev/null 2>&1; then \
CURRENT=$$(go env GOVERSION 2>/dev/null | sed 's/^go//'); \
OLDEST=$$(printf '%s\n%s\n' "$(GO_VERSION)" "$$CURRENT" | sort -V | head -n1); \
if [ "$$OLDEST" = "$(GO_VERSION)" ] && [ -n "$$CURRENT" ]; then \
echo " go$$CURRENT already installed (>= $(GO_VERSION)), skipping."; \
exit 0; \
fi; \
echo " go$$CURRENT is older than $(GO_VERSION), upgrading."; \
else \
echo " no Go toolchain on PATH, installing."; \
fi; \
DEB_ARCH=$$(dpkg --print-architecture); \
case "$$DEB_ARCH" in \
amd64) GOARCH=amd64 ;; \
arm64) GOARCH=arm64 ;; \
armhf) GOARCH=armv6l ;; \
*) echo " unsupported architecture: $$DEB_ARCH" >&2; exit 1 ;; \
esac; \
TARBALL="go$(GO_VERSION).linux-$$GOARCH.tar.gz"; \
URL="https://go.dev/dl/$$TARBALL"; \
echo " downloading $$URL"; \
curl -fsSL -o "/tmp/$$TARBALL" "$$URL"; \
echo " installing to /usr/local/go"; \
$$SUDO rm -rf /usr/local/go; \
$$SUDO tar -C /usr/local -xzf "/tmp/$$TARBALL"; \
rm -f "/tmp/$$TARBALL"; \
echo " installed $$(/usr/local/go/bin/go version)"
# install-deps-go-tools installs the three Go binaries this repo calls
# out to during `make proto` and `make lint`. protoc-gen-go and
# protoc-gen-go-grpc pin to specific upstream release branches; golangci-
# lint pulls @latest (the v2 install path) and then we assert the
# installed version parses as >= GOLANGCI_LINT_VERSION so a stale binary
# in $GOPATH/bin from a previous dev session doesn't silently get used
# against Go 1.25 code it can't parse. Run `make install-deps
# GOLANGCI_LINT_VERSION=2.0.0` if you want to enforce a tighter floor.
install-deps-go-tools:
@set -eu; \
if ! command -v go >/dev/null 2>&1; then \
export PATH="/usr/local/go/bin:$$PATH"; \
fi; \
echo "==> Installing Go tools via 'go install'"; \
echo " google.golang.org/protobuf/cmd/protoc-gen-go"; \
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest; \
echo " google.golang.org/grpc/cmd/protoc-gen-go-grpc"; \
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest; \
echo " github.com/golangci/golangci-lint/v2/cmd/golangci-lint"; \
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@latest; \
GOBIN="$$(go env GOBIN)"; \
if [ -z "$$GOBIN" ]; then GOBIN="$$(go env GOPATH)/bin"; fi; \
echo "==> Asserting golangci-lint version >= $(GOLANGCI_LINT_VERSION)"; \
if ! "$$GOBIN/golangci-lint" version >/dev/null 2>&1; then \
echo " ERROR: $$GOBIN/golangci-lint is not executable" >&2; \
exit 1; \
fi; \
INSTALLED=$$("$$GOBIN/golangci-lint" version 2>&1 | sed -En 's/.*has version v?([0-9][0-9.]*).*/\1/p' | head -n1); \
if [ -z "$$INSTALLED" ]; then \
echo " ERROR: could not parse golangci-lint version output" >&2; \
"$$GOBIN/golangci-lint" version >&2; \
exit 1; \
fi; \
OLDEST=$$(printf '%s\n%s\n' "$(GOLANGCI_LINT_VERSION)" "$$INSTALLED" | sort -V | head -n1); \
if [ "$$OLDEST" != "$(GOLANGCI_LINT_VERSION)" ]; then \
echo " ERROR: golangci-lint $$INSTALLED is older than the required $(GOLANGCI_LINT_VERSION)" >&2; \
echo " The tool understands Go 1.25 syntax only from v1.64.0 / v2.x onward." >&2; \
exit 1; \
fi; \
echo " golangci-lint $$INSTALLED (>= $(GOLANGCI_LINT_VERSION)) OK"
tests/.venv: tests/requirements.txt
python3 -m venv tests/.venv
tests/.venv/bin/pip install -q -r tests/requirements.txt
robot-test: build tests/.venv
tests/rf-run.sh docker $(TEST)
clean:
rm -rf build/
rm -f $(GEN_FILES)