Fix pause to cancel probe goroutine; add Robot Framework integration tests
Pause semantics - PauseBackend now cancels the probe goroutine so no HTTP/TCP/ICMP traffic is sent while the backend is paused. Previously the goroutine kept running and results were silently discarded. - ResumeBackend launches a fresh probe goroutine on the existing worker, preserving transition history. The backend re-enters unknown state. Integration tests (tests/01-maglevd/) - Containerlab topology with 3 nginx:alpine backends on a dedicated management network (172.20.30.0/24) with static IPs. - maglevd config with 200ms HTTP health-check interval for fast test convergence (rise=2, fall=2). - 8 test cases: deploy lab, start maglevd, all backends reach up, nginx logs confirm probes arriving, pause stops probes (probe count stable), resume restarts probes, disable stops probes, enable restarts probes. VPP dataplane test (tests/02-vpp-lb/) - Rewrite 01-e2e-lab.robot to match the actual single-VPP topology: test client-to-server ping through VPP bridge domains and verify nginx is serving on all app servers. The previous version referenced a non-existent topology file and tested OSPF/BFD between two VPP nodes that don't exist in this lab. Build infrastructure - Add 'make robot-test' target with TEST= for suite selection. - Add tests/.venv target for Robot Framework virtualenv. - Make IMAGE optional in rf-run.sh. - Add .gitignore entries for test output, venv, logs, and clab state.
This commit is contained in:
135
tests/01-maglevd/01-healthcheck.robot
Normal file
135
tests/01-maglevd/01-healthcheck.robot
Normal file
@@ -0,0 +1,135 @@
|
||||
*** Settings ***
|
||||
Library OperatingSystem
|
||||
Library Process
|
||||
Resource ../common.robot
|
||||
|
||||
Suite Setup Setup Suite
|
||||
Suite Teardown Cleanup Suite
|
||||
|
||||
|
||||
*** Variables ***
|
||||
${lab-name} maglevd-test
|
||||
${lab-file} maglevd-lab/maglevd.clab.yml
|
||||
${config-file} maglevd-lab/maglev.yaml
|
||||
${runtime} docker
|
||||
${GRPC_PORT} 9091
|
||||
|
||||
|
||||
*** Test Cases ***
|
||||
Deploy maglevd-test lab
|
||||
${rc} ${output} = Run And Return Rc And Output
|
||||
... ${CLAB_BIN} --runtime ${runtime} deploy -t ${CURDIR}/${lab-file}
|
||||
Log ${output}
|
||||
Should Be Equal As Integers ${rc} 0
|
||||
|
||||
Start maglevd
|
||||
${handle} = Start Process ${MAGLEVD}
|
||||
... --config ${CURDIR}/${config-file}
|
||||
... --grpc-addr :${GRPC_PORT}
|
||||
... --log-level debug
|
||||
... alias=maglevd stdout=${EXECDIR}/tests/out/maglevd.log
|
||||
... stderr=STDOUT
|
||||
Set Suite Variable ${MAGLEVD_HANDLE} ${handle}
|
||||
Sleep 3s Wait for nginx containers and probes to converge
|
||||
|
||||
All backends reach up state
|
||||
[Template] Backend Should Be Up
|
||||
nginx1
|
||||
nginx2
|
||||
nginx3
|
||||
|
||||
Health checks are reaching all backends
|
||||
[Template] Probe Count Should Be Positive
|
||||
nginx1
|
||||
nginx2
|
||||
nginx3
|
||||
|
||||
Pause backend stops probing
|
||||
Maglevc set backend nginx1 pause
|
||||
Backend Should Have State nginx1 paused
|
||||
Sleep 1s
|
||||
${before} = Get Probe Count nginx1
|
||||
Sleep 2s Wait to confirm no new probes arrive
|
||||
${after} = Get Probe Count nginx1
|
||||
Should Be True ${after} == ${before}
|
||||
... Probe count for nginx1 grew while paused: ${before} → ${after}
|
||||
|
||||
Resume backend restarts probing
|
||||
Maglevc set backend nginx1 resume
|
||||
${before} = Get Probe Count nginx1
|
||||
Sleep 2s Wait for resumed probes to accumulate
|
||||
${after} = Get Probe Count nginx1
|
||||
Should Be True ${after} > ${before}
|
||||
... Probe count for nginx1 did not grow after resume: ${before} → ${after}
|
||||
Wait Until Keyword Succeeds 5s 500ms
|
||||
... Backend Should Be Up nginx1
|
||||
|
||||
Disable backend stops probing
|
||||
Maglevc set backend nginx2 disable
|
||||
Backend Should Have State nginx2 removed
|
||||
Backend Should Be Disabled nginx2
|
||||
Sleep 1s
|
||||
${before} = Get Probe Count nginx2
|
||||
Sleep 2s Wait to confirm probes stopped
|
||||
${after} = Get Probe Count nginx2
|
||||
Should Be True ${after} == ${before}
|
||||
... Probe count for nginx2 grew while disabled: ${before} → ${after}
|
||||
|
||||
Enable backend restarts probing
|
||||
Maglevc set backend nginx2 enable
|
||||
${before} = Get Probe Count nginx2
|
||||
Sleep 2s Wait for re-enabled probes to accumulate
|
||||
${after} = Get Probe Count nginx2
|
||||
Should Be True ${after} > ${before}
|
||||
... Probe count for nginx2 did not grow after enable: ${before} → ${after}
|
||||
Wait Until Keyword Succeeds 5s 500ms
|
||||
... Backend Should Be Up nginx2
|
||||
|
||||
|
||||
*** Keywords ***
|
||||
Setup Suite
|
||||
${arch} = Run go env GOARCH
|
||||
Set Suite Variable ${ARCH} ${arch}
|
||||
Set Suite Variable ${MAGLEVD} ${EXECDIR}/build/${ARCH}/maglevd
|
||||
Set Suite Variable ${MAGLEVC} ${EXECDIR}/build/${ARCH}/maglevc
|
||||
|
||||
Cleanup Suite
|
||||
Run Keyword And Ignore Error Terminate Process maglevd kill=true
|
||||
Run ${CLAB_BIN} --runtime ${runtime} destroy -t ${CURDIR}/${lab-file} --cleanup
|
||||
|
||||
Maglevc
|
||||
[Documentation] Run a maglevc command and return its output.
|
||||
[Arguments] ${cmd}
|
||||
${rc} ${output} = Run And Return Rc And Output
|
||||
... ${MAGLEVC} --server\=localhost:${GRPC_PORT} --color\=false ${cmd}
|
||||
Log ${output}
|
||||
Should Be Equal As Integers ${rc} 0
|
||||
RETURN ${output}
|
||||
|
||||
Backend Should Be Up
|
||||
[Arguments] ${name}
|
||||
${output} = Maglevc show backends ${name}
|
||||
Should Match Regexp ${output} state\\s+up
|
||||
|
||||
Backend Should Have State
|
||||
[Arguments] ${name} ${expected_state}
|
||||
${output} = Maglevc show backends ${name}
|
||||
Should Match Regexp ${output} state\\s+${expected_state}
|
||||
|
||||
Backend Should Be Disabled
|
||||
[Arguments] ${name}
|
||||
${output} = Maglevc show backends ${name}
|
||||
Should Match Regexp ${output} enabled\\s+false
|
||||
|
||||
Get Probe Count
|
||||
[Documentation] Return the number of HTTP health-check requests seen in a backend's nginx log.
|
||||
[Arguments] ${name}
|
||||
${output} = Run docker logs clab-${lab-name}-${name} 2>/dev/null | grep -c "GET /" || echo 0
|
||||
${count} = Convert To Integer ${output.strip()}
|
||||
RETURN ${count}
|
||||
|
||||
Probe Count Should Be Positive
|
||||
[Arguments] ${name}
|
||||
${count} = Get Probe Count ${name}
|
||||
Should Be True ${count} > 0
|
||||
... No health-check requests found in nginx logs for ${name}
|
||||
43
tests/01-maglevd/maglevd-lab/maglev.yaml
Normal file
43
tests/01-maglevd/maglevd-lab/maglev.yaml
Normal file
@@ -0,0 +1,43 @@
|
||||
maglev:
|
||||
healthchecker:
|
||||
transition-history: 5
|
||||
|
||||
healthchecks:
|
||||
http-check:
|
||||
type: http
|
||||
port: 80
|
||||
params:
|
||||
path: /
|
||||
response-code: "200"
|
||||
interval: 200ms
|
||||
fast-interval: 100ms
|
||||
down-interval: 1s
|
||||
timeout: 1s
|
||||
rise: 2
|
||||
fall: 2
|
||||
|
||||
backends:
|
||||
nginx1:
|
||||
address: 172.20.30.11
|
||||
healthcheck: http-check
|
||||
nginx2:
|
||||
address: 172.20.30.12
|
||||
healthcheck: http-check
|
||||
nginx3:
|
||||
address: 172.20.30.13
|
||||
healthcheck: http-check
|
||||
|
||||
frontends:
|
||||
http-vip:
|
||||
description: "Test HTTP VIP"
|
||||
address: 192.0.2.1
|
||||
protocol: tcp
|
||||
port: 80
|
||||
pools:
|
||||
- name: primary
|
||||
backends:
|
||||
nginx1: {}
|
||||
nginx2: {}
|
||||
- name: fallback
|
||||
backends:
|
||||
nginx3: {}
|
||||
20
tests/01-maglevd/maglevd-lab/maglevd.clab.yml
Normal file
20
tests/01-maglevd/maglevd-lab/maglevd.clab.yml
Normal file
@@ -0,0 +1,20 @@
|
||||
name: maglevd-test
|
||||
|
||||
mgmt:
|
||||
network: maglevd-test-net
|
||||
ipv4-subnet: 172.20.30.0/24
|
||||
|
||||
topology:
|
||||
nodes:
|
||||
nginx1:
|
||||
kind: linux
|
||||
image: nginx:alpine
|
||||
mgmt-ipv4: 172.20.30.11
|
||||
nginx2:
|
||||
kind: linux
|
||||
image: nginx:alpine
|
||||
mgmt-ipv4: 172.20.30.12
|
||||
nginx3:
|
||||
kind: linux
|
||||
image: nginx:alpine
|
||||
mgmt-ipv4: 172.20.30.13
|
||||
Reference in New Issue
Block a user