Update the build to multi-arch packages with buildx and qemu on arm64,amd64

README.md

@@ -1,5 +1,80 @@
 # VPP Containerlab Docker image
 
+## User Documentation
+
+The file `vpp.clab.yml` contains an example topology existing of two VPP instances connected each to
+one Alpine linux container, in the following topology:
+
+![learn-vpp](learn-vpp.png)
+
+This container ships with both Bird2 and FRRouting as controlplane agents.
+
+You can deploy:
+*   Bird2: `containerlab deploy --topo vpp-bird.clab.yml`.
+*   FRR: `containerlab deploy --topo vpp-frr.clab.yml`.
+
+three relevant files for VPP are included in this repository:
+1.   `config/vpp*/vppcfg.yaml` configures the dataplane interfaces, including a loopback address.
+1.   `config/vpp*/bird-local.conf` configures the controlplane to enable BFD and OSPF.
+1.   `config/vpp*/frr.conf` configures the controlplane to enable BFD and OSPF.
+
+Once the lab comes up, you can SSH to the VPP containers (`vpp1` and `vpp2`) which will have your
+SSH keys installed (if available). Otherwise, you can log in as user `root` using password `vpp`.
+
+VPP runs its own network namespace called `dataplane`, which is very similar to SR Linux default
+`network-instance`. You can join it to take a look:
+
+```bash
+pim@summer:~/src/vpp-containerlab$ ssh root@vpp1
+root@vpp1:~# nsenter --net=/var/run/netns/dataplane
+root@vpp1:~# ip -br a
+lo               DOWN           
+loop0            UP             10.82.98.0/32 2001:db8:8298::/128 fe80::dcad:ff:fe00:0/64 
+eth1             UNKNOWN        10.82.98.65/28 2001:db8:8298:101::1/64 fe80::a8c1:abff:fe77:acb9/64 
+eth2             UNKNOWN        10.82.98.16/31 2001:db8:8298:1::1/64 fe80::a8c1:abff:fef0:7125/64 
+
+root@vpp1:~# ping 10.82.98.1 ## The vpp2 IPv4 loopback address
+PING 10.82.98.1 (10.82.98.1) 56(84) bytes of data.
+64 bytes from 10.82.98.1: icmp_seq=1 ttl=64 time=9.53 ms
+64 bytes from 10.82.98.1: icmp_seq=2 ttl=64 time=15.9 ms
+^C
+--- 10.82.98.1 ping statistics ---
+2 packets transmitted, 2 received, 0% packet loss, time 1002ms
+rtt min/avg/max/mdev = 9.530/12.735/15.941/3.205 ms
+```
+
+The two clients are running a minimalistic Alpine Linux container, which doesn't ship with SSH by
+default. You can enter the containers as following:
+
+```bash
+pim@summer:~/src/vpp-containerlab$ docker exec -it client1 sh
+/ # ip addr show dev eth1
+531235: eth1@if531234: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 9500 qdisc noqueue state UP 
+    link/ether 00:c1:ab:00:00:01 brd ff:ff:ff:ff:ff:ff
+    inet 10.82.98.66/28 scope global eth1
+       valid_lft forever preferred_lft forever
+    inet6 2001:db8:8298:101::2/64 scope global 
+       valid_lft forever preferred_lft forever
+    inet6 fe80::2c1:abff:fe00:1/64 scope link 
+       valid_lft forever preferred_lft forever
+/ # traceroute 10.82.98.82
+traceroute to 10.82.98.82 (10.82.98.82), 30 hops max, 46 byte packets
+ 1  10.82.98.65 (10.82.98.65)  5.906 ms  7.086 ms  7.868 ms
+ 2  10.82.98.17 (10.82.98.17)  24.007 ms  23.349 ms  15.933 ms
+ 3  10.82.98.82 (10.82.98.82)  39.978 ms  31.127 ms  31.854 ms
+
+/ # traceroute 2001:db8:8298:102::2
+traceroute to 2001:db8:8298:102::2 (2001:db8:8298:102::2), 30 hops max, 72 byte packets
+ 1  2001:db8:8298:101::1 (2001:db8:8298:101::1)  0.701 ms  7.144 ms  7.900 ms
+ 2  2001:db8:8298:1::2 (2001:db8:8298:1::2)  23.909 ms  22.943 ms  23.893 ms
+ 3  2001:db8:8298:102::2 (2001:db8:8298:102::2)  31.964 ms  30.814 ms  32.000 ms
+```
+
+From the vantage point of `client1`, the first hop represents the `vpp1` node, which forwards to
+`vpp2`, which finally forwards to `client2`.
+
+## Developer Documentation
+
 This docker container creates a VPP instance based on the latest VPP release. It starts up as per
 normal, using /etc/vpp/startup.conf (which Containerlab might replace when it starts its
 containers). Once started, it'll execute `/etc/vpp/bootstrap.vpp` within the dataplane. There are
@@ -14,18 +89,33 @@ two relevant files:
 For Containerlab users who wish to have more control over their VPP bootstrap, it's possible to
 bind-mount `/etc/vpp/bootstrap.vpp`.
 
-## Building
+### Building
+
+To build, this container uses Docker's `buildx`, for which on Debian Bookworm it's required to use
+the upstream (docker.com) packages described [[here](https://docs.docker.com/engine/install/debian/)].
+To allow the buildx to build for multi-arch, it's also required to install the Qemu `binfmt`
+emulators, with:
+
+```bash
+docker run --privileged --rm tonistiigi/binfmt --install all
+```
+
+Then, ongoing builds can be cross-platform and take about 1500 seconds on an AMD64 i7-12700T
+The buildx invocation will build 'latest' and then tag it with the current VPP package release,
+which you can get from `vppcfg show version`, like so:
 
 ```bash
 IMG=git.ipng.ch/ipng/vpp-containerlab
 TAG=latest
-docker build --no-cache -f docker/Dockerfile.bookworm -t $IMG docker/
+docker buildx build --push --platform linux/arm64,linux/amd64 \
-docker image tag $IMG $IMG:$TAG
+  --tag $IMG:$TAG -f docker/Dockerfile docker/
-docker push $IMG
+
-docker push $IMG:$TAG
+TAG=v25.02-release
+docker buildx build --push --platform linux/arm64,linux/amd64 \
+  --tag $IMG:$TAG -f docker/Dockerfile docker/
 ```
 
-## Testing the container standalone
+### Testing standalone container
 
 ```bash
 docker network create --driver=bridge clab-network --subnet=192.0.2.0/24 \
@@ -35,11 +125,11 @@ docker run --cap-add=NET_ADMIN --cap-add=SYS_NICE --cap-add=SYS_PTRACE \
            --device=/dev/net/tun:/dev/net/tun \
            --device=/dev/vhost-net:/dev/vhost-net \
            --privileged --name clab-pim \
-           docker.io/pimvanpelt/vpp-containerlab:latest
+           git.ipng.ch/ipng/vpp-containerlab:latest
 docker network connect clab-network clab-pim
 ```
 
-### A note on DPDK
+#### A note on DPDK
 
 DPDK will be disabled by default as it requires hugepages and VFIO and/or UIO to use physical
 network cards. If DPDK at some future point is desired, mapping VFIO can be done by adding this:
@@ -49,10 +139,10 @@ network cards. If DPDK at some future point is desired, mapping VFIO can be done
 
 or in Containerlab, using the `devices` feature:
 
-```
+```yaml
 my-node:
-  image: vpp-containerlab:latest
+  image: git.ipng.ch/ipng/vpp-containerlab:latest
-  kind: vpp
+  kind: fdio_vpp
   devices:
     - /dev/vfio/vfio
     - /dev/net/tun
@@ -71,14 +161,14 @@ $ sudo modprobe uio_pci_generic
 Particularly the VFIO driver needs to be present before one can attempt to bindmount
 `/dev/vfio/vfio` into the container!
 
-## Configuring VPP
+### Configuring VPP
 
 When Containerlab starts the docker containers, it'll offer one or more `veth` point to point
 network links, which will show up as `eth1` and further. `eth0` is the default NIC that belongs to
 the management plane in Containerlab (the one which you'll see with `containerlab inspect`). Before
 VPP can use these `veth` interfaces, it needs to bind them, like so:
 
-```
+```bash
 docker exec -it clab-pim vppctl
 ```
 
@@ -97,7 +187,3 @@ Containerlab will attach these `veth` pairs to the container, and replace our Do
 that waits for all of these interfaces to be added (typically called `if-wait.sh`). In our own CMD,
 we then generate a config file called `/etc/vpp/clab.vpp` which contains the necessary VPP commands
 to take control over these `veth` pairs.
-
-In addition, you can add more commands that'll execute on startup by copying in
-`/etc/vpp/manual-pre.vpp` (to be executed _before_ the containerlab stuff) or
-`/etc/vpp/manual-post.vpp` (to be executed _after_ the containerlab stuff).

config/lab-frr.env

@@ -0,0 +1,2 @@
+BIRD_ENABLED=false
+FRR_ENABLED=true

config/vpp1/frr.conf

@@ -0,0 +1,34 @@
+frr version 10.3
+frr defaults traditional
+hostname vpp1
+log syslog informational
+service integrated-vtysh-config
+!
+ip router-id 10.82.98.0
+!
+interface eth2
+ ip ospf bfd
+ ip ospf cost 10
+ ip ospf network point-to-point
+ ipv6 ospf6 area 0
+ ipv6 ospf6 bfd
+ ipv6 ospf6 cost 10
+ ipv6 ospf6 network point-to-point
+exit
+!
+interface loop0
+ ip ospf passive
+exit
+!
+router ospf
+ redistribute connected
+ network 10.82.98.0/24 area 0
+exit
+!
+router ospf6
+ redistribute connected
+exit
+!
+bfd
+exit
+!

config/vpp1/vppcfg.yaml

@@ -6,7 +6,7 @@ interfaces:
     addresses: [ 10.82.98.65/28, 2001:db8:8298:101::1/64 ]
   eth2:
     description: 'To vpp2'
-    mtu: 9000
+    mtu: 9216
     lcp: eth2
     addresses: [ 10.82.98.16/31, 2001:db8:8298:1::1/64 ]
 loopbacks:

config/vpp2/frr.conf

@@ -0,0 +1,31 @@
+frr version 10.3
+frr defaults traditional
+hostname vpp2
+log syslog informational
+service integrated-vtysh-config
+!
+ip router-id 10.82.98.1
+!
+interface eth2
+ ip ospf bfd
+ ip ospf cost 10
+ ip ospf network point-to-point
+ ipv6 ospf6 area 0
+ ipv6 ospf6 bfd
+ ipv6 ospf6 cost 10
+ ipv6 ospf6 network point-to-point
+exit
+!
+interface loop0
+ ip ospf passive
+exit
+!
+router ospf
+ redistribute connected
+ network 10.82.98.0/24 area 0
+exit
+!
+router ospf6
+ redistribute connected
+exit
+!

config/vpp2/vppcfg.yaml

@@ -6,7 +6,7 @@ interfaces:
     addresses: [ 10.82.98.81/28, 2001:db8:8298:102::1/64 ]
   eth2:
     description: 'To vpp1'
-    mtu: 9000
+    mtu: 9216
     lcp: eth2
     addresses: [ 10.82.98.17/31, 2001:db8:8298:1::2/64 ]
 loopbacks:

docker/Dockerfile

@@ -0,0 +1,30 @@
+FROM ubuntu:noble
+ARG DEBIAN_FRONTEND=noninteractive
+ARG VPP_INSTALL_SKIP_SYSCTL=true
+ARG REPO=release
+EXPOSE 22/tcp
+RUN apt-get update && apt-get -y install curl procps tcpdump iproute2 iptables \
+  iputils-ping net-tools git python3 python3-pip vim-tiny openssh-server bird2 \
+  mtr-tiny traceroute && apt-get clean
+
+# Install VPP
+RUN mkdir -p /var/log/vpp /root/.ssh/ && \
+    curl -s https://packagecloud.io/install/repositories/fdio/${REPO}/script.deb.sh | bash && \
+    apt-get -y install vpp vpp-plugin-core && apt-get clean
+
+# Build vppcfg
+RUN pip install --break-system-packages build netaddr yamale argparse pyyaml ipaddress && \
+    git clone https://git.ipng.ch/ipng/vppcfg.git && cd vppcfg && python3 -m build && \
+    pip install --break-system-packages dist/vppcfg-*-py3-none-any.whl
+
+# Install FRR
+RUN curl -s -o /usr/share/keyrings/frrouting.gpg https://deb.frrouting.org/frr/keys.gpg && \
+    echo deb '[signed-by=/usr/share/keyrings/frrouting.gpg]' https://deb.frrouting.org/frr bookworm frr-stable \
+         > /etc/apt/sources.list.d/frr.list && \
+    apt -y update && apt -y install frr frr-pythontools && apt clean
+
+# Config files
+COPY files/etc/ /etc/
+COPY files/init-container.sh /sbin/
+RUN chmod 755 /sbin/init-container.sh
+CMD ["/sbin/init-container.sh"]

docker/Dockerfile.bookworm

@@ -1,25 +0,0 @@
-FROM debian:bookworm
-ARG DEBIAN_FRONTEND=noninteractive
-ARG VPP_INSTALL_SKIP_SYSCTL=true
-ARG REPO=release
-EXPOSE 22/tcp
-RUN apt-get update && apt-get -y install curl procps tcpdump iproute2 iptables \
-  iputils-ping net-tools git python3 python3-pip vim-tiny openssh-server bird2 \
-  mtr-tiny traceroute && apt-get clean
-
-# Install VPP
-RUN mkdir -p /var/log/vpp /root/.ssh/
-RUN curl -s https://packagecloud.io/install/repositories/fdio/${REPO}/script.deb.sh |  bash
-RUN apt-get update && apt-get -y install vpp vpp-plugin-core && apt-get clean
-
-# Build vppcfg
-RUN pip install --break-system-packages build netaddr yamale argparse pyyaml ipaddress
-RUN git clone https://github.com/pimvanpelt/vppcfg.git && cd vppcfg && python3 -m build && \
-    pip install --break-system-packages dist/vppcfg-*-py3-none-any.whl
-
-# Config files
-COPY files/etc/vpp/* /etc/vpp/
-COPY files/etc/bird/* /etc/bird/
-COPY files/init-container.sh /sbin/
-RUN chmod 755 /sbin/init-container.sh
-CMD ["/sbin/init-container.sh"]

docker/files/etc/frr/daemons

@@ -0,0 +1,17 @@
+# These are the daemons that FRR will use for VPP Containerlab
+# NOTE: we need to run in the 'dataplane' network namespace, and use the `dplane_fpm_nl` plugin
+bgpd=yes
+ospfd=yes
+ospf6d=yes
+bfdd=yes
+ldpd=yes
+
+vtysh_enable=yes
+watchfrr_options="--netns=dataplane"
+zebra_options="  -A 127.0.0.1 -s 67108864 -M dplane_fpm_nl"
+bgpd_options="   -A 127.0.0.1"
+ospfd_options="  -A 127.0.0.1"
+ospf6d_options=" -A ::1"
+staticd_options="-A 127.0.0.1"
+bfdd_options="   -A 127.0.0.1"
+ldpd_options="   -A 127.0.0.1"

docker/files/etc/frr/frr.conf

@@ -0,0 +1,10 @@
+# This is the VPP Containerlab default FRR configuration.
+
+frr defaults traditional
+log syslog informational
+ip forwarding
+ipv6 forwarding
+service integrated-vtysh-config
+!
+ip router-id 192.0.2.1
+!

docker/files/init-container.sh

@@ -5,6 +5,7 @@ CLAB_VPP_FILE=${CLAB_VPP_FILE:=/etc/vpp/clab.vpp}
 VPPCFG_VPP_FILE=${VPPCFG_VPP_FILE:=/etc/vpp/vppcfg.vpp}
 NETNS=${NETNS:="dataplane"}
 BIRD_ENABLED=${BIRD_ENABLED:="true"}
+FRR_ENABLED=${FRR_ENABLED:="false"}
 
 echo "Creating dataplane namespace"
 /usr/bin/mkdir -p /etc/netns/$NETNS
@@ -25,6 +26,13 @@ if [ "$BIRD_ENABLED" == "true" ]; then
   /usr/bin/nsenter --net=/var/run/netns/$NETNS /usr/sbin/bird -u bird -g bird
 fi
 
+if [ "$FRR_ENABLED" == "true" ]; then
+  echo "Starting FRRouting in $NETNS"
+  ROUTERID=$(ip -br a show eth0 | awk '{ print $3 }' | cut -f1 -d/)
+  sed -i -e "s,^ip router-id .*,ip router-id $ROUTERID," /etc/frr/frr.conf
+  /etc/init.d/frr start
+fi
+
 echo "Generating $CLAB_VPP_FILE"
 : > $CLAB_VPP_FILE
 MTU=9216

vpp-bird.clab.yml

@@ -0,0 +1,39 @@
+name: learn-vpp
+prefix: ""
+
+topology:
+  kinds:
+    fdio_vpp:
+      image: git.ipng.ch/ipng/vpp-containerlab:latest
+      startup-config: config/__clabNodeName__/vppcfg.yaml
+      binds:
+        - config/__clabNodeName__/bird-local.conf:/etc/bird/bird-local.conf:ro
+    linux:
+      image: alpine:latest
+
+  nodes:
+    vpp1:
+      kind: fdio_vpp
+    vpp2:
+      kind: fdio_vpp
+    client1:
+      kind: linux
+      exec:
+        - ip link set address 00:c1:ab:00:00:01 dev eth1
+        - ip addr add 10.82.98.66/28 dev eth1
+        - ip route add 10.82.98.0/24 via 10.82.98.65
+        - ip addr add 2001:db8:8298:101::2/64 dev eth1
+        - ip route add 2001:db8:8298::/48 via 2001:db8:8298:101::1
+    client2:
+      kind: linux
+      exec:
+        - ip link set address 00:c1:ab:00:00:02 dev eth1
+        - ip addr add 10.82.98.82/28 dev eth1
+        - ip route add 10.82.98.0/24 via 10.82.98.81
+        - ip addr add 2001:db8:8298:102::2/64 dev eth1
+        - ip route add 2001:db8:8298::/48 via 2001:db8:8298:102::1
+
+  links:
+    - endpoints: ["vpp1:eth2", "vpp2:eth2"]
+    - endpoints: ["client1:eth1", "vpp1:eth1"]
+    - endpoints: ["client2:eth1", "vpp2:eth1"]

vpp-frr.clab.yml

@@ -0,0 +1,41 @@
+name: learn-vpp
+prefix: ""
+
+topology:
+  kinds:
+    fdio_vpp:
+      image: git.ipng.ch/ipng/vpp-containerlab:latest
+      startup-config: config/__clabNodeName__/vppcfg.yaml
+      binds:
+        - config/__clabNodeName__/frr.conf:/etc/frr/frr.conf
+      env-files:
+        - config/lab-frr.env
+    linux:
+      image: alpine:latest
+
+  nodes:
+    vpp1:
+      kind: fdio_vpp
+    vpp2:
+      kind: fdio_vpp
+    client1:
+      kind: linux
+      exec:
+        - ip link set address 00:c1:ab:00:00:01 dev eth1
+        - ip addr add 10.82.98.66/28 dev eth1
+        - ip route add 10.82.98.0/24 via 10.82.98.65
+        - ip addr add 2001:db8:8298:101::2/64 dev eth1
+        - ip route add 2001:db8:8298::/48 via 2001:db8:8298:101::1
+    client2:
+      kind: linux
+      exec:
+        - ip link set address 00:c1:ab:00:00:02 dev eth1
+        - ip addr add 10.82.98.82/28 dev eth1
+        - ip route add 10.82.98.0/24 via 10.82.98.81
+        - ip addr add 2001:db8:8298:102::2/64 dev eth1
+        - ip route add 2001:db8:8298::/48 via 2001:db8:8298:102::1
+
+  links:
+    - endpoints: ["vpp1:eth2", "vpp2:eth2"]
+    - endpoints: ["client1:eth1", "vpp1:eth1"]
+    - endpoints: ["client2:eth1", "vpp2:eth1"]

vpp.clab.yml

@@ -1,42 +0,0 @@
-name: learn-vpp
-prefix: ""
-
-topology:
-  kinds:
-    fdio_vpp:
-      image: git.ipng.ch/ipng/vpp-containerlab:latest
-    linux:
-      image: alpine:latest
-
-  nodes:
-    vpp1:
-      kind: fdio_vpp
-      binds:
-        - config/vpp1/vppcfg.yaml:/etc/vpp/vppcfg.yaml:ro
-        - config/vpp1/bird-local.conf:/etc/bird/bird-local.conf:ro
-    vpp2:
-      kind: fdio_vpp
-      binds:
-        - config/vpp2/vppcfg.yaml:/etc/vpp/vppcfg.yaml:ro
-        - config/vpp2/bird-local.conf:/etc/bird/bird-local.conf:ro
-    client1:
-      kind: linux
-      exec:
-        - ip link set address 00:c1:ab:00:00:01 dev eth1
-        - ip addr add 10.82.98.66/28 dev eth1
-        - ip route add 10.82.98.0/24 via 10.82.98.65
-        - ip addr add 2001:db8:8298:101::2/64 dev eth1
-        - ip route add 2001:db8:8298::/48 via 2001:db8:8298:101::1
-    client2:
-      kind: linux
-      exec:
-        - ip link set address 00:c1:ab:00:00:02 dev eth1
-        - ip addr add 10.82.98.82/28 dev eth1
-        - ip route add 10.82.98.0/24 via 10.82.98.81
-        - ip addr add 2001:db8:8298:102::2/64 dev eth1
-        - ip route add 2001:db8:8298::/48 via 2001:db8:8298:102::1
-
-  links:
-    - endpoints: ["vpp1:eth2", "vpp2:eth2"]
-    - endpoints: ["client1:eth1", "vpp1:eth1"]
-    - endpoints: ["client2:eth1", "vpp2:eth1"]

vpp.clab.yml

@@ -0,0 +1 @@
+vpp-bird.clab.yml