diff --git a/README.md b/README.md index 5390255..63d4da6 100644 --- a/README.md +++ b/README.md @@ -106,12 +106,13 @@ which you can get from `vppcfg show version`, like so: ```bash IMG=git.ipng.ch/ipng/vpp-containerlab +ARCH=linux/$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/') TAG=latest -docker buildx build --push --platform linux/arm64,linux/amd64 \ +docker buildx build --load --platform $ARCH \ --tag $IMG:$TAG -f docker/Dockerfile docker/ TAG=v25.10-release -docker buildx build --push --build-arg REPO=2510 --platform linux/arm64,linux/amd64 \ +docker buildx build --load --build-arg REPO=2510 --platform $ARCH \ --tag $IMG:$TAG -f docker/Dockerfile docker/ ``` @@ -125,20 +126,104 @@ do not expose NUMA topology via sysfs, such as OrbStack on Apple Silicon; VPP 26 Point `--build-context vppdebs=` at a directory containing `libvppinfra_*.deb`, `vpp_*.deb`, and `vpp-plugin-core_*.deb`. If the context is not provided, the build falls back to packagecloud as normal. The `.deb` files are bind-mounted during the build and never stored -in an image layer. +in an image layer. **Note:** the directory must contain `.deb` files for exactly one VPP version; +if multiple versions are present the glob patterns will match ambiguously and the build will fail. ```bash -# Build from locally compiled VPP packages (e.g. from ~/src/vpp after make build): +# Build from locally compiled VPP packages (e.g. from ~/src/vpp after make pkg-deb): IMG=git.ipng.ch/ipng/vpp-containerlab -docker buildx build --load --platform linux/arm64 \ - --build-context vppdebs=~/src/vpp/build-root \ +ARCH=linux/$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/') +VPPDEBS=~/src/vpp/build-root +docker buildx build --load --platform $ARCH \ + --build-context vppdebs=$VPPDEBS \ --tag $IMG:latest -f docker/Dockerfile docker/ # Build from packagecloud as normal (no --build-context needed): -docker buildx build --push --platform linux/arm64,linux/amd64 \ +docker buildx build --load --platform $ARCH \ --tag $IMG:latest -f docker/Dockerfile docker/ ``` +### Multiarch + +Building a combined `linux/amd64` + `linux/arm64` manifest requires two machines building natively +— one per architecture. The setup below uses `summer` (amd64, Linux) and `jessica` (arm64, macOS +running OrbStack). **VPP must be compiled on each machine before building the Docker image**, because +the sideloader mounts locally built `.deb` files that are architecture-specific. + +#### Setup + +On `jessica`, the Docker daemon runs inside OrbStack's Linux VM. Expose its SSH port so `summer` +can reach it. OrbStack listens on `127.0.0.1:32222`; add a jump-host entry to `~/.ssh/config` on +`summer`: + +``` +Host jessica-orb + HostName 127.0.0.1 + Port 32222 + User pim + ProxyCommand ssh jessica -W 127.0.0.1:32222 + IdentityFile ~/.ssh/jessica-orb-key + IdentitiesOnly yes + UserKnownHostsFile /dev/null + StrictHostKeyChecking no +``` + +Copy OrbStack's SSH key from `jessica` to `summer`: + +```bash +scp jessica:~/.orbstack/ssh/id_ed25519 ~/.ssh/jessica-orb-key +chmod 600 ~/.ssh/jessica-orb-key +``` + +Verify the full chain works: + +```bash +ssh jessica-orb 'uname -m && docker info | head -3' +# expected: aarch64 +``` + +Create the multiarch builder (run once on `summer`): + +```bash +docker buildx create --name multiarch --driver docker-container --platform linux/amd64 --node summer-amd64 +docker buildx create --append --name multiarch --driver docker-container --platform linux/arm64 --node jessica-arm64 ssh://jessica-orb +docker buildx inspect multiarch --bootstrap +``` + +#### Build + +Build VPP on both machines first (`make pkg-deb` in your VPP source tree on both `summer` and the +OrbStack VM on `jessica`). When sideloading `.deb` files, Docker sends the build context from the +client to every builder node — meaning `summer`'s amd64 debs would be sent to `jessica-orb` for +the arm64 build (wrong arch). The solution is to build each platform separately on its native +machine and combine them into a manifest. + +```bash +IMG=git.ipng.ch/ipng/vpp-containerlab +VPPDEBS=~/src/vpp/build-root + +# Step 1: build amd64 on summer, push with platform tag +docker buildx build --platform linux/amd64 \ + --build-context vppdebs=$VPPDEBS \ + --push --tag $IMG:latest-amd64 \ + -f docker/Dockerfile docker/ + +# Step 2: build arm64 natively on jessica-orb, push with platform tag +# (repo and VPP debs must be present on jessica-orb at the same paths) +ssh jessica-orb "cd ~/src/vpp-containerlab && \ + docker buildx build --platform linux/arm64 \ + --build-context vppdebs=$VPPDEBS \ + --push --tag $IMG:latest-arm64 \ + -f docker/Dockerfile docker/" + +# Step 3: combine into a single multi-arch manifest and push in one step +# (docker buildx build --push produces manifest lists, so use imagetools, not docker manifest) +docker buildx imagetools create \ + --tag $IMG:latest \ + $IMG:latest-amd64 \ + $IMG:latest-arm64 +``` + ### Testing standalone container ```bash