Shipping it

Until September, the only place xdp-bfd had ever run was my own test setup, built from source. I wanted it to be something someone else could install next to their FRR and just run.

Packages

I made packages for Debian, Ubuntu and Fedora, and I wanted them to run without root. So I went through what the engine actually needs, and it came down to four capabilities. Two are for loading the XDP program, one is for attaching it to an interface, and the last is for the raw socket that echo packets go out on. The service runs as its own user with just those four.

Older distros couldn’t even build it, because it needed clang 20. The problem was a 32-bit atomic operation that older clang versions can’t compile for BPF. Making those atomics 64-bit fixed it, and it builds with clang 17 now.

Which kernels

For kernels, I stopped going by version numbers and just checked whether the XDP program loads. Most kernels from 6.1 up are fine. AlmaLinux 9 says it’s 5.14 and still loads it, because Red Hat backports the BPF parts of newer kernels. Debian 11 and the stock kernel on Ubuntu 22.04 don’t.

Which FRR

The FRR version mattered more, so I ran the FRR tests against 13 releases and master. Everything older than 10.1.2 connected to xdp-bfd and then dropped the connection about once a second, and no session ever came up. bfdd was going into its reconnect code right after connecting and closing the socket. That was fixed upstream in 10.1.2, so that’s the oldest version that works.

Even on newer releases, two of the bugs from my FRR post still apply, because the fixes are only on master. The UNIX socket doesn’t work, so it has to be TCP, and bfdd only gets about 58 sessions across to xdp-bfd each time it connects.

VyOS

Then I tried it on VyOS, which comes with its own FRR. Sessions came up on the fast path, except the ones configured without a local address. bfdd was handing those to xdp-bfd with the local address left blank. XDP looks sessions up by both addresses, so nothing ever matched, and the peer’s packets got dropped as unknown. Now xdp-bfd fills in the address the kernel would use to reach that peer.

Restarts and the README

The restart handover from part 6 went into the package too, so systemctl reload and package upgrades would hand over without dropping anything. Both passed CI. Neither actually worked on a real install, which I only found out in part 9.

The README had grown to 786 words by then. I cut it down to 266, just what xdp-bfd is and how to build and run it, and moved everything else to the wiki.

Next, running it on real hardware.