Enabling echo mode took the session down

Commit 7786df1467, “bfdd: gate IPv6 echo reflection on known sessions”, is a good change. An unguarded echo reflector will bounce a self-addressed packet back to whoever sends one, which is an amplification primitive. Requiring the packet’s source and local address to name an existing BFD session closes that.

The problem is that the sending side never picks a source address, so the guard also rejects the legitimate echoes of its own peer. Turning on echo-mode takes a working IPv6 session out of service.

Who chooses the source

ptm_bfd_echo_snd() sends IPv6 echoes through the shared per-VRF socket bvrf->bg_echov6. That socket is not bound to any address. So the kernel chooses a source from the outgoing interface, following RFC 6724 source address selection.

On the receiving end, bp_bfd_echo_in() builds its lookup key from that chosen source together with the local address, and calls bfd_key_lookup(). If the kernel picked anything other than the session’s configured local-address, the key does not match a session, the echo is dropped, and the sender’s echo detection eventually declares the path dead.

The session goes down. Not a degraded feature: an outage caused by enabling a feature.

Why most people never see it

RFC 6724 does longest-prefix match first. If your addresses live in different prefixes, the rule picks the one in the peer’s prefix, which is the right one, and everything works.

You need more than one address inside the peer’s own prefix for the selection to become ambiguous. That happens with service or anycast addresses on the link, during renumbering, or with privacy extensions turned on. Common enough to matter, rare enough that the guard shipped without anyone hitting it.

IPv4 is unaffected, and for an interesting reason. bp_udp_send_fp() builds the frame with source equal to destination, so the neighbour’s forwarding plane loops it. The received source is therefore always the peer’s own address, by construction. The v4 path never asks the kernel to choose.

The fix, and why it is not a setsockopt

The source is attached per message as an IPV6_PKTINFO control message rather than set once with setsockopt:

if (pktinfo != NULL) {
        size_t used = msg.msg_controllen ? CMSG_SPACE(sizeof(ttlval)) : 0;

        msg.msg_control = msgctl;
        msg.msg_controllen = used + CMSG_SPACE(sizeof(*pktinfo));

        cmsg = used ? CMSG_NXTHDR(&msg, CMSG_FIRSTHDR(&msg)) : CMSG_FIRSTHDR(&msg);
        cmsg->cmsg_level = IPPROTO_IPV6;
        cmsg->cmsg_type = IPV6_PKTINFO;
        cmsg->cmsg_len = CMSG_LEN(sizeof(*pktinfo));
        memcpy(CMSG_DATA(cmsg), pktinfo, sizeof(*pktinfo));
}

The used dance at the top is because the hop-limit cmsg may already be in the buffer, so this appends rather than overwrites. That is the sort of detail that is invisible in a description and unavoidable in the code.

It cannot be sticky. The same socket carries every session’s echoes and the reflection path. A source set for one session would leak into another session’s reflection, which trades one wrong-source bug for a different one. The reflection call site passes NULL and keeps the kernel’s choice, which is correct there.

The caller side is three lines and says exactly what it is defending against:

/*
 * The echo socket is shared and unbound, so the kernel would
 * otherwise pick any address on the outgoing interface. The peer
 * reflects only echoes whose source names a known session, so an
 * arbitrary source gets the echo dropped there.
 */
memcpy(&pktinfo.ipi6_addr, &bfd->key.local, sizeof(pktinfo.ipi6_addr));
if (bfd->ifp)
        pktinfo.ipi6_ifindex = bfd->ifp->ifindex;
pktinfop = &pktinfo;

Writing a test for source address selection

This was the genuinely fiddly part, and most of the diff.

The topotest needs the kernel to pick the wrong address on purpose. My first instinct was address ordering: add the decoy afterwards and let the rules sort it out. That does not work. Among equally preferred candidates, Linux keeps selecting the address that has been on the interface longest, so adding a decoy later has no effect at all.

What does work is RFC 6724 rule 3, which prefers non-deprecated addresses. Put a second global address on r1 in the session’s prefix, then mark the session’s own local-address deprecated with preferred_lft 0. Now the kernel actively prefers the decoy.

The test also asserts, as a precondition, that the kernel really does select the decoy. Without that assertion, a future change to source selection would make the test pass because the bug can no longer be triggered, rather than because the bug is fixed. A test that quietly stops testing anything is worse than no test.

Echo mode is configured on both routers, because bfdd only opens the per-VRF IPv6 echo socket when a session uses echo. With echo on the sender alone, the peer never reaches the reflection path at all.

Verified both ways: patched, everything passes. With the fix reverted, the reflection and stability assertions fail, r1 logs up -> down reason:echo-failed about every 1.3 seconds, and the source-selection precondition still passes, which is exactly the signature you want from a regression test.

Open as #22920.

What I took from it

The hardening commit was right, and it exposed a latent gap rather than creating one: the sender was always relying on the kernel to guess, and nothing had checked the guess before. Adding a correctness requirement on the receive side turned a silent assumption on the send side into an outage.

And the test taught me more than the fix. Making a kernel reliably do the wrong thing, on demand, took several attempts and a rule in RFC 6724 I had not needed before.