Multihop, and a test that validated nothing

Multihop looks like the smallest milestone in the project.

Single-hop BFD proves adjacency with the TTL: a control packet must arrive at 255, so anything routed has been decremented and is rejected. Multihop gives that up by definition, since packets cross routers and arrive lower, and the receiver enforces a configured minimum instead.

One comparison changes from == 255 to >= minimum. The plumbing was already there and being thrown away: the dataplane registration carries a TTL byte and a multihop flag, bfdd fills both, and the engine parsed the byte and discarded it while the reject mask refused any session carrying the flag.

Where the comparison can live

The GTSM check sits in the parser, before any session lookup, and that placement is the point: a spoofed flood at TTL 64 dies in a handful of instructions having touched no map.

But a per-session minimum is not known until the session is found. Enforcing it there would mean every rejected packet paying for a hash lookup first, trading away the cheapest rejection path in the engine to support a mode most deployments never enable.

The resolution is a single bit in a flags map saying whether any multihop session exists at all. When it is clear, the parser behaves exactly as before, byte for byte. When it is set, the verdict for a below-255 packet defers to after the config lookup:

/* Deferred GTSM. A control packet that did not arrive at 255 is
 * acceptable only if it names a configured session whose minimum
 * admits it. An unconfigured pair must still drop: the
 * promiscuous PASS above exists for observation, not to relax
 * GTSM. Single-hop sessions carry min_ttl 255, so nothing below
 * 255 reaches them and their behaviour is unchanged. */
__u8 pttl = iph ? iph->ttl : (ip6 ? ip6->hop_limit : 0);

if (pttl != 255) {
        __u32 mt = (cfg && cfg->min_ttl) ? cfg->min_ttl : 255;

        if (!cfg || pttl < mt) {
                count(3);
                return XDP_DROP;
        }
}

The !cfg half of that condition matters more than it looks. The lookup’s miss path returns XDP_PASS for the standalone observer, so dropping only on pttl < mt would have quietly started leaking low-TTL packets for unconfigured address pairs straight to userspace, undoing one of the hardening guarantees from part 6. A relaxation added for one feature reopening a hole closed by another is the kind of interaction that no single test looks for.

Note also min_ttl defaulting to 255 when there is no config. Single-hop sessions carry 255, so the comparison is unchanged for them and the multihop code costs them nothing but a branch that is never taken.

The test that matters is not that multihop works. It is that a packet at TTL 200 aimed at a single-hop session is still dropped while multihop is active elsewhere on the same box. Enabling a relaxation for one session must not relax anything for another.

Then the milestone taught the same lesson the project keeps teaching

At this point I had a clean change, a defensible design for where the check lives, and a harness that exercised all three cases: above the minimum accepted, below it dropped, single-hop still strict. Everything passed. I was ready to call it done in an afternoon, which should have been the warning.

It validated nothing. The injector was sending to UDP 3784, and RFC 5883 multihop runs on 4784.

Real multihop traffic was reaching the parser, failing a port check that only knew about single-hop, and being passed to a stack where nothing was listening for it. The capture said so immediately: the neighbour transmitting to 4784, the engine replying to 3784, tcpdump labelling one Multihop and the other Control.

A test that agrees with you because it exercises the wrong thing is worse than no test, and the only defence is to check what the wire actually carries rather than what the harness was told to send.

The reply had a subtler version of the same problem

The kernel bounce rewrites the received frame in place and sends it back, which for single-hop is correct by accident: the packet arrives at 255 and leaves at 255.

For multihop the reply would go out already decremented, lose more crossing back, and be measured against the peer’s own minimum. The session would establish in one direction and fail in the other, and it would present as a peer that flaps rather than as a TTL problem.

RFC 5883 asks for multihop to be sent at 255 precisely so the receiver can count hops, which the userspace path had always done via a socket option. The bounce now matches it, with an incremental checksum update for v4 and a plain assignment for v6.

One IPv6 asymmetry worth recording

The single-hop socket sets IPV6_MINHOPCOUNT to 255, so the kernel discards low-hop packets before userspace ever sees them. A good defence there and a fatal one for multihop, where every packet arrives below 255 by definition.

The multihop socket deliberately omits it and lets XDP enforce the per-session minimum instead, which loses nothing, because that is where the minimum is known anyway.

Reading the validation took a trick

Both families were validated the same way: above the minimum accepted, below it dropped, a single-hop session still strict at 255 with multihop live, and an injected packet below 255 reflected back at 255.

The last of those needed a trick to read, because injected packets and their replies are indistinguishable from live session traffic by address and port. But scapy leaves the IPv6 traffic class at zero while FRR sets it, and the bounce preserves whatever arrived, so the replies to injected packets identify themselves.

That surfaced a small trap of its own: tcpdump omits the class and flowlabel fields entirely when both are zero, so a pattern written against the normal output format silently matches none of them. Twenty replies sat in the capture looking, to a careless grep, like nothing at all.

Next

The method, which is the only part of this project I would defend without qualification.