Adaptive Routing with BIER

Introduction ECMP is extensively used in modern AI DC networking, often with so called Adaptive Routing mechansim, with which the load/congestion/capacity information for the nexthop, next nexthop, or even the entire path is considered to dynamically adjusts routing paths based on changes in global network conditions, thereby optimizing network performance and resource utilization . Consider the following spine and leaf network (due the limitation of ASCII art, some S-L links are not drawn, but every spine is connected to every leaf):

If L1 detects that S1-L2 link is congested but L1-S2 and S2-L2 are not, it can avoid S1 and choose S2 for the L1-L2 traffic. That's the concept of adaptive routing in a nut shell. Now consider multicast using BIER . Suppose the source is attached to L1, and the receivers are attached to L2 and L3. Without adaptive routing, L1 can send the BIER packet to any of the spine switches, say S1. S1 would replicate it to L2 and L3, but since the S1-L2 link is congested, L2 will experience packet losses. With adaptive routing for BIER, L1 can avoid S1 but send the copy to S2, who then replicate to L2 and L3. If none of the spines has uncongested paths to both L2 and L3, then L1 can replicate to two spines, who then forward to L2 or L3 separately. This can be achieved without changing the BIER forwarding algorithm. Oly the calculation of F-BM and some BIFT entries need to be changed, as described below. Note that, while the example and the text below uses congestion as the criterion, other criteria can be used for adaptive routing as well , and the scheme in this document also works with that - how a BFR neighbor is determined usable or not for a BFER from adaptive routing aspect is orthogonal. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Mode of Operation This is an information document for local behaviors, and there is no nomaltive specification. This secion describe how the adaptive routing is done with BIER. BIER forwarding is based on the F-BM associated with a BFR neighbor, representing all the BFERs reachable via that neighbor. The index of the lowest bit that is set in the incoming bitstring is used to look up a BIFT entry, resulting in a set of BFR neighbors that can be used to reach the BFER represented by that bit. One of the neighbors is chosen, and its F-BM is used to determine the outgoing bitstring for the copy replicated to it. The F-BM is also used to mask bits off the incoming bitstring, and the forwarding iterates for the remaining set bits. The F-BM includes all the bits for BFERs reachable via that neighbor, as described in . Using the extremely simple example of 4-bit bitstring (for only 4 BFERs in the example), L1's BIFT includes four entries: Key Neighbor 1 (L1) self (pop)
2 (l2) S1,S2,S3,S4 3 (l3) S1,S2,S3,S4 4 (L4) S1,S2,S3,S4 The F-BM is 1111 for all S1~S4 when all the links are up and adaptive routing is not used (i.e., congestion is ignored). Now let's say S1-L2 is congested and all other links are fine. With adaptive routing, the second entry then becomes: 2 (l2) S2,S3,S4 S1's F-BM also becomes 1101, because it no longer has an uncongested path to L2. Other entries and F-BMs do not change. When L1 forwards the 0110 packet (for L2 and L3), the 2nd entry is first looked up beause the index of the lowest set bit is 2. S1 is not in the entry and one of (S2, S3,S4) is chosen. Now consider that S2-L3 is also congested. The BIFT becomes: Key Neighbor 1 (L1) self (pop)
2 (L2) S2,S3,S4 3 (L3) S1, S3,S4 4 (L4) S1,S2,S3,S4 S1 F-BM: 1101 S2 F-BM: 1011 For the same packet, in the first lookup, if S2 is chosen out of (S2, S3, S4) in the 2nd entry, the outgoing bitstring becomes 0010 (0110 \& 1011), and another lookup is done for the remaining bits 0100, leading to the 3rd entry. This time a copy is sent to one of (S1, S3, S4) who all have an uncongested path to L3. Now consider that all the Sx-L3 links are congested - there is just no congested path to L3. Now the 3rd entry goes back to the following, and all the F-BMs have L3's bit added back: 3 (L3) S1,S2,S3,S4 Now we can give the formular for the calculation of the F-BM for a neighbor when adaptive routing is used: Let F-BM0 represent all BFERs reachable via the neighbor (the vanilla F-BM in ). Let F-BM1 represent all BFERs reachaable via the neighbor without congestion, based on unicast adaptive routing. Let F-BM2 represent all BFERs that can be reached via all the neighbors without congestoin. ~F-BM2 then represents all BFERs that can't be reached without congestion in any way. F-BM2 = Logical OR of F-BM1 of all neighbors F-BM = F-BM1 | F-BM0 & ~F-MB2 The change to BIFT entries: If a BFER can be reached via some neighbors without congestion, only those neighbors are included in the BIFT entry. When a lookup hits the entry, one of those neighbors will be used. If a BFER can not be reached via any neighbor without congestion, all the neighbors via which the BFER can be reached w/o considering congestion are included in the entry. This means that if there is no uncongested path to a BFER, we'll take chance to foward traffic towards it after all, even though this may sustain or further contribute to the congestion. If it is desired to drop the packet for the BFER, then F-BM1 in step 2 instead of F-BM in step 4 can be used, and the BFER's entry in the BIFT becomes a discard one.

Security Considerations To be added.

IANA Considerations No IANA consideration is needed

Acknowledgments The authors want to thank Duncan Roweth for his collaboration on this adaptive routing scheme for BIER.