Multicast Routing In Fat Trees

Introduction Because of the simple north-south regular topology in Fat Tree networks, the PIM-Bidir solution is extended for multicast in RIFT in this document, referred to as MRIFT. The following is a summary of the changes and adaptations compared to PIM-Bidir. With PIM-Bidir, PIM joins are sent towards a Rendezvous Point Address (RPA), which could be an address not belonging to any router. The RPA does belong to a RP Link (RPL), which could be attached to a single router or multiple routers (e.g. RPL is a LAN). With MRIFT, there is no concept of RPA any more (joins are simply sent northbound). The joins are terminated on some sub-TOF nodes and the RPL is simulated by a spanning tree among some TOF and sub-TOF nodes. Instead of (*,G) trees in PIM-Bidir, MRIFT uses (*,G-Prefix) trees, where the G-Prefix could be *, G, or anything in between (e.g., 225.1.1.0/24). For light flows, they could just follow the (*,*) tree. For heavy flows, individual (*,G) trees could be built. For medium flows, some (*,G-prefix) trees could be shared. All the First Hop Routers (FHRs, connecting to sources) and the Last Hop Routers (LHRs, connecting to receivers) of a particular (*,G) flow must agree on whether a (*,*) or (*,G) or (*,G-prefix) tree is used for the flow so that they all join the same tree. This is done via out of band control outside the scope of this document. Because of the rich connections in Fat Trees, a router has to choose one of its many north neighbors to send join to. This is done through hashing. The hashing algorithm should lead to several but not too many routers choosing the same north neighbor, so that fewer routers are involved in multicast traffic forwarding, yet none of those routers are overburdened by replicating to too many downstream neighbors. Instead of PIM messages, RIFT's own TIEs are used, similar to the concept in . This introduces the concept of neighbor-scoped flooding - a multicast TIE is sent only to a chosen upstream north neighbor that consumes it and then regenerates a new TIE for the next upstream. When a join reaches a sub-TOF node, the normal join process stops. This forms a sub-tree rooted at this sub-TOF node. Multiple sub- trees of the same tree may be joined by a single TOF node, or they may have to be connected by a spanning tree serving as the RPL. For example, in the following topology, in normal situations the two sub- tree roots for the two pods, say Spine111 and Spine121, may be joined by TOF21, but if the TOF21-Spine121 link is down, then TOF22 may be used, and if the TOF22-Spine111 link is also down, then Spine111 and Spine121 will have to be joined via Spine111-TOF21-Spine112-TOF22-Spine121.

E/W . | | | | | | | | | . P111/2| |P121 | | | | S v . ^ ^ ^ ^ | | | | . | | | | | | | | . +--------------+ | +-----------+ | | | +---------------+ . | | | | | | | | . South +-----------------------------+ | | ^ . | | | | | | | All TIEs . 0/0 0/0 0/0 +-----------------------------+ | . v v v | | | | | . | | +-+ +<-0/0----------+ | | . | | | | | | | | .+-+----++ optional +-+----++ ++----+-+ ++-----++ .| | E/W link | | | | | | .|Spin111+----------+Spin112| |Spin121| |Spin122| .+-+---+-+ ++----+-+ +-+---+-+ ++---+--+ . | | | South | | | | . | +---0/0--->-----+ 0/0 | +----------------+ | . 0/0 | | | | | | | . | +---<-0/0-----+ | v | +--------------+ | | . v | | | | | | | .+-+---+-+ +--+--+-+ +-+---+-+ +---+-+-+ .| | (L2L) | | | | Level 0 | | .|Leaf111~~~~~~~~~~~~Leaf112| |Leaf121| |Leaf122| .+-+-----+ +-+---+-+ +--+--+-+ +-+-----+ . + + \ / + + . Prefix111 Prefix112 \ / Prefix121 Prefix122 . multi-homed . Prefix .+---------- Pod 1 ---------+ +---------- Pod 2 ---------+ ]]>

Specifications

Multicast Capability A new optional field is added to the NodeCapabilities to indicate that the node is enabled for multicast:

Optional Per-neighbor Flooding Scope This document introduces an optional per-neighbor flooding scope for TIEs:

When a node originates a TIE with a per-neighbor flooding scope, it is sent to the specified neighbor only. When a node receives a TIE with per-neighbor flooding scope, it is accepted only if the node is the specified neighbor, and it is not reflooded any further.

Multicast TIE Currently the multicast TIEs are only N-TIEs with per-neighbor flooding scope except on TOFs and sub-TOFs. If a multicast TIE is received from a node south of sub-TOFs without the per-neighbor flooding scope specified, it MUST be discarded.

sub_tof_children; } /** Type of TIE. ... */ enum TIETypeType { ... TIETypeIPMulticast = 11, TIETypeMaxValue = 12, } /** Single element in a TIE. ... */ union TIEElement { ... /** IP multicast elements. */ 10: optional IPMulticastTIEElement ip_multicast; } ]]>

Building Spanning Tree among TOFs and sub-TOFs Note: this is still subject to further discussion/change. It may be replaced by another scheme upon further discussions. If a sub-TOF node is the root of a sub-tree for a (*, G-prefix) tree, it hashes to a TOF neighbor as its parent for the tree, and originates a corresponding multicast N-TIE with the per-neighbor flooding scope, targeted at the chosen ToF neighbor. A receiving TOF node originates a corresponding S-TIE without the per-neighbor flooding scope. The sub_tof_children list all of sub- TOF nodes that have chosen the TOF as parent. When a sub-TOF node that is the root of a sub-tree receives from TOF neighbors the S-TIEs for the same tree, it chooses, from all those S-TIEs, the one with the highest originating system-id as the root. If the chosen root is not already its parent, it re-parents to the chosen root. After the above steps, if a non-root TOF node remains as the chosen parent of some sub-TOF nodes, the TOF node needs to join towards the root through some intermediate sub-TOF and TOF nodes. If it has a sub-TOF neighbor listed in the sub_tof_children of the root, it originates an S-TIE with the per- neighbor flooding scope set to the sub-TOF neighbor, i.e. the sub-TOF neighbor now becomes the parent of the TOF node (that is a parent of some other sub-TOF nodes) and now the two sub-trees are joined at the the sub-ToF neighbor. In case the TOF node does not have a neighbor listed in the sub_tof_children of the S-TIE for the root, it finds a sub-ToF node that is a next hop of the route to the root, and originates an S-TIE with per-neighbor flooding scope set to the sub-ToF neighbor, as in the above case where the neighbor is a sub_tof_child of the root. When the neighbor receives the S-TIE join, it further joins towards the root, either direcly or indirectly via another ToF, by originating a neighbor flooding scoped N-TIE. In the indirect case, the ToF repeats the above steps and joins another sub-ToF node.

Root Resilience Considerations While the sub-trees eventually converges to a single root, if the root fails at the node level or loses all its links, a new root will be automatically chosen by the above procedure.

Multiplane Considerations Because the ultimate root of the tree is on a single ToF node, the planes must be connected via ToF rings, which is assumed for multi-plane RIFT. In the single plane case, or with the ToF rings for multi-planes, it is guaranteed that the ToFs and sub-ToFs are contiguous due to the south reflection and east-west ring connection, unless the topology is partitioned due to wide-spread failures, in which case the multicast trees are also partitioned.

Security Considerations To be provided.

Acknowledgements The authors thank Bruno Rijsman and Antoni Przygenda for their review and suggestions.