BGP Signaling for Multipath Traffic Engineering Junction States

Introduction describes the architecture and framework for Multipath Traffic Engineering (MPTE). A signaling approach was described, which could be implemented via extensions to RSVP, PCEP, or BGP. This document specifies how to signal MPTE Directed Acyclic Graphs (DAGs), in particular, how to provision junctions that make up an MPTE DAG, using a new AFI/SAFI, the MPTE AFI/SAFI, in BGP. specifies the BGP extensions to signal multicast replication states to multicast tree nodes. Much of the concepts and extensions can be used to signal MPTE Junction states. This section describes how that is achieved, and the difference between multicast signaling and MPTE signaling.

Mode of Operation While the BGP signaling for MPTE is not limited to Data Centers (DCs), a DC using EBGP signaling is used as an example. Assume the EBGP sessions between switches in the DC support the MPTE SAFI for the signaling of junction states. A future revision of this document will describe the use of Route Reflectors to a) isolate MPTE from the other functions of the EBGP mesh (basic routing), and b) scale sessions. For each DAG, its Signaling Source (SS), which could be a controller or an ingress switch, originates a set of BGP routes of the MPTE SAFI, one for each junction node. The route is referred to as a Junction route, and MUST carry a Route Target (RT) to target the route at the corresponding junction node. Once the route is propagated to the targeted node, the matched RT causes the route to be imported by the node and stopped from being propagated further. Before the matching, the route is propagated by the BGP infrastructure. Before a junction node has at least one path set up to an egress, its upstream node should not start sending traffic to it. This ordered control is preferably done in a hop-by-hop fashion, like in the RSVP-TE case. When a junction has its local state set up for a DAG (starting with an egress node), it originates a RESV route for each of its PHOPs for the DAG, carrying the encapsulation information (e.g., an MPLS label) the PHOP should use to reach it (see "Ordered Control" below; bandwidth is not carried in the RESV route). The upstream node repeats the process, and eventually the ingress node can start sending traffic. Note that a junction node may originate RESV routes before it receives from all its NHOPs. When more or updated RESV routes are received from its downstream, or when some of its downstream nodes are removed or no longer reachable, it will send updated RESV routes to its PHOPs. As an option, the ordered control could be done by the signaling source (SS). In this case, the encapsulation information (e.g., MPLS labels) can be assigned by the SS and included in the junction routes (the label assignment options are detailed in ). After a junction node installs the forwarding state, it sends a Junction Confirmation route to the SS (see "Sending Junction Confirmation" below), which will tally the result and notify the ingress when and how much traffic can be put onto the DAG. This is as if the junction nodes were programmed with static routes, which shifts the burden/complexity to the SS.

Collecting Topology/TE Information Typically, Traffic Engineering uses the IGP (via TE extensions) to distribute topology and TE information. That is not an option for a DC that uses BGP signaling. BGP-LS is a mechanism using BGP extensions to collect link state and TE information that has been signaled by IGP. Typically, the information is distributed to a few collectors (e.g., controllers) from a few distributors (e.g., IGP border routers). This document suggests using BGP LS to distribute TE information for MPTE. Every switch is a distributor of its local information. If distributed calculation is used, each switch is also a collector of other switches' local information. More details will be provided.

Considerations for BGP Signaling outlined the information carried in the JUNCTION message. When implemented in BGP, the MC Address, MPTED ID, Junction Node Address, and Originating Node Address are encoded in the NLRI of a new SAFI (see "AFI/SAFI and NLRI" below); the Tunnel Type is implicit (an MPTE DAG uses a single, uniform tunnel/encapsulation type for all its junctions) and is not separately encoded. The Junction route (Type 1, see "AFI/SAFI and NLRI" below) additionally carries, as a non-key trailing part of the NLRI present only on reachable announcements (omitted on withdrawals), the Junction Bandwidth and a DAG Instance ID identifying the current (possibly in-place-updated) version of this junction's signaling state. Junction Bandwidth is the bandwidth to be admitted/reserved at this junction (not necessarily uniform across the whole DAG); it is not split per-NHOP on the wire. Instead, each node derives, for every NHOP of the junction, the bandwidth to admit/reserve on the corresponding TE-link by multiplying this Junction Bandwidth by the NHOP's load share (the Tunnel Weight sub-TLV, below) -- i.e., only the junction's aggregate bandwidth is carried explicitly on the wire, while the per-NHOP split is reconstructed locally at each hop. The route also carries no protocol next-hop: since all of a junction's forwarding information (the PHOP/NHOP list, labels, and load shares) is carried in the TEA described below, a conventional, RIB-resolved next-hop address would serve no forwarding purpose for this SAFI. Implementations SHOULD omit the next-hop entirely (encode a zero-length next-hop in the MP_REACH_NLRI attribute, as some other SAFIs, e.g. Flow-Spec, already do in some configurations) rather than populate it with an address that is never used to forward traffic and would otherwise need RIB resolution and a "show route" nexthop display for no benefit. All the NHOP and PHOP information is encoded into a Tunnel Encapsulation Attribute (TEA) , with extensions specified in and this document. A TEA encodes a list of "tunnels", each of which could be a real tunnel or just an interface or neighbor. As explained in , when a TEA is attached to an NLRI of MCAST-TREE SAFI, corresponding traffic is replicated across the downstream tunnels in the TEA. Otherwise (including in the MPTE case), traffic is load-balanced across the downstream (NHOP in the case of MPTE) tunnels. Other than that, most of the TEA extensions defined in are applicable to MPTE, with the following notes: All tunnel types and sub-TLVs mentioned in can be used. A tunnel with an RPF sub-TLV is for a PHOP; a tunnel without one is a NHOP. There is no separate "direction" field -- presence or absence of the RPF sub-TLV is the sole indication of whether a given tunnel entry is a PHOP or a NHOP. The NHOP load share is encoded in the Tunnel Weight sub-TLV. In the case of labeled MPTE tunnels, a single Tree Label sub-TLV is reused for both directions, since RPF presence/absence already disambiguates the tunnel's role: On a NHOP tunnel (no RPF sub-TLV), the Tree Label sub-TLV signals the outgoing label this junction uses toward that NHOP. On a PHOP tunnel (with RPF sub-TLV), the Tree Label sub-TLV instead signals the incoming label this junction expects to receive from that PHOP. Implementations MUST NOT rely on a distinct sub-TLV type to distinguish outgoing vs. incoming label; the RPF sub-TLV is authoritative for that. Every PHOP and NHOP tunnel carries an MPTE Hop Identity sub-TLV (see "MPTE Hop Identity sub-TLV" below) identifying the peer interface/node at the other end of that hop (replacing identification by separate interface-address or link-index sub-TLVs), and an MPTE Hop Version sub-TLV (see "MPTE Hop Version sub-TLV" below) carrying a per-hop convergence/version value used to detect and reconcile stale hop state across successive updates to the same junction. For the MCAST-TREE case, only one tunnel has an RPF sub-TLV, and either there is only one tunnel with a Tree Label sub-TLV in the case of P2MP tunnel, or each tunnel has a Tree Label sub-TLV in the case of MP2MP tunnel. For the MPTE case, every PHOP tunnel for labeled MPTE tunnels has a Tree Label sub-TLV encoding the expected incoming label, unless ordered control is used (see "Ordered Control" below), in which case the incoming label is instead learned via a Junction RESV route. The indication of an egress point (on a pure egress or on a bud node) is an Any-Encapsulation tunnel without either the RPF sub-TLV or any sub-TLV that identifies a downstream interface/tunnel. In the bud node case, this tunnel has the Tunnel Weight sub-TLV, indicating the load share since the traffic is load-balanced between local delivery and other NHOP tunnels.

Specification The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Tunnel Weight sub-TLV This document defines a Tunnel Weight sub-TLV with type TBD0 allocated from the BGP Tunnel Encapsulation Attribute Sub-TLVs registry. Note that it is deferent from the Weight sub-TLV from the SR Policy Segment List Sub-TLVs registry.

AFI/SAFI and NLRI This document defines a new SAFI type MPTE with value TBD1 for signaling MPTE junction states. When it is used with AFI 1, the IP addresses in the NLRI are IPv4. When it is used with AFI 2, the addresses in the NLRI are IPv6. The NLRI is encoded as follows:

Route Type, Length, MC Address, MPTED ID, Junction Node Address, and Originating Node Address make up the NLRI key; there is no Tunnel Type field in the key (an MPTE DAG uses a single encapsulation/tunnel type throughout, so it does not need to be part of the per-junction NLRI key). MPTED ID identifies the DAG (persistent for the life of the MPTED tunnel). DAG Instance ID and Junction Bandwidth are a non-key trailing part of the NLRI: they are included (and the Length field increased accordingly) only on reachable announcements, and omitted on withdrawals, since they do not participate in route identity. DAG Instance ID identifies the current (possibly in-place-updated) instance/version of this junction's signaling state, matching the terminology of . Junction Bandwidth is the bandwidth to be admitted/reserved at this junction (it is a per-junction, not DAG-wide, quantity); it is meaningful only on the Junction route (Type 1, below) -- a junction further derives each of its own NHOPs' bandwidth locally, as the product of Junction Bandwidth and that NHOP's load share (see "Considerations for BGP Signaling" above and the Tunnel Weight sub-TLV, below). On Junction Confirmation and Junction RESV routes (Types 2 and 3, below) these two fields are both encoded as zero and MUST be ignored by the receiver. This document defines the following Route Types:

The semantics of the MC Address, Junction Node Address, and Originating Node Address fields, and of the Route Target used to direct the route to its intended recipient, differ by Route Type. Note that Originating Node Address, across all Route Types, is simply the address of whichever node originated that particular route instance (the SS for Type 1, the confirming junction for Type 2, the sending junction for Type 3): Junction route (Type 1): originated by the SS toward every node that owns a junction of the DAG. MC Address identifies the MC (MPTE Computer) that computed the DAG. Junction Node Address is the address of the junction node this route describes. Originating Node Address is the SS's own address. The Route Target is an IP-address-specific RT with the Global Administrator field set to the Junction Node Address, so that only that node imports the route. The route carries the junction's PHOP/NHOP list in a Tunnel Encapsulation Attribute, as described above. Junction Confirmation route (Type 2): originated by a junction node back toward the SS once that node's local forwarding state for the junction has been programmed (i.e., the junction has come up). MC Address is unchanged from the corresponding Junction route. Junction Node Address is the confirming node's own address, and Originating Node Address is likewise set to the confirming node's own address. The Route Target is an IP-address-specific RT with the Global Administrator field set to the SS's address, so that only the SS imports it. This route's existence (origination) is itself the confirmation signal; it additionally carries a TEA with this node's own real, currently-in-use PHOP/NHOP addresses and labels (as opposed to the planned topology/labels carried in the Type 1 route), letting the SS populate per-hop label detail (e.g., for a "show mpte tunnel detail" style display) that it would not otherwise learn. This TEA is omitted when the junction's local state is no longer available (e.g., torn down before this deferred route could be sent) or when global-label mode is in use (every node already deterministically derives the same label from the DAG Instance ID, so no round-trip is needed). Junction RESV route (Type 3): used only under Ordered Control (see "Ordered Control" below); originated by a junction node toward one of its PHOPs to carry the label the junction has locally allocated for that PHOP's traffic -- the BGP-signaled equivalent of an RSVP Resv message. MC Address is unchanged from the corresponding Junction route. Junction Node Address is set to the target PHOP's address (where the route is directed), and Originating Node Address is set to the sending junction's own address (not the PHOP's). The Route Target is an IP-address-specific RT with the Global Administrator field set to the PHOP's router ID when known, falling back to MC Address otherwise (so that, in the fallback case, anything importing the SS's Type-1/Type-2 RT also picks up the route). The TEA carries a single PHOP-direction tunnel entry identifying the target PHOP (via the MPTE Hop Identity sub-TLV) and the allocated label (via the Tree Label sub-TLV).

MPTE Hop Identity sub-TLV The MPTE Hop Identity sub-TLV identifies the peer interface/node at the far end of a PHOP or NHOP hop carried in a tunnel entry of the TEA. It replaces identifying a hop purely by an interface or neighbor address: by separately carrying the peer's router ID (loopback) in addition to its interface address, it lets a receiver disambiguate hops to the same neighbor router ID across multiple parallel links.

On a NHOP tunnel entry (no RPF sub-TLV), Peer Interface Address is the downstream neighbor's interface address (on the link toward it) and Peer Router ID is that neighbor's own (loopback) address. On a PHOP tunnel entry (with RPF sub-TLV), the same two fields instead identify the upstream neighbor's interface and router ID, respectively.

MPTE Hop Version sub-TLV The MPTE Hop Version sub-TLV carries an opaque, monotonically increasing per-hop value, set by the node that owns (originates state for) that hop, used by a receiver to detect and reconcile stale hop state when a Junction route for the same junction is updated -- e.g., to recognize that a previously-seen PHOP or NHOP entry that is no longer present in a newer update (because its version did not advance, or because it is simply absent) should be treated as removed, even if the corresponding Junction route itself was not withdrawn.

Procedures

Originating Junction Routes After the MC calculates an MPTED, the SS originates a Junction route (Type 1) for each junction node, as described under "AFI/SAFI and NLRI" above: an IP-Address-Specific RT with the Global Administrator field set to the junction node's address and the Local Administrator field set to 0, and no protocol next-hop. The route carries a Tunnel Encapsulation Attribute (TEA). Each tunnel in the TEA corresponds to a PHOP or NHOP: Each tunnel is an Any-Encapsulation tunnel, carrying an MPTE Hop Identity sub-TLV identifying the peer at the other end of that hop, and an MPTE Hop Version sub-TLV (in addition to other sub-TLVs). Each PHOP tunnel MUST also include an RPF sub-TLV (to indicate it is a PHOP) and, unless Ordered Control is used, a Tree Label sub-TLV encoding the incoming label this junction expects from that PHOP. Each NHOP tunnel MUST NOT include an RPF sub-TLV, and MUST, unless Ordered Control is used, include a Tree Label sub-TLV encoding the outgoing label this junction uses toward that NHOP. It MAY include a Tunnel Weight sub-TLV to encode the NHOP's load share. If the Tunnel Weight sub-TLV is not present, then the load share is 0.

Receiving Junction Routes Each node X that receives an MPTE route with an RT whose Global Administrator field does not match its loopback address propagates the route to all its neighbors (except the one from which it received the route). If the RT matches its own loopback address, X MUST import it, and MUST stop re-advertising the route upon match and importation. Once a Junction route (Type 1) is imported, X installs forwarding states as described in the following sections, in the case of MPLS when ordered control is not used (other tunnel types or ordered control will be specified in a future revision).

Building Forwarding Nexthop When a data packet is received, an IP address or MPLS label lookup is done to produce the forwarding information about how the packet should be forwarded. The forwarding information is referred to as forwarding nexthop in this document, or simply nexthop when it is not ambiguous. The forwarding nexthop for a junction is built by checking each NHOP tunnel. The MPTE Hop Identity sub-TLV identifies the outgoing interface/neighbor, and the Tree Label sub-TLV identifies the outgoing label. The Tunnel Weight sub-TLV provides the load-balancing share for the link, and bandwidth reservation on that NHOP's TE-link is done by multiplying the Junction Bandwidth carried in the Junction route's NLRI (see "AFI/SAFI and NLRI" above) by the NHOP's load share.

Installing Routes For each PHOP tunnel in the TEA, a label route is installed with the label value in the Tree Label sub-TLV, pointing to the forwarding nexthop built as specified above.

Sending Junction Confirmation Once a junction node has programmed its local forwarding state for a junction (i.e., the junction has come up), it sends a Junction Confirmation route (Type 2, see "AFI/SAFI and NLRI" above) back to the SS. Unless Ordered Control is used, the SS makes sure that all junctions have sent their confirmation before the tunnel is put into use. The Junction Confirmation route's existence (origination) is itself the confirmation signal, and its withdrawal signals that the junction is no longer up (e.g., because the corresponding Junction route was itself withdrawn, or because the junction's local forwarding state could no longer be programmed). Unlike the Junction route, which carries the SS's planned PHOP/NHOP topology and labels, the Junction Confirmation route's TEA (when present, see "AFI/SAFI and NLRI" above) carries the confirming node's own real, currently-in-use PHOP/NHOP addresses and labels, so that the SS can additionally learn and display per-hop label detail that it would not otherwise have visibility into.

Ordered Control When hop-by-hop Ordered Control is used, the Junction route's PHOP and NHOP tunnels in the TEA omit the Tree Label sub-TLV (carrying only the MPTE Hop Identity/Version and, for NHOPs, Tunnel Weight sub-TLVs), and a junction's forwarding state is not installed, nor is its own incoming label allocated, until the junction has converged -- which requires having learned, via a Junction RESV route (Type 3), the label to use toward at least one of its own NHOPs (a junction with no NHOPs, i.e. the DAG's egress, is trivially converged and uses the implicit-null label). Once a junction has converged and locally allocated (or, at egress, implicitly assigned) its own incoming label, it originates a Junction RESV route toward each of its PHOPs, carrying that label so the PHOP knows what label to push for traffic destined to this junction. As described under "AFI/SAFI and NLRI" above, the route's Junction Node Address targets the PHOP, its Originating Node Address identifies the sending junction itself, and its Route Target directs the route to that PHOP (by router ID, falling back to the SS's RT if the PHOP's router ID is not yet known). The route's TEA carries a single PHOP-direction tunnel entry with an MPTE Hop Identity sub-TLV (identifying the target PHOP -- i.e., which of possibly several PHOPs toward the same upstream junction this label applies to) and a Tree Label sub-TLV carrying the allocated label. A junction that receives a Junction RESV route from one of its NHOPs records the carried label as the outgoing label to use toward that NHOP, and, once converged by virtue of at least one such label being known, proceeds to allocate and propagate its own label upstream as above. This leaf-to-root label propagation is the BGP-signaled equivalent of RSVP Resv propagation.

Route Update and Withdrawal When a junction is updated (e.g., with added/removed/updated PHOPs/NHOPs), the corresponding Junction route is updated accordingly. If a junction is deleted, the corresponding Junction route is withdrawn. Corresponding Junction Confirmation and Junction RESV routes are updated, originated, or withdrawn accordingly.

Routes For Other Messages To be added.

Security Considerations To be added.

IANA Considerations To be added.

Acknowledgments The authors thank Vishnu Pavan Beeram, Chandrasekar Ramachandran, Sudharsana Venkataraman, and Jai Hari M K for their comments and suggestions.