Distributed Bandwidth Management for Segment Routing

Introduction Segment Routing (SR) steers packets through a network using an ordered list of segments. An SR Policy is associated with one or more candidate paths, and in many deployments each SR Policy has an associated bandwidth requirement. Distributed bandwidth management is a model in which the computation of SR Policy paths and the management of link bandwidth are performed by the network nodes in a distributed manner, without relying on a single centralized controller. Each head-end independently computes a path for its SR Policy that satisfies the policy's bandwidth requirement, using the traffic engineering (TE) information distributed by the IGP. Because head-ends compute paths independently and concurrently, path computation alone cannot guarantee that the bandwidth one head-end accounts for on a link is consistent with the reservations made by other head-ends. To make a reservation authoritative, signalling is used to reserve the bandwidth for each SR Policy on every transit node along the computed path. The signalling establishes per-SR-Policy reservation state on the transit nodes and confirms that the requested bandwidth is available end to end. When two or more head-ends concurrently compute paths over the same link, the same bandwidth may be provisionally accounted for by more than one head-end before the IGP has re-advertised the updated link bandwidth; that is, the bandwidth may be "double-booked". The signalling-based reservation resolves such races: if the requested bandwidth cannot be reserved on a transit node because it has already been committed to another SR Policy, the reservation, and therefore the SR Policy setup, fails. The head-end then recomputes the SR Policy path using the updated bandwidth information and retries the reservation. Distributed path computation and admission control rely on the TE bandwidth attributes that the IGP advertises for each link:

Maximum Link Bandwidth: the capacity of the link.
Maximum Reservable Link Bandwidth: the maximum bandwidth that may be reserved on the link, which may exceed the maximum link bandwidth when over-subscription is configured.
Unreserved (available) Bandwidth: the bandwidth not yet reserved, advertised per priority level.

These sub-TLVs were introduced for IS-IS in (carried in the Extended IS Reachability TLV) and for OSPFv2 in (carried in the TE Opaque LSA); the equivalent OSPFv3 extensions are defined in . When a transit node reserves bandwidth for an SR Policy, the Unreserved Bandwidth of the affected link changes. The transit node re-originates and floods the updated bandwidth sub-TLVs through the IGP so that all head-ends obtain a consistent view of the remaining reservable bandwidth. Subsequent SR Policy computations use this updated information, reducing the likelihood of double-booking on that link.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology

SR: Segment Routing.
SID: Segment Identifier.

Motivation The current Segment Routing architecture does not define a signalling protocol. As a result, it cannot provide network resource management , such as bandwidth admission control and reservation on its own, and can do so only with the aid of an off-box application such as a centralized network controller. For this reason, some SR traffic-engineering deployments rely on a centralized controller to compute paths, perform admission control, and manage link bandwidth for the SR Policies in the network. While a centralized controller provides a convenient global view, this model has some drawbacks that become more pronounced as the network grows:

Low scalability: a single controller (or controller cluster) must hold the state of, and compute and continually re-optimize paths for, every SR Policy in the network. As the number of nodes, links, and policies increases, the associated computation, state, and churn become a bottleneck.
Large failure blast radius: because the controller is on the critical path for provisioning and re-optimization, a controller failure, network partition, or software defect can simultaneously affect a large portion of the network.

These limitations of centralized WAN control, and the benefits of distributing control functions closer to the network nodes, are examined for large operator networks in . A model in which path computation and signalling-based bandwidth reservation are performed by the network nodes themselves, has been deployed for many years. The computation and reservation state is spread across the head-end and transit nodes rather than concentrated in one place, and the failure of any single node affects only the SR Policies that traverse it, substantially reducing the blast radius. Distributing the control functions does not by itself address forwarding state. In the model described in this document, however, the forwarding plane uses SR SIDs and the segment-stacking model: each path is encoded as a stack of SIDs carried in the packet, and transit nodes forward on shared per-link (Adjacency-SID or End.X SID) entries rather than on per-path forwarding state. This preserves the stateless-core property of Segment Routing and thereby solves the scalability issues posed by maintaining per-path forwarding state in the network core.

Distributed Bandwidth Management Architecture In the distributed bandwidth management architecture, the per-SR-Policy bandwidth admission control, reservation, and accounting are performed by the network nodes themselves rather than by a centralized controller. The head-end, transit, and egress nodes are responsible for computing paths and for reserving and maintaining bandwidth along the path. shows the functional components and their interactions.

Distributed Bandwidth Management Components | Transit | ... | Transit | --> | Egress | | (R1) | | (R2) | | (Rn) | | (Rz) | +-----------+ +-----------+ +-----------+ +-----------+ Data path (forwarding): R1 --> R2 --> ... --> Rn --> Rz, using the SID stack imposed by the head-end. Reservation signalling: initiated by the head-end (R1) to reserve bandwidth on every node along the path to the egress (Rz). IGP flooding: TE attributes and unreserved bandwidth are flooded among all nodes. ]]>

Head-End The head-end is the node at which an SR Policy is instantiated, and is the owner of the SR Policy, consistent with the SR Policy architecture defined in . The role of the head-end in this architecture is the same as the head-end role defined there. The SR Policy constructs defined in are unchanged by this document. In particular, an SR Policy remains identified by the tuple (head-end, color, endpoint); each SR Policy has one or more candidate paths; a candidate path is distinguished by its protocol origin, originator, and discriminator (distinguisher); and the active candidate path is selected by preference. These concepts remain the same when the associated bandwidth is managed in a distributed manner. When one or more candidate paths are enabled for distributed bandwdith management, each candidate path reserves bandwidth along the path and hence multiple candidate paths have to be used carefully to avoid overbooking the network resources. In addition to the role defined in , the head-end in this architecture:

Performs path computation for the candidate path, subject to the locally configured Explicit Route Object (ERO) and other path constraints (for example, affinities and metric bounds).
Initiates the signalling that reserves the required bandwidth for the SR Policy on every node along the computed path, as described in .
On reservation success, builds the SID stack (segment list) that encodes the forwarding path, as described in .
On reservation failure, recomputes the path using the updated bandwidth information and retries, as described in .

Transit Node A transit node is any node along the computed path between the head-end and the egress. For each SR Policy reservation that traverses it, a transit node:

Performs bandwidth admission control: it checks whether the requested bandwidth is available on the relevant outgoing link at the requested priority.
Reserves the requested bandwidth on the link when admission succeeds, establishing per-SR-Policy reservation state. When the requested bandwidth is not available, the request is rejected, which may trigger pre-emption of lower-priority reservations as described in .
Updates the IGP: after the reserved bandwidth on a link changes, the transit node re-originates and floods the updated unreserved bandwidth sub-TLVs so that all head-ends obtain a consistent view of the remaining reservable bandwidth.

Egress Node The egress node is the endpoint (tail-end) of the SR Policy, at which the SR Policy's forwarding path terminates.

Signalling A signalling protocol is used to install per-SR-Policy bandwidth reservation state on the transit nodes along a computed path and to perform hop-by-hop admission control. The signalling confirms that the requested bandwidth is available on every link of the path and commits the reservation. If any node along the path cannot satisfy the request, the setup fails and the head-end recomputes the SR Policy path as described in . RSVP-TE is the signalling protocol for this architecture. It is a proven and widely deployed control plane for bandwidth reservation and admission control, and already provides the foundation for operating it on a shared SR forwarding plane: it describes how to signal RSVP-TE tunnels on a shared MPLS forwarding plane, which allows RSVP-TE to perform bandwidth reservation and admission control while the data-plane forwarding is provided by SR. In this architecture, RSVP-TE maintains only the per-SR-Policy reservation (control-plane) state on each node along the path; it creates no per-SR-Policy forwarding state, because forwarding follows the shared SR segments as described in . extends to signal RSVP-TE tunnels directly over the SR-MPLS forwarding plane, using IGP-advertised Adjacency SIDs (Adj-SIDs) as the forwarding-plane labels, and provides the equivalent mechanism for the SRv6 forwarding plane using End.X SIDs. Together these define how RSVP-TE reserves bandwidth for SR Policies on both the SR-MPLS and SRv6 data planes.

Forwarding Plane Once the bandwidth reservation for an SR Policy succeeds, the head-end builds the forwarding path by encoding an explicit segment list that follows the reserved links. In SR-MPLS this is a stack of Adjacency-SIDs , and in SRv6 it is a list of End.X SIDs ; each such segment identifies a specific link (adjacency) on the reserved path. In SRv6, when the segment list is long, it can be encoded more compactly using the micro-SID (uSID) flavor of compressed SRv6 segment list encoding , which packs multiple micro-segments into each 128-bit SID and thereby reduces the number of SIDs, and the encapsulation overhead, needed to express the path. Because the path is expressed entirely as a segment list carried in the packet, the transit nodes forward the traffic solely on the basis of the top (active) segment, which represents a link rather than a policy. Consequently, although the control plane maintains per-SR-Policy signalling (reservation) state on each transit node along the path, the forwarding plane holds no per-SR-Policy state: a transit node's forwarding entry for an Adjacency-SID or End.X SID is shared by all SR Policies that traverse that link, preserving the stateless-core property of Segment Routing. The per-SR-Policy signalling state in the control plane is required primarily to facilitate pre-emption: it records how much bandwidth each SR Policy has reserved on a link and at what priority, so that when a higher-priority SR Policy requests bandwidth that is not otherwise available, the node can identify the lower-priority reservations to pre-empt. This state also enables per-policy bandwidth accounting and the correct release of bandwidth when an SR Policy is torn down.

Auto-Bandwidth Auto-bandwidth is a mechanism by which the head-end automatically adjusts the bandwidth reserved for an SR Policy so that the reservation tracks the actual measured traffic demand, instead of relying on a statically configured bandwidth value. Auto-bandwidth allows reservations to follow real demand, improving the utilization of network capacity while preserving the admission guarantees provided by the distributed reservation mechanism. As a feature, auto-bandwidth encompasses a broad set of functions, including traffic statistics collection, bandwidth demand estimation and prediction, and periodic optimization of the path and its reservation. The detailed specification of these functions is outside the scope of this document and is expected to be described in a separate document. Traffic is measured, and bandwidth is adjusted, on a per-segment-list basis: the head-end independently tracks the demand for each segment list of the SR Policy, because each segment list is computed, reserved, and signalled independently along its own path. All segment lists within the same SR Policy use the same auto-bandwidth parameters, so that the segment lists of a policy are managed consistently.

Resignalling Bandwidth Based on Auto-Bandwidth The traffic carried by a segment list may increase or decrease over time. When the auto-bandwidth-derived demand for a segment list changes significantly relative to its current reservation, the head-end resignals the reservation for that segment list using the new value:

An increase causes the head-end to signal a reservation at the higher bandwidth. This is subject to admission control on every node along the path; if the increased bandwidth cannot be admitted, the head-end recomputes the path and retries, as described in .
A decrease causes the head-end to signal a reservation at the lower bandwidth, releasing the excess so that the freed capacity is returned to the links as unreserved bandwidth and flooded by the IGP.

SR Policy Containers An SR Policy container is an SR Policy whose bandwidth demand is realized by one or more parallel segment lists that together carry the policy's traffic, with the demand load-balanced across the active segment lists. Modelling the SR Policy as a container of segment lists allows the aggregate demand of the policy to exceed the bandwidth that a single segment list can reserve on a single path, and allows the number of segment lists to adapt as the demand changes (typically driven by auto-bandwidth, see ). Details of the containers and splitting and merging will be specified in a seperate document in future.

Operations, Administration, and Maintenance (OAM) This architecture does not introduce a new data plane. As described in , the forwarding path of an SR Policy is expressed entirely as a segment list of standard SR SIDs ,Adjacency-SIDs in SR-MPLS and End.X SIDs in SRv6 and transit nodes forward solely on those shared per-link segments, with no per-SR-Policy forwarding state. Because the data plane is unchanged, the existing Segment Routing OAM mechanisms apply to SR Policies managed by this architecture without any modification. This section summarizes the applicable mechanisms and the documents that define them. Seamless BFD (S-BFD) can be used, unchanged, to verify the continuity and liveness of the forwarding path of an SR Policy. An S-BFD session runs over the segment list imposed by the head-end. No new S-BFD procedures are required by this document. On-demand connectivity verification and fault isolation using MPLS LSP Ping and Traceroute apply unchanged. Because an SR Policy path is a stack of Adjacency-SIDs that need not correspond to a single routed FEC, the Nil FEC is well suited to validating such label stacks, and the SR-specific target FEC stack sub-TLVs for Prefix-SIDs and Adjacency-SIDs are defined in . For the SRv6 data plane, the corresponding ping, traceroute, and S-BFD procedures are defined in . Active performance measurement, such as delay and packet-loss measurement, also applies unchanged. The Two-Way Active Measurement Protocol (TWAMP) and the Simple Two-Way Active Measurement Protocol (STAMP) can measure the performance of the path taken by an SR Policy. In all of these cases the OAM packets traverse the same SR segment list as the monitored traffic and are processed by the standard SR data plane, so no OAM-specific extensions are introduced by the distributed bandwidth management mechanism defined in this document.

Operational Considerations TODO: Scaling, convergence, backward compatibility, and manageability.

Security Considerations TODO: Describe threats and mitigations. Reference the security considerations of the underlying SR data planes and IGP/BGP-LS advertisements as applicable.

Acknowledgements TODO.