Internet-Draft C-SCORE November 2026
Joung, et al. Expires 5 May 2027 [Page]
Workgroup:
DetNet Working Group
Internet-Draft:
draft-ietf-detnet-stateless-fair-queuing-02
Published:
Intended Status:
Standards Track
Expires:
Authors:
J. Joung
Sangmyung University
J. Ryoo
ETRI
T. Cheung
ETRI
Y. Li
Huawei
P. Liu
China Mobile

Latency Guarantee with Stateless Fair Queuing

Abstract

This document specifies the implementation of the stateless fair queuing framework defined in Recommendations ITU-T Y.3129 [Y.3129] and Y.3148 [Y.3148]. The framework provides end-to-end (E2E) latency bounds to flows without requiring core nodes to maintain per-flow state. For each packet, the entrance node of a flow computes the ideal service completion time under a fluid model, referred to as the Finish Time (FT), and encodes it in the packet header. Each subsequent core node updates the FT by adding a delay factor that is determined by the flow and the nodes. Schedulers serve queued packets in ascending order of FT. This mechanism provides strong isolation among flows. The E2E latency bound of a flow depends on the flow's own parameters, such as its maximum burst size and service rate, and on link parameters, such as the link capacity and the maximum packet length among the flows sharing the link. This document specifies the required metadata and its encoding formats, the admission control procedure, and an approximation of stateless fair queuing using a strict priority (SP) scheduler.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 5 May 2027.

▲

Table of Contents

1. Introduction

Recommendation ITU-T Y.3129 [Y.3129] specifies the requirements and framework for Deterministic Networking (DetNet) using a set of work-conserving packet schedulers that guarantee end-to-end (E2E) latency bounds to flows. Recommendation ITU-T Y.3148 [Y.3148] further specifies the functional architecture, functional entities, operational procedures, and packet metadata required to fulfill the requirements and framework specified in Y.3129.

In this framework, core nodes do not maintain per-flow state. The edge node through which a flow enters the network, referred to as the entrance node, computes for each packet the ideal service completion time under a fluid model, referred to as the Finish Time (FT), and records it in the packet header. Each subsequent core node updates the FT, without per-flow state, by adding a delay factor that is a function of the parameters of the flow and of the upstream nodes. The scheduler serves queued packets in ascending order of FT. This mechanism is referred to as stateless fair queuing, and the resulting framework is referred to as Work-conserving Stateless Core Fair Queuing (C-SCORE) [C-SCORE].

C-SCORE provides strong isolation among flows. For a certain choice of the delay factor, a closed-form expression for the E2E latency bound of a flow can be derived. This bound is identical to that of a network in which every node employs a stateful fair queuing scheduler. It depends on the flow's intrinsic parameters, such as its maximum burst size and service rate, and on link parameters, such as the link capacity and the maximum packet length among the other flows sharing each output link with the flow.

The key component of C-SCORE is the packet state carried as metadata. Although C-SCORE does not maintain per-flow state at core nodes, it operates as a fair queuing scheduler, a class of schedulers known to provide the best flow isolation. The metadata carried in the packet header is simple and can be updated either while the packet waits in the queue or before it enters the queue.

To guarantee its performance, C-SCORE, like any other DetNet solution, requires that the total service rate of the flows on each link remain below the capacity of that link. To ensure this, admission control is typically performed for each flow before data transmission. Admission control consists of the following steps:

  1. signaling the service rate of the incoming flow;
  2. checking the current total service rate of each link along the path; and
  3. deciding whether to accept the flow.

[Y.qos-sre-rf] specifies a framework, based on Y.3129, for identifying the service rates of active flows for admission control without maintaining per-flow state. This stateless service rate estimation improves the scalability of the admission control process. The framework relies on the metadata specified in Y.3129.

[Editor's Note: The current versions of Y.3129 and Y.3148 define three metadata fields. They are expected to be revised to define four.]

This document specifies the implementation details for realizing C-SCORE, including:

2. Terminology

2.1. Terms Used in This Document

2.2. Abbreviations

3. Conventions Used in This Document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

4. Assumptions

This document assumes that there are only two classes of traffic. High-priority traffic, equivalently DetNet traffic, requires guaranteed latency upper bounds. All other traffic is considered low-priority, or Best Effort (BE), traffic. This document addresses only high-priority traffic. However, a BE packet in transmission is not preempted by high-priority traffic.

All flows of DetNet traffic conform to their traffic specification (TSpec) parameters. That is, for flow i with maximum burst size B_i and arrival rate a_i, the cumulative arrival from flow i during any time interval [t_1, t_2], where t_1 < t_2, does not exceed B_i + (t_2 - t_1)a_i.

The service rate actually allocated to flow i, denoted r_i, is greater than or equal to its arrival rate a_i. As shown in Equation (4) in Section 5.1, the E2E latency bound of a flow can be adjusted by adjusting its service rate. The symbol r is used interchangeably with r_i to denote the service rate of the flow under observation. Similarly, B, a, and L are used to denote the parameters of the flow under observation.

The total service rate allocated to all flows at a node does not exceed the capacity of the node's output link. Consequently, resource reservation and admission control are required.

The following terms are used in this document:

5. Framework

5.1. Core Stateless Fair Queuing

Fair queuing (FQ) schedulers assign each packet a Finish Time (FT), which determines its service order: the packet with the smallest FT in the buffer is served first. The key idea of FQ is to compute the ideal service completion time of each packet in a hypothetical fluid service model and to use it as the service order in the actual packet-based scheduler.

Although FQ schedulers provide excellent flow isolation, they require per-flow state, namely F(p-1), the FT of the preceding packet of the same flow. Upon the arrival of each packet, the scheduler must identify the flow and retrieve the FT of the preceding packet of that flow. The scheduler must also update the flow state to F(p) upon the departure of the packet.

[Y.3129] and [Y.3148] specify an FQ scheduler that does not require per-flow state at core nodes. At the entrance node, the FT of packet p is computed as shown in Equation (1), where the subscript 0 denotes the entrance node of the flow under observation.

  F_0(p) = max{F_0(p-1), A_0(p)} + L(p)/r                    (1)

where:

  • A_0(p): The arrival time of packet p at the entrance node.
  • L(p): The length of packet p.
  • r: The service rate allocated to the flow.

The FT of packet p for core node h is obtained by adding a delay factor d_(h-1)(p), which depends on the upstream node h-1 and on the packet, to the FT for node h-1, as shown in Equation (2).

  F_h(p) = F_(h-1)(p) + d_(h-1)(p)                           (2)

The framework achieves a bounded E2E latency by defining the delay factor as shown in Equation (3).

  d_h(p) = L_h/R_h + L/r + delta_h(p)                        (3)

where:

  • L_h: The maximum packet length observed at node h over all flows.
  • R_h: The link capacity of node h.
  • L: The maximum packet length of the flow.
  • r: The service rate allocated to the flow.
  • delta_h(p): The time difference of packet p between node h and node h+1, defined as the arrival time of p at node h+1 minus the service completion time of p at node h (see Section 6.4).

The time difference delta_h(p) includes the clock discrepancy and the propagation delay between node h and node h+1. Since it is relatively stable across packets, it can be approximated by a constant, delta_h.

When d_h(p) is defined as in Equation (3), the latency bound in Equation (4) holds [KAUR] [C-SCORE].

                         h                        h-1
  D_h(p) <= (B-L)/r + SUM {L_j/R_j + L/r}  +  SUM delta_j(p)  (4)
                        j=0                       j=0

where D_h(p) is the latency experienced by packet p from its arrival at node 0 to its departure from node h, and B is the maximum burst size of the flow to which p belongs. The last term of Equation (4) accounts for the clock discrepancies and the propagation delays between the nodes. When the clock discrepancies are negligible, for example, with an absolute time reference, this term effectively represents the E2E propagation delay.

The latency bound in Equation (4) is the same as that of a network in which every node employs a stateful FQ scheduler. All parameters in the bound are intrinsic to the flow, except L_h/R_h and the propagation delays.

5.2. Stateless Service Rate Estimation

The C-SCORE framework can also be used to estimate the total service rate of the flows on a link. This is based on the following two observations:

  1. The FTs of consecutive packets of a flow partition the time axis into non-overlapping intervals. Therefore, for any given time instant, exactly one packet of the flow has an interval that contains that instant.
  2. The difference between the FTs of consecutive packets of a flow remains the same across nodes, since, with delta_h approximated as a constant, the FTs of all packets of the flow are increased by the same amount at each node.

If each packet carries both its FT and the difference from the FT of the preceding packet, the packet whose interval contains a given time instant can be identified from the packet metadata alone. By choosing a reference time for estimation, exactly one such packet is identified per flow. Since the service rate of the flow is also carried in the packet as metadata, the service rates of these packets can be summed to obtain the total service rate of the link.

[Y.qos-sre-rf] specifies such a framework, based on [Y.3129], for identifying the service rates of active flows for admission control without maintaining per-flow state. This framework relies on the metadata specified in [Y.3129], in particular the difference between the FTs of the preceding and current packets, denoted as DeltaF(p) in this document.

6. Operational Procedures

6.1. Metadata

[Y.3129], [Y.3148], and [Y.qos-sre-rf] specify the following metadata to be carried in each packet of a flow:

  • L: The maximum packet length of the flow.
  • r: The service rate allocated to the flow.
  • F_h(p): The FT of packet p to be used at node h.
  • DeltaF(p): The difference between the FT of packet p and the FT of the preceding packet of the same flow, that is, F_0(p) - F_0(p-1).

L, r, and DeltaF(p) are set by the entrance node and remain unchanged along the path. F_h(p) is dynamic and is updated at every node in accordance with Equation (2).

[Editor's Note: The current versions of Y.3129 and Y.3148 define three metadata fields: L, r, and F_h(p). They are expected to be revised to also include DeltaF(p).]

When packet p arrives at core node h, it carries F_h(p), L, r, and DeltaF(p). F_h(p) has been calculated in advance by node h-1 using d_(h-1)(p), as shown in Equation (2). Node h uses F_h(p) to determine the service order of the packet. Before transmitting the packet, node h calculates F_(h+1)(p) = F_h(p) + d_h(p) and replaces F_h(p) with F_(h+1)(p) in the packet. As shown in Equation (3), d_h(p) is the sum of the flow-specific term L/r, which is obtained from the packet metadata, and the node-specific terms L_h/R_h and delta_h, which are maintained by node h. This update can be performed either before the packet enters the queue or while the packet waits in the queue.

The entrance node calculates F_0(p) using Equation (1) and uses it to determine the service order of the packet at the entrance node. It then calculates F_1(p) = F_0(p) + d_0(p) and records F_1(p), together with L, r, and DeltaF(p), in the packet.

The detailed operations for creating and updating the metadata at the entrance node and core nodes are specified in [Y.3129] and [Y.3148]. These references also describe considerations for reducing the complexity of the entrance node.

6.3. Admission Control for Latency Guarantee

As described in Section 4, the total service rate allocated to the flows on each link must not exceed the capacity of the link. This section describes recommended admission control procedures for the C-SCORE framework. The procedures use RSVP-like signaling with a PATH message sent from the entrance node toward the egress node and a RESV message sent in the reverse direction. This document does not mandate a specific signaling protocol or message encoding.

In both procedures, only the entrance node maintains per-flow state. Core nodes do not maintain per-flow state; they obtain the total service rate of their outgoing links using the stateless service rate estimation mechanism specified in [Y.qos-sre-rf] (see Section 5.2).

6.3.1. Signaling Information

A flow request from an application includes the following information:

  • Traffic Specification (T-Spec): The arrival rate a, the maximum packet length L, and the maximum burst size B of the flow.
  • Requested Specification (R-Spec): The E2E latency bound D_req and, optionally, the E2E jitter bound required by the application.

In addition to the T-Spec and R-Spec, a PATH message carries the following fields, which are updated hop by hop:

  • Hop count (H): The index of the last node that has processed the PATH message. The entrance node sets H to 0, and each subsequent node increments H by 1. When the PATH message reaches the egress node, the path consists of H+1 nodes, indexed from 0 to H.
  • Accumulated link term (S_LR): The sum of L_h/R_h over the nodes that have processed the PATH message.
  • Accumulated time difference (S_delta): The sum of delta_h over the links that the PATH message has traversed. Each node obtains delta_h as described in Section 6.4.
  • Requested service rate (r): Used in the basic procedure (Section 6.3.2).
  • Desired Rate (r_desired), Minimum Acceptable Rate (r_min), and Path-Available-Rate (r_path): Used in the rate discovery procedure (Section 6.3.3).

Using these fields, the E2E latency bound in Equation (4) for a service rate r can be written as follows:

  D(r) = (B + H*L)/r + S_LR + S_delta                        (5)

Accumulating S_LR and S_delta separately, rather than a single per-hop latency value, allows the egress node to evaluate the latency bound for any candidate service rate. This is required for the rate discovery procedure.

6.3.2. Basic Admission Procedure

In the basic procedure, the application requests a specific service rate r, where r >= a.

  1. The application requests a flow from the entrance node with the T-Spec, the R-Spec, and the requested service rate r.
  2. The entrance node acts as the admission controller. It checks whether r is within the available capacity of its own outgoing link. If so, it creates tentative per-flow state for the flow, sets H = 0, S_LR = L_0/R_0, and S_delta = 0, and sends a PATH message toward the egress node. Otherwise, it rejects the request.
  3. Each core node h estimates the total service rate of its outgoing link using the mechanism specified in [Y.qos-sre-rf] and checks whether adding r keeps the total within the aggregate bandwidth threshold of the link. If not, the core node returns a PATH error message to the entrance node and does not forward the PATH message.
  4. If the check succeeds, the core node increments H, adds delta_(h-1) to S_delta and L_h/R_h to S_LR, and forwards the PATH message to the next node.
  5. The egress node performs the same check and update as a core node, and then computes the E2E latency bound D(r) using Equation (5). If D(r) <= D_req, the egress node generates a RESV message carrying r. Otherwise, it returns a PATH error message to the entrance node.
  6. The egress node sends the RESV message to the entrance node. Since core nodes do not maintain per-flow state, they do not need to process the RESV message.
  7. When the RESV message reaches the entrance node, the flow is admitted. The entrance node commits r to its per-flow state and starts marking the packets of the flow with the metadata specified in this document, performing initial shaping if necessary.

6.3.3. Admission Procedure with Rate Discovery

The RSVP-like signaling can also be used to discover and negotiate an appropriate service rate for a flow. The PATH message identifies the link with the smallest residual capacity on the path, so the maximum rate that can be supported without violating the latency bounds of existing flows is determined in a single round trip.

  1. The application requests a flow from the entrance node with the T-Spec, the R-Spec, a Desired Rate r_desired, and a Minimum Acceptable Rate r_min, where r_desired >= r_min >= a.
  2. The entrance node acts as the admission controller. It computes the residual capacity r_avail_0 of its own outgoing link, sets r_path = min(r_desired, r_avail_0), and initializes H, S_LR, and S_delta as in Section 6.3.2. If r_path < r_min, it rejects the request. Otherwise, it creates tentative per-flow state and sends a PATH message toward the egress node.
  3. Each core node h computes the residual capacity r_avail_h of its outgoing link as the aggregate bandwidth threshold of the link minus the total service rate of the flows already admitted. The total service rate is estimated using the mechanism specified in [Y.qos-sre-rf].
  4. The core node updates r_path = min(r_path, r_avail_h). If the updated r_path < r_min, the core node returns a PATH error message to the entrance node and does not forward the PATH message. Otherwise, it updates H, S_LR, and S_delta as in Section 6.3.2 and forwards the PATH message.
  5. When the PATH message reaches the egress node, r_path represents the maximum service rate that the entire E2E path can support at that moment.
  6. The egress node selects the service rate r_opt as described in Section 6.3.3.1. If no rate satisfies the conditions, it returns a PATH error message to the entrance node.
  7. The egress node sends a RESV message carrying r_opt to the entrance node.
  8. When the RESV message reaches the entrance node, the flow is admitted.
  9. The entrance node commits r = r_opt to its per-flow state and starts marking the packets of the flow.
6.3.3.1. Selection of the Service Rate

The highest supportable rate is not always the most appropriate rate. The egress node selects r_opt such that r_opt is not greater than r_path and not smaller than the arrival rate a, the Minimum Acceptable Rate r_min, and the rate required for the E2E latency bound in Equation (5) to satisfy D_req. If no such rate exists, the egress node rejects the request.

Within this range, the egress node can select r_opt based on the following criteria:

  • Latency requirements: A higher service rate results in smaller FT increments and thus a smaller per-hop latency bound. A rate higher than the minimum required rate provides a margin against variations in delta_h.
  • Buffer constraints: The buffer required at core nodes depends on the maximum burst size B and the service rate r. The selected rate needs to be consistent with B so that the buffers at core nodes do not overflow.
  • Network availability: The egress node can choose a rate lower than r_path to leave capacity for flows that may join later.

Since core nodes need only the committed rate r carried in the packet metadata, the rate discovery and selection logic is confined to the control plane and the edge nodes. In large-scale networks, the same mechanism can be used for aggregate rate discovery, where r_path represents the available capacity for a bundle of flows that share the same path. In this way, the admission process maps the current congestion state of the path onto a single service rate parameter.

6.3.4. Operational Considerations

  • Concurrent admission requests: The service rate estimation in [Y.qos-sre-rf] reflects a flow only after its packets are transmitted. If multiple PATH messages traverse the same link at about the same time, each of them may observe the same residual capacity, and the link may be overbooked. To prevent this, a core node can maintain a single aggregate counter of pending reservations, rather than per-flow state. The core node adds the requested rate (r, or r_path in the rate discovery procedure) to the counter when it forwards a PATH message and includes the counter when computing the residual capacity. The rate is removed from the counter after a timeout that is long enough for the packets of the newly admitted flow to be reflected in the service rate estimation.
  • Error handling: When the entrance node receives a PATH error or RESV error message, it removes the tentative per-flow state and notifies the application that the request was rejected. The application can retry with relaxed requirements, for example, a larger D_req or a smaller r_min.
  • Flow removal: When a flow terminates, the entrance node removes the per-flow state and stops marking the packets of the flow. Since core nodes do not maintain per-flow state, no explicit teardown message is required at core nodes. The flow is no longer counted by the service rate estimation once its last packet has been transmitted.
  • Idle flows: An admitted flow that is temporarily idle may not be counted by the service rate estimation. If the flow resumes transmission, the total service rate of a link may exceed the aggregate bandwidth threshold. Operators can mitigate this by setting the aggregate bandwidth threshold sufficiently below the link capacity, or by having the entrance node maintain the presence of the flow during idle periods.

6.4. Compensation of Time Difference between Nodes

As described in Section 5, there is a time difference between adjacent nodes, which includes the clock discrepancy and the propagation delay between them. The time difference of packet p between node h and node h+1 is defined as follows:

  delta_h(p) = A_(h+1)(p) - C_h(p)                           (6)

where A_(h+1)(p) is the arrival time of packet p measured by the clock of node h+1, and C_h(p) is the service completion time of packet p measured by the clock of node h.

The FT does not need to be precise, since it is used only to determine the service order of packets. If the propagation delay between the nodes is constant and the clocks of the nodes do not drift relative to each other, delta_h(p) is the same for all packets and can be replaced by a constant, delta_h. In practice, clocks may drift slowly. Therefore, delta_h needs to be updated only occasionally, for example, periodically or when a change in the measured value exceeds a configured threshold.

6.4.1. Measurement of Time Difference

To obtain delta_(h-1), node h needs C_(h-1)(p), the service completion time of a packet at the upstream node h-1. Node h computes delta_(h-1) from C_(h-1)(p) and its own measurement of A_h(p), using Equation (6). Since both the clock discrepancy and the propagation delay are included in delta_(h-1), node h does not need to separate them.

C_(h-1)(p) can be provided to node h in one of the following ways:

  • Piggybacking: Node h-1 records C_(h-1)(p) in the metadata of selected packets. C_(h-1)(p) can be provided on demand, at the request of node h, or periodically.
  • Explicit messages: Nodes h-1 and h exchange dedicated timing messages. The message format can follow that of the Network Time Protocol (NTP) [RFC5905]. However, the procedure is simpler than that of NTP. NTP estimates the clock offset alone, using round-trip measurements under the assumption of symmetric delay. Here, only the combined one-way value delta_(h-1) is required, so a single one-way measurement is sufficient.

6.4.2. Time Difference between Non-adjacent Nodes

The time difference between non-adjacent nodes can be obtained in the same manner. This is useful when nodes that do not support C-SCORE exist between two C-SCORE nodes. In this case, however, the measured time difference also includes the variable queuing delays at the non-compliant nodes and is therefore not stable across packets. One possible approach is to sample the time difference over a sufficiently long interval and to use the maximum sampled value as delta_h. The latency bound in Equation (4) then holds only if the maximum queuing delay at the non-compliant nodes is bounded and is captured by the sampled maximum.

7. Characteristics

7.1. Taxonomy

According to the taxonomy suggested by [I-D.ietf-detnet-dataplane-taxonomy], the framework in this document, C-SCORE, is a flow-level, rate-based, work-conserving, asynchronous, non-periodic, and in-time solution.

[I-D.ietf-detnet-dataplane-taxonomy] also defines seven suitable categories for deterministic networking. A category is defined as a set of solutions grouped by one or more criteria, where a criterion is a principle or standard by which a solution can be judged or assigned to a certain category.

According to this categorization, C-SCORE belongs to the "flow-level rate-based unbounded" category, which is one of the seven suitable categories.

7.2. Strengths

The dominant factor in the per-hop latency of C-SCORE is the maximum packet length of the flow divided by its service rate. This factor is independent of the parameters of other flows. As such, the most distinguishing strength of C-SCORE is its flow isolation capability. A fine-tuned E2E latency bound can be assigned to a flow by controlling the flow's own parameters, such as its service rate. Once a latency bound is assigned to a flow, it remains almost the same in spite of changes in the network situation, such as other flows joining and leaving.

C-SCORE is work-conserving and thus benefits from statistical multiplexing gain without wasting bandwidth, which is a key to the success of the Internet. The consequence is a smaller average latency. The observed maximum latency is also much smaller than the theoretical latency bound. Note that, with a work-conserving solution, observing the theoretical latency bound is extremely difficult in real situations. This is because the worst-case latency results from a combination of multiple rare events, e.g., a maximum burst from a flow colliding with the maximum bursts from all other flows at every node. In contrast, with non-work-conserving solutions, latencies close to the bounds are commonly observed.

C-SCORE is rate-based, and thus its admission control is simple, since the admission condition depends only on the service rates of flows. This process aligns well with existing protocols.

Overall, C-SCORE is suitable for large-scale networks at any utilization level, with various types of flows joining and leaving dynamically.

8. Approximate C-SCORE Using Strict Priority Schedulers

8.1. General Description

C-SCORE requires a priority queue that sorts the packets in the queue in accordance with their FTs. Compared with the strict priority (SP) schedulers used in current switching nodes, this may limit the overall data plane scalability as well as the deployability of the solution. SP schedulers are commonly implemented in the hardware chips of current switching nodes. They usually consist of 8 to 32 queues and serve packets in a higher-priority queue first whenever such packets are present. Packets in the same queue are served on a first-in, first-out (FIFO) basis.

It is therefore desirable to implement C-SCORE with such an SP scheduler. This section specifies the architecture and algorithms of the approximate C-SCORE using rotating SP schedulers, as well as the E2E latency bound of a network of approximate C-SCORE nodes.

8.2. Transit Node Architecture

This section specifies the architecture of an approximate C-SCORE transit node, in which an SP scheduler with a limited number of queues behaves as an approximate priority queue.

The input port module classifies packets into DetNet flows and best-effort (BE) flows. Packets of DetNet flows are put into one of N queues in the output port module, which operate as rotating SP scheduler queues. If queue k, where 0 <= k <= N-1, has the highest priority at time t, then queue (k+N-1) mod N has the lowest priority at t. Packets of BE flows are put into a separate BE queue in the output port module. The BE queue is served only when there is no packet in queues 0 to N-1.

Time is partitioned into discrete intervals of uniform length S, called slots. Each slot is allocated to one of the N queues. Let T_i denote the end boundary of slot i. An arriving packet p is assigned to slot i if its FT, F_h(p), falls within the interval (T_(i-1), T_i]. The packet is buffered in the corresponding queue and served in FIFO order within the queue. The SP scheduler gives precedence to the queue that corresponds to the earliest slot. At the end of each slot, the priorities of the queues rotate: the queue of the slot that has just ended becomes the lowest-priority queue and is reused for the slot N slots later. The scheduler is work-conserving and serves packets at the line rate whenever the system is backlogged.

This section assumes that the FT of every packet falls within the range covered by the N slots. In other words, it is assumed that no packet has an FT beyond the slot boundaries covered by the N queues, or equivalently, that N*S is large enough to cover the FTs of all packets that can be present in the node at the same time.

8.3. Algorithms

To provide differentiated quality of service, the per-hop latency needs to be adjusted according to the service rates of the flows. With a sufficiently small slot length S, the system can provide tiered latency bounds based on the number of slots spanned by L/r, the maximum virtual service interval of a flow. For a flow with (n-1)S < L/r <= nS, the per-hop latency in an idealized preemptive system is bounded by (n+1)S. Flows with higher service rates, that is, smaller L/r, therefore receive lower latency, which aligns latency performance with bandwidth allocation.

In an idealized preemptive system, the scheduler ensures that all packets mapped to a slot are fully served before the end boundary of the slot, T_i. Consequently, the service interval of any packet is contained within its assigned slot. In a realistic non-preemptive system, however, the service completion can be delayed by up to L_h/R_h, where L_h is the maximum packet length and R_h is the link capacity at node h. Thus, the service of the packet is guaranteed to complete no later than T_i + L_h/R_h.

The slot length S is chosen such that S >= min_p{L(p)/r(p)}. While a smaller S generally reduces the E2E latency, reducing S below this lower bound, which is determined by the minimum packet transmission time relative to the flow rate, yields no additional scheduling benefit. In practice, N hardware queues are allocated and managed cyclically to accommodate the slots. Consequently, the selection of S involves a trade-off: a larger S allows a smaller number of physical queues N, at the expense of reduced scheduling precision.

The approximate C-SCORE follows the architecture of C-SCORE. That is, Equations (1) and (2) are still used. However, the delay factor in Equation (3) is replaced by the one in Equation (7).

  d_h(p) = L_h/R_h + (n_h+1)S_h + delta_h(p)                (7)

where S_h is the slot length at node h, and n_h is the integer that satisfies (n_h-1)S_h < L/r <= n_h*S_h for the flow under observation, that is, n_h = ceiling(L/(r*S_h)).

8.4. E2E Latency Bound of the Approximate C-SCORE

In a network of approximate C-SCORE nodes, the E2E latency of packet p is bounded as shown in Equation (8).

                   H                           H-1
  D_H(p) <= B/r + SUM {(n_j+1)S_j + L_j/R_j} + SUM delta_j(p)   (8)
                  j=0                          j=0

where D_H(p) is the latency experienced by packet p from its arrival at node 0 to its departure from node H, the last node on the path, and B is the maximum burst size of the flow. As in Equation (4), the last term accounts for the clock discrepancies and the propagation delays between the nodes. When the clock discrepancies are negligible, for example, with an absolute time reference, this term effectively represents the E2E propagation delay.

Since n_j*S_j < L/r + S_j, the per-hop term (n_j+1)S_j in Equation (8) exceeds the corresponding term L/r in Equation (4) by less than 2S_j. Therefore, as S_j decreases, the latency bound of the approximate C-SCORE approaches that of C-SCORE, except for an additional L/r in the first term.

9. Considerations for Non-compliant Nodes

A network can contain non-compliant end nodes and transit nodes. For example, some end nodes may not support the signaling required for admission control. The network can also contain legacy nodes, or nodes that implement data plane enhancement solutions other than C-SCORE. The compensation of the time difference across non-compliant transit nodes is described in Section 6.4.2.

[Editor's Note: How the presence of non-compliant nodes can be compensated for, and how much these nodes degrade the performance of C-SCORE, will be discussed in a future version of this document.]

10. Relationships to ITU-T Standards

This document is based on two ITU-T Recommendations, [Y.3129] and [Y.3148], and one draft ITU-T Recommendation, [Y.qos-sre-rf].

Recommendation ITU-T Y.3129 specifies the requirements and framework for C-SCORE. The requirements address the generation, update, and properties of the FT. The framework describes a mechanism that guarantees E2E latency bounds while meeting these requirements. It specifies how to obtain the FT at core nodes, how to select the delay factor, and how to configure a network for latency guarantees.

Recommendation ITU-T Y.3148 specifies the functional architecture, functional entities, operational procedures, and packet metadata required to fulfill the requirements and framework specified in Recommendation ITU-T Y.3129. The operational procedures cover stateless FQ at the entrance node and core nodes, including the creation and update of the FT values of packets.

Draft Recommendation ITU-T Y.qos-sre-rf specifies the requirements and framework for identifying the service rates of active flows for admission control without maintaining per-flow state, based on the packet metadata defined in [Y.3129].

This document bridges the gap between these ITU-T standards and deployable protocols. Specifically, it specifies:

11. IANA Considerations

There might be matters that require IANA considerations associated with metadata. If necessary, relevant text will be added in a later version.

12. Security Considerations

This section will be described later.

13. Acknowledgements

14. Contributor

15. References

15.1. Normative References

[I-D.ietf-mpls-mna-detnet]
Song, X., Mirsky, G., Varga, B., Gandhi, R., and Q. Xiong, "MPLS Network Action for Deterministic Networking", Work in Progress, Internet-Draft, draft-ietf-mpls-mna-detnet-01, , <https://datatracker.ietf.org/doc/html/draft-ietf-mpls-mna-detnet-01>.
[I-D.ietf-mpls-mna-ps-hdr]
Rajamanickam, J., Gandhi, R., Zigler, R., Dong, J., and J. Bhattacharya, "MPLS Network Action (MNA) Post-Stack Header Specification", Work in Progress, Internet-Draft, draft-ietf-mpls-mna-ps-hdr-19, , <https://datatracker.ietf.org/doc/html/draft-ietf-mpls-mna-ps-hdr-19>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/info/rfc2119>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/info/rfc8174>.
[RFC8200]
Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, , <https://www.rfc-editor.org/info/rfc8200>.
[Y.3129]
International Telecommunication Union, "Requirements and framework for stateless fair queuing in large scale networks including IMT-2020 and beyond", ITU-T Recommendation Y.3129, .
[Y.3148]
International Telecommunication Union, "Functional architecture for stateless fair queuing in large scale networks including IMT-2020 and beyond", ITU-T Recommendation Y.3148, .
[Y.qos-sre-rf]
International Telecommunication Union, "Requirements and framework for service rates estimation with packet metadata for stateless admission control", ITU-T Draft Recommendation Y.qos-sre-rf, .

15.2. Informative References

[C-SCORE]
Joung, J., Kwon, J., Ryoo, J., and T. Cheung, "Scalable flow isolation with work conserving stateless core fair queuing for deterministic networking", IEEE Access, vol. 11, pp. 105225 - 105247, doi:10.1109/ACCESS.2023.3318479, .
[I-D.ietf-detnet-dataplane-taxonomy]
Joung, J., Geng, X., Peng, S., and T. T. Eckert, "Dataplane Enhancement Taxonomy", Work in Progress, Internet-Draft, draft-ietf-detnet-dataplane-taxonomy-07, , <https://datatracker.ietf.org/doc/html/draft-ietf-detnet-dataplane-taxonomy-07>.
[KAUR]
Kaur, J. and H.M. Vin, "Core-stateless guaranteed rate scheduling algorithms", in Proc. INFOCOM, vol.3, pp. 1484-1492, .
[RFC5905]
Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, , <https://www.rfc-editor.org/info/rfc5905>.

Authors' Addresses

Jinoo Joung
Sangmyung University
Jeong-dong Ryoo
ETRI
Taesik Cheung
ETRI
Yizhou Li
Huawei
Peng Liu
China Mobile