]> Ericsson

marcus.ihlar@ericsson.com

Ericsson

mirja.kuehlewind@ericsson.com

Web and Internet Transport SCONE measurement delay RTT on-path This document defines POEM Delay, a delay measurement companion to the Periodic On-path Explicit Measurement (POEM) protocol for QUIC. A delay sample packet, coalesced with ordinary QUIC packets, bounces between the endpoints. On-path observers timestamp it in each direction to measure the round-trip time and its components on either side of the observer. Each sample carries a coarse indication of how long the sending endpoint held it, so observers can largely exclude endpoint delay. A report packet lets observers tell endpoints how network delay is split between the two sides of the observer. About This Document Status information for this document may be found at . Discussion of this document takes place on the SCONE Working Group mailing list (), which is archived at . Subscribe at .

Introduction QUIC encrypts the transport header fields that on-path observers use to measure delay for TCP. This document defines POEM Delay, an explicit delay signal for QUIC. Like the existing signals, it trades accuracy against how often a measurement succeeds:

• The spin bit () yields a measurement every round trip for any traffic, but includes endpoint idle time, which can be long when the application limits sending. Reordering and loss cause further errors.
• The Delay bit uses a single marked packet, the delay sample, that bounces between client and server. An endpoint that cannot reflect the sample within 1 ms drops it, so measurements are accurate, but many samples are dropped when the application limits sending or when acknowledgments are delayed by more than 1 ms.
• POEM Delay keeps the accuracy of the Delay bit while more measurements succeed. Endpoints can hold the sample up to 30 ms and report the coarse response delay, which observers subtract.

POEM Delay carries the delay sample in a QUIC long header packet coalesced with ordinary QUIC packets, following SCONE and the loss measurement protocol of , called POEM Loss here. As in those protocols, all information is in the first octet. Compared with the Delay bit, it adds:

• a Sample ID, which keeps observers from pairing unrelated samples, such as a lost sample and its replacement, and
• a coarse response delay, which observers subtract to measure network delay rather than network plus endpoint delay.

A delay report packet, analogous to the POEM Loss report, lets an observer tell an endpoint how network delay is split between the two sides of the observer. POEM Loss and POEM Delay are independent and can be used together.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document uses the following terms:

Delay sample:

The packet that bounces between the endpoints ().

Sample chain:

A delay sample generated by the client and its reflections, all carrying the same Sample ID.

Generation:

Sending a delay sample that starts a new chain. Only the client generates.

Reflection:

Sending a delay sample in response to receiving one.

Response delay:

The time an endpoint holds a delay sample before reflecting it.

Observer:

An on-path network element that timestamps delay samples or writes into delay report packets.

Component:

The round-trip delay between the observer and one endpoint.

Network RTT:

The round-trip time excluding the response delay of both endpoints.

Overview After negotiating support (), the client generates a delay sample. Each endpoint reflects a received sample in the next datagram it sends, so the sample crosses the path once in each direction per round trip. Each sample carries a 2-bit Sample ID and a 4-bit Response Delay, which indicates in coarse steps how long the sender held it. An observer timestamps each sample as it passes ().

Delay Sample Operation | t0 | | |--------------------->| | | | D1 | |<----- DS(id,D1) -----| |<--------------------| t1 | D2 | | | |---- DS(id,D2) ----->| t2 | | |--------------------->| | | | Server component (network) = t1 - t0 - D1 Client component (network) = t2 - t1 - D2 Network RTT = t2 - t0 - D1 - D2 ]]>

An observer that sees both directions measures both components and the network RTT. An observer that sees one direction measures the RTT, but can only subtract the response delay of one endpoint (). Endpoints can also send delay reports (), into which an observer writes the fraction of the network RTT on the report sender's side.

Delay Sample Packets

Delay Sample Packet Format A delay sample packet is a QUIC long header packet that follows the QUIC invariants; see .

Delay Sample Packet Format

Header Form:

Set to 1 (0x80), indicating a QUIC long header packet.

Reserved:

The next bit (0x40) is reserved and can be set according to .

Sample ID:

A 2-bit value (0x30) identifying the chain.

Response Delay:

A 4-bit code (0x0f) for the response delay of the sender ().

Version:

Set to 0xTBD3, identifying a POEM Delay sample packet.

Destination Connection ID Length and Destination Connection ID:

Set to the same value as other packets in the datagram; see .

Source Connection ID Length and Source Connection ID:

Set to match the Source Connection ID of the packet that follows in the datagram, or empty if that packet has a short header.

Delay sample packets MUST be coalesced with other QUIC packets and MUST be the first packet in the UDP datagram. As in SCONE and POEM Loss , all signals are in the first octet. This leaves the space after the connection IDs to the coalesced QUIC packet and lets observers read the signals at a fixed offset, without parsing the variable-length connection IDs. SCONE packets and POEM Loss packets also need to be first, so a datagram carries at most one of these, a delay sample, or a delay report. A reflection that cannot go in the next datagram for this reason is sent in a later one, and the wait counts as response delay. Endpoints SHOULD NOT add an optional packet of this kind, such as a delay report, to a datagram while a reflection is pending.

Response Delay The response delay runs from receiving the datagram carrying a delay sample to sending the datagram carrying its reflection. Typically, an endpoint records the receive time when it extracts the sample and computes the response delay when it builds the reflection. A receive time recorded before the coalesced QUIC packet is processed is discarded if that packet is not successfully processed. The Response Delay field encodes the response delay in 2 ms steps: Response Delay Codes

Code	Response delay	Observer estimate
0	0 ms to < 2 ms	1 ms
1	2 ms to < 4 ms	3 ms
n	2n ms to < 2n+2 ms	2n+1 ms
14	28 ms to < 30 ms	29 ms
15	Not reported	(none)

The 2 ms granularity balances measurement accuracy against exposure of endpoint-internal timing, such as cryptographic processing time (). An endpoint MUST NOT reflect a sample if the response delay would reach MAX_RESPONSE_DELAY, which is 30 ms. The sample is dropped and the chain ends. This value covers the default max_ack_delay of 25 ms in QUIC versions 1 and 2 (), and a fixed limit lets the client choose T_Max () without knowing the server's acknowledgment behavior. Generated samples carry code 15. An endpoint MAY also send code 15 in a reflection, for example if it does not want to expose its response delay.

Generation Only the client generates delay samples. When and whether to generate is up to the client; the server reflects any chain it receives. A client that wants continuous measurement starts a new chain:

• in one of its first datagrams after receiving the server's transport parameter,
• on a new path, once it is permitted to send on it (),
• when T_Max has elapsed since it last sent a sample of the current chain without accepting a reflection, and
• when a chain ends because of the response delay limit or the chain length limit (below).

A generated sample carries the Sample ID of the previous chain on the path plus one, modulo 4 (any value for the first chain), and Response Delay 15. T_Max needs to exceed the RTT plus the server's response delay, so that the client does not replace chains still in flight. A suitable value is twice the smoothed RTT plus MAX_RESPONSE_DELAY, or 1 second until an RTT estimate is available, such as the smoothed RTT of QUIC versions 1 and 2 (). A client MUST NOT reflect a sample after accepting MAX_CHAIN_REFLECTIONS reflections of the current chain, where MAX_CHAIN_REFLECTIONS is 8. This ends the chain and bounds the effect of extra samples that the acceptance rules cannot detect (). If the client starts a new chain in its place, observers lose about one measurement per 8 round trips, since a generated sample is never the later sample of a pair. A client can stop generating at any time, for example when the connection is idle. If it also stops reflecting, the chain ends.

Reflection Each endpoint keeps, per path, the Sample ID of the current chain and whether it is awaiting a sample, that is, whether it has sent a sample of the chain and not yet accepted one in return. A server accepts a delay sample when:

• its Sample ID differs from the current chain, which starts a new chain;
• its Sample ID matches the current chain and the server is awaiting a sample; or
• the server has no chain on this path, for example after a NAT rebinding, which makes the sample's chain current.

A client accepts a delay sample when its Sample ID matches the current chain and the client is awaiting a sample. Other samples are discarded without changing the chain state. An endpoint can also decline an acceptable sample, for example because of load, which ends the chain. An endpoint reflects an accepted sample in the first datagram it sends on the same path that can carry it, copying the Sample ID and setting the Response Delay. It can hold the reflection longer to mask the RTT (), within MAX_RESPONSE_DELAY. If it accepts a sample of a new chain while holding one, it reflects only the new one. Endpoints record when they send each sample. With QUIC duplicate detection (), these rules normally keep one sample in flight per chain and let the client end stale chains by not reflecting them. A late sample of an old chain costs at most one reflection, or the current chain if the server is holding a sample. The rules cannot detect a second sample with the current Sample ID, for example one injected on path or a stale sample after the Sample ID wraps; MAX_CHAIN_REFLECTIONS bounds how long such a sample circulates.

Reflection Opportunities An endpoint MUST NOT send a datagram, or send it earlier, solely to carry a delay sample. Delay samples therefore never generate traffic or affect idle timeouts. If no datagram is sent within MAX_RESPONSE_DELAY, the chain ends. When they have a choice, endpoints SHOULD carry delay samples in datagrams that the peer will respond to promptly, giving the peer an opportunity to reflect. In QUIC versions 1 and 2, these are datagrams containing ack-eliciting packets, which the peer acknowledges within its max_ack_delay ().

Receiving Delay Samples A delay sample MUST be discarded unless another QUIC packet in the same datagram is successfully processed. QUIC packets discarded as duplicates are not successfully processed, so endpoints ignore duplicated samples. This relies on the QUIC version discarding duplicates, as versions 1 and 2 do (); delay samples MUST NOT be used with versions that do not.

Connection Migration Delay sample state is per path. After migrating, the client starts a new chain once it is permitted to send on the new path. Endpoints discard chain state for paths no longer in use. Endpoints SHOULD NOT send delay samples on a path while the QUIC version restricts sending on it, such as before path validation in QUIC versions 1 and 2. Delay samples count toward any anti-amplification limit, such as that in .

Observing Delay Samples An observer identifies delay sample packets by the long header bit (0x80) and the version 0xTBD3, and determines their direction from the UDP 4-tuple. Observers do not modify delay samples. Below, D is the observer estimate for the Response Delay of the later sample of a pair (). Each subtraction of D has an error of up to 1 ms.

Bidirectional Observers An observer that sees both directions measures a component from two consecutive samples of the same chain in opposite directions, seen at t1 and t2: This is the network component toward the endpoint the earlier sample was sent to. Two consecutive components add up to the network RTT.

Unidirectional Observers Any observer can measure the RTT from two consecutive samples of the same chain in the same direction, seen at t1 and t2: This subtracts the response delay of the endpoint sending the observed samples, but not that of the other endpoint, which is carried in the direction the observer does not see. The result is the network RTT plus up to MAX_RESPONSE_DELAY.

Validity of Measurements Observers MUST only pair samples with the same Sample ID. A sample with a new Sample ID starts a chain and is never the later sample of a pair. If the later sample has Response Delay 15, D is 0 and the result includes that endpoint's response delay, which is less than MAX_RESPONSE_DELAY. Such results are comparable to Delay bit measurements. Samples of a chain alternate in direction. If a bidirectional observer sees two samples in the same direction with none in between, one is a duplicate or a sample was missed. It MUST NOT use either for a component, and resumes with the next sample in the opposite direction. Otherwise, observers cannot detect duplicates or missed samples, and chains they do not see can bring the Sample ID back to the same value. Observers therefore discard implausible results, such as negative values or intervals much shorter or longer than recent measurements.

Observer State Per flow, an observer keeps the Sample ID of the current chain and, per direction, the time and Response Delay of the most recent sample, marked invalid after a repeat in the same direction.

Delay Report Packets

Delay Report Packet Format A delay report packet is a QUIC long header packet into which an observer writes delay localization information for the receiver.

Delay Report Packet Format

Header Form, Reserved, and Connection ID fields:

As for delay sample packets ().

Upstream Delay Ratio:

A 6-bit value (0x3f). Values 0 to 62 represent the fraction value/62 of the network RTT between the report sender and the observer. The value 63 means no observer wrote the field; endpoints MUST send 63.

Version:

Set to 0xTBD4, identifying a POEM Delay report packet.

Delay report packets follow the coalescing rules for delay sample packets ().

Sending Delay Report Packets Either endpoint MAY send delay reports. Endpoints SHOULD NOT send more than one per network round trip; one every 4 to 8 round trips is RECOMMENDED.

Observing Delay Report Packets An observer identifies delay report packets by the long header bit (0x80) and the version 0xTBD4. The upstream component is the one toward the report sender. An observer that has not measured both components MUST NOT write the field. Otherwise, it MAY write: rounded and clamped to 0-62, using network components from consecutive pairs of the same chain. Observers SHOULD average over multiple round trips to reduce the effect of transient queuing and response delay quantization; the method is up to the observer. Any observer MAY overwrite the field. As in POEM Loss , the receiver sees the value from the last observer on the path.

Receiving Delay Report Packets A delay report packet MUST be discarded unless another QUIC packet in the same datagram is successfully processed. A value of 63 means no observer wrote the field. Otherwise, the receiver can combine the ratio with its own network RTT estimate over a similar period:

• Component toward the report sender = network_rtt * ratio / 62
• Component toward the receiver = network_rtt * (1 - ratio / 62)

RTT estimates from QUIC loss recovery () include endpoint delay and would bias the result. Receivers SHOULD average over multiple reports.

Application Use Applications can use delay localization, for example, to distinguish access network delay from delay elsewhere on the path, to inform path selection when multiple network accesses are available, or for diagnostics. How applications use this information is out of scope.

Applicability

Observer Location As in POEM Loss , reports do not identify the observer or its location. Endpoints SHOULD NOT assume a particular observer location.

Measurement Rate and Accuracy A bidirectional observer gets at most one component measurement per one-way trip, and any observer at most one RTT measurement per round trip. Sample loss and long response delays reduce this rate, and delay variation within an RTT cannot be measured. The coarse Response Delay adds an error of up to 1 ms per subtracted response delay, which averaging reduces when response delays vary.

Bidirectional Requirement Unlike POEM Loss, POEM Delay requires both endpoints to reflect samples. Components, network RTT, and reports require an observer that sees both directions. With asymmetric routing, observers measure an RTT that includes one endpoint's response delay.

Per-Flow Scope Measurements apply to a single flow identified by the UDP 4-tuple.

Relationship to the Spin Bit Delay samples and the spin bit are independent. Observers can prefer delay sample measurements, which largely exclude endpoint delay and tolerate reordering, and fall back to the spin bit when none are recent; see .

Negotiating POEM Delay An endpoint indicates that it can receive and reflect delay samples by including the poem_delay_supported transport parameter (0xTBD). Endpoints MUST NOT send delay sample packets unless both endpoints included the transport parameter. An endpoint MAY send delay report packets if its peer included it. The transport parameter MUST be empty. A non-zero length MUST be treated as a connection error of type TRANSPORT_PARAMETER_ERROR. This transport parameter is valid for QUIC versions 1 and 2 and any other version that recognizes the versions and transport parameters registries established in Sections and of . Endpoints MUST NOT remember the transport parameter for 0-RTT, so delay sample and delay report packets cannot be sent until it is received.

Security Considerations

On-Path Modification Delay reports rely on on-path elements modifying packets, the same trust model as SCONE and POEM Loss : an element that can modify packets can also drop or delay them. Delay sample fields are not integrity protected. An on-path element can change the Response Delay to mislead other observers, change the Sample ID to end the chain or revive a stale one, or add a second sample with the current Sample ID. The effect is limited to incorrect or lost measurements until the chain ends, after at most MAX_CHAIN_REFLECTIONS client reflections ().

Fake Packets Endpoints only accept delay samples and reports from datagrams with a successfully processed QUIC packet, which off-path attackers cannot construct.

Misleading Reports An observer can write misleading ratios. Endpoints SHOULD treat reports as advisory and compare them with their own measurements.

Falsified Delay Signals Endpoints can falsify delay samples:

Inflated response delay:

Observers underestimate network delay, possibly producing negative values, which are discarded.

Deflated response delay:

Endpoint delay is attributed to the network between the observer and that endpoint.

Extra samples:

Generating samples at a high rate or reflecting samples that should not be accepted puts multiple samples in flight, producing meaningless measurements.

Observers cannot verify endpoint claims. Network operators MUST NOT use POEM Delay signals as the sole input to automated network policy, assurance, or engineering decisions, and should compare measurements from untrusted endpoints with other data.

Privacy Considerations

Flow Identification Delay sample and report packets add no flow identifiers beyond the UDP 4-tuple and connection IDs. Chain state is not carried across paths, and a 2-bit Sample ID cannot meaningfully link paths.

Endpoint Roles An observer that sees a new Sample ID appear can identify the client, which is the only generator. Observers that see the handshake already know this, and no measurement here uses it.

Delay Exposure Delay samples expose the RTT and, to bidirectional observers, the RTT to each endpoint. The spin bit already exposes the RTT; discusses the privacy implications. A precise response delay would directly reveal endpoint-internal timing, such as cryptographic processing time, scheduling, or load. The 2 ms steps avoid this, but do not prevent inference: observers see the total interval between samples, so where network delay is stable they can infer response delay variation from timing alone, as with the spin bit and Delay bit. Over many samples, jitter around step boundaries can reveal finer detail, and the code itself distinguishes processing paths that differ by more than 2 ms. describes masking the RTT with a random reflection delay kept constant across connections for some time. A client can do so without including the added delay in the Response Delay, so that observers attribute it to the network on the client side. The added delay counts toward MAX_RESPONSE_DELAY. Endpoints can send code 15 to withhold their response delay, or omit the transport parameter to expose no delay information.

IANA Considerations This document requests the following registrations.

POEM Delay Version Numbers This document registers the following entries in the "QUIC Versions" registry:

Value:

0xTBD3

Status:

permanent

Specification:

This document

Change Controller:

IETF (iesg@ietf.org)

Contact:

QUIC Working Group (quic@ietf.org)

Notes:

POEM Delay Sample Packet

Value:

0xTBD4

Status:

permanent

Specification:

This document

Change Controller:

IETF (iesg@ietf.org)

Contact:

QUIC Working Group (quic@ietf.org)

Notes:

POEM Delay Report Packet

poem_delay_supported Transport Parameter This document registers the following entry in the "QUIC Transport Parameters" registry:

Value:

0xTBD

Parameter Name:

poem_delay_supported

Status:

Permanent

Specification:

This document

Change Controller:

IETF (iesg@ietf.org)

Contact:

QUIC Working Group (quic@ietf.org)

Notes:

(none)

References Normative References This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. This document defines the properties of the QUIC transport protocol that are common to all versions of the protocol. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document describes a method for negotiating the ability to send an arbitrary value for the second-most significant bit in QUIC packets. This document specifies QUIC version 2, which is identical to QUIC version 1 except for some trivial details. Its purpose is to combat various ossification vectors and exercise the version negotiation framework. It also serves as a template for the minimum changes in any future version of QUIC. Note that "version 2" is an informal name for this proposal that indicates it is the second version of QUIC to be published as a Standards Track document. The protocol specified here uses a version number other than 2 in the wire image, in order to minimize ossification risks. Informative References This document describes loss detection and congestion control mechanisms for QUIC. Ericsson Ericsson This document defines the Periodic On-path Explicit Measurement (POEM) protocol, which enables passive on-path measurement of packet loss for QUIC flows. POEM uses periodic marker packets, coalesced with ordinary QUIC packets, that allow on-path network elements to measure upstream packet loss by counting packets between markers. Marker packets also carry a sender-reported loss count, enabling observers to distinguish upstream from downstream loss. Additionally, POEM defines a report mechanism through which network elements communicate measurement results back to endpoints. Mozilla Private Octopus Inc. Fastly Meta Ericsson This document describes a protocol where on-path network elements can communicate their perspective on the maximum sustainable throughput for QUIC flows to endpoints. This throughput advice suggests an upper bound on long-term average throughput, independent of and complementary to real-time congestion control signals. This document describes protocol-independent methods called Explicit Host-to-Network Flow Measurement Techniques that can be applicable to transport-layer protocols between the client and server. These methods employ just a few marking bits inside the header of each packet for performance measurements and require the client and server to collaborate. Both endpoints cooperate by marking packets and, possibly, mirroring the markings on the round-trip connection. The techniques are especially valuable when applied to protocols that encrypt transport headers since they enable loss and delay measurements by passive, on-path network devices. This document describes several methods that can be used separately or jointly depending of the availability of marking bits, desired measurements, and properties of the protocol to which the methods are applied.

Acknowledgments This work builds on the Delay bit defined in and the communication model of the SCONE protocol .