]> CableLabs

858 Coal Creek Circle Louisville Colorado 80027 US g.white@cablelabs.com http://www.cablelabs.com

Ciena

gregimirsky@gmail.com

ZTE Corp.

Nanjing CN xiao.min2@zte.com.cn

Operations and Management IP Performance Measurement stamp DSCP ECN Congestion The Simple Two-Way Active Measurement Protocol (STAMP) enables one-way and round-trip measurement of network metrics between IP hosts, and has a facility for defining optional extensions. This document updates the definition of the Class of Service TLV (originally defined in RFC 8972) to enable the measurement of manipulation of the value of the Explicit Congestion Notification (ECN) field of the IP header by middleboxes between two STAMP hosts, and to enable discovery and measurement of paths that provide differential treatment of packets depending on the value of their ECN field.

Introduction defined several extensions to the Simple Two-way Active Measurement Protocol (STAMP). Among those is a Class of Service (CoS) TLV that enables measurements that utilize the Differentiated Services Code Point (DSCP) marking in both directions. Also, the CoS TLV supports outbound measurements that utilize the Explicit Congestion Notification (ECN) field, but it lacked support for such measurements on the return path. Experience deploying STAMP and its extensions demonstrated that it is helpful to an operator to monitor ECN's consistency in both directions. This specification updates the definition of the CoS TLV in a backward compatible manner to support monitoring of ECN in the return path of the STAMP test session. The ECN monitoring capabilities defined in this document can help operators detect ECN bleaching and investigate its location along a path. These capabilities can also complement hop-by-hop measurements using on-path telemetry, as described in . discusses these uses. Another STAMP extension header defines a mechanism that can be used by a Session-Sender to request that the Session-Reflector copy the entire IP header from the received test packet into the reflected test packet. That capability could be used in place of the CoS TLV to observe the DSCP and ECN values in the forward direction in situations where there is no need to control the return path DSCP or ECN value.

Conventions Used in This Document

Acronyms

CoS:

Class of Service

DSCP:

Differentiated Services Code Point

ECN:

Explicit Congestion Notification

STAMP:

Simple Two-way Active Measurement Protocol

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Class of Service TLV

TLV Definition The STAMP Session-Sender MAY include a CoS TLV in the STAMP test packet. The format of the CoS TLV is presented in .

Class of Service TLV

Where the fields are defined as follows.

• STAMP TLV Flags: eight-bit field; format presented in .
• CoS (Class of Service) Type: one-octet field; the value MUST be set to 4.
• Length: two-octet field; set equal to the value 4 (octets).
• DSCP1: DSCP value intended by the Session-Sender to be used as the DSCP value of the reflected test packet.
• DSCP2: received value in the DSCP field at the ingress of the Session-Reflector.
• EC2: received value in the ECN field at the ingress of the Session-Reflector.
• RPD (reverse path DSCP): two-bit field indicating whether the Session-Reflector used DSCP1 or DSCP2 as the DSCP value of the reflected test packet; a Session-Sender MUST set the value of the RPD field to 0b00 on transmission.
• EC1: ECN value intended by the Session-Sender to be used as the ECN value of the reflected test packet.
• RPE (reverse path ECN): two-bit field indicating whether the Session-Reflector used EC1 as the ECN value of the reflected test packet; a Session-Sender MUST set the value of the RPE field to 0b00 on transmission.
• Reserved: twelve-bit field; MUST be zeroed on transmission and ignored on receipt.

Session-Reflector Behavior A STAMP Session-Reflector that receives a test packet with the CoS TLV MUST include the CoS TLV in the reflected test packet. The Session-Reflector MUST copy the value of the DSCP and ECN fields of the IP header of the received STAMP test packet into the DSCP2 and EC2 fields, respectively, in the CoS TLV in the reflected test packet. A Session-Reflector might have a local policy configuration that limits the DSCP values that it can use for reflected test packets. This local policy could be (for example) a system default policy, a global policy for the Session-Reflector, or a policy that is configured for specific destination addresses or networks. The Session-Reflector MUST verify whether the use of the value of the DSCP1 field is permitted in the reflected test packet. If it is, the Session-Reflector MUST set the DSCP field's value in the IP header of the reflected test packet equal to the value of the DSCP1 field of the received test packet. Otherwise, the Session-Reflector MUST use the DSCP value of the received STAMP packet and set the value of the RPD field to 0b01. Upon receiving the reflected packet, if the value of the RPD field is 0b00, the Session-Sender will save the DSCP value for analysis of the CoS in the reverse direction. If the value of the RPD field in the received reflected packet is 0b01, only CoS in the forward direction can be analyzed. A Session-Reflector might have a local policy configuration that limits the ECN values that it can use for reflected test packets. This local policy could be (for example) a system default policy, a global policy for the Session-Reflector, or a policy that is configured for specific destination addresses or networks. Additionally, a Session-Reflector could have platform limitations that prevent it from setting the ECN value in reflected test packets. The Session-Reflector MUST set the ECN value in the IP header of the reflected STAMP test packet to the value of the EC1 field, if it is permitted and capable to do so. If the Session-Reflector is able to set the ECN value in the IP header of the reflected STAMP test packet to the EC1 value, it MUST then set the RPE field in the CoS TLV in the reflected test packet to the value 0b11. If the Session-Reflector is unable to set the ECN value in the IP header of the reflected STAMP test packet to the EC1 value, it MUST instead set the ECN value in the IP header to 0b00 and set the RPE field in the CoS TLV to the value 0b10 in the reflected test packet. If the Session-Reflector is unable to set the ECN value in the IP header of the reflected STAMP test packet to the EC1 value, and the Session-Reflector is unable to set the ECN value in the IP header of the reflected STAMP test packet to 0b00, it MUST set the Conformant Reflected Packet STAMP TLV flag () to 1. As a result, the Session-Sender can detect whether the EC1 value was used by inspecting the value of the RPE field and the Conformant Reflected Packet STAMP TLV flag in the received reflected test packet.

Interoperability with RFC 8972 The extended CoS TLV defined in this draft is backward compatible with the specification in . The handling of the DSCP1, DSCP2, EC2 and RPD fields defined here is identical to the handling defined for the equivalent fields in . Consider a case when an implementation that supports this specification performs as Session-Sender, and the intended Session-Reflector's support of the CoS TLV is according to . If the operator requires monitoring ECN in the reverse direction, the value of the EC1 field will be set to a non-zero value. Because the Session-Reflector would treat the EC1 field as part of the Reserved field, it would ignore its value as per . Further, the Session-Reflector would treat the RPE field as part of the Reserved field and thus it would send the value 0b00 in the reflected STAMP packet, rather than sending the value 0b10 or 0b11. Consequently, the Session-Sender will determine that the Session-Reflector does not implement the current version of the CoS TLV, and that the ECN value in the IP header of the reflected test packet was not set as requested. Also, this specification supports the case when the Session-Reflector supports the extended CoS TLV as defined in this specification and the Session-Sender supports the CoS TLV according to . In that scenario, the Session-Sender will set its Reserved field to zeros. The Session-Reflector will interpret the first two bits of that field as the EC1 field, as shown in and thus will set the value of the ECN field in the IP header of the reflected packet to 0b00. Further the Session-Reflector will set the RPE field to 0b11. The Session-Sender will treat the RPE field as part of the Reserved field and will ignore its value.

Congestion Response and require applications that send packets marked with the ECT0 or ECT1 codepoints to implement a response to congestion notifications from the network. While the STAMP protocol doesn't include the concept of a connection between a Session-Sender and a Session-Reflector, there are situations in which the Session-Sender may send multiple STAMP packets to the same Session-Reflector within the round-trip time between them, and there are situations in which the Session-Sender may elicit multiple STAMP packets from a Session-Reflector in response to a single sent STAMP packet. The Session-Sender therefore generally controls both its own sending rate and the rate of packets sent in response by an individual Session-Reflector, and is responsible for implementing an appropriate congestion response. When using this CoS TLV with either an ECT0 or an ECT1 value in the ECN field of the IP header of the STAMP packet, the Session-Sender gains the ability (via observing a CE reflected value in the EC2 field) to detect that congestion was present on the path from the Session-Sender to the Session-Reflector. Similarly, when using this CoS TLV with either an ECT0 or an ECT1 value in the EC1 field, the Session-Sender gains the ability (via observing a CE in the ECN field of the IP header of the reflected packet) to detect that congestion was present on the path from the Session-Reflector to the Session-Sender. If a Session-Sender sends multiple STAMP packets with the CoS TLV to a Session-Reflector within an RTT with either the ECT0 or ECT1 value in the ECN field of the IP header, it MUST observe the reflected EC2 field and be capable of reducing its sending rate upon observation of a CE value. If a Session-Sender sends multiple STAMP packets with the CoS TLV to a Session-Reflector within an RTT with either the ECT0 or ECT1 value in the EC1 field, it MUST observe the ECN value in the IP header of the reflected packets and be capable of reducing its sending rate upon observation of a CE value. If a Session-Sender sends a STAMP packet containing both the CoS TLV and the Reflected Test Packet Control TLV () to a Session-Reflector, and specifies either the ECT0 or ECT1 value in the EC1 field, it MUST observe the ECN value in the IP header of the reflected packets and be capable of adjusting the Reflected Test Packet Control parameters in any future STAMP packet sent to the same Session-Reflector based on the observation of CE values. The specific congestion response behavior is left to the implementer, but it MUST be consistent with the congestion response requirements defined in and . The CoS TLV can be used to characterize the CE-marking behavior of a path under a range of traffic conditions, including overload. Such measurements may require a Session-Sender to maintain a specified test traffic rate or pattern despite observing CE markings. The congestion response capabilities described above MUST be enabled except when conducting such measurements in a controlled environment with the agreement of the operators of the networks under test. In these cases, operators SHOULD establish limits on the rate and duration of test traffic, including reflected traffic, and take measures to prevent unintended impact on traffic outside the scope of the test.

Operational Considerations This document extends the STAMP Class of Service (CoS) TLV to support the measurement of Explicit Congestion Notification (ECN) treatment in both the forward and reverse directions of a STAMP test session. The extension does not introduce new protocol state or session establishment procedures, but it does introduce additional operational considerations for deployment, monitoring, and troubleshooting.

Deployment Considerations To obtain meaningful ECN measurements, both the Session-Sender and Session-Reflector SHOULD support the extended CoS TLV specified in this document. In deployments where a Session-Reflector only supports the CoS TLV defined in , a Session-Sender can detect the absence of reverse-path ECN support through the behavior of the RPE field and SHOULD limit its analysis accordingly. Operators SHOULD verify that the platform hosting the Session-Reflector is capable of setting and reporting ECN values in reflected test packets. Platform limitations or operating system restrictions may prevent the requested ECN value from being applied.

Configuration and Policy Considerations The behavior of a Session-Reflector may be influenced by local policy governing the use of DSCP and ECN values in reflected packets. Such policies can be configured globally or on a per-destination basis. Operators SHOULD understand and document these policies because policy-enforced remarking may otherwise be incorrectly interpreted as network impairment or middlebox behavior. When measurements are performed across administrative domains, operators SHOULD coordinate ECN handling policies to facilitate consistent interpretation of measurement results.

Monitoring and Observability The extension improves observability of ECN processing by providing information about ECN handling on both directions of the measurement path. Specifically, a Session-Sender can determine:

• The ECN value received by the Session-Reflector.
• Whether the Session-Reflector applied the requested reverse-path ECN value.
• Whether Congestion Experienced (CE) markings were observed on either path.

Operators SHOULD collect and analyze the EC2 field, the ECN value observed in reflected packets, and the RPE field. These values can assist in identifying ECN bleaching, ECN remarking, differential treatment of ECN-marked packets, or congestion occurring along the measured path. ECN bleaching, in which a network device clears the ECN field to Not-ECT, has been observed in deployed networks . Operators can investigate this failure mode using the CoS TLV. For example, when a Session-Sender transmits an ECT(0)- or ECT(1)-marked test packet and the reflected EC2 field reports Not-ECT, the measurement reveals bleaching on the forward path. Operators can use this information to identify and correct misconfigurations that prevent ECN from operating as intended. When these measurements reveal ECN bleaching, operators can use complementary diagnostic tools to help locate the responsible devices. For example, traceroute-like techniques can inspect the quoted IP headers in ICMP responses to detect changes to the ECN field along a path; describes a tool for investigating such header modifications. While these techniques are independent of STAMP and do not require the use of STAMP packets or the CoS TLV, differences in probe traffic and paths can affect how their results relate to the STAMP measurements. ECN monitoring using the CoS TLV can be combined with hop-by-hop measurements as described in . The CoS TLV itself does not provide observations from intermediate nodes. These hop-by-hop measurements require additional on-path telemetry encapsulations, such as IPv6 Hop-by-Hop Options for Alternate Marking or In Situ OAM (IOAM), or MPLS Network Action (MNA) Sub-Stacks, together with support and configuration at participating nodes. Correlating the resulting telemetry with ECN observations can provide additional information for investigating congestion and differential packet treatment along the measured path. For example, the extensions defined in can be used to return collected IOAM data in IPv6 extension headers to the Session-Sender, while the CoS TLV provides ECN monitoring. The deployment and security considerations of the selected telemetry mechanism also apply.

Troubleshooting Considerations The CoS TLV can be used to compare performance metrics, such as latency and packet loss, for test packets sent with different ECN codepoints. Interpretation of these measurements depends on the packets retaining the ECN capability indicated by the selected codepoint along the path under test, while allowing for congestion marking from ECT(0) or ECT(1) to CE. ECN bleaching or other unintended remarking can prevent packets from receiving the treatment being evaluated and lead to unexpected or misleading performance results. The ECN observations described in can help operators establish whether this prerequisite is met. Unexpected ECN-related measurement results may be caused by:

• Middleboxes modifying or clearing ECN markings.
• Traffic conditioning or QoS policies.
• Session-Reflector platform limitations.
• Network congestion resulting in CE markings.

When investigating unexpected behavior, operators SHOULD examine the EC2 and RPE fields together with the ECN field of reflected packets in order to distinguish between network-induced behavior and Session-Reflector policy enforcement.

Performance and Congestion Considerations This extension enables the use of ECT(0) and ECT(1) markings in STAMP measurements. Implementations using these codepoints are required to monitor for CE indications and react appropriately to congestion conditions, subject to the exception for controlled measurements described in . Operators SHOULD ensure that automated measurement systems utilizing ECN-marked STAMP packets comply with the congestion response requirements discussed in , including the conditions for controlled measurements, and do not generate excessive measurement traffic outside those conditions when CE markings are observed.

Interoperability Considerations The extension is designed to be backward compatible with implementations. Mixed deployments may therefore be introduced incrementally. Operators SHOULD verify interoperability between Session-Senders and Session-Reflectors during deployment and SHOULD confirm that reverse-path ECN functionality is available before relying on bidirectional ECN measurement results.

IANA Considerations This document includes no request to IANA.

Security Considerations This document extends the functionality of the CoS TLV () and inherits all the security considerations applicable to the base STAMP specification and . As this specification defines a mechanism to set ECN values, this document inherits all the security considerations discussed in and . Monitoring and optional control of ECN for a reflected STAMP test packet using the extended CoS TLV may be used across the Internet such that the Session-Sender and the Session-Reflector are located in different domains. In environments where a middlebox may tamper with the CoS TLV, this could affect the accuracy of ECN measurements. Implementers should consider the security implications of such tampering and take appropriate measures to mitigate potential risks. These measures could include the use of the HMAC TLV defined in and/or the use of UDP port numbers from the Dynamic Ports (aka Private or Ephemeral Ports) range () as the destination UDP port number in the STAMP test packets. The use of dynamic ports makes it harder for a middlebox to detect STAMP packets compared with when it is assigned the well-known UDP port number 862.

References Normative References Informative References

Acknowledgements TBD

Contributors Karthik Sundaresan, William Hawkins III, Gorry Fairhurst, Rüdiger Geib