bpm L. Contreras Internet-Draft A. Bazan Intended status: Informational Telefonica Expires: 6 May 2027 2 November 2026 Calibration of Measured Time Values between Network Elements draft-contreras-bpm-calibration-00 Abstract Network devices are incorporating capabilities for time stamping certain packets so that such time stamps can reference the time at which a packet passes through a given device (at ingress or egress). Those stamps or marks are relevant to calculating the measured delay and can be used for traffic engineering purposes. To ensure consistency and accuracy across different network element implementations, a benchmarking procedure is necessary to calibrate the timestamps. Such a procedure can permit the identification and correction of any deviations or biases in the time stamps produced by diverse devices. This document proposes a methodology for Calibrating the measurements from different network element implementations in a common measurement scenario. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 6 May 2027. Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. Contreras & Bazan Expires 6 May 2027 [Page 1] Internet-Draft Time measurement calibration November 2026 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Baseline measurement scenario . . . . . . . . . . . . . . . . 4 3. Network Element Calibration Tests . . . . . . . . . . . . . . 4 3.1. Single network element test . . . . . . . . . . . . . . . 5 3.2. Paired network elements test . . . . . . . . . . . . . . 6 3.3. Measurement conditions . . . . . . . . . . . . . . . . . 6 3.4. Resolution of the calibration process . . . . . . . . . . 7 3.5. Repeatability and Stability Assessment . . . . . . . . . 8 3.6. Calibration Under Operational Load Conditions . . . . . . 8 4. Further discussions . . . . . . . . . . . . . . . . . . . . . 9 4.1. Consideration of hybrid methods (e.g., IOAM) as measurement mechanisms for calibration assessment . . . . . . . . . . 9 4.2. Usage of instrumentation instead of physical fiber spools for determining the baseline value . . . . . . . . . . . 9 4.3. Cross-Method Validation . . . . . . . . . . . . . . . . . 10 5. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 5.1. Normative References . . . . . . . . . . . . . . . . . . 10 5.2. Informative References . . . . . . . . . . . . . . . . . 11 Appendix A. Measurement data normalization . . . . . . . . . . . 12 Appendix B. Calibration Considerations for Link Aggregation Groups (LAGs) . . . . . . . . . . . . . . . . . . . . . . . . . 13 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 13 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 1. Introduction Latency is becoming one of the major drivers for network traffic engineering and planning due to the introduction of novel services sensitive to both latency and its variability. Network devices incorporate capabilities for time-stamping certain packets so that such time stamps can reference the time at which a packet passes through a given device (at ingress or egress). Those stamps or marks are relevant to calculating the measured delay and can be used for traffic engineering purposes. That is the case of the Flex Algo algorithms [RFC9502] [RFC9351] aiming to represent low latency topologies being based on the time reference measured among Contreras & Bazan Expires 6 May 2027 [Page 2] Internet-Draft Time measurement calibration November 2026 devices. Furthermore, services sensitive to latency, such as the deterministic ones [RFC8557] or those following the network slicing paradigm [RFC9543], require from accurate information in order to place flows in the network. The measurements here considered are those performed between neighbor nodes in a network, so that an end-to-end delay metric can be composed by aggregating partial measurements. To ensure consistency and accuracy across different network element implementations, a benchmarking procedure is necessary to calibrate the timestamps. This allows correction of observed measurements and improves the reliability of latency-based traffic engineering decisions. Different protocols are implemented nowadays on network devices to perform such measurements, such as TWAMP-light [RFC5357] [RFC8545] or STAMP [RFC8972]. It is then important to ensure that the time stamp produced by the network device is accurate so that the traffic engineering decisions based on such measurements are correct. Differences in timestamp generation architecture may introduce systematic offsets and bias that affect measurement accuracy. Differences on accuracy of the measurements among devices can be due to different causes, such as different solutions for time stamping (e.g., hardware-based vs software-based), different module involved in the time stamping process (e.g., central processor vs line card), different type of component taking care of the timestamping (e.g., Network Processor, ASIC, hardware accelerator, etc), etc. The traffic engineering decisions will be typically implemented by centralized controllers, which can process the measurements and decide on the specific traffic engineering path. This can be the case of the Path Computing Element (PCE) [RFC5440]. It is important to note that any deviations or bias on the time stamps of the packets can lead to inaccurate calculations or decisions. An example is the following: Network Device A can introduce some delay (e.g., due to less powerful processing capabilities) in the timestamp versus Network Device B. In a dual-plane backbone, with similar characteristics in terms of physical path length, where one of the planes is built with network devices of type A while the other one is built with network devices of type B, the difference between the time stamps in A and B can lead to all the traffic being delivered by one of the planes. This is the case, for instance, in a dual-connect architecture enabling MPLS and SRv6 coexistence with Flex-Algo, operating over protected IP Backbone paths with identical physical path lengths. In this scenario hardware heterogeneity can introduce measurement asymmetries. Discrepancies in the timestamp processing capabilities implemented in the two types of network devices may occur. If a low-latency Flex-Algo topology is deployed Contreras & Bazan Expires 6 May 2027 [Page 3] Internet-Draft Time measurement calibration November 2026 and used for traffic engineering, this measurement offset can artificially skew the metric calculation, causing the path computation to steer the traffic exclusively over certain paths, thereby impacting the expected policy load distribution or path protection. The purpose of this document is to present a benchmarking procedure to calibrate the time stamps of different network device implementations so that the observed measured can be corrected if needed during the processing of the measured time-stamped data. 2. Baseline measurement scenario In order to create a reference or baseline to later on compare the measurements obtained from distinct network elements, a reference scenario is defined. This reference scenario will serve to calibrate the time stamps of the different network devices. The reference scenario assume an instrumentation device generating and receiving TWAMP-light or STAMP packets. Those packets are delivered on a fiber spool of a given known length, L. Typical lengths for testing purposes are in the order of 20 - 100 kms. It can be assumed a delay of 5 us/km on that fiber spool [ITU-T-G.114]. The measurement setup is as follows: +------------+ | | +------------| tester |<-------------+ | | | | | +------------+ | | | | | | Fiber spool of length L | +----------------------------------------+ Figure 1: Baseline measurement scenario The tester will generate the baseline measurement for the time spent on delivering packets through the link represented by the fiber spool. Such a reference value can be defined as T_baseline and will serve for comparison of the measurements performed by the distinct implementations of network elements for calibration purposes. 3. Network Element Calibration Tests As in the baseline scenario described previously, equivalent calibration procedures can be performed using actual network elements. The following test scenarios are considered. Contreras & Bazan Expires 6 May 2027 [Page 4] Internet-Draft Time measurement calibration November 2026 * Single network element test, performing the test with just one single network element, similar to the baseline scenario. This kind of test can be useful for characterizing the behavior of one specific network device implementation, including both its hardware and software components. * Paired network element test, performing the test between two network elements. This can be useful for determining interworking scenarios, considering either network elements from the same vendor but with different characteristics (e.g., network device model, hardware or software characteristics, etc), or network elements from different vendors. 3.1. Single network element test The scenario is similar to the baseline scenario, as follows. +------------+ | Network | +------------| Element |<-------------+ | | A | | | +------------+ | | | | | | Fiber spool of length L | +----------------------------------------+ Figure 2: Single network element test As before, the network element generates measurement packets through the link represented by the fiber spool. The obtained value can be referred as T_ne. The calibration exercise results from the comparison between T_baseline and T_ne. Thts is, the difference between T_ne and T_baseline provides an indication of the measurement accuracy and potential timestamping bias associated with the network element under test. The characteristics of both ports and line cards used to terminate the fiber spool may influence the observed measurements. Different hardware implementations can introduce additional latency or timestamping variations. Therefore, calibration tests should document the specific interface and line card used during the measurement process. Contreras & Bazan Expires 6 May 2027 [Page 5] Internet-Draft Time measurement calibration November 2026 3.2. Paired network elements test This scenario considers the measurement to be performed between a pair of network elements, as follows. +------------+ +------------+ | Network | Fiber spool of length L | Network | | Element |<----------------------->| Element | | A | | B | +------------+ +------------+ Figure 3: Paired network elements test The network elements may belong to the same or different vendor. The purpose of this test is to calibrate in interoperability scenarios where the the devices involved represent different implementations of the same network device functionality, including the protocols used for time stamping. In this case, one of the network elements generates measurement packets over the link represented by the fiber spool of known length (and previously measured in the reference scenario). The value obtained from the measurement packets from Network Element A to Network Element B can be referred as T_ne_ab. Similarly, it is possible to obtain T_ne_ba on the other direction. The calibration procedure consists of comparing the measured values against the baseline reference value T_baseline. Assuming that both T_ne_ab and T_ne_ba produce the same result, the calibration exercise results from the comparison between T_baseline and T_ne_ab (or T_ne_ba). Whatever observed deviation will provide an indication of the combined effect of the measurement accuracy and timestamping characteristics of the network elements participating in the test. On the other hand, differences observed between T_ne_ab and T_ne_ba may indicate asymmetric forwarding behavior, differences in timestamp generation mechanisms, synchronization inaccuracies, or implementation-specific characteristics affecting the measurement process. 3.3. Measurement conditions The measurements defined in this document should specify the following information: Contreras & Bazan Expires 6 May 2027 [Page 6] Internet-Draft Time measurement calibration November 2026 * Network device model, including a description of the hardware and software version used during the test. * Length of the fiber spool used as calibration reference for the measurement. * Description of the ports and line card where the fiber spool is connected, since different line cards could produce measurements with different accuracy levels. * Protocol used in the measurement (e.g., TWAMP-light, STAMP, ...). * Clock source (e.g., GNSS, PTP, SyncE, NTP, holdover, etc). * Duration of the test for statistical consistency (e.g., period for average, min, max calculations) * Network tester or instrumentation used for generating the baseline values. In addition, if available, the exact timestamp insertion point is of interest since this information is relevant for interpreting differences observed implementations and hardware architectures. For instance, insertion points such as Network Processing Unit (NPU), forwarding ASIC, line card CPU, route processor, etc. (Note: to consider scalability aspects on the collected measurements that could impact the calibration exercise. For instance, if multiple sessions are running in parallel from one node to several neighbor nodes, with one session per neighbor). 3.4. Resolution of the calibration process The calibration process of the time-stamps observed during the benchmarking methodology needs to take into account the time units of relevance for the purpose of the usage of such measurements. The timestamp format used in TWAMP, TWAMP Light, and STAMP protocols is based on the NTP format, as defined in [RFC8877]. These protocols employ a 64-bit timestamp, where: * The first 32 bits represent the number of seconds since the NTP epoch (January 1, 1900) * The remaining 32 bits represent fractional seconds. This structure provides a theoretical resolution of approximately 233 picoseconds. Contreras & Bazan Expires 6 May 2027 [Page 7] Internet-Draft Time measurement calibration November 2026 For instance, with this in mind, if the purpose is the generation of FlexAlgo topologies based on delay, it should be taken into account that the delay field advertised when using IS-IS [RFC8570] or OSPF [RFC7471] is expressed in microseconds. Thus, differences below a microsecond are not relevant. OSPF or IS-IS with traffic engineering extensions encode the "delay" value in microseconds using 24-bit fields. This naturally results in lower precision compared to the timestamp format used in active measurement protocols. When calibrating measurements for use in network operations or control plane mechanisms, it is crucial to consider the specific use case of the timestamp to ensure appropriate granularity and accuracy. For example, in the case of constructing low-latency FlexAlgo topologies, it is necessary to align the measurement resolution to microseconds, as this is the resolution that the IGP protocol can propagate towards the control plane generating the low-latency topology. To achieve this, the measured delay collected by means of TWAMP, TWAMP Light, and STAMP protocols SHOULD be adjusted using a ceiling function, i.e., rounding up to the nearest integer value in microseconds. Other use cases, however, may require different resolution levels, rounding techniques, or adjustments depending on operational goals. 3.5. Repeatability and Stability Assessment Repeatability is relevant in order to provide a more complete characterization of the calibration process. Accuracy obtained from a single execution does not guarantee the temporal consistency of the measurementa. The calibration process MUST include repeatability tests intended to evaluate the stability of the measurement results over time. This is related to the number of independent measurement runs and the number of samples per measurement run. 3.6. Calibration Under Operational Load Conditions Results obtained under idle or low-load conditions may not accurately represent the behavior of a network element operating in production environments. Factors such as increased CPU utilization, buffer occupancy, forwarding pipeline congestion, and telemetry processing overhead can affect timestamp generation and measurement accuracy. Therefore, the calibration process SHOULD be repeated under different levels of operational load in order to evaluate the stability and accuracy of the measurement system under realistic operating conditions. The following load levels are RECOMMENDED: Contreras & Bazan Expires 6 May 2027 [Page 8] Internet-Draft Time measurement calibration November 2026 +------------+----------------------+ | Condition | Resource Utilization | +------------+----------------------+ | Idle | 0% | | Low | 25% | | Medium | 50% | | High | 75% | | Saturation | 95% | +------------+----------------------+ The characteristics of the background traffic used to generate the load on the link SHOULD be documented as part of the calibration report. 4. Further discussions This is a placeholder for capturing topics that will be considered for next versions of the document. 4.1. Consideration of hybrid methods (e.g., IOAM) as measurement mechanisms for calibration assessment On-path hybrid methods, such as In situ Operations, Administration, and Maintenance (IOAM) [RFC9197] [RFC9326] allow the obtention of telemetry information that can be used for measuring delays in network links. The IOAM data can be embedded in different encapsulations. While these methods coud be used for collecting the metric of interest, in the context of calibrating hop-by-hop measurements, it would impose practical limitations. For instance, the need of generating individual flows per hop for the solely purpose of measuring the link delay. Furthermore, it would require the specification of the measurement conditions in terms of the flows used for embedding the telemetry data. 4.2. Usage of instrumentation instead of physical fiber spools for determining the baseline value Instrumentation equipment can introduce impairments in a link in a controlled manner, either as a deterministic or statistical manner. For the purpose of the calibration pursued in this document, the usage of instrumentation equipment can be considered instead of physical fiber spools by configuring deterministic delay values. While this can be a proper approach, it presents some practical shortcomings. For instance, the need of supporting all the variety of interface bit rates of interest in the platforms to be calibrated. Contreras & Bazan Expires 6 May 2027 [Page 9] Internet-Draft Time measurement calibration November 2026 4.3. Cross-Method Validation Multiple measurement solutions may coexist within the same operational environment. Differences in measurement collection, processing methodologies, timestamp generation mechanisms per solution, and data aggregation techniques may result in systematic deviations between different measurement methods. The objective of this analysis is to determine the level of convergence between different measurement methodologies and to identify potential systematic deviations among implementations of the different measurement solutions. 5. References 5.1. Normative References [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", RFC 5357, DOI 10.17487/RFC5357, October 2008, . [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation Element (PCE) Communication Protocol (PCEP)", RFC 5440, DOI 10.17487/RFC5440, March 2009, . [RFC7471] Giacalone, S., Ward, D., Drake, J., Atlas, A., and S. Previdi, "OSPF Traffic Engineering (TE) Metric Extensions", RFC 7471, DOI 10.17487/RFC7471, March 2015, . [RFC8545] Morton, A., Ed. and G. Mirsky, Ed., "Well-Known Port Assignments for the One-Way Active Measurement Protocol (OWAMP) and the Two-Way Active Measurement Protocol (TWAMP)", RFC 8545, DOI 10.17487/RFC8545, March 2019, . [RFC8570] Ginsberg, L., Ed., Previdi, S., Ed., Giacalone, S., Ward, D., Drake, J., and Q. Wu, "IS-IS Traffic Engineering (TE) Metric Extensions", RFC 8570, DOI 10.17487/RFC8570, March 2019, . [RFC8972] Mirsky, G., Min, X., Nydell, H., Foote, R., Masputra, A., and E. Ruffini, "Simple Two-Way Active Measurement Protocol Optional Extensions", RFC 8972, DOI 10.17487/RFC8972, January 2021, . Contreras & Bazan Expires 6 May 2027 [Page 10] Internet-Draft Time measurement calibration November 2026 [RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi, Ed., "Data Fields for In Situ Operations, Administration, and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197, May 2022, . [RFC9326] Song, H., Gafni, B., Brockners, F., Bhandari, S., and T. Mizrahi, "In Situ Operations, Administration, and Maintenance (IOAM) Direct Exporting", RFC 9326, DOI 10.17487/RFC9326, November 2022, . [RFC9351] Talaulikar, K., Ed., Psenak, P., Zandi, S., and G. Dawra, "Border Gateway Protocol - Link State (BGP-LS) Extensions for Flexible Algorithm Advertisement", RFC 9351, DOI 10.17487/RFC9351, February 2023, . [RFC9502] Britto, W., Hegde, S., Kaneriya, P., Shetty, R., Bonica, R., and P. Psenak, "IGP Flexible Algorithm in IP Networks", RFC 9502, DOI 10.17487/RFC9502, November 2023, . [RFC9534] Li, Z., Zhou, T., Guo, J., Mirsky, G., and R. Gandhi, "Simple Two-Way Active Measurement Protocol Extensions for Performance Measurement on a Link Aggregation Group", RFC 9534, DOI 10.17487/RFC9534, January 2024, . 5.2. Informative References [ITU-T-G.114] "One-way transmission time", May 2003. [RFC8557] Finn, N. and P. Thubert, "Deterministic Networking Problem Statement", RFC 8557, DOI 10.17487/RFC8557, May 2019, . [RFC8877] Mizrahi, T., Fabini, J., and A. Morton, "Guidelines for Defining Packet Timestamps", RFC 8877, DOI 10.17487/RFC8877, September 2020, . [RFC9543] Farrel, A., Ed., Drake, J., Ed., Rokui, R., Homma, S., Makhijani, K., Contreras, L., and J. Tantsura, "A Framework for Network Slices in Networks Built from IETF Technologies", RFC 9543, DOI 10.17487/RFC9543, March 2024, . Contreras & Bazan Expires 6 May 2027 [Page 11] Internet-Draft Time measurement calibration November 2026 Appendix A. Measurement data normalization As part of the comparison of the measurements produced by different implementations, a data normalization process could be required. The goal is to ensure that functionally equivalent metrics can be compared consistently in multivendor scenarios. The reason for it is that network elements from different vendors can use distinct metric names, units of measurement, temporal granularities, timestamp formats, and statistical methodologies to represent equivalent performance indicators. Such normalization process can imply the mapping of equivalent metrics, the conversion of reported units, the temporal granularity considered, the aligment of the statistical window used and the normalization of the timestamp format. The objective of metric normalization is to ensure that equivalent measurements collected from heterogeneous implementations are interpreted and processed in a consistent manner. An example of metric normalization is shown below: +----------+------------------+----------------+ | Vendor | Native Metric | Normalized KPI | +----------+------------------+----------------+ | Nokia | round-trip-delay | RTT | | Cisco | network-delay | RTT | | HPE | rtt-average | RTT | | Huawei | delay-average | RTT | +----------+------------------+----------------+ Once calibrated, normalized measurement data may be subsequently exported to observability platforms, analytics systems, Software- Defined Networking (SDN) controllers, automation frameworks, and data lake environments. This procedure can lead to accurate comparisons, consistent performance assessments, and correct Traffic Engineering decisions in multivendor operational environments. Contreras & Bazan Expires 6 May 2027 [Page 12] Internet-Draft Time measurement calibration November 2026 Appendix B. Calibration Considerations for Link Aggregation Groups (LAGs) Link Aggregation Groups (LAGs) are widely deployed in operational networks to increase available bandwidth and provide resiliency through the use of multiple physical links presented as a single logical interface. Although LAG member links are treated by the control plane as a single logical resource, individual member links may exhibit different performance characteristics, including variations in delay, packet loss, delay variation, and utilization. Performance differences among member links of the same LAG may occur due to different reasons, like different physical link lengths, intermittent failures affecting individual member links, etc. As consequence of that, measurements performed exclusively on the aggregated LAG may hide degradations affecting individual member links. [RFC9534] defines extensions to the Simple Two-Way Active Measurement Protocol (STAMP) that enable performance measurements on individual LAG member links through the use of micro sessions. These extensions allow measurement sessions to be associated with specific physical member links, enabling independent assessment of the performance of each member within the aggregation group. When the network element under test utilizes LAGs, the calibration process SHOULD evaluate each member link individually. The purpose of this is to identify performance differences among individual member links that may not be observable when measurements are performed exclusively on the aggregated logical interface. Measurement results SHOULD be reported individually for each member link prior to the calculation or presentation of aggregated LAG statistics. The calibration report SHOULD identify the member link associated with each measurement and document any observed differences among members. Acknowledgements The author would like to thank Miguel Cros (Telefonica) for the discusions on the topic. Moreover the author would like also thank Giuseppe Fioccola, Tal Mizrahi, Carsten Rossenhoevel for the valuable comments received during the BMWG meetings. This work has been partially funded by the European Commission through the Horizon Europe SNS JU PREDICT-6G project (GA 101095890). Authors' Addresses Contreras & Bazan Expires 6 May 2027 [Page 13] Internet-Draft Time measurement calibration November 2026 Luis M. Contreras Telefonica Ronda de la Comunicacion, s/n 28050 Madrid Spain Email: luismiguel.contrerasmurillo@telefonica.com Adilson Bazan Telefonica Sao Paulo- Brazil Email: adilson.fernandes@telefonica.com Contreras & Bazan Expires 6 May 2027 [Page 14]