]> Cisco

Canada fluffy@iii.ca

Cisco

USA snandaku@cisco.com

art metrics moq Many systems produce significant volumes of metrics which either are not all needed at the same time for consumption by collection/aggregation endpoints or may also compete for bandwidth with the primary application, thus exacerbating congestion conditions especially in low-bandwidth networks. Delivering these over architectures enabled by publish/subscribe transport like Media over QUIC Transport (MOQT) , allows metrics data to be prioritized within the congestion context of the primary application as well as enabling local nodes to cache the metric value to be later retrieved via new subscriptions. This document specifies how to send metrics type information over MOQT.

Introduction Systems often share the network bandwidth used for metrics with realtime media, which impacts the media quality. Metrics compete with media for bandwidth, reducing the peak bandwidth available to the primary application and often causing congestion in low-bandwidth networks. There is a desire to transport metrics at an appropriate priority level over the same transport as the application media. This allows metrics to take advantage of times when the media bitrate is below the peak rate, without impacting the peak rate available for media. Publishing metrics over MOQT also allows MOQT relays to act as caches that store metrics close to where they are produced. Instead of sending every metric to collectors, metrics can be cached in relays and retrieved only when needed, enabling "just in time" delivery of metrics (see ).

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Architecture Overview This section describes two deployment models for metrics over MOQT. In both, resources publish metrics as MOQT tracks (see ) to a relay, and metrics collectors subscribe to them via relays. Relays cache metrics so that collectors can retrieve them later.

Metrics Sharing a Session with Media In this model, an endpoint such as a realtime conferencing client publishes its media and its metrics over the same MOQT session, and thus the same QUIC connection, to its ingress relay. Media and metrics share the congestion control context of that connection, and MOQT priorities (see ) determine what is sent when the available bandwidth is limited. At the relay, media and metrics are forwarded to different destinations: media to the other participants, and metrics to a metrics collector, possibly through other relays. The relay does not need to treat the two differently beyond the track names and properties. | Participants | | +--------+ |==================>| | +--------------+ | | Media | | | Relay | | +--------+ | metrics | | metrics +--------------+ | |Metrics | |==================>| (cache) |-------->| Metrics | | +--------+ | (pri 128..240) | | | Collector | +------------+ +---------+ +--------------+ ]]> When congestion occurs on the endpoint's connection, objects of the higher granularity metrics levels are the first to be delayed and, with OBJECT_DELIVERY_TIMEOUT, dropped, while media is unaffected. The relay continues to cache what it has received, and a collector can retrieve the metrics for a time window of interest later using FETCH (see ).

Metrics as the Primary Data In this model, the endpoints, such as servers, network devices or sensors, send metrics as their main data over MOQT. Each endpoint publishes its metrics tracks to a relay, which may aggregate many endpoints, and one or more metrics collectors subscribe to them. | Relay |------>| Relay |---->| Metrics | +------------+ | (cache) | | (cache) | | Collector | +---------+ +---------+ +-----------+ +------------+ ^ | | Endpoint N |--metrics--+ | +-----------+ +------------+ +--------->| Dashboard | +-----------+ ]]> Collectors discover endpoints and subscribe to the levels of interest (see ), while the remaining levels stay in relay caches until needed (see ). Multiple consumers, such as a collector and a dashboard, can subscribe to the same tracks through relays without the endpoint sending the data more than once.

Metrics Data Model Below picture captures relationship between the entities in the data model. | Value: String | | | +--------------------+ | +------------------+ | +--->|Attributes [0..n] | -----+ | +------------------+ +--------------------+ | +----->|Metric Name: String | | | +--------------------+ | +------------------+ | +--> | Metric |------+ +--------------------------+ +------------------+ | | TimeSeries 1..n | +--->| [{time, value}, ...] | +--------------------------+ ]]> The devices or systems publishing the metrics are referred to as "Resources" and have an unique "ResourceID". Each metric reported by a Resource has an associated timeseries. A metric's timeseries is time ordered set of values (observations at points in time) for the corresponding metric. Metric names are represented as strings and may contain ASCII letters, digits, underscores, and colon. A metric's value can be one of "Gauge" or "Counter". The "Gauge" value type represents a scalar value that always represents the current value being measured. The "Counter" value type is a cumulative value that represents a single monotonically increasing value that can increase or be reset to zero on restart. The Gauge and Counter can represent either a 64-bit floating point value or a 64-bit integer value. Resources also define zero or more Attributes. Attributes capture the dimensional data to identify any given combination of them for the metrics reported by the resource. Attributes are represented as key-value pairs and represented as strings. The data model specified in this section is consistent with the OpenTelemetry metrics data model when at rest (also known as Timeseries model). See for streaming model where MOQT is used for transporting the metric data.

ResourceID Each resource that creates metrics has a unique "ResourceID". This is created by taking the MAC address of the primary network interface in binary, computing the SHA-1 hash of it, and truncating to lower 64 bits. Note that SHA-1 does not provide any security properties here; it is just a hash that is widely implemented in hardware. If this is not possible, any other random stable 64-bit identifier MAY be used. The advantage of using the MAC address is that many other management systems use this address and it makes it easier to correlate. The disadvantage is that it reveals the MAC address. When carried in MOQT names, the ResourceID is encoded as exactly 16 lowercase hexadecimal ASCII characters.

Metrics and MOQT This section maps the metrics data model defined in this specification to the MOQT Object Data Model (). The core idea is that the names of the MOQT tracks are set up such that a subscriber can discover and subscribe to each resource reporting metrics separately, pick the right granularity level, and select individual metrics using MOQT filters, while relays cache and serve past metric values.

Track Naming Each resource publishes its metrics under the Track Namespace consisting of the following three Track Namespace Fields, each encoded as ASCII: ) ]]> The Track Namespace Prefix ("metrics.moq.arpa", "v1") is common to all resources, which allows a collector to discover all resources (see ). Within a resource's namespace the following tracks are defined:

• One data track per granularity level. The Track Name is a single ASCII digit identifying the level (see ).
• One catalog track with the Track Name catalog (see ).

Using the serialization format in , the Warning level track of the resource with ResourceID 8f2c4a1b0e9d7c63 is rendered as:

Granularity Levels Following granularity levels are defined in this specification, along with their associated track names and default publisher priorities (see ). Higher levels carry more detailed metrics: in this document, "lower levels" refers to Emergency (0) towards Error (3), and "higher levels" to Warning (4) towards Debug (7).

Level	Track Name	Default Publisher Priority
Emergency	0	128
Alert	1	144
Critical	2	160
Error	3	176
Warning	4	192
Notice	5	208
Informational	6	224
Debug	7	240

Mapping of metrics and its reporting frequency at the aforementioned levels by a resource is application defined. However, it can be typical of an implementation to report metrics that represent aggregation of values over larger time intervals or that represent erroneous conditions, to use granularity that is one of Emergency, Alert, Critical and Error. Such metrics have characteristics of being less frequent and hence consume lesser bandwidth. Metrics that are captured more frequently and capture detailed view of system being measured typically are reported at the Warning through Debug levels. Such metrics, however, consume high bandwidth when published.

Catalog Track The catalog track describes the resource and the metrics it publishes. Each catalog group is one version of the catalog, and its objects mirror the objects of data groups:

• Object ID 0 carries the catalog header.
• Object ID N, for N > 0, carries the Metric Description of the metric with Metric Index N.

The Metric Index of a metric is the Object ID of its Metric Description in the catalog group, and is also the Object ID that carries the metric's value in data groups (). It is not encoded in the Metric Description itself. Metric Indices are unique across all data tracks of the resource within a catalog version. A subscriber uses the Metric Description with index N to interpret Object ID N of any data group that refers to that catalog version. The catalog header and metric description are encoded as follows, using the notation of and the variable-length integer (vi64) of .

Catalog Header (Object ID 0)

• Version: The catalog format version. This document defines version 1.
• Attribute: A resource attribute, where Key and Value are UTF-8 strings.

Metric Description (Object ID N > 0)

• Metric Type: 0x0 for Gauge, 0x1 for Counter (see ).
• Value Representation: The encoding of the metric value in data objects (see ): 0x0 for an unsigned integer, 0x1 for a signed integer, 0x2 for a floating point number.
• Name: The metric name, in ASCII (see ).
• Unit: The unit of the value following conventions, or empty if not specified.

Receivers MUST ignore any bytes following the fields defined in this document, to allow future extension. The Group ID of a catalog group is the catalog version number. The first version has Group ID 1, and each new version increments it by one, so the reference carried in every data group (see ) stays small. The publisher MUST publish a catalog group before any data group that refers to it. Metric Indices are scoped to a catalog version: a new catalog version MAY assign indices differently. Publishers SHOULD keep indices stable across versions, which keeps Object ID filters valid (see ) and allows unchanged metric descriptions to be reused. Because each metric is described by its own object, a subscriber does not need the complete catalog before decoding data. It can decode the value in data Object ID N as soon as it has catalog Object ID N of the referenced version, and it can retrieve only the descriptions it needs by applying the same Object ID filter to the catalog track as to the data track. A subscriber SHOULD subscribe to the catalog track in parallel with the data tracks, using FILL_PARAMETERS selecting the current group (), and hold data objects until the referenced catalog objects are received. Catalog versions rarely change, so subscribers MAY cache them per Track Namespace and catalog version and use a cached version without waiting. A subscriber that receives a data group referring to a catalog version it does not have retrieves it with FETCH. The catalog track MUST use subgroup delivery and SHOULD carry the following Track Properties:

• MAX_CACHE_DURATION no shorter than that of any data track of the resource, so that cached data remains interpretable.
• DEFAULT_PUBLISHER_PRIORITY numerically lower than that of any data track of the resource, so that catalog objects arrive ahead of the data that depends on them. This can place the catalog ahead of media using the default priority, which is acceptable given its small size and infrequent changes.
• No OBJECT_DELIVERY_TIMEOUT.

Groups and Objects Each group in a data track holds the set of metrics captured by the resource at one point in time. The Group ID is the capture time in milliseconds since the UNIX Epoch (00:00:00 UTC on 1 January 1970). This satisfies the requirement that Group IDs increase with time () and allows subscribers to express time ranges directly as Location filters (see ). Each group is independently useful: decoding a group requires only the objects in that group and the catalog version it refers to. Objects within a group are:

• Object ID 0 is REQUIRED and carries the Capture Info.
• Object ID N, for N > 0, carries the Metric Value of the metric with Metric Index N in the referenced catalog version, observed at the capture time.

Capture Info (Object ID 0)

• Capture Timestamp Offset: The nanoseconds to add to the Group ID, in milliseconds, to obtain the precise capture time. The value is less than 1000000.
• Catalog Version: The Group ID of the catalog version used by this group (see ).

The Metric Value is the entire object payload, encoded according to the Value Representation of the metric:

• Unsigned integer: a vi64.
• Signed integer: a 64-bit two's complement integer in network byte order.
• Floating point: an IEEE 754 binary64 value in network byte order.

A resource that does not report a given metric at a capture time simply omits that object. A subscriber or relay cannot infer anything from a missing Object ID (). The publisher MUST therefore include the Prior Object ID Gap Object Property () on the object that follows any skipped Object IDs, signaling that they do not exist in the group. The publisher MUST also end each group with an object with the End of Group Object Status () and an Object ID one greater than the largest Object ID carrying a metric value, so that the receiver knows that no further metrics exist in the group, including skipped metrics after the last value. Object Status is not carried in FETCH responses, which convey non-existent objects as described in . Resources typically do not capture metrics every millisecond, so Group IDs in a data track are sparse. The publisher SHOULD include the Prior Group ID Gap Object Property () on Object 0, set to the number of milliseconds since the previous capture minus one. This allows relays and fetches to know that the intervening groups will never exist, rather than leaving them in the unknown state. All objects of a group MUST be sent in a single subgroup with Subgroup ID 0, so that Object 0 is delivered ahead of the values that depend on it. The Datagram delivery mode MUST NOT be used. An example of a group is shown in .

Track Properties Publishers SHOULD include the following Track Properties () on each data track:

• DEFAULT_PUBLISHER_PRIORITY: see .
• DEFAULT_PUBLISHER_GROUP_ORDER: Ascending (0x1), so that collectors receive samples in capture order. Subscribers interested only in the freshest value, such as live dashboards, MAY request Descending group order.
• MAX_CACHE_DURATION: the retention time for which relays may serve cached samples. Lower granularity levels are typically retained longer than higher granularity levels.
• OBJECT_DELIVERY_TIMEOUT: for high granularity levels, a timeout after which a stale sample is dropped rather than delivered late.

Priorities Metrics are expected to yield to the primary application under congestion. The publisher SHOULD set the DEFAULT_PUBLISHER_PRIORITY Track Property of each data track to the value in , or another value that keeps lower granularity levels ahead of higher granularity levels. These values are in the lower priority half of the range, behind the MOQT default of 128 typically used for media, with Emergency at par with the default. Subscribers can adjust the relative importance of metrics against other subscriptions in the session using the SUBSCRIBER_PRIORITY parameter, which is considered before publisher priority.

Discovery and Subscription A collector discovers resources by sending SUBSCRIBE_NAMESPACE with the Track Namespace Prefix ("metrics.moq.arpa", "v1"). The relay responds with a NAMESPACE message for each resource namespace, and NAMESPACE_DONE when a resource stops publishing. For each discovered resource, the collector subscribes by name to the catalog track and to the data tracks of the levels it wants to receive continuously, for example levels 0 to 3. Resources that start publishing later are learned from subsequent NAMESPACE messages. Tracks of other levels are not subscribed to and remain available from the relay cache via FETCH (see ).

Filtering MOQT Range Filters () carried in SUBSCRIBE, FETCH or FILL_PARAMETERS allow a collector to further reduce the data delivered within a track:

• An Object ID range filter selects specific metrics by Metric Index. The filter MUST include Object ID 0, which identifies the catalog version. For example, ranges [0, 0] and [3, 5] deliver Object 0 and the metrics with indices 3 to 5. The same filter applied to the catalog track delivers only the catalog header and the descriptions of those metrics. A subscriber updates its filters when a new catalog version assigns indices differently.
• An Object Property range filter selects individual samples based on application-defined Object Properties. For example, an application can mark samples that cross a configured threshold with an Object Property from the application-specific range, so that a collector subscribed to a high granularity level receives only those samples. Such properties are outside the scope of this document.

Range Filters are only available when the relay advertises a non-zero MAX_FILTER_RANGES. Collectors therefore treat filtering as an optimization and MUST be able to process unfiltered tracks.

Retrieval of Past Metrics Since Group IDs are capture times in milliseconds, a time range of metrics corresponds directly to a range of Locations:

• A collector retrieves past samples for a time window [T1, T2] by sending FETCH with a Location filter from {T1, 0} to {T2, max}. The request is served from the relay cache, subject to MAX_CACHE_DURATION.
• A collector that wants recent history followed by live samples sends a SUBSCRIBE with a Location filter starting at the Next Object, together with FILL_PARAMETERS whose Location filter selects the starting point of the history. The relay delivers the history on a fill fetch stream and live samples on the subscription, without gaps or duplicates ().
• A collector that only needs live samples subscribes with a Location filter starting at the Next Group.

This enables the "just in time" collection model: resources publish all levels to a nearby relay, while collectors subscribe to the low granularity levels and fetch the high granularity levels for the time windows of interest, for example around an alert.

Examples This example follows a resource that reports two metrics on its Warning level track: cpu_usage_percentage and cpu_temperature.

Catalog The resource publishes catalog version 1 as Group 1 of its catalog track.

Catalog track example

Data The resource captures both metrics at 15:59:51.000123456 UTC on 7 July 2024 (Group 1720367991000). One second later it captures only cpu_temperature (Group 1720367992000).

Data track example

Result A collector holding catalog version 1 turns the two groups into the following time series (), with capture times in UTC on 7 July 2024: Decoded time series

Capture time	Metric	Value	Unit
15:59:51.000123456	cpu_usage_percentage	99	%
15:59:51.000123456	cpu_temperature	45.1	Cel
15:59:52.000004211	cpu_temperature	44.8	Cel

Use with MSF An MOQT Streaming Format (MSF) catalog can direct a client to publish metrics using a publishTracks entry with a packaging of "moqmetrics" and a role of "metrics". In such an entry:

• namespace is the Track Namespace of rendered using , with %resourceId% as a placeholder for the client's ResourceID: metrics.2emoq.2earpa-v1-%resourceId%.
• name is the data track for a granularity level (). A catalog can include one entry per level it wants the client to publish.

A client publishing any data track in this way also publishes the catalog track () in the same namespace. The connectionUri and token of the entry apply to the catalog track as well.

IANA Considerations This document has no IANA actions.

Security Considerations TODO

References Normative References Cisco Google Google Meta This document defines Media over QUIC Transport (MOQT), a publish/ subscribe protocol that runs over QUIC and WebTransport. MOQT leverages the features of these transports, such as streams, datagrams, priorities, and partial reliability. MOQT operates both point-to-point and through intermediate relays, enabling scalable low-latency delivery. Despite its name, MOQT is media agnostic and can be used for a wide range of use cases. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. Informative References OpenTelemetry n.d. Akamai Cisco This document specifies the MOQT Streaming Format, designed to operate on Media Over QUIC Transport.

Changes from the ietf122 Version

• Aligned with draft-ietf-moq-transport-22.
• Added an Architecture Overview describing metrics sharing a session with media, and metrics as the primary data.
• Track Namespace is now the tuple ("metrics.moq.arpa", "v1", resourceID); the string form uses the MOQT serialization format instead of a moq:// URL.
• Group ID is the capture time in milliseconds since the UNIX Epoch as a full 64-bit value.
• Attributes and metric names moved to a versioned catalog track with one object per metric. Each data group refers to the catalog version that maps its Object IDs to metrics, which removes the dependency on previous groups.
• Object payloads use a binary encoding following the MOQT notation.
• Each group is sent as a single subgroup, ending with an End of Group object, and skipped metrics are signaled with Prior Object ID Gap.
• Added the use of Prior Group ID Gap, Track Properties for priority, group order, caching and delivery timeouts, SUBSCRIBE_NAMESPACE for discovery, range filters for metric selection, and FETCH and fill for time range retrieval.
• Added use with MSF.

Acknowledgments Thanks to TODO for contributions and suggestions to this specification.