| Internet-Draft | mix | November 2026 |
| Nandakumar & Jennings | Expires 6 May 2027 | [Page] |
This document specifies MoQ Interconnect Exchange (MIX), a protocol that allows Media Over QUIC (MoQ) relays to discover each other, exchange reachability and subscription information, and forward MoQ objects across administrative and topological boundaries. MIX carries its control state as MoQ tracks.¶
This work is outside the scope of the IETF Media Over QUIC (MoQ) working group. It is discussed on that working group's mailing list, moq@ietf.org, because that is where the people interested in MoQ relay interconnection are.¶
This note is to be removed before publishing as an RFC.¶
The latest revision of this draft can be found at https://suhasHere.github.io/moq-relay-interconnect/draft-nandakumar-moq-mix.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-nandakumar-moq-mix/.¶
Discussion of this document takes place on the Media Over QUIC Working Group mailing list (mailto:moq@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/moq/. Subscribe at https://www.ietf.org/mailman/listinfo/moq/.¶
Source for this draft and an issue tracker can be found at https://github.com/suhasHere/moq-relay-interconnect.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 6 May 2027.¶
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Media Over QUIC Transport [MoQTransport] (MOQT) defines how endpoints publish and subscribe to media tracks through relays. MOQT itself is scoped to a single relay: it does not specify how relays exchange the reachability and subscription information they need to forward objects between their respective subscriber populations.¶
Related work ([MOQ-CLUSTER]) addresses relay coordination within a single cluster or CDN but does not define cross-domain discovery, peering negotiation, or policy-governed fan-out.¶
This document specifies MoQ Interconnect Exchange (MIX), which closes that gap by reusing MOQT for the inter-relay control plane and defining a data plane that operates both within a single administrative domain and across administrative boundaries.¶
MIX uses MOQT facilities where their existing semantics fit the control plane, so the request, fan-out, scheduling, and object-retrieval behavior of the control plane comes directly from MOQT. MIX does not assume that every MOQT relay caches objects or that MOQT itself supplies relay-to-relay routing.¶
Namespace discovery and separate tracks. Relays publish MIX
namespaces with PUBLISH_NAMESPACE and discover peers via
SUBSCRIBE_NAMESPACE / NAMESPACE / NAMESPACE_DONE. nodes, publishers,
and subscribers are independent tracks so a peer can subscribe to only the
state it needs and MIX can apply distinct retention, priority, and
admission policy to each.¶
Live updates plus state repair. SUBSCRIBE carries live updates.
Draft 22's FILL_PARAMETERS mechanism, when backed by a cache,
recovers retained objects after a reconnect or missed update. Because
MOQT caching is optional, MIX needs its own recovery for the case
where a fill is incomplete (Section 6.2).¶
Priority scheduling and delivery deadlines. MOQT subgroup streams
and datagrams, publisher priorities, and OBJECT_DELIVERY_TIMEOUT /
SUBGROUP_DELIVERY_TIMEOUT let MIX prioritize routing-critical state
and bound how long an advertisement consumes delivery effort.
Priorities influence scheduling only, and deadlines may drop state
(Section 6.2).¶
Direct use of MOQT controls. MIX uses these MOQT controls as defined, without re-encoding them in MIX messages. MIX-specific messages carry only what MOQT does not represent: originator identity, freshness, path, and target-set information.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The following terms are used throughout this document:¶
A MIX-capable MOQT relay. Each Node is identified by a 128-bit Node ID, described in Section 3.¶
A Node that connects MOQT application publishers or subscribers to the inter-relay network. It originates publish and subscribe state for its local clients and injects or delivers their objects through MIX.¶
A Node that carries relay reachability and forwarding state and forwards objects along selected inter-relay paths. It typically does not terminate application publisher or subscriber sessions.¶
A Node that exchanges Publisher and Subscriber Table state on behalf of other Nodes and
does not forward objects. Control Nodes are used in the split
deployment model (Section 12.2) and advertise type = 2 in
node_info (Section 8.3.1).¶
A MIX Edge Node or MIX Transit Node; any Node that is not a Control Node.¶
A QUIC connection between two Nodes used to exchange MIX control-plane
tracks, including announce, nodes, publishers, and subscribers (Section 5).
This connection also carries the MOQT control messages needed to
establish and manage its subscriptions and publications.¶
A separate QUIC connection between two Nodes used to forward
inter-relay data (Section 7). It also uses MOQT control messages
to establish the session and its delivery-node-sets track subscription. Those MOQT messages
manage the transport session; they do not make it a MIX control
Session.¶
A MIX relationship between two Nodes. It consists of a Control
Session carrying the Node, Publisher, and Subscriber Table tracks in
the peering's scope
(Section 6.1) and, when that scope includes the Node Table and data is
to be forwarded, a separate Data Session. Each
connection has its own MOQT session state. An delivery-node-sets track is associated
with the Data Session whose downstream targets it describes.¶
In this document, an unqualified Session means a Control Session. When the Data Session is intended, it is named explicitly.¶
The Nodes this Node knows about, one entry per Node ID. Each entry
holds that Node's attributes and public key, and one candidate path to
it from each peer. It answers "which Nodes exist, and how do I reach
each one?" It is filled by NODE_INFO_ADVERTISEMENT and NODE_INFO_WITHDRAWAL.¶
The tracks that are published, one entry per originating Edge Node
and track. Each entry says that the Edge Node has a local publisher
for the track, and which forwarding mode it wants. It answers "where
does track T come from?" It is filled by PUBLISH_INFO_ADVERTISEMENT and
PUBLISH_INFO_WITHDRAWAL.¶
The tracks that are wanted, one entry per originating Edge Node and
track. Each entry says that the Edge Node has local subscribers for
the track. It answers "who wants track T?" It is filled by
SUBSCRIBE_INFO_ADVERTISEMENT and SUBSCRIBE_INFO_WITHDRAWAL.¶
Together, the Publisher Table says where a track comes from, the Subscriber Table says where it must go, and the Node Table says how to get between them. Full details are in Section 4.¶
The Node that a Node, Publisher, or Subscriber Table entry describes, and the only Node that can create
or withdraw it. For a Node Table entry it is the Node itself (node_info.id).
For a Publisher or Subscriber Table entry it is the Edge Node whose local publisher or
subscriber the entry describes (source_node_id). The originator
signs the entry (Section 10); other Nodes only re-publish it. The
term follows BGP route reflection [RR], where ORIGINATOR_ID names
the router that injected a route.¶
An advertisement installs or updates an entry in a Node, Publisher, or Subscriber Table; a withdrawal removes it. Both are signed by the entry's originator. The messages for each table are defined in Section 8.1.¶
Forwarding state for one track on one Data Session: the Edge Nodes
with subscribers for the track that it must reach by way of that Data
Session's downstream Node. Each Data Session has one Delivery Node Set
per track it carries. The upstream Node builds it from its Subscriber Table (who
wants the track) and its Node Table (which next hop leads to each of them),
and sends it on the Data Session's delivery-node-sets track. Each
downstream Node splits it for its own next hops, so the set narrows
at every hop and the track is copied only where paths split. For
example, if track T has subscribers behind B1 and B2, both reached
through Transit X, the Delivery Node Set for T on the Data Session to
X lists {B1, B2}; X then sends {B1} toward B1 and {B2} toward B2.
See Section 4.5.¶
A 32-bit label that the upstream Node of a Data Session assigns to one Delivery Node Set on that Data Session. In pipelined mode it travels with every forwarded object (Section 7.2), so the downstream Node can find the set without parsing the track identity. It has meaning only on one Data Session. See Section 4.5.¶
This section is informative.¶
Each MIX participant is a Node: an MOQT relay that also implements MIX. A Node uses two kinds of Session, each a separate QUIC connection. A Control Session carries MIX state as MOQT tracks: each Node publishes its Node, Publisher, and Subscriber Tables as tracks to which its peers subscribe. A Data Session carries forwarded objects and is opened only when a track needs to flow to a downstream peer, and only between Nodes that exchange Node Tables. Whether a peering also carries the Publisher and Subscriber Tables depends on the deployment: every peering carries all three when every Node runs both planes, while a large deployment can move the exchange of the Publisher and Subscriber Tables to a small set of Control Nodes (Section 12).¶
A track flows through MIX as follows:¶
Discovery. Each Node publishes an announce track under a
well-known namespace prefix. Nodes learn each other's Node IDs by
subscribing to that prefix (Section 5).¶
Peering. Two Nodes subscribe to each other's Node, Publisher, and Subscriber Table tracks over a Control Session. Which of Node, Publisher, and Subscriber Tables a peering carries is its scope (Section 6.1).¶
Flooding. Each Node advertises its own Node Table entry and re-advertises entries learned from peers, appending itself to the path. Publishers originate Publisher Table entries and subscribers originate Subscriber Table entries (Section 4). Each advertisement is signed by the Node that originated it (Section 10).¶
Path selection. Each Node selects a best egress Session per destination Node, considering only paths on which every hop supports the track's requested forwarding mode (Section 4.4).¶
Fan-out setup. For each track it forwards, a Node groups the track's destination Nodes by their best egress. It opens (or reuses) a Data Session to each of those next hops and publishes on it the track's Delivery Node Set: the destinations to be reached through that next hop, labelled with a Delivery Node Set Id (Section 5.1.3).¶
Delivery. Objects follow the selected path in the track's forwarding mode (Section 7.1). At each hop, the downstream Node finds the Delivery Node Set for an arriving object (by its Id in pipelined mode, by track in native mode), repeats the grouping of step 5 for its own next hops, and forwards the object under its own Delivery Node Set Ids (Section 7.3).¶
Each entry in the Node, Publisher, and Subscriber Tables then moves through a few simple states:¶
Node Table entry: is there a path to its Node?
| advertisement
v
+-------------+
+--| REACHABLE |
| +-------------+
| | ^
| | path | path
| | lost | back
| v |
| +-------------+
withdrawal | | UNREACHABLE |
| +-------------+
| |
| | ORIGINATOR_UNREACH_HOLD expires
| v
| +-------------+
+->| REMOVED |
+-------------+
Publisher or Subscriber Table entry: is its originator reachable?
| advertisement
v
+-------------+
+--| ACTIVE |
| +-------------+
| | ^
| | lost | back
| | |
| v |
withdrawal | +-------------+
or | | UNUSABLE |
restart | +-------------+
| |
| | ORIGINATOR_UNREACH_HOLD expires
| v
| +-------------+
+->| REMOVED |
+-------------+
A Publisher or Subscriber Table entry is UNUSABLE exactly while its originator's Node Table
entry is UNREACHABLE. "Restart" means the originator came back with a
newer origin_epoch. Both holds are ORIGINATOR_UNREACH_HOLD. In a
split deployment, a Forwarding Node also removes reflected entries if
it has had no Control Node for CONTROL_STALE_HOLD. The full rules are
in Section 4.2 and Section 4.3.¶
Each Node has an Ed25519 [RFC8032] signing key pair. The Node ID is a 128-bit value derived from the Node's Ed25519 public key:¶
node_id = first 16 bytes of SHA-256(ed25519_public_key),
as a 128-bit value in network byte order
Because the Node ID is bound to the Node's public key, no separate
identity-to-key mapping is required: a receiver that has the Node's
public key can verify that any Node ID claim is authentic. This makes
MIX originator authentication (Section Section 10) self-verifying: an
originating advertisement carries a signature by the Ed25519 private
key whose derived Node ID matches the advertisement's source_node_id.¶
On receipt, a Node MUST verify that the public key in node_info
(Section 8.3.1) derives the advertised ID using the formula above, in addition to
verifying the signature and local trust policy. A valid signature by
an unrelated key MUST NOT authenticate a claimed Node ID.¶
The canonical textual form of a Node ID is four groups of eight lowercase hexadecimal digits separated by colons; the colons MAY be omitted for a compact form. Groups MUST NOT be shortened by trimming leading zeros.¶
<group> ::= 8HEXDIG <canonical> ::= <group> ":" <group> ":" <group> ":" <group> <compact> ::= 32HEXDIG
Examples: 1a2b3c4d:5e6f7081:9200aabb:ccddeeff,
1a2b3c4d5e6f70819200aabbccddeeff.¶
Nodes MAY be configured with a static Ed25519 key pair or MAY generate one at startup. Once generated, the key pair SHOULD be persisted across restarts to preserve Node ID stability.¶
Each Node maintains a Node Table, a Publisher Table, and a Subscriber Table and, for each of its Data Sessions, a set of Delivery Node Sets:¶
| State | Describes | Keyed by | Stores |
|---|---|---|---|
| Node Table | Reachable Nodes | Node ID | Most recent node_info, one candidate path per peer, and the computed best egress Session |
| Publisher Table | Published tracks | (source_node_id, full_name_hash) |
Most recent publish_info
|
| Subscriber Table | Requested tracks | (source_node_id, full_name_hash) |
Most recent subscribe_info
|
| Delivery Node Set | Where each forwarded track must go, per Data Session | (Data Session, Delivery Node Set Id) | Track and destination Node IDs (Section 4.5) |
Each of these tables is filled by the advertisement and withdrawal messages defined in Section 8.1. On Session teardown a Node MUST remove every Node Table candidate learned from the departing peer (Section 4.2). Publisher and Subscriber Table entries are not withdrawn on Session teardown; their lifetime follows the reachability of their originator (Section 4.3).¶
Best-egress selection is computed per Node Table entry using the algorithm in Section 4.4.¶
Every advertisement and withdrawal carries two freshness fields, both set by its originator:¶
origin_epoch (64 bits) identifies the originator's current incarnation
(Section 10.2).¶
sequence (64 bits) counts the advertisements and withdrawals the originator has published for
one key in its Node, Publisher, or Subscriber Table
(Section 4). It increases with every message
for that key and MUST NOT wrap or be reused within an epoch.¶
An originator MUST either persist its epoch and per-key sequences across
restarts or advance origin_epoch before it publishes again.¶
A receiver orders messages for the same originator and key by the pair
(origin_epoch, sequence), comparing origin_epoch first. It accepts
a message only if its pair is greater than the stored one, and
re-floods only messages it accepts.¶
For a NODE_INFO_ADVERTISEMENT (Section 8.3), this comparison covers only the originator-signed
fields of node_info. Its path is unsigned and tracked per peer
(Section 4.2), so an advertisement whose pair equals the stored one is not
accepted as new but still replaces the sending peer's candidate path.¶
A Node can usually reach another Node through more than one peer. For
each Node Table entry E, it therefore keeps one candidate path per peer: the
most recent path that peer advertised for E.id. Path selection
(Section 4.4) chooses the best egress among these candidates. When
the best one disappears, the Node fails over to the next without
waiting for the originator to re-advertise.¶
A candidate changes only on one of four events, each tied to a single
peer S:¶
| Event on Session S | Effect |
|---|---|
NODE_INFO_ADVERTISEMENT for E.id (Section 8.3) whose (origin_epoch, sequence) equals or exceeds the stored one |
S's candidate becomes the received path. If that path contains the local Node ID, S reaches E.id only through this Node, so S's candidate is removed instead. |
NODE_UNREACH for E.id (Section 8.4) |
S's candidate is removed. |
S enters CLOSED
|
S's candidate is removed from every entry. |
NODE_INFO_WITHDRAWAL from E.id (Section 8.3) |
E is removed with all its candidates, and the withdrawal is re-flooded. |
After any of these events, the Node re-runs path selection for E and
tells its own peers what changed:¶
If the best path changed materially, it re-publishes E on its nodes
track with its own element appended.¶
If no candidate is left, E is unreachable. If the Node had
advertised E, it publishes NODE_UNREACH for E.id on its nodes
track, so that its peers remove their path through it.¶
The same rules then apply at each peer, so the loss of a path spreads hop by hop and stops at Nodes that still have another candidate.¶
An unreachable entry keeps its originator-signed fields for
ORIGINATOR_UNREACH_HOLD (default 90 s) and is then removed. If a path
returns within that time, an advertisement with the same (origin_epoch,
sequence) restores a candidate without the originator having to
re-advertise.¶
NODE_UNREACH needs no signature because it can only remove the
candidate of the Session it arrived on, which is the path the sender
itself advertised. The sender could have the same effect by never
advertising that path, so accepting an unsigned NODE_UNREACH gives it
no new power.¶
Holding one candidate per peer costs memory in proportion to the number of Nodes times the number of peers. A Node MAY keep fewer candidates than it has peers, at the cost of slower failover.¶
Publisher and Subscriber Table entries describe their originator, not the path they arrived on. A Publisher Table entry says that the originator publishes a track; a Subscriber Table entry says that the originator has subscribers for one. A Node therefore keeps an entry when the peer that delivered it goes away. It removes the entry only when the originator's statement is no longer true, or the Node can no longer count on hearing about it if it changes.¶
The originator ends an entry in one of two ways:¶
Withdrawal. The originator sends PUBLISH_INFO_WITHDRAWAL
(Section 8.5) or SUBSCRIBE_INFO_WITHDRAWAL (Section 8.6)
when its last local publisher or subscriber for the track leaves. The
withdrawal floods like any other message, and each Node removes the entry
when it accepts the withdrawal.¶
Restart. When a Node accepts a NODE_INFO_ADVERTISEMENT from the originator
with a newer origin_epoch, it removes every Publisher and Subscriber Table entry from
that originator with an older epoch. The restarted originator re-advertises
the entries that are still valid.¶
A Node also ends entries on its own when it can no longer rely on hearing from the originator:¶
Originator unreachable. While the originator's Node Table entry is unreachable
(Section 4.2), the originator's entries are kept but not used for
path setup. If the originator is still unreachable after
ORIGINATOR_UNREACH_HOLD, they are removed.¶
No Control Node. In a split deployment, a Forwarding Node that
has had no ESTABLISHED Reflection peering for CONTROL_STALE_HOLD
removes every entry it did not originate, because it may have missed
withdrawals (Section 6.5).¶
These local removals are not flooded. Every Node reaches the same conclusion from its own Node Table or peering state.¶
When an unreachable originator comes back, no special recovery is needed. The Sessions that re-form across the repaired link fill from each other (Section 6.2), the Nodes there accept the originator's entries as new, and normal flooding carries the entries on.¶
Path selection chooses, for each destination Node E.id, the peer
through which this Node sends data toward it: E's best egress. It
works on E's candidates (Section 4.2), one per peer, and runs
again whenever they change.¶
A candidate is eligible unless:¶
its peer is a Control Node, which never forwards data; or¶
it is being considered for a track whose Publisher Table entry requests
pipelined mode, and some hop on its path does not set the pipelined
bit (bit 1) of data_modes (Section 8.3.1). Every Forwarding Node
supports native mode, so a native-mode track can use any eligible
candidate.¶
Among eligible candidates, the Node prefers:¶
the shortest path, counted as the sum of hop_count over its
elements. Without path aggregation (Section 14) this is
the number of elements; then¶
the lowest cumulative smoothed round-trip time (sRTT): the sum of srtt_us over the path plus
the Node's own sRTT to the peer that advertised it.¶
Hop count decides first, and sRTT only breaks ties.¶
The chosen peer's Session is cached on E as its best egress. The
Node recomputes it when that Session closes, when its candidate is
removed, or when a materially better candidate arrives. A change of
best egress moves E.id from the Delivery Node Set on the old egress Data Session to
the Delivery Node Set on the new one, for every track forwarded toward it (Section 7.3) and can lead the Node to re-publish E (Section 4.2).¶
To judge "materially better", implementations SHOULD round sRTT values (for example, to whole milliseconds) before comparing them, so that sub-millisecond jitter does not make the best egress, and the advertisements that depend on it, oscillate.¶
The Node, Publisher, and Subscriber Tables say which Nodes exist, which tracks are published, and which Edge Nodes have subscribers. To forward a track, a Node also needs to know, on each Data Session, where the objects it sends there must end up. That is the Delivery Node Set.¶
An Delivery Node Set belongs to one track on one Data Session. It names the destination Nodes, the Edge Nodes with subscribers for the track, that the track must reach through the downstream Node of that Data Session.¶
Delivery Node Sets are built hop by hop. The publisher's Edge Node takes the destinations for the track from its Subscriber Table, groups them by best egress (Section 4.4), and creates one Delivery Node Set per egress Data Session. Each downstream Node receives that Delivery Node Set, groups its destinations again by its own best egress, and creates its own Delivery Node Sets. At each hop a Delivery Node Set keeps only the destinations still ahead on that branch, so a track is copied exactly where its destinations' paths diverge:¶
E1's Subscriber Table: track T has subscribers behind B1, B2, B3
{B1,B2} {B1}
[E1] --------------> [X] ------------> [B1]
| |
| | {B2}
| +--------------> [B2]
|
| {B3}
+------------------------------------> [B3]
Each Delivery Node Set is labelled with an Id that the upstream Node chooses and that is unique on that Data Session. In pipelined mode every forwarded object carries the Delivery Node Set Id (Section 7.2), so the downstream Node finds the Delivery Node Set with one lookup instead of parsing the track identity. Delivery Node Set Ids have no meaning beyond their Data Session: each hop assigns its own on its egress Data Sessions.¶
An Delivery Node Set changes when a destination is added or removed (a Subscriber Table change)
or when a destination's best egress changes (a Node Table change). The
upstream Node then publishes the updated Delivery Node Set under the same Id, or
withdraws it when no destinations remain. Delivery Node Sets are published on each
Data Session's delivery-node-sets track (Section 5.1.3) as DELIVERY_NODE_SET_ADVERTISEMENT
and DELIVERY_NODE_SET_WITHDRAWAL (Section 8.7). How a Node forwards objects
using them is specified in Section 7.3.¶
MIX carries all of its state as MOQT tracks. Every track sits under the
namespace prefix ("mix","v1"). The rest of the namespace tuple says
whose state the track holds and, for delivery-node-sets, which Data Session it
belongs to:¶
| Namespace | Track | Published by | Carried on | Contents |
|---|---|---|---|---|
("mix","v1","announce","<node-id>")
|
announce
|
Every Node | Control Session | The Node's CONNECT, for discovery (Section 8.2) |
("mix","v1","<node-id>")
|
nodes
|
Every Node | Control Session | Node Table advertisements and withdrawals, and NODE_UNREACH (Section 8.3, Section 8.4) |
("mix","v1","<node-id>")
|
publishers
|
Every Node | Control Session | Publisher Table advertisements and withdrawals (Section 8.5) |
("mix","v1","<node-id>")
|
subscribers
|
Every Node | Control Session | Subscriber Table advertisements and withdrawals (Section 8.6) |
("mix","v1","<node-id>")
|
publishers-mesh, subscribers-mesh
|
Control Nodes only | Control Session | Publisher and Subscriber Table entries reflected to other Control Nodes (Section 12.2.1) |
("mix","v1","<node-id>","<sid>")
|
delivery-node-sets
|
Upstream Node of the Data Session | That Data Session | One Delivery Node Set per track forwarded on that Data Session (Section 8.7) |
<node-id> is the textual form of the publishing Node's ID
(Section 3). <sid> is a 128-bit Data Session Identifier, chosen by
the Node that opens the Data Session (Section 6) and written in
the same textual form. The publishers-mesh and subscribers-mesh tracks carry the
same objects as publishers and subscribers and follow the same track semantics.¶
Each Data Session has its own namespace, so its delivery-node-sets track is kept
apart from every other Data Session's while the track name stays the
fixed literal "delivery-node-sets". A Node MAY list its active Data Sessions by
issuing a MOQT SUBSCRIBE_NAMESPACE on its own prefix
("mix","v1","<node-id>"): it receives one NAMESPACE message per
Data Session and a NAMESPACE_DONE when one is withdrawn.¶
Discovery uses the separate announce prefix, so a Node can find every
other Node without a directory service. A MOQT SUBSCRIBE_NAMESPACE on
("mix","v1","announce") returns one NAMESPACE message per Node in
the mesh, and a NAMESPACE_DONE when a Node withdraws. A MOQT
SUBSCRIBE to a Node's announce track then retrieves its CONNECT
(Section 6).¶
How each track numbers its groups and objects, its priority, and how to subscribe to it are given in Section 5.1.¶
Each MIX control track specifies an Object priority value for its
objects. MOQT uses lower numeric priority for higher scheduling
precedence, with 0 the highest ([MoQTransport]). Priority 0 is
reserved for MOQT control traffic, and MIX tracks MUST NOT use it.¶
The nodes and delivery-node-sets tracks use priority 1, because they gate
forwarding: path selection cannot proceed without reachability, and
data egress is held back until the target set is known. The publishers and
subscribers tracks carry per-track demand that does not gate delivery, and
use priority 2. The bootstrap announce track also uses priority
2; after initial discovery, its refresh is a liveness heartbeat
rather than a routing decision.¶
nodes Track
The nodes track distributes reachability. A Node publishes on it its own
NODE_INFO advertisement and the advertisements it has accepted from
peers, so that every Node learns which other Nodes exist, the path to
each, and the per-hop sRTT along that path.¶
A monotonically increasing epoch, incremented when self-info materially changes (Node type, contact, geo) and at process start. Advertisements for other Nodes reuse the current epoch.¶
0.¶
A single MIX control message (Section 8) tagged
NODE_INFO_ADVERTISEMENT, NODE_INFO_WITHDRAWAL, or NODE_UNREACH.¶
1. Node Table gates path selection.¶
As in Section 6.2.¶
Loop prevention is inherited from the payload: an advertisement whose
path contains the receiver's own Node ID removes the sending peer's
candidate instead of adding one (Section 4.2). When a Node
re-publishes an advertisement on its own nodes track, it appends its own
element to the path (Section 8.3.1).¶
A Control Node MUST NOT re-publish Node Table advertisements it has accepted from peers. It accepts them to learn public keys (Section 10.3), but it never appears in a data-plane path.¶
publishers and subscribers Tracks
The publishers track distributes publications: each entry states that a track
is available at its originator and the forwarding mode requested for
it. The subscribers track distributes demand: each entry states that a Node has
subscribers for a track. A Node publishes on these tracks the entries
it originates and the entries it has accepted from peers. Together they
let each Node determine which tracks need to flow toward it and from
where. In the split deployment model, which accepted entries a Node
re-publishes depends on its role (Section 12.2.1).¶
A monotonically increasing MOQT Group ID assigned by the Node publishing this track. It is a local transport location, separate from the MIX freshness fields, and MUST NOT wrap or be reused during the track's lifetime. A Node that re-publishes an entry assigns its own MOQT locations and preserves the signed MIX payload fields.¶
0.¶
A MIX control message tagged SUBSCRIBE_INFO_ADVERTISEMENT,
SUBSCRIBE_INFO_WITHDRAWAL, PUBLISH_INFO_ADVERTISEMENT, or PUBLISH_INFO_WITHDRAWAL.¶
2.¶
MOQT Group IDs and Object IDs are local transport locations, not MIX sequence numbers (Section 4.1); this keeps MOQT location ordering and FETCH ranges independent of per-originator freshness.¶
delivery-node-sets Track
The delivery-node-sets track carries one Delivery Node Set for each track forwarded on its Data
Session: the destination Nodes that the track must reach through the
downstream Node, labelled with a Delivery Node Set Id. The track is scoped to that Data Session: <sid> in the namespace tuple identifies the
Data Session, and both endpoints agree on it when that session is opened
(Section 6). The track name is the fixed literal "delivery-node-sets".¶
The 32-bit Delivery Node Set Id assigned by the sender, carried in the low 32 bits of the MOQT Group ID (upper bits zero). MIX reuses the same Delivery Node Set Id when the target set changes, so each Delivery Node Set Id maps to one MOQT group whose most recent object is the current set.¶
0.¶
A monotonically increasing counter starting at 0 for the first
object in the group and incremented by one for each subsequent
update to the set. Consumers treat the object with the greatest
Object ID in the group as the current state for that Delivery Node Set Id.¶
DELIVERY_NODE_SET_ADVERTISEMENT or DELIVERY_NODE_SET_WITHDRAWAL.¶
The Delivery Node Set Id is a 32-bit identifier. It is carried as a 32-bit field in
DELIVERY_NODE_SET_ADVERTISEMENT payloads (Section 8) and as a
variable-length integer ([RFC9000]) in the data-plane MIX Data Header
(Section 7.2). Both forms carry the same 32-bit value.¶
As in Section 6.2.¶
1. Data cannot be forwarded until its Delivery Node Set is known.¶
Tied to the Data Session. On teardown the publisher unpublishes the track and the subscriber implicitly loses its subscription.¶
announce Track
The announce track lets Nodes discover each other. Each Node
publishes on it a CONNECT message describing itself, including its
Node ID and public key, so that a prospective peer can identify the
Node and verify its signed advertisements before subscribing to its
Node, Publisher, and Subscriber Table tracks.¶
A monotonically increasing epoch, bumped whenever the publisher
publishes a new CONNECT object (see Section 5.1.5). The
initial group after process start is 0.¶
0.¶
A monotonically increasing counter reset to 0 at each new group.¶
A CONNECT message (Section 8) carrying the publisher's
node_info.¶
2.¶
As in Section 6.2; the fill yields the current CONNECT.¶
See Section 5.1.5.¶
A Node discovers peers by issuing a MOQT SUBSCRIBE_NAMESPACE with
prefix ("mix","v1","announce") and receiving one NAMESPACE message
per known peer, followed by NAMESPACE_DONE on withdrawal. It then
attaches to each peer's announce track with a MOQT SUBSCRIBE (see
Section 6).¶
If a peer withdraws its announce namespace while A still holds an
ESTABLISHED peering with that peer, the withdrawal is treated as an
intent-to-depart signal. A SHOULD begin the CLOSED transition
(Section 6.4) for that peering unless a replacement CONNECT
arrives on a re-published namespace within the staleness window
defined in Section 5.1.5. The peering's Control Session and any
Data Sessions remain open until the state machine completes the
transition; this prevents a flapping announce from immediately
tearing down in-flight data.¶
announce Refresh and Timeout
A CONNECT object is a soft-state advertisement: subscribers use it to
learn the publisher's public key, contact, geo, and capability bits, and
they need a bounded assurance that the currently held object still
reflects the publisher. Two rules govern the publisher's rewrite
schedule:¶
Timer-driven refresh. A publisher MUST re-publish its CONNECT
object at least once per ANNOUNCE_REFRESH_INTERVAL. The default
value is 30 s; deployments MAY use a shorter interval where faster
detection of stale peers matters, or a longer interval on
resource-constrained deployments, provided all peers agree via
configuration.¶
Change-driven refresh. A publisher MUST re-publish its CONNECT
object as soon as any field of node_info covered by the originator
signature (Section 10) changes — Node type, contact string, geo
coordinates, public key, data_modes, or origin_epoch. Path
accumulation is not a change (path is not signed).¶
Each rewrite advances the MOQT Group ID by one and resets Object ID to
0. Because the announce track is latest-state oriented, older
groups need not be retained beyond the configured recovery window.¶
A subscriber MUST treat a CONNECT object as stale when it has not
observed a new group within
ANNOUNCE_STALE_MULTIPLIER * ANNOUNCE_REFRESH_INTERVAL (default
multiplier 3, giving a 90 s default staleness bound). A stale
CONNECT on an active peering is a hint that the peer is
unhealthy; the subscriber SHOULD initiate CLOSED transition
(Section 6.4) if no refresh arrives within one additional
interval.¶
The CONNECT refresh has a monotonically increasing MIX sequence within
the current originator epoch. origin_epoch changes only when the Node
starts a new incarnation that may reuse sequence state. Subscribers
use the (origin_epoch, sequence) pair for replay defence independently
of MOQT group progression.¶
Peering between two Nodes A and B is established using standard MOQT
control messages. The mapping below is normative; any additional MOQT
messages required by [MoQTransport] for these exchanges (SETUP,
REQUEST_OK / REQUEST_ERROR, PUBLISH_OK / PUBLISH_ERROR, and so
on) are used as defined there.¶
Publish namespaces and tracks. On startup, each Node issues a MOQT
PUBLISH_NAMESPACE for its per-Node namespace
("mix","v1","<node-id>") and a second PUBLISH_NAMESPACE for the
bootstrap namespace ("mix","v1","announce","<node-id>").
It then issues MOQT PUBLISH requests for nodes, publishers, subscribers, and
announce tracks and publishes its CONNECT message as an object
on announce. PUBLISH_NAMESPACE announces namespace availability;
it does not by itself publish a track or its objects.¶
Discover peers. To find peers, Node A issues a MOQT
SUBSCRIBE_NAMESPACE with prefix ("mix","v1","announce"). It
receives one NAMESPACE message per Node currently in the mesh
(and a NAMESPACE_DONE when a peer's namespace is subsequently
withdrawn), learning each peer's Node ID.¶
Attach control-plane tracks. To peer with B, Node A:¶
Issues a MOQT SUBSCRIBE with FILL_PARAMETERS for the
announce track of B (("mix","v1","announce","<B-node-id>")
/ "announce") to obtain B's current CONNECT body and receive
later refreshes.¶
Issues MOQT SUBSCRIBE requests for the tracks under
("mix","v1","<B-node-id>") that the peering's scope calls for
(Section 6.1), drawn from nodes, publishers, and subscribers, to
receive current state and live updates (Section 6.2). A
needs no further discovery for this: B's namespace follows from
the Node ID that A learned in step 2, and the track names are
fixed by this document (Section 5). MOQT allows a
subscriber to subscribe to a track without first receiving a
PUBLISH_NAMESPACE for its namespace ([MoQTransport]).¶
When the peering's scope includes the Node Table and A selects B as the
egress for a track, A opens a separate Data Session to B and
chooses its <sid>. On that connection, A issues a MOQT
PUBLISH_NAMESPACE for ("mix","v1","<A-node-id>","<sid>") and
a PUBLISH for its delivery-node-sets track, and B subscribes to that track
(Section 6.2). The MOQT control messages for this run on the
Data Session. A opens the Data Session when a track first needs
to flow to B and reuses it for later tracks.¶
The scope of a peering is the set of Node, Publisher, and Subscriber Table tracks that each Node subscribes to from the other. Scope is set by local configuration for each peering and is not negotiated on the wire. Three scopes are defined:¶
| Scope | A subscribes to B's | Used between | Data Sessions |
|---|---|---|---|
| Full |
nodes, publishers, subscribers
|
Forwarding Nodes, integrated model | Yes |
| Topology |
nodes
|
Forwarding Nodes, split model | Yes |
| Reflection | Forwarding Node and Control Node: publishers, subscribers, plus nodes when A is the Control Node. Two Control Nodes: publishers-mesh, subscribers-mesh
|
A Forwarding Node and a Control Node, or two Control Nodes | No |
The following rules apply regardless of deployment model:¶
A Forwarding Node MUST NOT subscribe to the nodes track of a Control
Node.¶
A Node MUST open a Data Session to a peer only over an ESTABLISHED
peering with Full or Topology scope. Path selection (Section 4.4)
yields only such peers.¶
A Control Node MUST NOT open or accept Data Sessions.¶
A Control Node's subscription to a Forwarding Node's nodes track lets
the Control Node learn the public keys of every Forwarding Node that
Forwarding Node can reach, which it needs to verify the Publisher and Subscriber Tables
entries it receives (Section 10).¶
For the nodes, publishers, and subscribers tracks, subscribers SHOULD issue a MOQT
SUBSCRIBE whose Location Filter starts at the Next Object. To also
acquire retained state, the initial SUBSCRIBE SHOULD include
FILL_PARAMETERS with a location filter ending at Largest Object.
The fill range and the live subscription filter are distinct in draft
22. Implementations MUST process both fill-delivered and
subscription-delivered objects using the same MIX freshness and
duplicate-suppression rules. A relay MAY return an incomplete fill
when objects are unavailable; MIX therefore requires a full-state
snapshot or equivalent recovery mechanism after a cache miss (see the
editor's note below).¶
The same pattern applies to the announce track and to the
per-Data-Session delivery-node-sets track (Section 5.1.3). The delivery-node-sets cache lifetime
SHOULD cover expected peer setup and recovery intervals.¶
Priorities influence scheduling only. A Node MUST NOT treat an
advertisement whose delivery timed out under OBJECT_DELIVERY_TIMEOUT
or SUBGROUP_DELIVERY_TIMEOUT ([MoQTransport]) as delivered.¶
Editor's note: This document does not yet define how a Node recovers full state when a fill is incomplete because the publishing relay has no usable cache.¶
B performs the subscriptions to A that the peering's scope calls for (Section 6.1). For Full and Topology scope these mirror A's.¶
Two Nodes MAY open Control Sessions to each other at the same time. A
pair of Nodes keeps at most one Control Session. Once each Node has
received the other's CONNECT, the connection opened by the Node with
the numerically lower Node ID is kept and the other is closed. Both
directions of subscription then run over the kept connection. Closing
the duplicate connection is not a peering close: neither Node removes
state learned from the other (Section 6.4).¶
Each peering (an ordered pair of Nodes A and B, viewed from A's side) progresses through a small state machine. States describe the
control-plane relationship only; data-plane subscriptions
(Section 6 step 3) are opened independently once ESTABLISHED
is reached on a peering whose scope includes the Node Table.¶
+---------+
| INIT |
+---------+
|
| subscribe to B's announce track
v
+---------+
| OPENING |
+---------+
|
| receive B.CONNECT object
v
+---------+
| HELLO |-------> REJECTED
| RECVD | signature / policy reject
+---------+
|
| subscribe to tracks in scope
|
v
+-------------+
| ESTABLISHED |
+-------------+
| | |
| | | signature failure at ingress
| | | / peer close / QUIC error
v v v
+-------------+
| CLOSED |
+-------------+
State transitions:¶
INIT -> OPENING: A issues a MOQT SUBSCRIBE for B's announce
track (("mix","v1","announce","<B-node-id>") / "announce").¶
OPENING -> HELLO_RECVD: A receives B's CONNECT object on the
announce track. A does not yet trust B; the object's signature MUST
verify against a pubkey that either (a) is derived from B's Node ID
(Section 3) or (b) was already installed as a trusted key.¶
HELLO_RECVD -> ESTABLISHED: A applies the simultaneous-peer
tiebreaker and issues MOQT SUBSCRIBE requests with
FILL_PARAMETERS for the B tracks in the peering's scope
(Section 6.1) as specified in Section 6.2. B receives A's
advertisements through its own subscriptions to A.¶
HELLO_RECVD -> REJECTED: signature verification failed, the peer
pubkey does not match A's local pinning policy, or admission control
declines the peering. A cancels its subscription to B's announce
track and does not subscribe to B's Node, Publisher, or Subscriber
Table tracks. The state is terminal
from A's perspective; B may retry after a backoff.¶
ESTABLISHED -> CLOSED: A observes any of (i) a MOQT
PUBLISH_DONE on any control-plane track from B, an explicit
cancellation of A's SUBSCRIBE, or, when the peering's scope
includes Node Table, a STOP_SENDING on a Data Session to B; (ii) QUIC
transport failure; (iii) sustained signature failure at ingress; or
(iv) a local admin action. A connection closed by the
simultaneous-peer tiebreaker (Section 6.3) does
not close the peering. On entry to CLOSED, A removes every Node Table
candidate it learned from B (Section 4.2). Publisher and Subscriber Table entries
follow Section 4.3, and those learned over a Reflection peering
also follow Section 6.5.¶
Tying a Topology or Full peering to its Data Sessions keeps Node Table and the data plane in step: a peer that can no longer forward is withdrawn from path selection rather than left as a black hole. A Reflection peering has no Data Sessions, so data-plane events never close it.¶
CLOSED is a terminal state; recovery is a fresh INIT on a new
session identifier. The Node ID of the peer is unchanged, so any
previously distributed originator pubkey remains valid across
reconnection.¶
Losing a Reflection peering does not by itself remove any Publisher or Subscriber Table entry. The entries are signed by their originators and stay valid while those originators are reachable (Section 4.3), and a Forwarding Node peered with more than one Control Node keeps receiving updates from the others.¶
Reachability here comes from the Node Table, not from the Control Nodes. An
originator is reachable while the Node's Node Table entry for it has at least
one candidate (Section 4.2). A Forwarding Node builds its Node Table
over its Topology peerings with its data-plane neighbors, so it keeps
tracking originators even when it has no Control Node at all. A
Control Node builds its Node Table from its subscriptions to the nodes tracks
of the Forwarding Nodes it serves (Section 6.1).¶
A Forwarding Node that has no ESTABLISHED Reflection peering receives
no updates, and so misses any withdrawals issued in the meantime. To
bound that, a Forwarding Node that has had no ESTABLISHED Reflection
peering for CONTROL_STALE_HOLD removes every Publisher and Subscriber Table entry it did
not originate. The default is 90 s. It SHOULD exceed the time needed
to re-establish a Reflection peering and complete its fill
(Section 6.2). The fill on the next Reflection peering restores
the entries.¶
The data plane between two Nodes uses a Data Session: a QUIC connection
separate from the Control Session of the peering between them, whose
scope includes the Node Table (Section 6.1). Keeping the two
connections separate prevents media flow control from delaying
control-plane updates, allows a Data Session to be opened only when a
track needs to flow, and lets each connection use its own congestion
control and priority settings. Each Data
Session uses MOQT control messages to set up and manage its delivery-node-sets track
subscription, but carries forwarded objects separately from the MIX
control-state tracks. Two forwarding modes are defined
(Section 7.1), and each Publisher Table entry carries the mode requested by
its originator. A Node MUST select an
end-to-end path only if every hop supports the mode requested for the
track.¶
MIX defines two ways to carry a track's objects between Nodes. In native mode every hop forwards whole MOQT objects. In pipelined mode every hop forwards bytes as they arrive, behind a small MIX Data Header. The track's originator picks the mode in its Publisher Table entry, and a path is usable for pipelined mode only if every hop on it supports it.¶
| Native mode | Pipelined mode | |
|---|---|---|
| What crosses each hop | A MOQT object, framed per hop | A MIX Data Header, then the object's bytes |
| Publisher's Edge Node | Forwards MOQT objects | Re-frames each object with an delivery_set_id
|
| Transit Nodes | Receive whole objects and MAY cache them | Forward bytes before the object boundary is known |
| Subscriber's Edge Node | Delivers MOQT objects | Rebuilds the MOQT object |
| Support | Required of every Forwarding Node |
OPTIONAL, advertised in data_modes
|
In native mode, the inter-relay data-plane connection runs the same MOQT
[MoQTransport] object framing that is used between application
publishers/subscribers and their edge relays. Each application object is
forwarded as a MOQT object, preserving its track identity and location
(group_id, subgroup_id, object_id) across hops. Track Aliases and
request IDs are local to each MOQT session and MUST be mapped per hop.
Publisher priority is preserved as the priority of the forwarded object
subject to relay policy. The MOQT stream mapping is:¶
A subgroup maps to a QUIC unidirectional stream per hop, as defined by [MoQTransport]. Each relay may use a different stream and local Track Alias on egress while preserving the object locations.¶
Objects delivered via QUIC datagrams remain in datagrams.¶
FETCH responses ([MoQTransport]) traverse each hop as a single unidirectional stream carrying the FETCH object sequence.¶
Intermediate relays MAY cache objects and serve retained ranges with
FETCH or fill fetch streams when supported. MOQT does not require a
relay to cache, so a peer must be able to repair state after a cache
miss; this is not yet defined (see the editor's note in
Section 6.2). Priority scheduling and delivery-timeout behavior follow
[MoQTransport] on each session; ordering is per subgroup or as
otherwise specified by a request's Group Order, not a global order
across tracks or streams.¶
Native mode is the default. Forwarding Nodes MUST support native mode.¶
In pipelined mode, the inter-relay data-plane connection replaces MOQT subgroup framing with the MIX Data Header (Section 7.2) followed by verbatim application bytes. The stream mapping is:¶
Each MOQT subgroup arriving at the publisher's Edge Node is re-framed
into one unidirectional QUIC stream on the inter-relay connection.
The stream carries a new_stream_header (Section 7.2) followed by
back-to-back object bodies from the subgroup, each later one preceded
by its own existing_stream_header. Intermediate relays forward the
stream's bytes hop by hop, rewriting only the Delivery Node Set Id in each header,
since Delivery Node Set Ids are local to a Data Session. The terminal edge relay reconstructs a MOQT
subgroup stream toward each local subscriber, preserving the
subgroup identifier.¶
Objects that the publisher's Edge Node received in MOQT datagrams are re-emitted as
QUIC datagrams, each beginning with a datagram_header. Each
hop rewrites the Delivery Node Set Id and re-enqueues the datagram.¶
FETCH responses are always carried in native framing regardless of
the track's forwarding_mode, because FETCH requires bounded
cache-and-replay semantics at every hop.¶
Intermediate relays MUST forward bytes as they arrive: they SHOULD NOT wait for MOQT-level object boundaries. Intermediate relays do not maintain a per-object cache and cannot replay a partially-forwarded object.¶
Pipelined mode is OPTIONAL. Support is signalled per Node via the
data_modes field in node_info (Section 8.3.1) and requested per
track via the forwarding_mode field in publish_info
(Section 8.5.1).¶
A route from the publisher's Edge Node to a subscriber is admissible for a pipelined-mode
track only if every intermediate Node advertises pipelined-mode support
in its Node Table entry. If no such path exists at the time of a SUBSCRIBE
(or a PUBLISH in publisher-initiated tracks), the publisher's Edge Node
MAY downgrade the track to native mode for that subscriber and MUST
record the downgrade in its local subscribe state so that the
subscriber sees stock MOQT object framing.¶
When a Data Session terminates mid-stream, the upstream Node re-runs path selection (Section 4.4) and opens a Data Session on the replacement path. If a peering closes, the Node removes the Node Table candidates learned over it (Section 4.2), and path selection uses the remaining candidates.¶
In native mode, each affected relay MAY replay cached MOQT objects onto
the new egress subgroup stream so that subscribers see a small gap
rather than a lost group. A subscriber MAY also recover the missing
range with a bounded FETCH ([MoQTransport]) served by any relay
whose cache still contains the range.¶
In pipelined mode, bytes that were in-flight beyond the last completed
MIX stream boundary are lost; the receiving relay MUST NOT attempt to
re-assemble a partially-received MOQT object. The terminal edge relay
signals the loss to the subscriber by advancing to a new group or
object_id as defined by MOQT.¶
In pipelined mode, every datagram and every stream between Nodes begins with a MIX Data Header. A stream also carries one before each later object on it. The header tells the receiving Node which Delivery Node Set the bytes belong to, so it can forward them before the object ends. There are three types of header, using the notation of Section 1.3 of [RFC9000]:¶
datagram_header {
type (8) = 0x00,
header_length (8),
delivery_set_id (i),
data_length (i),
}
new_stream_header {
type (8) = 0x02,
header_length (8),
delivery_set_id (i),
priority (8),
ttl_us (i),
data_length (i),
}
existing_stream_header {
type (8) = 0x01,
header_length (8),
delivery_set_id (i),
}
Which header this is. 0x00 begins a QUIC datagram. 0x02 begins a
QUIC stream and precedes its first object. 0x01 precedes each later
object on the same stream.¶
The number of bytes in the header after this field. A receiver that does not need every field can use it to skip to the payload.¶
The Id of the track's Delivery Node Set on this Data Session (Section 4.5). The receiving Node looks it up to find where to forward the bytes (Section 7.3).¶
The MOQT publisher priority of the subgroup carried on the stream ([MoQTransport]).¶
A lifetime for the stream's objects, in microseconds.¶
The number of payload bytes that follow the header: the datagram's payload, or the first object on a new stream.¶
Editor's note: The meaning of ttl_us is not yet specified. An
existing_stream_header carries no length, so a receiver cannot yet
find where each later object on a stream ends.¶
Native-mode streams do not carry a MIX Data Header; they carry MOQT object frames as defined in [MoQTransport].¶
Each Node derives its egress Delivery Node Sets from its ingress Delivery Node Sets as described in Section 4.5; a destination that is the Node itself is delivered locally. The result maps each ingress Delivery Node Set Id to a set of egress Data Sessions and their Delivery Node Set Ids.¶
For each ingress unidirectional stream, a Node looks up the ingress
delivery_set_id in that mapping (Section 5.1.3), then opens one
egress unidirectional stream per selected downstream Data Session. Egress
streams are opened lazily, on the first byte of the ingress stream, so
that a relay does not consume QUIC flow-control budget on peers that
have no active subscribers.¶
ingress stream for track T, delivery_set_id = 42
|
v
+---------------------------+
| Delivery Node Sets for |
| the ingress Data Session: |
| Id 42 -> {B2, C1} |
| Id 57 -> {D1} |
+-------------+-------------+
|
+---------+---------+
v v
egress to B2 egress to C1
Datagrams are fanned out by rewriting the Delivery Node Set Id per egress and queuing a fresh QUIC datagram frame on each downstream Data Session.¶
In native mode the Delivery Node Set is found by track (full_name_hash) rather than
by Delivery Node Set Id, and selects the set of downstream Data Sessions in the same
way, but the egress carries a MOQT subgroup stream (or datagram,
or FETCH response) as defined by [MoQTransport] rather than a MIX
Data Header stream. The relay preserves track identity and object
locations on egress. Track Alias and request state are mapped to the
local MOQT session.¶
All MIX control messages appear as MOQT object payloads. Structures in this section use the notation of Section 1.3 of [RFC9000], as [MoQTransport] does. Each object carries exactly one message:¶
mix_control_message {
tag (8),
body (..),
}
tag identifies the message type, and body is the structure listed
for that type below. Because MOQT frames each object with a length, the
body needs no length prefix of its own.¶
| Tag | Name | Track | Body | Defined in |
|---|---|---|---|---|
| 0x01 | CONNECT | announce |
node_info
|
Section 8.2 |
| 0x04 | NODE_INFO_ADVERTISEMENT | nodes |
node_info
|
Section 8.3 |
| 0x05 | NODE_INFO_WITHDRAWAL | nodes |
node_info (empty path) |
Section 8.3 |
| 0x06 | SUBSCRIBE_INFO_ADVERTISEMENT | subscribers |
subscribe_info
|
Section 8.6 |
| 0x07 | SUBSCRIBE_INFO_WITHDRAWAL | subscribers |
subscribe_info (empty subscribe_data) |
Section 8.6 |
| 0x08 | PUBLISH_INFO_ADVERTISEMENT | publishers |
publish_info
|
Section 8.5 |
| 0x09 | PUBLISH_INFO_WITHDRAWAL | publishers |
publish_info (empty parameters) |
Section 8.5 |
| 0x0A | DELIVERY_NODE_SET_ADVERTISEMENT | delivery-node-sets |
delivery_node_set_info
|
Section 8.7 |
| 0x0B | DELIVERY_NODE_SET_WITHDRAWAL | delivery-node-sets | empty | Section 8.7 |
| 0x0C | NODE_UNREACH | nodes |
node_unreach
|
Section 8.4 |
All fixed-width integers are in network byte order. A 128-bit field
holds either a Node ID (Section 3) or a track hash (Section 9).
Fields marked (i) use the QUIC variable-length integer encoding
(Section 16 of [RFC9000]).¶
The Node, Publisher, and Subscriber Tables each have one advertisement
and one withdrawal message: NODE_INFO_ADVERTISEMENT and
NODE_INFO_WITHDRAWAL for Node Table, PUBLISH_INFO_ADVERTISEMENT and PUBLISH_INFO_WITHDRAWAL for
Publisher Table, and SUBSCRIBE_INFO_ADVERTISEMENT and SUBSCRIBE_INFO_WITHDRAWAL for Subscriber Table. The
following rules apply to all six:¶
The entry's originator creates and signs the
message (Section 10) and publishes it on its own nodes,
publishers, or subscribers track.¶
A Node that accepts the message (Section 4.1) re-publishes it on its own track of the same kind without changing any signed field, unless its role rules this out: Control Nodes do not re-publish Node Table entries, and in a split deployment Forwarding Nodes do not re-publish Publisher or Subscriber Table entries (Section 12.2.1).¶
A withdrawal uses the same body as the advertisement with its variable-length part emptied, as stated for each message below.¶
Announces a Node to prospective peers. Each Node publishes a CONNECT
on its announce track at startup and refreshes it as described in
Section 5.1.5. Peers use it to discover the Node and to learn
its public key before subscribing to its Node, Publisher, and
Subscriber Table tracks (Section 6).¶
The body is the Node's node_info (Section 8.3.1). CONNECT and
NODE_INFO_ADVERTISEMENT share the same body and signature input
(Section 10) and differ only in the tag byte and the track they
appear on.¶
A NODE_INFO_ADVERTISEMENT states that a Node exists, gives its attributes and
public key, and gives one path to it. A Node originates its own
NODE_INFO_ADVERTISEMENT at startup and again whenever a signed field of its
node_info changes. A Forwarding Node that accepts one re-publishes it,
with its own path element appended, whenever its best path to the
originator changes (Section 4.2). A receiver stores the signed
fields in the Node Table entry for the originator and the path as the sending
peer's candidate.¶
A NODE_INFO_WITHDRAWAL states that a Node is leaving the mesh. Only the Node
itself sends it, with an empty path. A receiver removes the Node Table entry
and all its candidates and re-floods the withdrawal. A Node that has only lost
its path to another Node sends NODE_UNREACH instead
(Section 8.4).¶
The node_info structure is:¶
node_info {
id (128),
type (8),
contact_len (8),
contact (..),
longitude (64),
latitude (64),
pubkey (256),
origin_epoch (64),
sequence (64),
data_modes (8),
path_len (i),
path (..) ...,
origin_sig (512),
}
path_element {
flags (8),
node_id_or_token (128),
srtt_us (64),
hop_count (16),
}
0 for a MIX Transit Node, 1 for a MIX Edge Node, 2 for a MIX
Control Node. Other values are reserved.¶
An optional UTF-8 contact string of contact_len bytes (0 to 253),
without a NUL terminator.¶
Optional location, each an IEEE 754 binary64 value in network byte order.¶
The originator's Ed25519 public key (Section 10).¶
Freshness fields (Section 4.1, Section 10.2).¶
The forwarding modes the Node supports (see below).¶
The number of path elements (0 to 32), followed by that many
path_element structures, ordered from the originator to the most
recent hop (see below).¶
The originator's Ed25519 signature (Section 10).¶
Path element flags (see below).¶
The Node ID of this hop, or an aggregation token when flags bit 0 is set (Section 14).¶
The hop's sRTT to the previous hop, in microseconds, or the sum over an aggregated run.¶
The number of Nodes this element represents: 1, or more for an
aggregated element.¶
The data_modes byte advertises which inter-relay forwarding modes the
Node supports:¶
| Bit | Meaning |
|---|---|
| 0 | Native mode (see Section 7.1). Set by every Forwarding Node. |
| 1 | Pipelined mode (see Section 7.1). |
| 2-7 | Reserved; MUST be sent as 0 and ignored on receive. |
A Forwarding Node MUST set bit 0. A Control Node does not forward
objects and MUST send data_modes = 0. A receiver MUST treat a Node
whose type is a value other than 0, 1, or 2 as a Control Node for
path selection. Bit 1 is set only if the Node's implementation
performs MIX Data Header framing and byte-pipeline forwarding on the
data plane.¶
The flags byte in each path element encodes:¶
| Bit | Meaning |
|---|---|
| 0 | Aggregated: node_id_or_token is an opaque aggregation token |
| 1-7 | Reserved; MUST be sent as 0 and ignored on receive |
hop_count gives the number of Nodes represented by the element. For
non-aggregated elements it is always 1. For aggregated elements it
records the interior hop count so downstream selection can still
compute a meaningful path length. See Section 14.¶
The originator publishes its own NODE_INFO_ADVERTISEMENT with a single path
element for itself (srtt_us = 0, hop_count = 1). Each Node that
re-publishes the advertisement appends its own element, with srtt_us
set to its sRTT to the peer it received the advertisement from.
Elements therefore run from the originator to the most recent hop.¶
The 32-element maximum on path_len bounds propagation diameter and
functions as a hop-count TTL. A Node that receives a NODE_INFO_ADVERTISEMENT
whose path_len is 32 MUST NOT re-flood it (appending its own hop
would exceed the maximum); the entry is still accepted locally. A
receiver MUST discard any advertisement that arrives with
path_len > 32 as malformed.¶
States that the sender no longer has a path to the Node named by
node_id, and retracts the path it last advertised for that Node. A
Node sends it on its own behalf, not an originator's, so it is not signed.
When a Node sends it and how a receiver applies it are specified in
Section 4.2.¶
node_unreach {
node_id (128),
}
The Node ID of the Node that is no longer reachable.¶
A PUBLISH_INFO_ADVERTISEMENT states that a track is available at its originator,
and the forwarding mode the originator requests for it
(Section 7.1). An Edge Node originates one when a local MOQT
publisher makes a track available. A receiver stores it in its Publisher Table.¶
A PUBLISH_INFO_WITHDRAWAL states that the track is no longer available at
the originator. The originator sends it when its local publisher stops
publishing. A receiver removes the Publisher Table entry.¶
The publish_info structure is:¶
publish_info {
sequence (64),
source_node_id (128),
origin_epoch (64),
track_full_name_hash (..),
forwarding_mode (8),
parameters_len (i),
parameters (..),
origin_sig (512),
}
track_full_name_hash {
namespace_hashes_len (16),
namespace_hashes (128) ...,
name_hash (128),
}
Freshness fields (Section 4.1, Section 10.2).¶
The Node ID of the originator.¶
The per-component hashes of the track's namespace tuple
(namespace_hashes_len of them) and of its name (Section 9),
so receivers can do prefix matching. The single-value
full_name_hash used in Publisher and Subscriber Table keys is derived from these
components.¶
The mode the originator selected for this track: 0 for native, 1
for pipelined (Section 7.1). It is covered by the originator
signature (Section 10), and an intermediate Node MUST NOT
rewrite it.¶
Additional MOQT PUBLISH / PUBLISH_NAMESPACE parameters as defined
in [MoQTransport], opaque to MIX.¶
The originator's Ed25519 signature (Section 10).¶
A PUBLISH_INFO_WITHDRAWAL uses the same encoding with parameters_len = 0;
receivers ignore its forwarding_mode field.¶
A SUBSCRIBE_INFO_ADVERTISEMENT states that its originator has local subscribers for
a track. An Edge Node originates one when its first local subscriber
for the track subscribes. A receiver stores it in its Subscriber Table.¶
A SUBSCRIBE_INFO_WITHDRAWAL states that the originator no longer has subscribers
for the track. The originator sends it when its last local subscriber
leaves. A receiver removes the Subscriber Table entry.¶
The subscribe_info structure is:¶
subscribe_info {
sequence (64),
source_node_id (128),
origin_epoch (64),
track_full_name_hash (..),
subscribe_data_len (i),
subscribe_data (..),
origin_sig (512),
}
sequence, source_node_id, origin_epoch, track_full_name_hash,
and origin_sig are as in publish_info (Section 8.5.1).¶
Subscription data of subscribe_data_len bytes.¶
A SUBSCRIBE_INFO_WITHDRAWAL uses the same encoding with
subscribe_data_len = 0.¶
A DELIVERY_NODE_SET_ADVERTISEMENT carries one Delivery Node Set: for one track on the Data
Session whose delivery-node-sets track carries it (Section 5.1.3), the destination
Nodes the track must reach through the downstream Node. The upstream Node of the Data Session sends it, and a later advertisement
for the same Delivery Node Set Id replaces the earlier one. A
DELIVERY_NODE_SET_WITHDRAWAL removes the Delivery Node Set whose Id is given by the
MOQT group it appears in; its body
is empty. Both are local to one Data Session and are not signed.¶
The delivery_node_set_info structure is:¶
delivery_node_set_info {
delivery_set_id (32),
full_name_hash (128),
nodes_len (i),
nodes (128) ...,
}
The Id this Delivery Node Set is published under (Section 4.5).¶
The number of destination Nodes, followed by their Node IDs.¶
All MIX track identifiers (namespace hash, name hash, full-name hash) are 128-bit values computed with keyed SipHash-2-4 [SipHash] in two-word output mode. The SipHash key is a fixed 16-byte value shared by every MIX Node:¶
MIX_SIPHASH_KEY = 4d 49 58 74 72 61 63 6b 68 61 73 68 4b 31 00 00
("MIXtrackhashK1" zero-padded to 16 bytes)
A single fixed key across the network suffices because MIX's security posture treats track hashes as identifiers, not as capabilities: an attacker who can guess a track name learns the corresponding hash and gains nothing beyond what they'd gain by observing an advertisement on the wire. The key is fixed to guarantee interoperability. Deployments MAY additionally rekey by rolling a new protocol version.¶
MOQT track identifiers are tuples: an ordered list of namespace strings plus a name string. MIX hashes each component individually.¶
For a namespace tuple ns = (ns_0, ns_1, ..., ns_k-1) and a name
string nm:¶
namespace_hash_i = SipHash-2-4-128(MIX_SIPHASH_KEY, ns_i) name_hash = SipHash-2-4-128(MIX_SIPHASH_KEY, nm)¶
The namespace_hashes[] field of publish_info and subscribe_info is
exactly this per-component sequence.¶
For contexts that require a single value (Publisher and Subscriber Table keys, Delivery Node Set
lookup), the full_name_hash is:¶
full_name_hash = SipHash-2-4-128(
MIX_SIPHASH_KEY,
ns_0 || 0x00 || ns_1 || 0x00 || ... || ns_{k-1} || 0x00 || nm)
¶
The 0x00 separator prevents boundary-injection attacks (namespaces
("ab","c") and ("a","bc") produce distinct hashes).¶
MIX hashes identify tracks; they are not authenticators. Compared to
the 64-bit FNV-1a64 used in earlier designs, SipHash-128 raises the
cost of accidental or adversarial collisions by many orders of
magnitude: FNV-1a64 collisions can be constructed in seconds, whereas
constructing a 128-bit SipHash collision requires on the order of
2^64 work (birthday bound).¶
The SipHash key is a published constant, so MIX hashes offer no secret-key protection against chosen inputs. Authorization of advertisements is handled separately by originator signatures (Section 10) and operator-level ACLs. Implementations MUST NOT treat MIX hashes as capabilities, authenticators, or authorization tokens.¶
Implementations SHOULD validate their SipHash output against the test vectors in [SipHash].¶
Originator authentication is a core requirement of MIX. Every CONNECT,
NODE_INFO_ADVERTISEMENT, SUBSCRIBE_INFO_ADVERTISEMENT, PUBLISH_INFO_ADVERTISEMENT, and their
respective _WITHDRAWAL variants MUST carry a 64-byte Ed25519 [RFC8032]
signature produced by the originating Node's private key. There is no
unsigned encoding of these messages. NODE_UNREACH (Section 4.2)
is not an originator's advertisement: it retracts only the sender's own path
and is not signed. Every Node MUST verify the signature before
installing the entry in its Node, Publisher, or Subscriber Table and
before re-flooding. Advertisements
that fail signature verification MUST be discarded and MUST NOT be
re-flooded.¶
Signature verification restricts trust in the peering fabric to the
originator of each advertisement rather than to every transit peer. A
compromised transit Node can suppress or delay advertisements it
receives (a routing availability failure) but cannot forge NODE_INFO,
SUBSCRIBE_INFO, or PUBLISH_INFO on behalf of another Node.¶
The signature covers a canonical byte string over the immutable portion
of the advertisement. The mutable portion of a NODE_INFO_ADVERTISEMENT — its
path accumulated during propagation — is excluded from signature
input. Specifically:¶
For CONNECT, NODE_INFO_ADVERTISEMENT, and NODE_INFO_WITHDRAWAL:¶
node_info_sig_input {
domain_separator (..) = "MIX/node_info/v1",
tag (8),
id (128),
type (8),
contact_len (8),
contact (..),
longitude (64),
latitude (64),
pubkey (256),
origin_epoch (64),
sequence (64),
data_modes (8),
}
Note that path is not part of the signature input: paths accumulate
during propagation and cannot be pre-signed by the originator. Therefore,
the originator signature does not protect path contents or metrics. MIX
implementations MUST treat path data as an assertion by the immediate
authenticated peer unless a future version defines per-hop integrity.
Inter-provider deployments MUST NOT use an unsigned path as a sole
authorization or policy signal. Path aggregation does not solve this
integrity limitation.¶
For SUBSCRIBE_INFO_ADVERTISEMENT and SUBSCRIBE_INFO_WITHDRAWAL:¶
subscribe_info_sig_input {
domain_separator (..) = "MIX/subscribe_info/v1",
tag (8),
sequence (64),
source_node_id (128),
origin_epoch (64),
track_full_name_hash (..),
subscribe_data_len (i),
subscribe_data (..),
}
For PUBLISH_INFO_ADVERTISEMENT and PUBLISH_INFO_WITHDRAWAL:¶
publish_info_sig_input {
domain_separator (..) = "MIX/publish_info/v1",
tag (8),
sequence (64),
source_node_id (128),
origin_epoch (64),
track_full_name_hash (..),
forwarding_mode (8),
parameters_len (i),
parameters (..),
}
Each field takes its value from the message being signed, and tag is
that message's tag (Section 8). domain_separator is the
ASCII string shown, with no length prefix or terminator; it prevents a
signature over one message type from being reused for another.
Variable-length integers appear in their shortest encoding.¶
Every advertisement includes a 64-bit origin_epoch set by the
originating Node. It identifies a Node incarnation and MUST increase
before that Node publishes after a restart that could reuse a sequence.
It MUST NOT be derived solely from an unsynchronized wall clock. The
pair (origin_epoch, sequence) distinguishes a new process incarnation
from successive updates in that incarnation; sequence is wide enough
to avoid modular comparison and wrap handling.¶
Freshness comparison is defined in Section 4.1. Withdrawals carry the same originator, key, and freshness fields and an originator signature, so a stale withdrawal cannot erase newer state.¶
The CONNECT object includes the Node's public key and signed
node_info. A receiver MUST check both the signature and the binding
between the public key and Node ID before using the key to validate
advertisements. Deployments MAY also distribute trusted keys out of
band.¶
An implementation MUST NOT install a signed advertisement whose signature verifies against a public key it has never confirmed for that Node ID via at least one of:¶
A prior CONNECT object received over an authenticated, authorized
MOQT session.¶
An out-of-band configuration source (for example, a peering registry).¶
A first-flight trust-on-first-use policy, if local policy explicitly permits it.¶
Deployments in inter-provider settings SHOULD NOT rely on trust-on-first- use.¶
As in BGP [BGP], the Node, Publisher, and Subscriber Tables at each Node hold the most recent advertisement it has accepted for each key, and the mesh has converged when no Node has anything new to publish. To keep re-flooding to a minimum and avoid oscillation, Nodes SHOULD:¶
Suppress repeats. Keep a sent-state record: for each key, the
last advertisement or withdrawal the Node published on its own
nodes, publishers, or subscribers track. BGP calls the
equivalent state the Adj-RIB-Out. A Node does not publish a message
that matches its sent-state record.¶
Ignore sRTT jitter when deciding whether a best path changed materially (Section 4.4).¶
Rate-limit what it publishes on those tracks; a token bucket suffices.¶
Damp flapping keys with the Minimum Route Advertisement Interval (MRAI) extension (Section 15) where it is deployed.¶
These techniques are informative. Conforming implementations MAY use other mechanisms provided that convergence is guaranteed in finite time in the absence of continuing topology change.¶
This section is informative, except Section 12.2.1, which is normative for Nodes in a split deployment. It describes the two deployment shapes for MIX and the trade-offs between them.¶
MIX separates topology state (the Node Table), which grows with the number of Nodes, from per-track state (the Publisher and Subscriber Tables), which grows with the number of tracks and subscribers. The Node Table has to follow the data-plane topology because path selection runs over it; the Publisher and Subscriber Tables do not. Operators can therefore exchange all three tables among the Forwarding Nodes, the simplest shape to operate, or move the exchange of the Publisher and Subscriber Tables to a small, dedicated set of Control Nodes in front of a larger forwarding fabric. The choice shapes failure blast radius, the trust model between control and data roles, and which parts of the operational toolchain are involved in a given MIX event.¶
In the integrated model, every Node is a Forwarding Node and every peering has Full scope (Section 6.1). The same pair of Nodes exchanges Node, Publisher, and Subscriber Tables over a Control Session and, when a track needs to flow, forwards objects over a separate Data Session.¶
--- Control Session: Node, Publisher, and Subscriber Tables === Data Session: Delivery Node Sets and objects +---------+ +---------+ +---------+ +---------+ | Edge A1 |---|Transit X|---|Transit Y|---| Edge B1 | | |===| |===| |===| | +---------+ +---------+ +---------+ +---------+ Every Node exchanges all three tables and forwards objects.
Properties:¶
One role per Node. Each Node holds its own Node, Publisher, and Subscriber Tables and forwards objects for the Delivery Node Set Ids it is responsible for.¶
Failure blast radius = the relay. Losing a Node withdraws its table entries and tears down its data-plane peerings together.¶
Simplest to operate. No Control Nodes to place or provision.¶
Typical use: small-to-medium meshes within a single operator, pre-production environments, and reference implementations.¶
In the split model, a small, redundant set of Control Nodes holds and
distributes Publisher and Subscriber Table state for a larger fabric of Forwarding Nodes.
Forwarding Nodes exchange only their Node Tables with their data-plane neighbors,
so path selection (Section 4.4) runs over the data-plane topology.
Publisher and Subscriber Table entries reach Forwarding Nodes through the Control Nodes.
Each entry names its originator in source_node_id, and a Forwarding Node
finds the path toward that originator in its own Node Table. Control Nodes never
appear in a data-plane path.¶
Peerings are configured as follows:¶
Each Forwarding Node holds Topology peerings with its data-plane neighbors, and Reflection peerings with at least one Control Node. It SHOULD hold Reflection peerings with at least two, so that the loss of one is covered by Section 6.5.¶
Control Nodes hold Reflection peerings with each other and with the Forwarding Nodes they serve.¶
--- Control Session === Data Session
Publisher and
+------+ Subscriber Tables +------+ Control Nodes:
| C1 |--------------------| C2 | Publisher and
+------+ +------+ Subscriber Tables only,
| | no objects
+------+------+-------------+------+------+
| | | |
+---------+ +---------+ +---------+ +---------+
| Edge A1 |---|Transit X|---|Transit Y|---| Edge B1 |
| |===| |===| |===| |
+---------+ +---------+ +---------+ +---------+
Reflection peerings (to C1 and C2): Publisher and Subscriber
Tables, plus the Node Table toward the Control Nodes. Topology
peerings between Forwarding Nodes: Node Table only. Data
Sessions: Delivery Node Sets and objects.
Properties:¶
Two roles. Control Nodes hold Publisher and Subscriber Tables for the whole domain. Forwarding Nodes hold the full Node Table, which grows only with the number of Nodes, plus the Publisher and Subscriber Table entries the Control Nodes send them.¶
Failure blast radius is split. Losing one Control Node is covered by the remaining ones (Section 6.5). Losing a Forwarding Node closes its Topology peerings; its neighbors withdraw it from Node Table and re-run path selection without involving the Control Nodes.¶
Independent scaling. Per-track state scales on the Control Nodes; Forwarding Nodes scale with the number of Nodes and concurrent flows. In the integrated model every Node carries both.¶
Typical use: large multi-site deployments, operators who already separate control and forwarding planes elsewhere in their stack, and inter-provider interconnects where the control-plane trust surface is intentionally smaller than the forwarding fabric.¶
Three things a split deployment needs are out of scope for this specification and are left to configuration or other specifications: how a Forwarding Node finds the Control Nodes for its domain and replaces one it has lost; how a Control Node verifies that a Forwarding Node may inject Publisher and Subscriber Table entries for its local publishers and subscribers, beyond the originator signature; and which Publisher and Subscriber Table entries a Control Node sends to which Forwarding Nodes, since sending every entry to every Forwarding Node scales poorly.¶
Control Nodes reflect Publisher and Subscriber Table entries in a single hop over a full mesh, in the manner of BGP route reflection [RR]. The Control Nodes serving a domain MUST hold a Reflection peering with every other Control Node in that domain.¶
A Control Node publishes two tracks in addition to publishers and subscribers:
publishers-mesh and subscribers-mesh (Section 5). Its Forwarding Node
peers subscribe to publishers and subscribers; its Control Node peers subscribe
to publishers-mesh and subscribers-mesh (Section 6.1). Which accepted
entries a Node re-publishes depends on its role and where it learned
the entry:¶
| Node | Entry learned from | Re-published on |
|---|---|---|
| Control Node | A Forwarding Node |
publishers/subscribers and publishers-mesh/subscribers-mesh
|
| Control Node | A Control Node (publishers-mesh/subscribers-mesh) |
publishers/subscribers only |
| Forwarding Node | Any Reflection peering | Not re-published |
A Control Node needs two pairs of tracks because a MOQT track delivers
the same objects to every subscriber. Its clients need every entry it
holds, while other Control Nodes need only the entries learned from its
own clients. If other Control Nodes subscribed to publishers and subscribers
instead, each would re-publish what it received, and every Control Node
would receive every entry once from each of the others. Nothing would
loop, because copies that are not newer are dropped
(Section 4.1), but each copy still costs a signature check.
With the mesh tracks, an entry from Edge A, a client of C1, travels as
follows:¶
+---------+
| Edge A | originator, a client of C1
+---------+
| publishers
v
+---------+
+----------| C1 |-----------+
| +---------+ |
| publishers publishers-mesh |
v v
+-------------+ +-------------+
| C1's other | | C2, C3 |
| clients | +-------------+
+-------------+ | publishers only,
| never publishers-mesh
v
+-------------+
| C2's and |
| C3's clients|
+-------------+
Because entries learned from another Control Node never return to the
mesh, every entry crosses at most two Control Nodes on its way from
originator to Forwarding Node. A Forwarding Node that holds a Reflection
peering MUST NOT hold Full peerings, so its publishers and subscribers tracks
carry only the entries it originates.¶
The same entry normally arrives more than once: a Control Node
receives it from each Control Node that serves its originator, and a
Forwarding Node receives it from each Control Node it peers with. A
copy whose (origin_epoch, sequence) equals the stored one is not
accepted as new (Section 4.1) and is not re-flooded.¶
The sections before this one, other than those marked informative, form the mandatory MIX core. Every conforming implementation MUST implement them, except for features marked OPTIONAL where they are defined. The sections that follow define OPTIONAL extensions; a Node MAY implement any subset. Each extension is self-describing on the wire (via existing capability bits, path-element flags, or policy configuration) so that a Node without a given extension interoperates unchanged with a Node that has it.¶
This extension allows a Transit Node to replace a contiguous run of
path elements with a single aggregated element. The aggregated
element carries an aggregator-scoped token (Section 14.1) in place of
the interior Node IDs, and the sum of the interior sRTTs in place of
the per-hop values. Loop detection (Section 14.3) and
best-path selection continue to work across the aggregated element.¶
Path aggregation is intended for an operator that provides transit for
third parties and does not want to reveal the interior topology of its
network to external peers. It also shortens path on
advertisements sent across wide-area peerings. In a single-operator
mesh it provides no benefit and SHOULD be disabled.¶
A Transit Node MAY aggregate a contiguous run of path elements that it originated or transited, provided all of those elements belong to a single administrative aggregation domain recognized by local policy.¶
The aggregation token is a 128-bit HMAC-SHA-256 output truncated to the first 16 bytes so that it shares the wire slot with a Node ID:¶
agg_token = truncate128(HMAC-SHA-256(
key = aggregator_secret,
input = "MIX/agg/v1" || aggregator_scope_id))
¶
aggregator_secret is a per-aggregator 32-byte value SHOULD be rotated
periodically. aggregator_scope_id is a 128-bit value identifying the
aggregation domain (typically the aggregator's own Node ID, but a
distinct value MAY be used when a single operator runs multiple
disjoint aggregation domains).¶
agg_token is stable per aggregator for the lifetime of
aggregator_secret, which allows a downstream Node to detect a loop
that transits the aggregator twice: the same agg_token reappears in
the accumulating path.¶
When a Transit Node V receives a NODE_INFO_ADVERTISEMENT and re-floods it to peer P,
V MAY apply aggregation:¶
Let interior be the contiguous suffix of path whose elements
V considers part of its own aggregation domain, followed by V's own
about-to-be-appended element.¶
If aggregation to P is permitted by local policy AND |interior| > 1,
V replaces interior in the outbound path with a single element:¶
flags = 0x01 ; aggregated bit set node_id_or_token = agg_token ; V's aggregation token srtt_us = sum(srtt_us over interior) hop_count = sum(hop_count over interior)¶
Otherwise, V appends its own element unchanged:¶
flags = 0x00 node_id_or_token = V.node_id srtt_us = observed sRTT to previous hop hop_count = 1¶
Aggregation MUST NOT re-order path elements outside the aggregated
suffix. Aggregation MUST NOT alter path elements that are already
aggregated by a different aggregator (recognizable by their flags
being set with a different node_id_or_token).¶
A receiver detects a loop as follows:¶
For a non-aggregated element (flags bit 0 clear), the receiver checks
whether node_id_or_token equals its own Node ID.¶
For an aggregated element (flags bit 0 set), the receiver checks
whether node_id_or_token equals any agg_token derived from its own
aggregator_secret and any aggregator_scope_id it uses.¶
If either check matches, the advertisement is a loop: it removes the sending peer's candidate instead of adding one (Section 4.2).¶
Best-path selection (Section 4.4) SHOULD use the sum of hop_count
across path elements as the "path length" comparison metric, not
path_len (the number of encoded elements). This keeps aggregated and
non-aggregated advertisements comparable.¶
Aggregation is a local policy decision at each Transit Node. Implementations MUST provide operator control over:¶
Whether aggregation is enabled at all.¶
Which peers receive aggregated advertisements (typically peers outside the aggregator's administrative domain).¶
The aggregator_secret and its rotation schedule.¶
A downstream Node cannot un-aggregate an aggregated path element. Its best-current selection algorithm operates on the aggregated metrics directly.¶
This extension, named after BGP's Minimum Route Advertisement Interval
(MRAI) [BGP], limits how often a Node re-advertises a key to a peer.
After sending an advertisement for a key to a peer, the Node holds
further advertisements for that key to that peer until a timer of
mrai_ms milliseconds expires (Section 15.1). Each key therefore
changes at most 1000 / mrai_ms times per second on each peering. The
sent-state record (Section 11) alone does
not provide this bound, because a key that genuinely flaps differs from
the record on every transition.
Withdrawals are not delayed (Section 15.2), so failures still
propagate immediately, and the worst-case convergence time is bounded
(Section 15.4).¶
MRAI is intended for inter-provider peering and for any deployment in which advertisement flaps from a peer could impose measurable work on the receiver. In a single-operator mesh, a Node MAY omit this extension.¶
An implementation of this extension maintains a per-peer configuration
value mrai_ms, an integer number of milliseconds. Deployment guidance:¶
| Deployment | Recommended mrai_ms |
|---|---|
| Closed-mesh, single operator |
0 (disabled) |
| Inter-provider peering |
>= 250
|
mrai_ms = 0 disables damping and reduces MRAI behavior to that of
the sent-state record plus the token-bucket rate limiter alone.
Closed-mesh deployments MAY use mrai_ms = 0 since keys are unlikely
to be adversarially flapped and every millisecond of extra latency
matters.¶
For each peer P and each key K (a Node ID for Node Table, or a
(source_node_id, full_name_hash) pair for Publisher and Subscriber Tables), a Node
maintains one timer T[P,K].¶
On sending an advertisement to P for K, the Node records the current time as the last-send time.¶
Before sending any subsequent advertisement to P for K, the Node checks
now - last_send >= mrai_ms. If not, the send is deferred until the
timer expires; only the most recent deferred state (if any) is
transmitted when it does.¶
If the sent-state record (Section 11) suppresses the send outright (state unchanged from last transmitted), the timer is not restarted.¶
A NODE_INFO_WITHDRAWAL, NODE_UNREACH, SUBSCRIBE_INFO_WITHDRAWAL, or
PUBLISH_INFO_WITHDRAWAL MUST NOT be held by the MRAI timer. Withdrawals are sent
immediately and reset
T[P,K] so that a subsequent advertisement for the same key is delayed by
mrai_ms (this preserves the anti-flap property even across a withdrawal-advertisement
cycle).¶
MRAI, the sent-state record, and the token-bucket rate limiter compose:¶
For a network with maximum diameter D hops and MRAI mrai_ms, the
worst-case convergence time for a single state change is
D * mrai_ms + propagation. Operators SHOULD size mrai_ms with this
in mind; a value larger than one round-trip across the mesh may cause
observable stall during correlated failures.¶
TODO¶
Initial version.¶
The design of MIX was informed by earlier work on inter-relay peering, in particular [LAPS], which explored path-appending Node Table, sequence-gated PIB/SIB, and session-scoped Subscribe Node Set (SNS) concepts. The authors thank the LAPS authors for their contributions to the problem space.¶