| Internet-Draft | Available Session Recovery Protocol | September 2026 |
| Luo & Yan | Expires 18 March 2027 | [Page] |
This document describes an experimental protocol named the Available Session Recovery Protocol (ASRP). The protocol is designed to optimize high-availability network cluster architectures, providing a superior high-availability solution for clusters offering stateful network services such as load balancing and Network Address Translation (NAT [RFC4787]). ASRP defines the procedures for session backup and recovery, as well as the message formats used during these interactions, enabling efficient and streamlined session state management.¶
In contrast to traditional high-availability techniques that back up session state within the cluster itself, the core innovation of ASRP lies in its distributed backup of state information to the client or server side. This approach offers multiple advantages: theoretically unlimited elastic scaling capacity; support for rapid recovery from multi-point failures; reduction of resource redundancy through the elimination of centralized backup nodes; and significant simplification of cluster implementation complexity.¶
The ASRP protocol provides a standardized method for constructing elastic service clusters, facilitating broader participation from software and hardware developers in building elastic cloud network service clusters.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 18 March 2027.¶
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Traditional high-availability network clusters based on a master-backup architecture rely on session state synchronization between the master and backup nodes. While functionally complete, this architecture faces challenges in the cloud era, such as insufficient flexibility for elastic scaling, resource redundancy, and high implementation complexity. To address these challenges, the industry has moved to the Elastic Stateful Cluster.¶
An Elastic Stateful Cluster is a high-availability network service cluster composed of multiple active nodes whose number can be elastically scaled, providing stateful network services such as load balancing (SLB) and Network Address Translation (NAT). In such a cluster the hard problem is session state: a design has to decide where the state lives, and how a node that receives a packet of a session it does not hold recovers that session.¶
High-availability mechanisms evolved together with the deployment model they serve.¶
Master/backup HA (e.g., VRRP [RFC5798]) solved the failover problem of the master/backup era: it provides fast takeover of a shared address between two nodes, with session continuity typically left to private synchronization between the pair. The model fits a two-node deployment. In a multi-node cluster where every node forwards traffic simultaneously, there is no natural mapping onto a master/backup pair, and the failover unit no longer corresponds to the service unit.¶
The industry answer is the Elastic Stateful Cluster itself: all nodes are active, and session state is shared through intra-cluster synchronization (e.g., LVS connection synchronization daemons, DPVS), typically as full-mesh or master/replica replication. Conventional Elastic Stateful Clusters realize this with a Fast/Slow Path design that separates session management from packet forwarding:¶
+--------------------------+
| +----------------------+ |
| | ... | |
| | ... | |
| | +----------------+ | |
| | | Slow Path Node | | |
| | +----------------+ | |
| | ... | |
| | ... | |
| +----------------------+ |
| ^ | |
| | V |
| +----------------------+ |
| | ... | |
+----------+ | | ... | | +----------+
| | | | +----------------+ | | | |
| Client | <--------> | Fast Path Node | <--------> | Server |
| | | | +----------------+ | | | |
+----------+ | | ... | | |
| | ... | |
| +----------------------+ |
+--------------------------+
The slow path nodes are responsible for session creation and synchronization, while the fast path nodes are responsible for rapid packet forwarding. The drawback is the weak elastic scaling capability of the slow path nodes: new sessions must be propagated to other nodes so that asymmetric traffic can be handled anywhere, the synchronization traffic grows with the session creation rate and the node count, and a newly added node starts without state. A typical implementation reference is the AWS Hyperplane NFV platform.¶
Stateless scheduling (e.g., Maglev-style consistent hashing, ECMP [RFC2991] [RFC2992]) avoids session state on the nodes entirely by hashing each flow to a node. Node failure is handled by rehashing flows, which works only when the service can tolerate a flow landing on a node that holds no state for it; stateful services such as NAT cannot.¶
These mechanisms are what make elastic clusters work in practice, but they are private. The contrast with the previous generation is telling: the master/backup architecture is served by a standard (VRRP), while the Elastic Stateful Cluster - the architecture that large deployments actually use today - has no standard protocol for its session-state layer. Routing, failure detection, and address takeover are individually standardized; session synchronization and recovery are not, and every production cluster ships its own.¶
ASRP fills this gap by removing the session state from the cluster altogether: the backup of a session is stored at one of the session's own endpoints, nodes share no protocol state with each other, and recovery state is pulled on demand (QS/RS) only when a node actually needs a session it does not have. This addresses the hardest problem of the elastic cluster and, as a side effect, simplifies the cluster itself: no inter-node protocol, no synchronization traffic, and no state warm-up for a newly added node. The tradeoff is that the client or server side must support the protocol. Within a single administrative domain - a data center for load balancing, an enterprise edge for SNAT - this is a deployment decision rather than a protocol limitation.¶
Table 1 summarizes the comparison.¶
| Architecture | Elasticity | Implementation | Standard protocol |
|---|---|---|---|
| Master/backup HA | No (backup idles) | Private pairwise sync | VRRP [RFC5798] |
| Elastic Stateful Cluster (conventional) | Fast path: easy; slow path: hard | Complex (intra-cluster synchronization) | None (private implementations) |
| Elastic Stateful Cluster (ASRP) | Yes (all nodes uniform) | Simple (state kept outside the cluster) | This document |
+----------------------+
| ... |
+----------+ | ... | +----------+
| | | +----------------+ | | |
| Client | <--------> | ASRP Node | <--------> | Server |
| | | +----------------+ | | |
+----------+ | ... | +----------+
| ... |
+----------------------+
The Available Session Recovery Protocol (ASRP) proposes an innovative high-availability solution, aiming to provide a standardized method for constructing elastic service clusters. This facilitates broader participation from software and hardware developers in building elastic cloud network service clusters. Its core idea is to innovatively distribute session state information to the client or server. The lifecycle of the backup state is synchronized with the real session, eliminating the need for independent keepalive and timeout mechanisms. This design ensures the timeliness and availability of the backup information.¶
ASRP defines corresponding session backup and recovery mechanisms. The protocol allows protocol messages to be transmitted together with the original service data packets, thereby reducing control overhead for state synchronization. In an elastic stateful cluster built on ASRP, network nodes possess atomic and mutually independent properties. There is no need for communication between nodes, nor is session synchronization required within the cluster. This fundamental design provides theoretically unlimited scaling capability and supports rapid recovery from multi-point failures.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
This document uses the following terms:¶
Network Node: A device that provides stateful network services (e.g., load balancing, NAT) and creates and maintains sessions for the traffic it forwards.¶
Client / Server: The two endpoints of a session. Depending on the operational mode, the server (PSV mode) or the client (ACT mode) also holds the backup of the session.¶
Backup Holder: The client or server that stores the backed-up state of a session.¶
Session: The state a network node maintains for a flow, keyed by the flow's 5-tuple.¶
Explicit First Packet: A packet whose characteristics explicitly indicate the start of a session, e.g., a TCP SYN or a DNS query.¶
Inline Message: An ASRP message carried within a packet of the flow itself, preceded by the ASRP Signature.¶
Standalone Message: An ASRP message carried in an independent UDP packet whose destination port is ASRP-PORT.¶
ASRP-PORT: The UDP destination port used for standalone ASRP messages.¶
For the ASRP protocol to function correctly, two prerequisites must be met. First, all network nodes within the cluster MUST run service software supporting the ASRP protocol. Second, the server or client responsible for backing up sessions MUST deploy a kernel module or an eBPF module that supports ASRP. Depending on whether this module is deployed on the server or the client, the protocol operates in one of two corresponding modes: Passive (PSV) Mode and Active (ACT) Mode.¶
In PSV mode, the network node is typically deployed in the same network domain as the server (e.g., inside a data center). Its typical service is load balancing.¶
In ACT mode, the network node is typically deployed in the same network domain as the client (e.g., an enterprise intranet). Its typical service is Source Network Address Translation (SNAT).¶
The operational mode (PSV or ACT) is determined by the deployment configuration of the network node and is not carried in ASRP messages.¶
Elastic
Stateful
Cluster
+------------------+
+----------+ | ... | +----------+
| | | +------------+ | | |
| Client | <----------> | node X | <----------> | Server |
| | | +------------+ | | |
+----------+ | ... | +----------+
+------------------+
Symmetric routing refers to the path mode where bidirectional traffic of the same session between a client and a server is always routed to the same node within the cluster.¶
Elastic
Stateful
Cluster
+------------------+
| ... |
+----------+ | +------------+ | +----------+
| | -----------> | node X | -----------> | |
| | | +------------+ | | |
| Client | | ... | | Server |
| | | +------------+ | | |
| | <----------- | node Y | <----------- | |
+----------+ | +------------+ | +----------+
| ... |
+------------------+
Asymmetric routing refers to the scenario where bidirectional traffic of the same session may be routed (e.g., by mechanisms such as ECMP [RFC2991], [RFC2992]) to different nodes within a cluster. In cloud networking environments, asymmetric routing is a common phenomenon, which imposes higher demands on the implementation of elastic stateful clusters.¶
ASRP achieves distributed backup and recovery of session state information by exchanging specific protocol messages among the client, server, and network nodes (such as load balancers or NAT devices). In a load-balancing scenario, session state is distributed and backed up to individual servers; in an SNAT scenario, session state is distributed and backed up to individual clients.¶
ASRP defines the following protocol messages: New Session message (NS), New Session Acknowledge message (NA), New Session Conflict message (NC), Query Session message (QS), Recover Session message (RS), Recovery no-session message (RX), Hello Session message (HS) and Push Session message (PS).¶
Generated by the network node, it is used to send session state information to a designated client (in ACT mode) or server (in PSV mode) for backup when creating a new session.¶
Generated by the server as a response to an NS message in PSV mode, acknowledging the receipt of the NS message.¶
Generated by the client or server when the session state carried in a received NS message conflicts with an existing local session, indicating the session conflict to the network node.¶
Generated by the network node, it is used to query the client or server for backup session state information when a received packet cannot match any local session and a session cannot be directly created.¶
Generated by the client or server holding the backup as a response to a QS message, it contains the state information required to recover the session. The network node parses the RS message and reconstructs or marks the local session, thereby achieving failure recovery.¶
Generated by the client or server holding the backup as a response to a QS message, indicating that the session queried by the QS message was not found. Except for the MsgType field, the RX message is identical to the corresponding QS message.¶
Generated by the client, it is used in ACT mode to announce to the network node its capability to support the ASRP protocol and to trigger the network node to return an NS message to complete session backup.¶
Generated by the server, it is used in PSV mode to push session state information to the network node. In the case of asymmetric routing, the network node utilizes the PS message to create/update sessions for fast packet forwarding.¶
ASRP messages can be transmitted in two modes: Inline Message and Standalone Message.¶
The Inline Message is transmitted with the same 5-tuples as the flow, either inserted into the original packet or carried in a separate packet that copies the original packet header. When the total packet length does not exceed the MTU, NS, HS, and PS messages SHOULD be inserted into the original packet; otherwise, they are carried in a separate packet. HS messages are sent after forwarded packets, and PS messages are sent before forwarded packets.¶
Traveling with the flow's 5-tuple is a structural requirement for HS and PS: both are sent from an endpoint towards the cluster, and under asymmetric routing the sender cannot predict which network node handles the corresponding direction of the flow. Since per-flow scheduling (e.g., ECMP hashing) maps one 5-tuple to one node, a message carried with the flow's 5-tuple - whether inserted into a packet of the flow or in a separate packet copying its header - is delivered to the same node as the flow itself, which is the node that holds (for HS) or needs (for PS) the session. NS, by contrast, is sent from a network node to a known endpoint (the selected server in PSV mode, or the client in ACT mode) and can also be transmitted as a Standalone Message; carrying it inline is a performance optimization that piggybacks on a packet about to be forwarded.¶
Inserting a message into the first packet of a session makes that packet carry data (e.g., a TCP SYN with a non-empty payload). Some middleboxes drop such packets; this behavior has been observed in the deployment of TCP Fast Open [RFC7413]. Where such interference is known or detected on a path, the implementation SHOULD use the separate-packet form instead of inserting into the original packet, since the separate packet carries the message without altering the characteristics of the session's own packets.¶
The Standalone Message is transmitted in an independent packet with its own 5-tuples. It is encapsulated using IP/UDP [RFC0768], with IP addresses configured to ensure mutual reachability and a fixed destination port ASRP-PORT (e.g., 51200). QS, RS, RX, NA and NC messages typically use this transmission mode.¶
In Inline Message mode, an ASRP Signature is used to indicate that a packet contains an ASRP message.¶
Inspired by the Proxy Protocol, an 8-byte ASRP Signature is used:¶
0x41, 0x53, 0x52, 0x50, 0x0D, 0x0A, 0x00, 0xF1.¶
The signature consists of the ASCII string "ASRP" followed by 0x0D, 0x0A, 0x00, 0xF1. The "ASRP" prefix keeps the marker readable and distinct from other in-stream protocols; the trailing bytes reduce the chance of a false match against application data that happens to begin with the ASCII string.¶
The ASRP Signature is a fixed marker for message identification; it is not a cryptographic signature. It is inserted in front of the ASRP message. In Standalone Message mode, the UDP destination port is ASRP-PORT, and the ASRP Signature is not required.¶
This section elaborates on, through a series of typical scenarios, how the ASRP protocol achieves session backup and recovery via message interaction in the event of network node failures under different operational modes. Each scenario details the involved protocol message flows and the processing steps of each entity.¶
ASRP Cluster
+----------+ +--------------+ +----------+
| | ---1:PKT--> | | -----2:NS----> | |
| | | | <----3:NC----- | |
| client | | ASRP Nodes | | server |
| | | | | |
| | <--5:PKT--- | | <----4:PS----- | |
+----------+ +--------------+ +----------+
This scenario describes that, in PSV mode, a network node receives an explicit first packet (i.e., a packet whose characteristics explicitly indicate the start of a session) and directly creates a session flow. Common examples of explicit first packets include TCP SYN [RFC9293] and DNS [RFC1034] [RFC1035] query, among others.¶
The processing flow is as follows:¶
Session Creation: Upon receiving a packet from the client (e.g., TCP SYN), the network node first creates a new session and then sends an NS message to the selected server.¶
Server Response: Upon receiving the NS message, the server stores the session state information and associates it with its local session. If a conflict occurs, it sends an NC message. In the case of asymmetric routing, when sending its first response packet, the server sends a PS message to the network node.¶
Session Recovery: Upon receiving the PS message, the network node restores the session and forwards packets accordingly.¶
The session state information backed up by the server is released upon local session termination, without requiring any additional teardown message.¶
ASRP Cluster
+----------+ +--------------+ +----------+
| | | | <----1:PKT---- | |
| | | | | |
| client | <--4:PKT--- | ASRP Nodes | -----2:QS----> | server |
| | | | | |
| | | | <---3:RS/RX--- | |
+----------+ +--------------+ +----------+
This scenario describes the session recovery flow triggered by a server packet.¶
The processing flow is as follows:¶
Session Query: Upon receiving a packet from the server, the network node searches its local session table. If no matching session is found, the node SHOULD first buffer the packet for forwarding, then sends a QS message back to the server.¶
Server Response: After receiving the QS message, the server looks up the backed-up session state information and sends an RS/RX message.¶
Session Recovery: Upon receiving an RS message, the network node creates a new local session and forwards packets accordingly. Upon receiving an RX message, the node SHOULD discard the pending packets.¶
ASRP Cluster
+----------+ +------------+ +------------+
| | | | ----2:QS-----> | ... |
| | ---1:PKT--> | | <---3:RS/RX--- | +--------+ |
| | | | ... | | server | |
| | | | ... | +--------+ |
| client | | ASRP Nodes | ---4:PKT/NS--> | ... |
| | | | | +--------+ |
| | | | | | server | |
| | <--7:PKT--- | | <--5:NA/NC---- | +--------+ |
| | | | <----6:PS----- | ... |
+----------+ +------------+ +------------+
This scenario describes the session creation/recovery flow triggered by a client packet.¶
The processing flow is as follows:¶
Query Local Session: Upon receiving a packet from the client, if no matching session is found, the network node first obtains a list of candidate servers (possibly multiple) for querying.¶
Query Backup Session: The network node sends QS messages to each candidate server. Each server replies with an RS or RX message.¶
Process Query Results: If a session is found, the network node restores the session and forwards the packet. Otherwise, for TCP packets: drop the packet. For UDP packets: create a new local session and send an NS message to the selected server.¶
Server Creates New Session: Upon receiving an NS message, the server stores the session state information and associates it with its local session. If a conflict occurs, it sends an NC message; otherwise, if the F-NA flag is set, it immediately replies with an NA packet.¶
Session Recovery: In an asymmetric routing environment, when sending its first response packet, the server sends a PS message to restore the session at the network node.¶
In this scenario, obtaining the list of candidate servers is a key challenge. Two solutions are proposed:¶
The network node derives an ordered list of candidate servers from the packet's 5-tuple using a deterministic server-selection algorithm, such as consistent hashing or history-aware consistent hashing, and sends QS messages to the candidates in order.¶
Enhance the client so that client packets carry backend server information, allowing the network node to directly extract the target server from the client packet. One such enhancement is specified in [I-D.draft-cmcc-tcp-sro], which carries the selected backend identity in a TCP option.¶
For the first solution, the specific algorithm is a local implementation choice; what the protocol relies on are three properties of it:¶
Determinism: The mapping from a session's 5-tuple to its candidate list MUST be identical on every node of the cluster, so that any node receiving a packet of a session it does not hold computes the same candidates to query.¶
History awareness: When the backend pool changes (e.g., a server is added), the algorithm SHOULD keep the servers that previously served the affected hash region in the candidate list, so that sessions created before the change can still locate their backup holder. A removed server simply fails to answer; the recovery attempt moves on to the next candidate or is abandoned.¶
Bounded querying: The number of candidates queried for one recovery MAY be limited to a small constant, bounding the QS traffic and the recovery latency. When the limit is reached without an RS response, the node stops querying and falls back to its local handling of packets without a session (Section 3.6).¶
ASRP Cluster
+----------+ +--------------+ +----------+
| | -----1:HS----> | | | |
| | <----2:NS----- | | ---3:PKT--> | |
| client | | ASRP Nodes | | server |
| | <----5:QS----- | | <--4:PKT--- | |
| | ---6:RS/RX---> | | | |
+----------+ +--------------+ +----------+
This scenario describes session creation at a network node and server-initiated session restoration.¶
The processing flow is as follows:¶
Session Backup: When sending a packet without receiving an NS message, the client sends an HS message to the network node to request session backup (the client MAY rate-limit the HS messages it generates). Upon receiving the HS message, the node replies with an NS message.¶
Session Lookup: For packets from a server with no matching session, the network node identifies the target client and sends it a QS message.¶
Session Recovery: Upon receiving QS, the client sends an RS/RX message to the network node to restore the session.¶
In step 2, identifying the target client is challenging. Two solutions are proposed:¶
ASRP Cluster
+----------+ +--------------+ +----------+
| | ----1:PKT----> | | | |
| | | | | |
| client | <----2:QS----- | ASRP Nodes | ---4:PKT--> | server |
| | | | | |
| | ---3:RS/RX---> | | | |
+----------+ +--------------+ +----------+
This scenario describes the client-packet-triggered session recovery.¶
The processing flow is as follows:¶
ASRP does not need a distributed consistency protocol, because the session state has a single writer at any time and the backup is a read-only copy.¶
The backup of a session is created by the NS message at session creation and is never modified afterwards; it is released when the local session terminates. It therefore reflects the session state at creation time. For the state that ASRP targets - the flow-defining state of load balancing and NAT sessions, keyed by the session's 5-tuple - this snapshot remains valid for the lifetime of the session, since the key does not change. Implementations that maintain additional mutable state (e.g., timeouts or counters) recover a usable approximation of it.¶
Recovery is a local action of one network node: the node rebuilds the session from the backup delivered in an RS message and thereafter owns the session. If, under asymmetric routing, two nodes receive packets of the same session and each recovers it independently, both hold the session locally and each forwards the packets that reach it - which is exactly how an active-active cluster treats any session it holds. No node writes to another node's session state, so there is no split-brain condition to resolve.¶
If recovery fails (an RX response, or no response before the recovery attempt is abandoned), the network node falls back to its local handling of packets without a session, e.g., discarding them or creating a new session.¶
An ASRP message consists of ten fields in total. The message header has a fixed length and is composed of the first six fields. The message body comprises the remaining four fields: the fixed-length Protocol and Cookie fields, and the variable-length Session-Tuple and Session-Data fields. The fields are defined as follows:¶
1. Version: 1 octet. This document defines version 0 (Ver0).
2. MsgType: 1 octet, the message type: NS=0, NA=1, QS=2, RS=3,
RX=4, HS=5, PS=6, NC=7.
3. STType: 1 octet, the Session-Tuple layout carried in the
message body: NULL=0 (no Session-Tuple), ST44=1, ST66=2,
ST46=3, ST64=4, ST4=5, ST6=6.
4. Flags: 1 octet. Bits are allocated from the least significant
bit:
- F-LIP6 (0x1): the Node-Local-IP carried by the NS message is
an IPv6 address.
- F-CC (0x2): client-side session conflict (NC message).
- F-NA (0x4): NA acknowledgment requested (NS message).
Unassigned bits MUST be sent as zero and ignored on receipt.
5. Length: 2 octets, the total length of the ASRP message in
octets, from the first octet of the Version field through the
last octet of the message body.
6. Reserved: 1 octet, reserved for future use. It MUST be sent as
zero and ignored on receipt.
7. Protocol: 1 octet, transport-layer protocol (e.g., TCP, UDP).
8. Cookie: 8 octets, an opaque value used for message validation.
The sender of a request message SHOULD fill this field with a
per-request random value; the responder MUST echo it unchanged.
See "Cookie Handling" and the Security Considerations.
9. Session-Tuple (ST): source and destination addresses and ports.
The IP address type is IPv4/IPv6.
There are 6 types of ST, as follows:
- ST4: IPv4-only tuple;
- ST6: IPv6-only tuple;
- ST44/ST66: Pairs of ST4 or ST6;
- ST46/ST64: Mixed IPv4/IPv6 tuples.
10. Session-Data (SD): opaque session state information.
¶
Rules for validating these fields on receipt are specified in "Message Validation".¶
ST4(length 12) Format:¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source IP (IPv4) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination IP (IPv4) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
ST6(length 36) Format: Same structure as ST4, but with IPv6 ST44(length 24) Format: ST4 pair ST66(length 72) Format: ST6 pair ST46(length 48) Format: Mixed ST pair(ST4->ST6 sequence) ST64(length 48) Format: Mixed ST pair(ST6->ST4 sequence)¶
For messages carrying two Session-Tuples (NS, NA, NC, RS, and PS), the first Session-Tuple (ST1) MUST be the client-side tuple, and the second Session-Tuple (ST2) MUST be the server-side tuple. For messages carrying a single Session-Tuple (QS and RX), the Session-Tuple MUST be the 5-tuple of the triggering packet.¶
The NS message contains a Node-Local-IP field and two Session-Tuples.¶
The Node-Local-IP field is located at the end of the NS message and carries the local IP address of the network node that sends the NS message. Its length is 16 octets when the F-LIP6 flag is set, and 4 octets otherwise. The client or server uses this address as the destination IP to send the NA message back to the network node.¶
NS Message Format:¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | MsgType | STType | Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length | Reserved | Protocol | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Cookie | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Session-Tuples: ST44/ST66/ST46/ST64 ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ Session-Data ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ Node-Local-IP (IPv4 or IPv6) ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The two Session-Tuples represent the network node's connections with the client and the server, respectively.¶
The NA message acknowledges the receipt of an NS message that carries the F-NA flag.¶
The NA message has the same structure as the NS message, except that the Node-Local-IP field is not present.¶
The structure of the message is the same as that of the NA message.¶
In the NC message, the client-side Session-Tuple remains unchanged, while the server-side Session-Tuple may be modified to reflect the conflicting session. The F-CC flag indicates whether the conflict is on the client side or the server side.¶
QS Message Format:¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | MsgType | STType | Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length | Reserved | Protocol | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Cookie | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Session-Tuple: ST4 or ST6 ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ Session-Data ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
RS Message Format: The structure of the message is the same as that of the NA message.¶
RX Message Format: The structure of the message is the same as that of the QS message.¶
HS Message Format:¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | MsgType | STType | Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length | Reserved | Protocol | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Cookie | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
PS Message Format: The structure of the message is the same as that of the NA message.¶
A packet that carries a single ASRP message is referred to as an ASRP packet. Based on the two transmission modes of ASRP messages, the format of ASRP packets can also be classified into two types.¶
In Inline Message mode, the ASRP packet shares the 5-tuples with the flow. The ASRP message is either inserted into the original packet:¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Original packet header (IP + TCP/UDP) ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ASRP Signature | | (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ NS/HS/PS message ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ Original packet data ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
or carried in a separate packet that copies the original packet header, without the original packet data:¶
In Standalone Message mode, the ASRP packet uses its own 5-tuples and is encapsulated using IP/UDP. The packet format is as follows:¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ IP + UDP Header (with destination port: ASRP-PORT) ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ QS/RS/RX/NA/NC message ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
An inline ASRP message exists only on the wire between the sending side's ASRP module (which inserts it) and the receiving side's ASRP module (which strips it). The message occupies leading octets of the segment payload, but it is never part of either endpoint's TCP sequence space: the sender's transport stack generated the segment without the message, and the receiver's transport stack processes the segment without it. In this respect an inline message travels like application data - the signature and the message simply occupy the head of the payload - yet it is handled entirely below the transport layer, and neither the application nor either transport stack observes it.¶
Consequently, implementations MUST NOT modify TCP sequence or acknowledgment numbers when inserting or stripping an inline message. The only fields requiring adjustment are the IP total length and the transport checksums, which MUST be recomputed after insertion or removal. After stripping, the byte stream each endpoint's transport stack observes is identical to that of a session that never carried ASRP messages.¶
Standalone-ASRP packets are identified by the UDP destination port, while Inline-ASRP packets are identified by the ASRP Signature. Once an ASRP packet is identified, the ASRP messages within it are then parsed and processed.¶
Before processing any ASRP message, a receiver MUST perform the following checks and silently discard the message if any of them fails:¶
The Version field is a version the receiver implements.¶
The Length field matches the number of ASRP message octets actually present in the packet.¶
The STType field matches the MsgType: QS and RX carry a single Session-Tuple (ST4 or ST6); NS, NA, NC, RS, and PS carry a Session-Tuple pair (ST44, ST66, ST46, or ST64); HS carries no Session-Tuple (NULL).¶
The Reserved field and any unassigned flag bits are ignored.¶
A message that passes these checks is processed according to its MsgType as described in the following subsections.¶
When a client or server receives an NS message, it MUST first check for session conflicts on the client side and then on the server side. A client-side conflict occurs when the client-side tuple matches an existing backed-up session on either side. If a conflict is found, it MUST clear the F-NA flag and send an NC message, with the F-CC flag set to indicate a client-side conflict.¶
If no conflict is found, it MUST back up the session state information and hand the embedded packet (if present) over to the system. If the F-NA flag is set, it MUST send an NA message.¶
The F-NA flag SHOULD be set when the session is created from a non-explicit first packet; explicit first packets (e.g., TCP SYN or DNS query) do not require it.¶
If an NS message is lost, in PSV mode a TCP SYN retransmission or subsequent packets regenerate NS messages until an NA message is received; in ACT mode, subsequent packets generate HS messages, prompting the network node to retransmit the NS message.¶
NS messages may be generated in both PSV and ACT modes. The handling procedures are described in Figure 5, Figure 7, and Figure 8.¶
The destination IP of the NA packet MUST be set to the Node-Local-IP field carried in the received NS message, and the destination port MUST be set to ASRP-PORT.¶
Upon receiving an NA message, the network node considers that the corresponding NS message has been processed by the responder, and MUST NOT send any further NS messages for this session.¶
The destination IP of the NC packet MUST be set to the Node-Local-IP field carried in the received NS message, and the destination port MUST be set to ASRP-PORT.¶
Upon receiving an NC message, the network node considers that the session conflicts with an existing local session on the responder. If the F-CC flag is set, the conflict is on the client side; otherwise, it is on the server side. The network node SHOULD handle the conflict according to its local policy.¶
The source IP of the QS packet MUST be set to the network node's local IP (obtainable from configuration), and the destination IP MUST be set to the client's or server's IP (obtainable from the forwarded packet as described in Figure 6 and Figure 9, or derived via algorithmic mapping to the client or server as described in Figure 7 and Figure 8). The source port MUST be randomly generated, and the destination port MUST be set to ASRP-PORT.¶
The backup holder MUST index the backed-up session by both the client-side and the server-side 5-tuples. When a client or server receives a QS packet, it MUST look up the backed-up session whose client-side or server-side 5-tuple matches the Session-Tuple in the QS message. If a match is found, it MUST return an RS message; otherwise, it MUST return an RX message.¶
If a QS message is lost, subsequent packets will trigger the generation of new QS packets, continuing the attempt to recover the session.¶
QS messages may be generated in both PSV and ACT modes. The handling procedures are described in Figure 6, Figure 7, and Figure 9.¶
The RS packet MUST reuse the protocol header of the QS packet, with the source and destination IP addresses swapped.¶
When a network node receives an RS packet, it MUST recover the session and forward packets according to the session.¶
If an RS message is lost, subsequent QS messages will continue the attempt to recover the session, thereby triggering retransmission of the RS message.¶
RS messages may be generated in both PSV and ACT modes. The handling procedures are described in Figure 5, Figure 6, Figure 7, Figure 8, and Figure 9.¶
Upon receiving an RX message, the network node MUST follow the processing procedures described in the respective scenarios under PSV/ACT mode.¶
The network node identifies the session by the 5-tuple of the packet carrying the HS message. If a local session is found, it MUST return an NS message for that session to the client, addressed to the source IP of that packet; otherwise, no NS message is returned.¶
HS messages are only generated in ACT mode. The handling procedure is described in Figure 8.¶
The source IP, destination IP, and source port of the PS packet MUST be copied from the packet sent by the server.¶
When a network node receives a PS message, it MUST recover the session; it extracts the forwarded packet (if present) and forwards it according to the session.¶
PS messages are only generated in PSV mode. The handling procedure is described in Figure 5 Figure 7.¶
ASRP messages may be subject to forgery, replay, or flooding by attackers who are able to inject traffic into the network that carries them. ASRP deployments can counter this with two layers of message validation; both layers draw on information local to the validating node, so no protocol-level trust relationship between nodes is required.¶
Cookie-based validation: Every ASRP message carries an 8-octet Cookie field. A node that sends a request (a QS message, an NS message carrying F-NA, or an HS message) fills the cookie with a per-request random value and verifies that the corresponding response (RS/RX, NA/NC, or NS, respectively) echoes it. This verifies that the response belongs to an outstanding request of the receiving node; a forged response that guesses the cookie, or a replayed response from an earlier request, can be recognized and discarded. Cookie generation and verification rules are determined by the implementation and deployment; see "Cookie Handling".¶
Session-legitimacy validation: Nodes SHOULD validate the session referenced by a message against application-level policies, such as verifying that the session points to a known and healthy server in a load-balancing deployment, or that the address translation complies with configured rules in a NAT deployment. Nodes that perform this validation MUST discard messages that fail it and MUST NOT create a new session for them.¶
Implementations MAY apply rate limiting to ASRP messages as appropriate, e.g., to contain QS floods [RFC4987]. ASRP responses are comparable in size to the requests that trigger them, so the protocol itself provides no amplification.¶
ASRP messages carry the 5-tuple of the session they describe. This is the same information that is visible in the session's own packets; observing ASRP traffic reveals the existence of a session between two endpoints and their addresses, but nothing beyond what the carried traffic already exposes.¶
The Node-Local-IP field of an NS message discloses the address of the network node that created the session to the backup holder. This is information the backup holder requires in order to return NA/NC messages, and the network node address is in any case observable in the forwarded traffic of the session.¶
ASRP introduces no new identifiers that persist across sessions and no tracking mechanism. Operators who wish to limit the visibility of session metadata can apply the usual operational measures to ASRP traffic, e.g., restricting routing of the ASRP-PORT destination to the serving domain.¶
This document defines an application-layer protocol (ASRP). All values of the MsgType, STType, and Flags fields used by this version of the protocol are defined in this specification. New values MUST be defined via an update to this document or a successor specification; a future standards-track version of ASRP SHOULD request the creation of IANA registries for these fields. For the implementation of this protocol, a UDP destination port requires allocation:¶
QS/RS/RX/NA/NC messages are encapsulated within UDP datagrams for transmission. A fixed UDP destination port number is required so that the receiving end can identify and process such encapsulated packets.¶
Service Name: asrp¶
Port Number: 51200 (proposed value for current experimentation)¶
Transport Protocol: udp¶
Description: Used for receiving UDP-encapsulated ASRP protocol messages.¶
For experimental implementations and interoperability testing prior to IANA assignment, UDP port 51200 MAY be used as a temporary default. This port falls within the dynamic/private port range (49152-65535) reserved for local or temporary use and documentation examples [RFC6335].¶
IANA is requested to assign a permanent port number in the "User Ports" range (1024-49151) for the "asrp" service in the "Service Name and Transport Protocol Port Number Registry", with a reference to this document.¶
Note to the RFC Editor: Please remove this section before publication.¶
ASRP has been implemented twice, covering both roles the protocol defines, and the implementations have been interoperability tested on a three-node testbed consisting of a client, a load-balancing cluster node, and a backend server:¶
A Linux kernel module extending IPVS, implementing the network-node role in PSV mode: session creation with inline NS insertion, QS/RS exchange for session recovery, NA/NC handling, cookie verification of responses, and the ASRP-PORT UDP listener.¶
An eBPF implementation based on traffic-control programs, implementing the backup-holder role on the server: NS backup from session first packets, session reconstruction from PS messages, and response generation with cookie echoing.¶
The testbed exercised direct session creation (PSV-Scenario-1), recovery after cluster session-table loss simulating node failure (PSV-Scenario-2), and the fallback when no backed-up session exists (RX). Recovery was observed to be driven by the retransmission behavior of the recovered session, as described in the scenarios.¶
The authors would like to thank all individuals who have provided valuable feedback and contributions during the development of this document.¶