China Mobile

Beijing100053 China lizhiqiangyjy@chinamobile.com

China Mobile

Beijing100053 China duzongpeng@chinamobile.com

Centec

Shanghai201203 China wangjj@centec.com

Centec

Shanghai201203 China chengw@centec.com

Centec

Shanghai200000 China zhaoying@centec.com

SAIA

Shanghai200125 China chunhao.zhao@sh-aia.com

Inesa

Shanghai200030 China sunxun@inesa.com

Web and Internet Transport RDMA RDMAP KV cache remote memory atomic fetch-and-add offload Inference systems move key-value (KV) cache blocks between instances and cache tiers over RDMA. Existing practice forces a choice between statically registered remote memory, which is fast but wastes memory, and per-transfer remote procedure calls, which use memory well but involve the target's CPU in every transfer. This document specifies the KV Cache Block Transfer Protocol (KBTP), an upper-layer protocol over RDMA with three operations: Block Load, which places a stored block into the initiator's memory; Block Mapped Store, which writes a block into an already allocated block; and Block Unmapped Store, which allocates a free block at the target and writes it in one operation. KBTP defines the request and completion messages, a block table, and a free-block ring whose counters use masked atomic fetch-and-add. These structures are designed so that a target can execute loads and stores within its RDMA network interface, and allocations as well where the interface supports conditional execution, without involving its CPU per operation, while remaining interoperable with targets that process requests in software. KBTP is bound to RDMAP with the RDMA protocol extensions, and can also be used over InfiniBand and RoCEv2.

Introduction

Background The KV cache that attention layers keep for processed tokens is reused across decoding steps and, increasingly, across requests. Longer context windows make it larger than accelerator memory can hold, so serving systems store it in several tiers and move it between prefill instances, decode instances, and cache servers . Two organizations are common: a central KV store that inference instances write to and read from, and peer-to-peer transfer in which, for example, a prefill instance writes directly to the decode instance that will use the cache. In both, the data moves in fixed-size blocks (typically holding the cache of 16 to 512 tokens, from kilobytes to megabytes) over RDMA.

Problem Statement Three RDMA usage patterns are common today:

1. One-sided access to statically allocated memory: the target registers a large region in advance (often gigabytes) and the source writes or reads it directly. Access is fast, but the coarse static allocation leaves much memory unused.
2. One-sided access to dynamically allocated memory: the source asks the target, with a two-sided remote procedure call, to allocate a fine-grained region (for example 4 KiB to 1 MiB), then accesses it one-sidedly. Memory is used well, but every allocation involves the target CPU, which limits throughput and adds latency.
3. Two-sided transfer: the source sends the data in messages and the target CPU allocates memory and copies or registers it. Memory is used well, but every transfer involves the target CPU.

Comparison of Approaches

Approach	Memory utilization	Target CPU per operation	Operation cost
One-sided, static	low	no	low
One-sided, dynamic	medium	yes (allocation)	medium
Two-sided	high	yes	high
Target-executed block operations (this document)	high (fine-grained blocks)	not required per operation	one round trip per operation

No interoperable protocol provides fine-grained allocation and one-round-trip block access without per-operation involvement of the target CPU. Since inference instances and cache servers are built by different implementers, the message formats and the structure of target memory must be specified, not only the RDMA operations.

Overview A KBTP target manages a pool of equally sized blocks. It keeps a block table that maps block handles to blocks, and a free-block ring that lists unallocated blocks. An initiator sends one request message per operation; the target answers with a completion. The operations are:

Block Load:

the target looks up the handle and writes the block into a buffer named by the initiator.

Block Mapped Store:

the target looks up the handle and writes data supplied by the initiator into the block.

Block Unmapped Store:

the target takes a free block from the ring, binds it to the handle, and writes the data into it.

Block Free:

the target unbinds the handle and returns the block to the ring.

Each operation needs a fixed, short sequence of RDMA operations at the target whose addresses come from the request, the table, or the ring. The formats in this document are designed so that the target can pre-post these sequences and let its RDMA interface execute them when a request arrives (). A target that processes requests in software is equally conformant; the initiator cannot tell the difference. Optional direct modes let initiators that the target trusts read blocks and allocate them with one-sided operations only ().

Scope and Relationship to Other Work KBTP defines an upper-layer protocol in the sense of RDMAP , like the RDMA transport for RPC and iSER . It is bound normatively to RDMAP over DDP and MPA , with the atomic and immediate data operations of . A mapping to InfiniBand reliable connections and RoCEv2 is given in . KBTP does not define new RDMA operations. specifies RDMA extensions for remote persistence and visibility, which KBTP targets can use for persistent tiers. Block keys and cache indexing, which tell an initiator which handle holds a given block, are provided by or by local means. Typical uses are: storing the KV cache of external knowledge (retrieval-augmented generation documents) once and loading it into any instance; keeping the cache of very long contexts in a shared pool and reusing it across requests; and caches embedded in network devices of computing power networks.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document uses the RDMAP terms Steering Tag (STag), Tagged Offset, Send, RDMA Write, RDMA Read, and Remote Peer from , and FetchAdd from . In addition:

Initiator:

The endpoint that issues KBTP requests.

Target:

The endpoint that owns the block pool.

Block:

A fixed-size region of the target's block pool, identified by its Block Offset, the Tagged Offset of its first octet within the Pool STag.

Block Handle:

A 64-bit name, chosen by an initiator within the handle range assigned to it, that is bound to a block by an Unmapped Store.

Generation:

A 16-bit counter of a table entry that changes whenever the handle is bound or unbound, so that stale references are detected.

Sealed block:

A block whose contents will not change until it is freed.

All fields in messages and in memory structures read by remote peers are in network byte order, except the address fields listed in when the target selects little-endian address fields.

Target Memory Structures

Block Pool The target allocates F blocks of Block Size octets each within a memory region named by the Pool STag. Each block is identified by its Block Offset. Tables and ring slots store Block Offsets rather than block numbers, so that a value read from them can be used directly as the address of an RDMA operation without arithmetic.

Block Table The block table has one 16-octet entry per handle that the target serves. The entry for handle h is at Table Offset + 16 * (h - Table Base Handle) within the Table STag.

Block Table Entry 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Flags | Generation | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Data Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Block Offset (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Flags: 0x0001 VALID (the handle is bound), 0x0002 SEALED. Data Length is the number of valid octets in the block. Generation never takes the value zero. An entry is written only by the target; when it binds an entry, the target writes Data Length and Block Offset before Flags and Generation.

Free-Block Ring The free-block ring has N slots, where N = 2^n is a power of two not smaller than F, and two 64-bit counters, Head (used to allocate) and Tail (used to free), each aligned to 8 octets. Each slot is 16 octets:

Ring Slot 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence (Ticket) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Block Offset (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

A slot whose Block Offset is all ones is empty (consumed). Each counter holds two fields that advance together: a 32-bit Ticket in bits 63-32 and the Slot Offset, the byte offset of the current slot (16 times the slot number), in the low n+4 bits. Bits 31 to n+4 are zero. Counters are modified only with FetchAdd using: Add Data = 0x0000000100000010 Add Mask = bit 63 | bit 31 | bit (n+3) The Add Mask makes each field add independently and discard its carry, so one FetchAdd returns the current Ticket and Slot Offset and advances the Ticket by one and the Slot Offset by one slot, wrapping at the end of the ring. The returned Slot Offset can be used as an address offset without further arithmetic, which allows targets to execute the allocation in their RDMA interface. Counters are in the target's memory byte order, as RFC 7306 specifies for atomic operations; they are never read with RDMA Read. The rules are:

• Initialization: for s < F the target writes slot s = {Sequence s, Block Offset of the s-th free block}; for F <= s < N it writes {Sequence s - N (modulo 2^32), all ones}. It sets Head to Ticket 0 and Slot Offset 0, and Tail to Ticket F and Slot Offset 16 * (F mod N).
• Allocation: FetchAdd on Head returns ticket t and offset o. The slot at o holds the block for this allocation once its Sequence equals t and its Block Offset is not all ones; until then the allocator re-reads the slot. Having taken the Block Offset, the slot is marked consumed by setting its Block Offset to all ones: by the target itself for target-executed allocations, and by the target when it processes BIND for direct allocations ().
• Free: the target first updates the table entry (), then performs FetchAdd on Tail, obtaining ticket u and offset o. If the slot at o is consumed, the target writes the Block Offset and then Sequence u. If it is not yet consumed, because the allocation that owns it has taken its ticket but not yet read it, the target MUST defer the write until the slot is consumed; it MUST NOT overwrite the slot.

With these rules no block is ever lost or handed out twice: each slot is written only after its previous content has been consumed, and each allocation reads exactly the slot written for its ticket. Writing the Block Offset before the Sequence ensures that a reader that observes the new Sequence also observes the new Block Offset.

Allocation Quotas The target assigns each initiator an Allocation Quota at session setup and keeps the sum of quotas not larger than F. An initiator's usage is the number of handles it has bound plus the number of tickets it has taken and not yet bound. An initiator MUST NOT exceed its quota; the target releases one unit of usage when a FREE completes or when an abandoned ticket is reclaimed. When all initiators respect their quotas, the ring always holds a block for every ticket taken, so an allocation waits at most for an in-progress free to finish writing its slot. Target-executed allocations take blocks from a ring shared by all initiators that the target serves this way. For DIRECT_ALLOC (), the target instead gives each initiator its own ring and Head counter over a partition of the pool whose size equals the initiator's quota, so that every ticket on that ring belongs to that initiator and the quota is enforced by the ring itself. A ticket that is not bound by BIND within the Ticket Timeout advertised in HELLO_ACK is abandoned. Before reclaiming its block, the target MUST ensure that no RDMA Write of the initiator can still reach the block, by invalidating the initiator's Partition STag (), which disables DIRECT_ALLOC for the rest of the session, or by terminating the connection. It then consumes the slot, returns the block to the ring with a new ticket, and rejects a later BIND for that ticket with status TICKET_ABANDONED. If a target observes an allocation by an initiator that exceeds its quota, it completes the request with NO_SPACE and MAY terminate the session.

Session Establishment After the RDMA connection is established, the initiator sends HELLO and the target answers with HELLO_ACK, both as Send messages and both starting with the common header () in which Request ID, Block Handle, Generation, and Status are zero (Status reports an error in HELLO_ACK).

HELLO Body 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Requested Features | Completion Credits | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Initiator ID (128 bits) | | | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Completion Credits is the number of receive buffers the initiator has posted for COMPLETION messages. The Initiator ID is a stable random identifier of the initiator; a target uses it to restore the initiator's handle range and bindings after a reconnection. Because it gives access to those bindings, the Initiator ID MUST be generated with at least 128 bits of entropy and kept secret, MUST be sent only on connections authenticated as described in , and a target MUST reject a HELLO whose Initiator ID belongs to a session that is still active.

HELLO_ACK Body (Target Descriptor) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Features | Request Credits | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Block Size | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Max Inline | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Allocation Quota | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Handle Base (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Handle Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Ticket Timeout (ms) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Direct-mode descriptors (present per Features, 16 each) ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Features: 0x0001 DIRECT_LOAD, 0x0002 DIRECT_ALLOC, 0x0004 RECORD_COMPLETION, 0x0008 ADDR_LE. Each direct-mode descriptor is 16 octets: STag (32 bits), Tagged Offset (64 bits), and Count (32 bits). If DIRECT_LOAD is set, a Table descriptor (Count = number of entries) follows, then a Table Base Handle (64 bits) and 8 reserved octets, and then a Pool descriptor (Count = F) that grants read access only. If DIRECT_ALLOC is set, a Partition descriptor follows, naming the part of the pool reserved for this initiator (Count = its quota) through an STag that grants write access, is specific to this connection (for example a memory window bound to it), and covers nothing else; then a Ring descriptor (Count = N) and a Head descriptor (Count = 1; the Tagged Offset MUST be a multiple of 8, as required for FetchAdd) of the ring dedicated to this initiator (). Handles of different initiators never overlap, so each initiator can choose handles for Unmapped Store without coordination. Any initiator MAY load any handle the target authorizes it to read; handles are exchanged among initiators by means outside this protocol. If ADDR_LE is set, the initiator MUST encode the Block Handle and the STag, Tagged Offset, and Length fields of the Data and Completion Descriptors in little-endian byte order, and the Block Offset fields of table entries and ring slots are in little-endian byte order. A target sets ADDR_LE when its RDMA interface uses little-endian work request fields and it wants to place these fields into pre-posted work requests unchanged (). GOODBYE ends the session. With flag 0x0001 RELEASE, the target frees all handles bound by the initiator. Otherwise, and when a connection fails, the target keeps the initiator's bindings for a retention time of its choosing; an initiator that reconnects with the same Initiator ID within that time receives the same handle range and finds its bindings intact.

Messages

Common Header

KBTP Common Header (24 Octets) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Opcode | Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Request ID (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Block Handle (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Generation | Status | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Version is 1. Opcodes are listed in . Request ID is chosen by the initiator and echoed in the completion. Generation in a request is the generation the initiator expects (zero means "any"); in a completion it is the entry's current generation. Status is zero in requests. Flags are interpreted per opcode; for LOAD, STORE, ALLOC_STORE, and FREE they are:

• 0x0001 INLINE: data follows the request.
• 0x0002 SEAL: seal the block after the store.
• 0x0004 RECORD: complete with a completion record (); a Completion Descriptor is present.
• 0x0008 INVALIDATE: the target completes with Send with Invalidate , invalidating the STag of the initiator's Data Descriptor.
• 0x0010 ALLOW_UNSEALED: a load may return an unsealed block.

KBTP Opcodes

Opcode	Name	Body after the header
0x01	HELLO
0x02	HELLO_ACK
0x10	LOAD	Data Descriptor [, Completion Descriptor]
0x11	STORE (mapped)	Data Descriptor [, Completion Descriptor] [, inline data]
0x12	ALLOC_STORE (unmapped)	Data Descriptor [, Completion Descriptor] [, inline data]
0x13	FREE	[Completion Descriptor]
0x14	BIND	Ticket (32), Data Length (32), Block Offset (64)
0x20	COMPLETION	Data Length (32), Reserved (32), Block Offset (64)
0x30	GOODBYE	none

Descriptors A Data Descriptor is 16 octets: STag (32 bits), Tagged Offset (64 bits), and Length (32 bits), naming a buffer of the initiator. For LOAD it is the destination; for STORE and ALLOC_STORE it is the source that the target reads with RDMA Read, unless INLINE is set, in which case STag and Tagged Offset are zero, Length is the inline data length, and the data follows the message. A Completion Descriptor is 16 octets: STag (32 bits), Tagged Offset (64 bits), and 32 reserved bits, naming a 24-octet completion record slot of the initiator.

Operations A target processes the requests of one connection in order. It validates every request before touching memory: the handle MUST be within a range the initiator is authorized for (its own range for stores and frees), lengths MUST NOT exceed Block Size or the initiator's buffer, and, if the request Generation is non-zero, it MUST equal the entry's. A failed check completes the request with the corresponding status and has no other effect.

Block Load

1. The target reads the table entry of the handle. The entry MUST be VALID, and SEALED unless ALLOW_UNSEALED is set.
2. It RDMA-writes Data Length octets of the block to the Data Descriptor. Data Length MUST NOT exceed the descriptor Length.
3. It completes the request () with the Block Offset, Data Length, and Generation of the entry.

Block Mapped Store

1. The target reads the table entry, which MUST be VALID and not SEALED and MUST belong to the initiator's handle range.
2. It obtains the data, from the inline payload or by RDMA Read from the Data Descriptor, and writes it into the block.
3. It updates Data Length and, if SEAL is set, sets SEALED.
4. It completes the request.

Block Unmapped Store

1. The handle MUST be in the initiator's range and its entry MUST NOT be VALID.
2. The target allocates a block from the ring and marks the slot consumed ().
3. It obtains the data and writes it into the block.
4. It writes the table entry with the Block Offset and Data Length, then with VALID, SEALED if requested, and the next Generation (incremented by one, skipping zero).
5. It completes the request with the Block Offset and Generation.

The entry is written after the data, so any initiator that observes VALID in the entry also observes the data.

Block Free

1. The handle MUST be in the initiator's range and VALID.
2. The target clears VALID and SEALED and increments the Generation (skipping zero) in the table entry.
3. It returns the block to the ring from which it was allocated (), after the entry update is visible.
4. It completes the request.

Initiators that may still be loading the block detect the free by the changed Generation. If the block may next be allocated to an initiator of a different trust domain, the target clears its contents before returning it to the ring.

Completion By default the target completes a request with a COMPLETION message sent as a Send (or Send with Invalidate when INVALIDATE is set). In the RDMAP binding, the contents of a buffer written by RDMA Write are determinate once a subsequent Send has been delivered (rule 5 of ), so receipt of the completion implies that the loaded data is in the initiator's buffer. If the target advertised RECORD_COMPLETION and the request has the RECORD flag, the target instead writes a 24-octet completion record to the Completion Descriptor: Request ID (64 bits), Status (16 bits), Generation (16 bits), Data Length (32 bits), and Block Offset (64 bits). A target MUST use completion records only on bindings in which an RDMA Write cannot become visible before the data of earlier RDMA Writes on the same connection. RDMAP does not guarantee this (rules 3 and 5 of ), so a target MUST NOT advertise RECORD_COMPLETION in the RDMAP binding.

Direct Modes Direct modes expose target structures to the initiator and are OPTIONAL. A target MUST offer them only to initiators it trusts for all blocks reachable through the exposed STags. Direct Load (DIRECT_LOAD): through the read-only Table and Pool STags, the initiator RDMA-reads the table entry, then the block, then the entry again, issuing each read only after the previous one has completed. The data is valid if both entry reads show the same Generation with VALID and SEALED set, because a sealed block does not change until it is freed and a free changes the Generation. This costs two round trips but no target processing. Direct Allocation (DIRECT_ALLOC): the initiator performs FetchAdd on its Head with the Add Data and Add Mask of , RDMA-reads the slot at the returned offset (re-reading until the Sequence equals its ticket and the Block Offset is not all ones, and then once more to confirm both values, since a 16-octet read is not guaranteed to be atomic), RDMA-writes the data into the block, and then sends BIND with the ticket, Data Length, and Block Offset. The target verifies that the ticket has not been abandoned, that the slot of the ticket holds that Block Offset, and that the initiator is within its quota; it then marks the slot consumed, writes the table entry, and completes. The initiator never writes ring slots. Frees always use FREE.

Status Codes KBTP Status Codes

Value	Name	Meaning
0	OK	Success
1	INVALID_HANDLE	Handle outside authorized range
2	STALE_GENERATION	Generation mismatch
3	NOT_VALID	Entry not bound
4	ALREADY_BOUND	Unmapped Store to a bound handle
5	NO_SPACE	Quota exceeded or ring empty
6	SEALED	Store to a sealed block
7	NOT_SEALED	Load of an unsealed block
8	TOO_LARGE	Length exceeds block or buffer
9	MALFORMED	Request cannot be parsed
10	INTERNAL	Target failure
11	TICKET_ABANDONED	BIND for a ticket that was reclaimed

Target Execution Model This section is informative. It shows how the formats allow a target to execute operations without CPU involvement. A target pre-posts, for each request credit, a receive buffer followed by a chain of work requests that its RDMA interface starts when the receive completes. The receive scatter list places the request's Data Descriptor and handle directly into the address fields of the following work requests (hence the ADDR_LE option), and table and ring entries read by the chain are placed into later work requests in the same way. Example Work Request Chains

Operation	Chain at the target
Block Load	receive request; read table entry into the next request; write block to initiator; send completion
Block Mapped Store	receive request with inline data; read table entry; write data to block; write entry; send completion
Block Unmapped Store	receive request with inline data; masked fetch-and-add on Head; read slot; check Sequence (conditional execution); mark slot consumed; write data to block; write table entry; send completion

The masked FetchAdd of returns a slot offset that is directly usable as an address, and table entries and slots hold Block Offsets in fixed positions, which is what makes these chains possible. Loads and mapped stores need only data movement. An allocation also needs the Sequence check of and the consumption of the slot; a chain can perform them only if the interface supports conditional execution of work requests, and otherwise the target handles allocations in software. Frees, which update the Generation and may be deferred, are normally handled in software and are not latency-critical. The target CPU otherwise only refills the pre-posted chains, in batches. Implementations that cannot chain work requests process the same messages in software.

Transport Bindings

RDMAP Binding

• KBTP runs over an RDMAP Stream on DDP and MPA . Peers SHOULD use the enhanced connection establishment of . Targets that advertise DIRECT_ALLOC, and targets that execute allocations in their RDMA interface, MUST support the atomic operations of .
• Requests, HELLO, HELLO_ACK, and COMPLETION are carried in Send or Send with Solicited Event messages; INVALIDATE uses Send with Invalidate.
• Loads use RDMA Write; stores without INLINE use RDMA Read by the target.
• Ring counters use FetchAdd with the Add Mask of .
• Completions are always COMPLETION messages; RECORD_COMPLETION is not offered ().
• The security of the binding is that of ; when the network is not trusted, IPsec MUST be used as specified there.

InfiniBand and RoCEv2 Over an InfiniBand reliable connection or RoCEv2 , the same messages are carried with Send, RDMA Write, and RDMA Read. The InfiniBand specification defines an unmasked Fetch and Add; masked variants exist as vendor extensions. With unmasked Fetch and Add, the Head and Tail counters hold the Ticket in their high 32 bits and zero in their low 32 bits (Add Data 0x0000000100000000), and the slot is computed as 16 * (ticket mod N) by the initiator in DIRECT_ALLOC or by the target in software. Completion records on these transports MUST be written with RDMA Write with Immediate, so that the initiator learns of completion from its completion queue rather than by polling memory.

Operational Considerations

• Block size follows the KV cache block of the serving system (for example 16 tokens per block) or a multiple of it.
• Quotas and handle ranges are set at session setup and changed by reconnecting; targets reserve free blocks for the sum of quotas.
• Implementations SHOULD expose counters per operation and status, ring occupancy, and quota use per initiator.

Security Considerations The security considerations of apply. In the default mode, the target exposes no STag of its own memory to initiators: all access to the block pool, the table, and the ring is performed by the target after validation. The initiator exposes only the buffers named in its descriptors; it SHOULD use STags that are valid for a single request and SHOULD request INVALIDATE so that they are revoked by the completion. Handle ranges prevent an initiator from overwriting or freeing another initiator's blocks. Generations prevent use of stale handles. Quotas bound memory consumption per initiator. Direct modes expose the table and pool for reading, and the ring counters for atomic modification, to the trusted initiator. A misbehaving initiator with DIRECT_ALLOC can take tickets without binding them; the Ticket Timeout bounds how long such blocks are withheld, and quotas bound how many. Targets offer direct modes only to initiators within the same trust domain and verify every BIND against the ring. KV cache blocks contain information derived from user prompts. Connections SHOULD be confined to a trusted network or protected with IPsec, and targets SHOULD clear blocks when they are freed if the blocks may be allocated to initiators of a different trust domain.

IANA Considerations IANA is requested to create a registry group named "KV Cache Block Transfer Protocol (KBTP)" with the registries "KBTP Opcodes" (8 bits, initial contents from ), "KBTP Status Codes" (16 bits, initial contents from ), "KBTP Flags" (one entry per opcode and bit of the 16-bit Flags field; initial contents: for LOAD, STORE, ALLOC_STORE, and FREE, the flags of ; for GOODBYE, 0x0001 RELEASE), and "KBTP Features" (one entry per bit of the 16-bit Features field, initial contents from ). New values are assigned by Specification Required ; the upper quarter of each value space is reserved for Private Use. The designated experts should check that new opcodes define their validation rules, memory ordering, and completion. This document requests no port number; KBTP is identified by the RDMA connection over which it is established.

References Normative References Informative References Unaffiliated InfiniBand Trade Association Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP) 23rd USENIX Conference on File and Storage Technologies (FAST 25)

Examples ALLOC_STORE of 8 octets of inline data ("KVBLOCK!") to handle 0x4001, Request ID 7, flags INLINE: 01 12 00 01 00 00 00 00 00 00 00 07 00 00 00 00 00 00 40 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 4b 56 42 4c 4f 43 4b 21 Its COMPLETION (Generation 5, Status OK, Data Length 8, Block Offset 0x150000, which is block 42 of a pool of 32 KiB blocks): 01 20 00 00 00 00 00 00 00 00 00 07 00 00 00 00 00 00 40 01 00 05 00 00 00 00 00 08 00 00 00 00 00 00 00 00 00 15 00 00 Ring counter for N = 4 (n = 2) with 16-octet slots: Add Data 0x0000000100000010, Add Mask 0x8000000080000020. Successive FetchAdd results on Head, starting from zero, show the ticket advancing while the slot offset wraps after four slots: returned (ticket, offset) new counter value (0, 0) 0x0000000100000010 (1, 16) 0x0000000200000020 (2, 32) 0x0000000300000030 (3, 48) 0x0000000400000000 (4, 0) 0x0000000500000010 A shared ring of four free blocks 100 to 103 (N = F = 4) serves target-executed allocations. The target takes ticket 0 for an ALLOC_STORE of initiator X, and that work request chain is delayed before reading slot 0. Meanwhile ALLOC_STORE requests of initiator Y take tickets 1, 2, and 3 (blocks 101, 102, 103), and Y frees block 101; the free obtains ticket 4, whose slot is slot 0, which has not been consumed yet, so the free is deferred. X's allocation then reads slot 0, obtains block 100, and consumes the slot; the deferred free now writes slot 0 with Sequence 4 and block 101, which the next allocation (ticket 4) obtains. No block is lost or handed out twice.

Acknowledgments The authors thank colleagues working on RDMA offload and KV cache storage for their comments.