Network Working Group Z. Li Internet-Draft Z. Du Intended status: Standards Track China Mobile Expires: 6 May 2027 J. Wang W. Cheng Y. Zhao Centec C. Zhao SAIA X. Sun Inesa 2 November 2026 Speculative Decoding Coordination Protocol (SDCP) for Distributed Inference with Multi-Level Caches draft-li-mlc-inference-acceleration-00 Abstract Speculative decoding accelerates language model inference: a small draft model proposes several tokens and the large target model verifies them in one forward pass. In large deployments, draft models and the target model run on separate, independently scaled instances that exchange state through a multi-level cache. This document specifies the Speculative Decoding Coordination Protocol (SDCP), with which a router assigns requests to draft instances by load, draft instances submit proposals that reference draft state in the cache, and a verifier, the only party that commits output, checks proposals under a registered acceptance rule, returns the accepted tokens and the next-round speculation limit, and escalates persistently rejected sessions to a reduced speculation length, another draft model, or decoding by the target model itself depending on its load. SDCP defines a session contract, a committed-prefix digest chain, CBOR messages carried over HTTP, idempotent rounds, and epoch fencing for recovery. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Li, et al. Expires 6 May 2027 [Page 1] Internet-Draft SDCP November 2026 Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 6 May 2027. Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Background . . . . . . . . . . . . . . . . . . . . . . . 3 1.2. Problem Statement . . . . . . . . . . . . . . . . . . . . 3 1.3. Protocol Overview . . . . . . . . . . . . . . . . . . . . 4 1.4. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 5 3. Architecture . . . . . . . . . . . . . . . . . . . . . . . . 6 4. Session Model . . . . . . . . . . . . . . . . . . . . . . . . 6 4.1. Session Contract . . . . . . . . . . . . . . . . . . . . 6 4.2. Committed-Prefix Digest Chain . . . . . . . . . . . . . . 8 5. Messages and Resources . . . . . . . . . . . . . . . . . . . 8 5.1. HTTP Usage . . . . . . . . . . . . . . . . . . . . . . . 8 5.2. Message Definitions . . . . . . . . . . . . . . . . . . . 9 6. Procedures . . . . . . . . . . . . . . . . . . . . . . . . . 11 6.1. Session Creation and Assignment . . . . . . . . . . . . . 11 6.2. Drafting and Proposal . . . . . . . . . . . . . . . . . . 11 6.3. Verification and Commit . . . . . . . . . . . . . . . . . 12 6.4. Acceptance Rules . . . . . . . . . . . . . . . . . . . . 13 6.5. Speculation Control and Escalation . . . . . . . . . . . 14 6.5.1. Speculation Limit . . . . . . . . . . . . . . . . . . 14 6.5.2. Low Rounds and Escalation . . . . . . . . . . . . . . 14 6.6. Sampling Policy Updates . . . . . . . . . . . . . . . . . 15 6.7. Output Delivery . . . . . . . . . . . . . . . . . . . . . 15 6.8. Commit Log . . . . . . . . . . . . . . . . . . . . . . . 16 6.9. Failure Handling and Recovery . . . . . . . . . . . . . . 16 Li, et al. Expires 6 May 2027 [Page 2] Internet-Draft SDCP November 2026 7. Error Handling . . . . . . . . . . . . . . . . . . . . . . . 16 8. Operational Considerations . . . . . . . . . . . . . . . . . 17 9. Security Considerations . . . . . . . . . . . . . . . . . . . 18 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 10.1. Media Type . . . . . . . . . . . . . . . . . . . . . . . 18 10.2. SDCP Registries . . . . . . . . . . . . . . . . . . . . 19 10.2.1. SDCP Problem Types . . . . . . . . . . . . . . . . . 19 10.2.2. SDCP Acceptance Rules . . . . . . . . . . . . . . . 19 10.2.3. SDCP Actions . . . . . . . . . . . . . . . . . . . . 19 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 11.1. Normative References . . . . . . . . . . . . . . . . . . 19 11.2. Informative References . . . . . . . . . . . . . . . . . 21 Appendix A. Test Vectors . . . . . . . . . . . . . . . . . . . . 21 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 22 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 1. Introduction 1.1. Background Autoregressive decoding produces one token per forward pass of the target model. Each pass reads all model weights and the whole KV cache, so decoding is limited by memory bandwidth and leaves compute idle. Speculative decoding [LEVIATHAN] [CHEN] lets a cheaper draft model propose M tokens, which the target model then scores in a single pass. The longest acceptable prefix of the proposal is kept, and the target model contributes one more token, so each target pass yields between 1 and M+1 tokens. With the speculative sampling rule, the output follows exactly the distribution of the target model. Some models are trained with built-in multi-token prediction heads that serve as drafters [DEEPSEEKV3]. 1.2. Problem Statement When drafting and verification run in one process, the coordination is internal. Large deployments separate them: several draft model instances, possibly of different sizes, serve many concurrent requests and feed one or more target-model verifier instances, and both exchange token state and KV caches through a multi-level cache spanning accelerator memory, host memory, and networked storage [I-D.li-kvcache-efficient-storage]. In that setting the following must be agreed between independently implemented components: * how requests are assigned to draft instances according to their load, so that the requests of one batch can use different drafters; Li, et al. Expires 6 May 2027 [Page 3] Internet-Draft SDCP November 2026 * how many tokens a drafter proposes, given its history of accepted proposals, its load, and the bandwidth available to the cache; * how a proposal refers to draft state in the cache instead of carrying it, so that the verifier can assemble large verification batches from many drafters; * which acceptance rule the verifier applies, and how the rule can follow user preferences; * what happens when proposals keep being rejected: reduce the speculation length, switch to another drafter, let the target model decode by itself if it has spare capacity, or fail; and * how one authoritative output is maintained across retries, concurrent proposals, and instance failures. 1.3. Protocol Overview SDCP addresses these needs with the following elements: * an immutable Session Contract that fixes the models, the acceptance rule, limits, and the escalation policy (Section 4.1); * a committed-prefix digest chain that every proposal must extend, which makes stale or forked proposals detectable (Section 4.2); * assignment, proposal, verdict, and output messages in CBOR [RFC8949], described in CDDL [RFC8610] and exchanged over HTTP (Section 5); * a verification procedure with registered acceptance rules (Section 6.3); and * speculation-length control and load-aware escalation (Section 6.5). The verifier is the sole commit authority. Drafters, routers, and caches can fail or misbehave without changing the committed output, except as discussed in Section 9. Li, et al. Expires 6 May 2027 [Page 4] Internet-Draft SDCP November 2026 1.4. Scope SDCP does not define model architectures, sampling kernels, batching algorithms, or client-facing inference APIs. It assumes a multi- level cache that stores and serves opaque objects named by URI and supports leases, such as KVSP [I-D.li-kvcache-efficient-storage]. Draft-model training, tree-structured speculation, and multiple verifiers per session are out of scope for this version. 2. Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. Target model: The model whose output distribution the service promises. Draft model: A cheaper model, or auxiliary prediction head, that proposes tokens. Router: The component that admits requests, creates sessions, and assigns them to drafters. Drafter: An instance that runs a draft model. Verifier: An instance that runs the target model and commits output. Multi-Level Cache (MLC): Storage for prompts, KV caches, and draft artifacts across memory and storage tiers, reachable by all instances. Session: One generation task, identified by a 16-octet identifier. Committed prefix: The prompt followed by all output tokens committed so far; its length in output tokens is n. Round: One proposal and its verdict. Rounds of a session are numbered from 1. Speculation length (M): The number of tokens in a proposal. Acceptance rule: The algorithm that decides how many proposed tokens are kept (Section 6.4). Epoch: A counter that identifies the current verifier authority of a Li, et al. Expires 6 May 2027 [Page 5] Internet-Draft SDCP November 2026 session (Section 6.9). 3. Architecture requests output tokens Client -----------> +--------+ +-------------> Client | Router | | +--------+ | assignment / | \ events | v | \ | +-----------+ | +-----------+-+---+ | Drafter 1 | | | Verifier | +-----------+ | | (target model) | +-----------+ | +--------+--------+ | Drafter 2 |--+--proposal->|verdict +-----+-----+ <---------+ | | v draft KV, q rows v target KV, commits +-------------------------------------+ | Multi-Level Cache | +-------------------------------------+ Figure 1: SDCP Roles The router creates a session at a verifier, which computes or loads the KV cache of the prompt, and assigns the session to a drafter. The drafter proposes tokens to the verifier. The verifier answers each proposal with a verdict that both commits output and steers the next round. Draft state that is large, such as the draft model's KV cache and probability rows, stays in the MLC and is referenced by URI. 4. Session Model 4.1. Session Contract The Session Contract is fixed when the session is created: Li, et al. Expires 6 May 2027 [Page 6] Internet-Draft SDCP November 2026 contract = { target: hash32, ; SHA-256 of target model weights tokenizer: hash32, ; SHA-256 of tokenizer definition ? drafters: [+ hash32], ; permitted draft models rule: uint, ; acceptance rule (registry), fixed ? sampling: sampling, m_max: uint .le 64, ; upper bound on M max_new: uint, ; maximum output tokens stop: [* uint], ; stop tokens low_rounds: uint .ge 1, ; R, escalation period ? low_ratio: uint .le 100, ; percent, default 0 fallback: [+ ladder], ; escalation ladder ? self_load: uint .le 100, ; max verifier load for self-decode ? self_max: uint, ; max tokens per self-decode period ? scope: bstr .size 16 ; KVSP Scope ID of cache objects } sampling = { ? temperature_milli: uint, ; temperature x 1000 ? top_k: uint, ? top_p_milli: uint .le 1000, ? seed: uint } hash32 = bstr .size 32 action = a-continue / a-reduce / a-switch / a-self-decode / a-finish / a-fail ladder = a-reduce / a-switch / a-self-decode / a-fail a-continue = 0 a-reduce = 1 a-switch = 2 a-self-decode = 3 a-finish = 4 a-fail = 5 The contract digest is the SHA-256 of the contract encoded with the core deterministic encoding of Section 4.2.1 of [RFC8949], prefixed with a domain separator: contract_digest = SHA-256( "SDCP-CONTRACT-1" || 0x00 || deterministic_cbor(contract) ) The sampling profile defines the target distribution p used by the acceptance rule: p is the target model's next-token distribution after applying temperature, top-k, and top-p in that order; a temperature of zero selects the most probable token (ties broken in favor of the smallest token identifier). An absent "drafters" member permits any draft model that the router's local policy allows. The router can change the sampling profile at a round boundary (Section 6.6); such updates are carried separately from the contract, Li, et al. Expires 6 May 2027 [Page 7] Internet-Draft SDCP November 2026 which, including the acceptance rule, is never modified. The contract digest is computed once, over the contract as created. A contract that contains a key not defined by this document MUST be rejected with problem type "sdcp-bad-contract", so that all parties compute the contract digest over the same content. 4.2. Committed-Prefix Digest Chain The verifier maintains a digest over the committed prefix. For a prompt of P tokens t[0..P-1] and committed output tokens o[1..n]: d[0] = SHA-256( "SDCP-PREFIX-1" || 0x00 || contract_digest || uint32(P) || uint32(t[0]) || ... || uint32(t[P-1]) ) d[k] = SHA-256( d[k-1] || uint32(o[k]) ) for k = 1..n uint32() is four octets in network byte order and SHA-256 is defined in [RFC6234]. A proposal states the length n and digest d[n] of the prefix it extends. Because d[n] depends on every committed token, a proposal built on a different or outdated prefix is detected without comparing token lists. 5. Messages and Resources 5.1. HTTP Usage SDCP is an HTTP application following BCP 56 [RFC9205]. Each role has a base URI configured by the operator; SDCP defines no well-known URI. HTTP/2 [RFC9113] or HTTP/3 [RFC9114] over TLS 1.3 [RFC8446] with mutual certificate authentication is REQUIRED. Message bodies use the media type "application/sdcp+cbor". Errors use Problem Details [RFC9457]; clients handle unlisted status codes by class [RFC9110]. Li, et al. Expires 6 May 2027 [Page 8] Internet-Draft SDCP November 2026 +========+=============================+============+=============+ |Role | Method and Path |Body |Response | +========+=============================+============+=============+ |Verifier| POST sessions |session-open|session-state| +--------+-----------------------------+------------+-------------+ |Verifier| PUT sessions/{id}/drafter |assignment |session-state| +--------+-----------------------------+------------+-------------+ |Verifier| POST proposals |proposal |verdict | +--------+-----------------------------+------------+-------------+ |Verifier| GET |none |output | | | sessions/{id}/output?from=n | | | +--------+-----------------------------+------------+-------------+ |Verifier| POST sessions/{id}/policy |policy |session-state| +--------+-----------------------------+------------+-------------+ |Drafter | POST assignments |assignment |none (204) | +--------+-----------------------------+------------+-------------+ |Router | PUT instances/{id}/status |status |none (204) | +--------+-----------------------------+------------+-------------+ |Router | POST events |event |none (204) | +--------+-----------------------------+------------+-------------+ Table 1: SDCP Resources {id} in session paths is the base64url encoding without padding [RFC4648] of the session identifier. 5.2. Message Definitions session-id = bstr .size 16 cache-ref = tstr ; URI of an MLC object session-open = { session: session-id, epoch: uint, contract: contract, prompt: [* uint] / cache-ref, ; tokens or MLC reference ? prompt_kv: cache-ref, ; target KV of the prompt commit_log: cache-ref, ; durable commit records ? resume: { ; present on failover round: uint, committed: uint, digest: hash32, committed_tokens: cache-ref, ? target_kv: cache-ref } } session-state = { session: session-id, epoch: uint, round: uint, committed: uint, digest: hash32, ? state: uint } assignment = { session: session-id, epoch: uint, contract: contract, Li, et al. Expires 6 May 2027 [Page 9] Internet-Draft SDCP November 2026 verifier: tstr, ; verifier base URI drafter: tstr, ; drafter base URI prompt: [* uint] / cache-ref, round: uint, ; last round with a verdict ? sampling: sampling, ; current profile, if updated committed: uint, digest: hash32, ? committed_tokens: cache-ref, ; committed output so far m_init: uint } proposal = { session: session-id, epoch: uint, round: uint, drafter: tstr, ; drafter base URI base: uint, base_digest: hash32, tokens: [+ uint], ; M proposed tokens ? q_rows: cache-ref, ; draft rows, for rule 2 ? draft_kv: cache-ref, status: status } verdict = { session: session-id, epoch: uint, round: uint, accepted: uint, ; a, 0 <= a <= M emitted: [* uint], ; tokens committed this round committed: uint, digest: hash32, ; new n and d[n] action: action, m_cap: uint, low: uint, ; consecutive low rounds load: uint .le 100 ; verifier load, percent } output = { session: session-id, epoch: uint, from: uint, tokens: [* uint], final: bool, ? finish: action } status = { instance: tstr, role: 1 / 2, ; 1 drafter, 2 verifier load: uint .le 100, queue: uint, ? accept_milli: uint .le 1000, ; EWMA of a/M x 1000 ? cache_mbps: uint, ; available MLC bandwidth ttl_ms: uint } event = { session: session-id, epoch: uint, kind: action, committed: uint, digest: hash32 } policy = { session: session-id, from_round: uint, sampling: sampling } Li, et al. Expires 6 May 2027 [Page 10] Internet-Draft SDCP November 2026 The contract, sampling, hash32, and action types are defined in Section 4.1. Unknown map keys MUST be ignored, except in the contract (Section 4.1). A receiver that encounters an unknown action value MUST treat it as "fail". The "q_rows" object holds, for each proposed position, the draft distribution from which the proposed token was sampled, as a row of V IEEE 754 binary32 values in little-endian order, where V is the vocabulary size; rows are concatenated in position order. The drafter MUST sample each proposed token from exactly the row it publishes. 6. Procedures 6.1. Session Creation and Assignment The router selects a verifier and sends session-open with epoch 1 and a commit_log reference in the MLC. The verifier validates the contract (it MUST refuse a contract whose target digest does not match its model), computes or loads the KV cache of the prompt, computes d[0], and answers with session-state (round 0, committed 0). The router then selects a drafter whose model is permitted by the contract. It first informs the verifier with PUT sessions/{id}/drafter, and then sends the assignment, carrying the current round, committed length, and digest, to the drafter. The verifier accepts proposals only from the drafter named in the latest assignment it has received. Drafter selection is a local policy based on the drafters' status reports, for example preferring drafters with low load and queue and high recent acceptance. The router MUST NOT assign sessions to an instance whose status has expired. Each instance reports status at intervals shorter than its ttl_ms. 6.2. Drafting and Proposal A drafter holding an assignment prepares its own KV cache for the committed prefix (by computation or by loading from the MLC) and generates M tokens, with 1 <= M <= min(m_max, m_cap), where m_cap is the limit from the last verdict (m_init before the first). It writes the draft artifacts the verifier may need to the MLC and holds leases on them until it receives the verdict of the round. It then sends a proposal with base and base_digest equal to the committed length and digest it last learned, and round equal to the last verdict's round plus one. Li, et al. Expires 6 May 2027 [Page 11] Internet-Draft SDCP November 2026 A drafter MUST NOT have more than one outstanding proposal per session. It MAY prepare the next proposal speculatively, assuming full acceptance, but sends it only after the verdict confirms that assumption. Choice of M is local to the drafter. The following rule is RECOMMENDED. Let alpha be the drafter's exponentially weighted estimate of the per-token acceptance probability for the session, t_d the time to draft one token (including writing its artifacts at the cache bandwidth currently available), and t_v the expected verification time. The expected number of tokens committed per round is (1 - alpha^(M+1)) / (1 - alpha) for alpha < 1, and M + 1 for alpha = 1 [LEVIATHAN]; the drafter chooses the M within its limits that maximizes that quantity divided by (M * t_d + t_v). A loaded drafter thereby proposes fewer tokens, and a drafter whose cache bandwidth is scarce does the same. 6.3. Verification and Commit On receiving a proposal, the verifier performs these steps: 1. Authorization: the authenticated client MUST be the drafter named in the latest assignment the verifier received for the session; otherwise respond 403 with problem type "sdcp-not-assigned". 2. Fencing: if epoch differs from the session epoch, respond 409 with problem type "sdcp-stale-epoch". 3. Idempotency: if a verdict for this round already exists, return it unchanged; if the round is being verified for an identical earlier request, wait for that verdict and return it. 4. Base check: if base is not the committed length or base_digest is not d[base], respond 409 with problem type "sdcp-stale-base" and the current session-state, so that the drafter can resynchronize. If round is not the last round plus one, respond 409 likewise. 5. Preconditions of the rule: if the acceptance rule needs draft probabilities, the verifier MUST have obtained the rows for every proposed position, and checked that q[i](x[i]) is not zero for every position, before drawing any random value. If it cannot obtain them, it rejects the round with problem type "sdcp- q-unavailable" and commits nothing. Obtaining them first is necessary because abandoning a round only after a rejection was observed would bias the output distribution. The verifier MAY fetch the rows concurrently with scoring. Li, et al. Expires 6 May 2027 [Page 12] Internet-Draft SDCP November 2026 6. Scoring: the verifier computes the target distributions p[1..M+1] for the positions following the committed prefix, normally by a single forward pass over the proposed tokens using the target KV cache of the committed prefix. It MAY batch proposals of many sessions into one pass. 7. Acceptance: it applies the contract's acceptance rule (Section 6.4), yielding a accepted tokens and one additional token. It truncates the emitted tokens after the first stop token and at max_new. 8. Commit: it appends the emitted tokens to the committed prefix, updates n and d[n], and appends a commit record (epoch, round, emitted tokens, n, and d[n]) to the session's commit_log (Section 6.8); only after that append succeeds does it send the verdict and make the tokens available as output. If the append fails because the log has been fenced, the verifier MUST discard the round, release no output, and stop serving the session. The commit and the recording of the verdict MUST be atomic with respect to concurrent proposals for the session. 9. Control: it determines action and m_cap as specified in Section 6.5. After the verdict, the drafter MUST discard its draft state for proposed positions beyond the accepted ones and MAY release its leases. Its KV cache for the accepted tokens remains valid. The verifier MAY publish the target KV cache of committed tokens to the MLC so that another verifier can take over the session. 6.4. Acceptance Rules Rules are registered (Section 10.2.2). This document defines two. In both, p[i] is the target distribution at proposed position i, x[i] the proposed token, and positions are evaluated in order. Rule 1, greedy: x[i] is accepted if it equals argmax p[i], where ties are broken in favor of the smallest token identifier. Evaluation stops at the first position that is not accepted. The additional token is argmax p[a+1]. The output is identical to greedy decoding of the target model as computed by the verifier's scoring passes, whatever the drafter proposes. Rule 2, speculative sampling: For each position, draw r uniformly from [0, 1). Accept x[i] if r < min(1, p[i](x[i]) / q[i](x[i])), where q[i] is the row of q_rows for position i, that is, the draft distribution from which x[i] Li, et al. Expires 6 May 2027 [Page 13] Internet-Draft SDCP November 2026 was sampled. At the first rejected position, the additional token is drawn from the normalized residual distribution max(0, p[i] - q[i]). If all M tokens are accepted, the additional token is drawn from p[M+1]. With exact q values, the output has the distribution of the target model [LEVIATHAN] [CHEN]. The proposal MUST carry q_rows; a proposal for which q[i](x[i]) is zero is rejected, before any random draw, with problem type "sdcp-bad- proposal". The verifier's random values MUST be independent of those the drafter used. When the sampling profile contains a seed, the verifier derives each value from HMAC-SHA-256 [RFC2104] keyed with the seed (as eight octets in network byte order) over "SDCP- VERIFY-1" || 0x00 || session || uint32(round) || uint32(i) || uint8(purpose), where purpose is 0 for the acceptance test and 1 for drawing the additional token, taking the first 53 bits of the result divided by 2^53. Drafters MUST NOT use this derivation for their own sampling. Without a seed, the verifier uses a cryptographically secure random number generator. 6.5. Speculation Control and Escalation 6.5.1. Speculation Limit Each verdict carries m_cap, the largest M the drafter may use in the next round, with 1 <= m_cap <= m_max. The verifier lowers m_cap when its own load makes long proposals costly, and as specified for the "reduce" action below. It SHOULD restore m_cap gradually (for example by one per round) when rounds are no longer low and its load permits. 6.5.2. Low Rounds and Escalation A round is low if 100 * a < low_ratio * M, or, when low_ratio is zero or absent, if a = 0. The verifier keeps three counters per session, all starting at zero: c, the number of consecutive low rounds; g, the number of consecutive rounds that are not low; and j, the position in the contract's fallback list. After a low round, c is incremented and g reset; after a round that is not low, c is reset and g incremented, and when g reaches R, where R is low_rounds, j is reset to zero. When c reaches R, the verifier applies entry j of the fallback list (skipping entries that are not permitted, as described for self_decode, and advancing j past every entry it applies or skips), then resets c to zero. If no entry remains, it applies "fail". Otherwise it applies "continue". The fallback list MUST contain only the actions reduce, switch, self_decode, and fail; j persists across a change of drafter and across a self-decoding period. The actions are: Li, et al. Expires 6 May 2027 [Page 14] Internet-Draft SDCP November 2026 continue (0): The drafter proceeds with the next round. reduce (1): The verifier sets m_cap to max(1, floor(m_cap / 2)) and the drafter proceeds. switch (2): The verifier sends an event to the router, which assigns the session to another drafter with a permitted model; the current drafter MUST stop proposing for the session. self_decode (3): The verifier decodes the session itself with the target model. This action is permitted only if the contract contains self_load and the verifier's current load does not exceed it; otherwise the verifier skips it and applies the next entry of the fallback list. Each token committed during self-decoding is recorded as a round of its own, with no proposal, so round numbers stay consecutive. During self-decoding the verifier commits tokens one at a time, publishes them as output, and stops after self_max tokens (if present) or when its load exceeds self_load; it then sends an event so that the router assigns a drafter again. finish (4): Generation has ended because of a stop token or max_new. fail (5): The verifier ends the session with an error. Because self_decode is conditional on the verifier's load, the same fallback list leads to self-decoding on a lightly loaded verifier and to the next entry, such as switch, on a busy one. 6.6. Sampling Policy Updates The router can change the sampling profile of a session, for example to follow a preference stated by the user. It sends a policy message with from_round greater than the current round. The verifier applies the new profile to rounds numbered from_round and later, and the router includes it in the "sampling" member of subsequent assignments; the contract in those assignments is unchanged. The acceptance rule cannot be changed. Committed output is never changed by a policy update. 6.7. Output Delivery Committed tokens are delivered with GET output?from=n, which returns the tokens committed at output indices n and later, waiting up to a server-chosen time if none are available. Only tokens whose commit record has been written are returned. Indices are stable and every output message carries the session epoch, so a front end that retries never duplicates or loses tokens, and a front end that receives output with a lower epoch than one it has seen discards it. The Li, et al. Expires 6 May 2027 [Page 15] Internet-Draft SDCP November 2026 final output message has final set to true and finish set to "finish" or "fail". 6.8. Commit Log The commit_log is an MLC object to which records are appended with a conditional operation, for example an HTTP PUT with If-Match on the current entity tag [RFC9110], so that an append succeeds only if no other append has happened since the writer last read the log. Each record carries the epoch of its writer. A commit record appended by a verifier MUST carry the session's current epoch and follow the last record it wrote. A fence record carries only an epoch; after a fence record with epoch E+1, appends carrying an epoch lower than E+1 MUST fail. If the MLC cannot enforce this rule, the verifier MUST read the log after its conditional append and treat its append as failed if a fence record with a higher epoch precedes its own record. 6.9. Failure Handling and Recovery * Drafter failure: when a drafter's status expires, the router reassigns its sessions. The new drafter starts from the committed prefix in the assignment. Proposals from the failed drafter are rejected because it is no longer the assigned drafter. * Verifier failure: the router first appends a fence record with the new epoch to the commit_log, so that the old verifier, even if it is only partitioned, can no longer commit or release output; it then reads the last commit record and opens the session at a new verifier with that epoch and the "resume" member carrying that round, committed length, digest, and the target KV cache from the MLC if available. It then informs the verifier of the drafter and sends a new assignment. Proposals carrying an older epoch are rejected, and front ends discard output carrying an older epoch, so a recovered old verifier cannot extend the session. * Lost messages: proposals and their verdicts are idempotent per round, so a drafter retries a proposal whose verdict it did not receive. 7. Error Handling +=====================+========+==============================+ | Type suffix | Status | Meaning | +=====================+========+==============================+ | sdcp-stale-epoch | 409 | Epoch is not current | +---------------------+--------+------------------------------+ | sdcp-stale-base | 409 | Proposal does not extend the | | | | committed prefix | Li, et al. Expires 6 May 2027 [Page 16] Internet-Draft SDCP November 2026 +---------------------+--------+------------------------------+ | sdcp-q-unavailable | 409 | Draft probabilities required | | | | by the rule are unavailable | +---------------------+--------+------------------------------+ | sdcp-bad-contract | 422 | Contract invalid or not | | | | supported | +---------------------+--------+------------------------------+ | sdcp-limit | 422 | Proposal violates m_cap or | | | | m_max | +---------------------+--------+------------------------------+ | sdcp-session-closed | 410 | Session has finished or | | | | failed | +---------------------+--------+------------------------------+ | sdcp-bad-proposal | 422 | Proposal inconsistent with | | | | its draft distributions | +---------------------+--------+------------------------------+ | sdcp-not-assigned | 403 | Drafter is not assigned to | | | | the session | +---------------------+--------+------------------------------+ Table 2: SDCP Problem Types The type URI is "https://iana.org/assignments/sdcp#" followed by the suffix. These types are specific to SDCP and are therefore registered in an SDCP registry rather than in the "HTTP Problem Types" registry (Section 4.2 of [RFC9457]). Problem details for "sdcp-stale-base" include the member "session_state" carrying the current committed length and digest. 8. Operational Considerations * Batch windows: a verifier that waits to fill a batch adds that wait to every round. Windows of a fraction of the target model's forward-pass time are typical. * Cache placement: draft artifacts are short-lived and SHOULD be placed in the fastest tier that both drafter and verifier can reach. * Monitoring: implementations SHOULD report the mean accepted length, the distribution of M, escalation counts by action, stale- base rejections, and self-decoded tokens. * Incremental deployment: a verifier can serve sessions without drafters by decoding itself, so SDCP can be introduced one drafter pool at a time. Li, et al. Expires 6 May 2027 [Page 17] Internet-Draft SDCP November 2026 9. Security Considerations All exchanges use mutual TLS, and every role authorizes requests by authenticated identity: only the router opens sessions and assigns drafters, and only the assigned drafter submits proposals. Integrity of output. Under the greedy rule, the committed output depends only on the target model and the committed prefix; a malicious drafter can waste verifier capacity but cannot change the output. Under speculative sampling, the drafter supplies q, and a drafter that reports false probabilities can bias the output (for example, by reporting a small q for tokens it wants accepted). Speculative sampling MUST therefore be used only with drafters that are trusted to the same degree as the verifier; with other drafters the greedy rule MUST be used. Denial of service. Proposals cost a forward pass. Verifiers MUST enforce m_cap and SHOULD rate-limit rounds per session and per drafter. The escalation ladder also bounds the work a poorly matched or hostile drafter can cause. Cache objects. References in proposals point to the MLC; the verifier MUST fetch them only from caches it trusts for the session's scope, so that a proposal cannot direct the verifier to arbitrary URIs. Privacy. Prompts, KV caches, and outputs are user data. The privacy considerations of [I-D.li-kvcache-efficient-storage] apply to all MLC objects of a session. 10. IANA Considerations 10.1. Media Type IANA is requested to register the media type "application/sdcp+cbor" in the "Media Types" registry [RFC6838]: Type name: application; Subtype name: sdcp+cbor; Required parameters: N/A; Optional parameters: N/A; Encoding considerations: binary; Security considerations: see Section 9 of RFC XXXX; Interoperability considerations: none; Published specification: RFC XXXX; Applications that use this media type: distributed inference systems using speculative decoding; Fragment identifier considerations: the syntax and semantics of fragment identifiers specified for "application/ cbor" apply; Additional information: Deprecated alias names for this type: N/A, Magic number(s): N/A, File extension(s): N/A, Macintosh file type code(s): N/A; Person and email address to contact: IETF ; Intended usage: COMMON; Restrictions on usage: none; Author: IETF; Change controller: IETF. Li, et al. Expires 6 May 2027 [Page 18] Internet-Draft SDCP November 2026 10.2. SDCP Registries IANA is requested to create a registry group named "Speculative Decoding Coordination Protocol (SDCP)" at "https://www.iana.org/assignments/sdcp" with the registries below. For the numeric registries, values 0-191 are assigned by Specification Required [RFC8126] and values 192-255 are reserved for Private Use. The designated experts should check that a new acceptance rule states whether it preserves the target distribution and what trust it requires in drafters. 10.2.1. SDCP Problem Types Columns: Type Suffix, Title, Recommended HTTP Status Code, Reference. The type URI is "https://iana.org/assignments/sdcp#" followed by the suffix. Initial contents: the entries of Table 2, with reference RFC XXXX. Policy: Specification Required. 10.2.2. SDCP Acceptance Rules Initial contents: 0 Reserved; 1 greedy; 2 speculative-sampling; each with reference RFC XXXX. A registration MUST specify the acceptance test, the additional token, and the information a proposal must carry. 10.2.3. SDCP Actions Initial contents: 0 continue; 1 reduce; 2 switch; 3 self_decode; 4 finish; 5 fail. 11. References 11.1. Normative References [RFC2104] Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: Keyed- Hashing for Message Authentication", RFC 2104, DOI 10.17487/RFC2104, February 1997, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, . Li, et al. Expires 6 May 2027 [Page 19] Internet-Draft SDCP November 2026 [RFC6234] Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms (SHA and SHA-based HMAC and HKDF)", RFC 6234, DOI 10.17487/RFC6234, May 2011, . [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013, . [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018, . [RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, June 2019, . [RFC8949] Bormann, C. and P. Hoffman, "Concise Binary Object Representation (CBOR)", STD 94, RFC 8949, DOI 10.17487/RFC8949, December 2020, . [RFC9110] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, June 2022, . [RFC9113] Thomson, M., Ed. and C. Benfield, Ed., "HTTP/2", RFC 9113, DOI 10.17487/RFC9113, June 2022, . [RFC9114] Bishop, M., Ed., "HTTP/3", RFC 9114, DOI 10.17487/RFC9114, June 2022, . Li, et al. Expires 6 May 2027 [Page 20] Internet-Draft SDCP November 2026 [RFC9205] Nottingham, M., "Building Protocols with HTTP", BCP 56, RFC 9205, DOI 10.17487/RFC9205, June 2022, . [RFC9457] Nottingham, M., Wilde, E., and S. Dalal, "Problem Details for HTTP APIs", RFC 9457, DOI 10.17487/RFC9457, July 2023, . 11.2. Informative References [CHEN] Chen, C., Borgeaud, S., Irving, G., Lespiau, J.-B., Sifre, L., and J. Jumper, "Accelerating Large Language Model Decoding with Speculative Sampling", arXiv:2302.01318, February 2023, . [DEEPSEEKV3] DeepSeek-AI, "DeepSeek-V3 Technical Report", arXiv:2412.19437, December 2024, . [I-D.li-kvcache-efficient-storage] Li, Z., Du, Z., Wang, J., Cheng, W., Zhao, Y., Zhao, C., and X. Sun, "Key-Value (KV) Cache Storage Protocol (KVSP): Indexing, Placement, Prefetch, and Migration for Distributed Inference", Work in Progress, Internet-Draft, draft-li-kvcache-efficient-storage-00, 2 November 2026, . [LEVIATHAN] Leviathan, Y., Kalman, M., and Y. Matias, "Fast Inference from Transformers via Speculative Decoding", Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR 202, July 2023, . Appendix A. Test Vectors A contract with greedy rule, m_max 8, max_new 256, stop token 2, low_rounds 3, and fallback ladder [reduce, switch, self_decode, fail], with target and tokenizer digests equal to the KVSP test vector digests, has the following deterministic CBOR encoding (143 octets): Li, et al. Expires 6 May 2027 [Page 21] Internet-Draft SDCP November 2026 a8 64 72 75 6c 65 01 64 73 74 6f 70 81 02 65 6d 5f 6d 61 78 08 66 74 61 72 67 65 74 58 20 25 4e 52 8c 66 9a 20 84 c9 72 72 98 ca cb 83 e8 85 4e 0a 6f f8 0e 2b f1 80 96 b9 dc e0 f7 eb da 67 6d 61 78 5f 6e 65 77 19 01 00 68 66 61 6c 6c 62 61 63 6b 84 01 02 03 05 69 74 6f 6b 65 6e 69 7a 65 72 58 20 9f 42 a7 a2 9d ca 30 63 ad 26 d7 be 7f e6 a9 30 fa 38 af b1 a7 03 e4 f7 ff 53 8d b4 e1 33 b1 9c 6a 6c 6f 77 5f 72 6f 75 6e 64 73 03 For the prompt [1, 2, 3, 4] and committed output tokens 17 and 42: contract_digest = bcae60a92c622b03fa1c229bac9f5654 076957e2f1a8bedcb3549ce4a6328e1f d[0] = 186b373698d5baf27716830811ffa641 4887dd0a4f9d13db6c00e19bdb57b76e d[1] = 3372e0fd91b77611c3502da5b0a5c8a5 1540e055d7abdd0d1bcf91110e142c1c d[2] = 469bbe77ceb5b1540a4d61d826a3117e 357e097c5ac20271306633559fccdf1f Example round under the greedy rule: committed n = 2; the drafter proposes [7, 9, 11] (M = 3); the target argmax tokens at the three positions are 7, 9, and 13, and at the fourth position 5. Then a = 2, the additional token is 13 (the correction at the first mismatch), emitted = [7, 9, 13], and the new committed length is 5. The round is not low, so c is reset and the action is "continue". Acknowledgments The authors thank colleagues working on disaggregated inference serving for discussions on separating drafting from verification. Authors' Addresses Zhiqiang Li China Mobile Beijing 100053 China Email: lizhiqiangyjy@chinamobile.com Zongpeng Du China Mobile Beijing 100053 China Li, et al. Expires 6 May 2027 [Page 22] Internet-Draft SDCP November 2026 Email: duzongpeng@chinamobile.com Junjie Wang Centec Shanghai 201203 China Email: wangjj@centec.com Wei Cheng Centec Shanghai 201203 China Email: chengw@centec.com Ying Zhao Centec Shanghai 200000 China Email: zhaoying@centec.com Chunhao Zhao SAIA Shanghai 200125 China Email: chunhao.zhao@sh-aia.com Xun Sun Inesa Shanghai 200030 China Email: sunxun@inesa.com Li, et al. Expires 6 May 2027 [Page 23]