Block-Level Training Aims to Optimize Speculative Decoding

What the research proposes

A research paper associated with Grafy Research Lab proposes block-level training for speculative decoding, the technique in which a draft model proposes multiple tokens for a target model to verify in one pass, according to the source. The source was published on 2026-11-11 and describes the proposal as an effort to improve expected acceptance length, the metric the source says governs speculative decoding speedup. The work matters because standard training objectives may not directly target that metric, according to the source.

Why expected acceptance length matters

Expected acceptance length is presented by the source as a key measure in speculative decoding. The source explains that the speedup of the technique is tied to this value. Standard objectives such as cross-entropy, KL divergence, and total variation focus on distribution matching or single-step overlap, according to the source. The source adds that these objectives may not address EAL in the way that block-level methods attempt to do. The source says the block-level objectives are intended to aim directly at EAL.

What the source says about existing objectives

According to the source, cross-entropy, KL divergence, and total variation are standard training objectives. The source adds that block-level objectives have been proposed to optimize EAL directly. The source also states that the proposed objectives remain surrogates rather than exact formulations of EAL. The source mentions that some methods average later acceptance over target distribution, according to the source text. The source records comments noting 19 pages and 10 figures, and lists subject categories including machine learning and computation and language.

What the surrogate limitation means

The source indicates that block-level training objectives are proposed as alternatives, but the source qualifies that they are surrogates rather than exact formulations of EAL. That distinction matters because a surrogate may not be equivalent to the target metric. The source does not claim the block-level methods have fully solved the problem; it says they aim to optimize EAL and that they remain surrogates in the source's description. The source's concern appears to be whether optimizing a proxy can produce the desired acceptance behavior.

How Grafy researchers and developers might engage

For creators, developers, and researchers who use AI tools, the story sits at the intersection of model training, inference, and evaluation. Grafy Research Lab has been associated with work on models and evaluation, and Grafy has described running many cloud models through providers such as OpenRouter, Vertex AI, fal, and Replicate. A practitioner interested in speculative decoding might use Grafy Chat to compare how different models respond to the same prompts, though Grafy does not offer a speculative decoding training objective as a product feature. A researcher who wants to examine the source paper in depth can use Reader to read, search, and question the material.

Frequently asked questions

Are block-level training objectives exact formulations of expected acceptance length?

No, according to the source. The source reports that block-level objectives have been proposed to optimize expected acceptance length directly, but says they remain surrogates rather than exact formulations of that measure. That distinction matters because a surrogate may guide training without guaranteeing the precise behavior researchers want.

Do standard cross-entropy, KL divergence, and total variation objectives optimize expected acceptance length?

According to the source, they do not. Standard training objectives such as cross-entropy, KL divergence, and total variation focus on distribution matching or single-step overlap. The source adds that expected acceptance length governs speculative decoding speedup, so those objectives may leave the quantity directly targeted by newer block-level methods inadequately addressed.

How might researchers examine expected acceptance length in speculative decoding?

The source does not provide a test procedure, but it centers expected acceptance length as the governing metric for speculative decoding speedup. Researchers could examine how draft proposals are weighted or averaged and whether training changes acceptance behavior. The source notes that block-level objectives are proposed as alternatives, while cautioning that they are surrogates.

---

Source: Grafy Research Lab — REAL Loss: Optimizing Real Expected Acceptance Length for Speculative Decoding