Researchers propose DCOCO to improve speculative decoding throughput in large language models

Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are verified in parallel by the target model. Recent diffusion-based drafters make this paradigm attractive by combining strong predictions from deep attention-based neural backbones with the avoidance of sequential autoregressive costs. However, this parallelism removes intra-block dependencies that would normally be available under autoregressive generation, as tokens within a draft block are predicted without access to the realized identities of preceding draft tokens.

To address this limitation, the authors introduce DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores these dependencies while preserving parallel drafting. Instead of committing to each position’s top prediction, DCOCO retains multiple candidate tokens and evaluates their contextual compatibility across adjacent draft positions in parallel. This enables contextually informed token selection at low additional cost.

The approach was evaluated across four verifier backbones and ten benchmarks, where DCOCO achieved the highest average throughput in every evaluated configuration. Speedups of up to 6.6x over vanilla decoding were observed, spanning greedy and stochastic decoding with and without extended reasoning.

The advantage of DCOCO persists under concurrent serving, where it outperforms the strongest competing method by 6–23% across 4–32 concurrent requests. Other sequential correctors fall below uncorrected DFlash at these loads, demonstrating that recovering intra-block dependencies through parallel contextual scoring delivers consistent acceleration across decoding regimes and serving loads.