Skip to content
Research · Nov 6, 2026

Researchers propose BlockLock for reproducible LLM inference with fixed batch geometry

BlockLock maintains shared batched prefill while pinning only the numerical path of protected requests, reducing latency by up to 55% and improving throughput by up to 122% relative to vLLM Batch-Invariant in evaluations across five dense models and three NVIDIA GPUs.

Trust79
HypeLow hype

1 source · cross-referenced

ShareXLinkedInEmail
TL;DR
  • BlockLock introduces two modes—BlockLock-Rows and BlockLock-Pack—to preserve output reproducibility for protected requests in batched LLM inference.
  • Evaluated on five dense models (Qwen3-8B/14B, Llama-3.1-8B-Instruct, Gemma-3-4B/12B-IT) and three NVIDIA GPUs (A6000, A100, H100).
  • BlockLock-Rows reduces latency by 53.5–55.0% and improves throughput by 115.2–122.2% over vLLM Batch-Invariant at equal delivered work.
  • Both modes preserve target sequences and Top20-logit bits under stated per-deployment contracts.

Researchers from an unnamed institution propose BlockLock, a method to achieve reproducible inference for large language models (LLMs) in batched settings. The core idea is to fix the geometry that a protected request sees while allowing shared batched prefill, thereby preserving the numerical path of the target request even as batch companions change.

The approach introduces two modes: BlockLock-Rows, which uses fixed-width rows and retains batched decode for all requests, and BlockLock-Pack, which employs terminal-anchored packing with an isolated dummy segment and extracts the protected request’s KV state for target-only decode. Under a stated per-deployment contract, both modes preserve the target’s output bitwise when its companions change.

The authors evaluate BlockLock across five dense models—Qwen3-8B, Qwen3-14B, Llama-3.1-8B-Instruct, Gemma-3-4B, and Gemma-3-12B-IT—and three single-GPU deployments (NVIDIA A6000, A100, and H100). They report results on tinyBenchmarks and BFCL, including initial tensor-parallel experiments on two NVIDIA A6000 GPUs.

Compared with vLLM Batch-Invariant, BlockLock-Rows reduces latency by 53.5–55.0% and improves throughput by 115.2–122.2% at equal delivered work. Unlike batch-invariant kernels, BlockLock does not require custom kernel implementations and constrains only the protected requests, simplifying deployment.

The paper emphasizes that reproducibility is achieved without sacrificing shared prefill compute or resorting to padding in decode, addressing limitations of prior approaches such as isolating prefill with fixed-shape decode.

Sources
  1. 01arXiv cs.AI — BlockLock: Reproducible LLM Inference with Fixed Batch Geometry
Also on Research

Stories may contain errors. Dispatch is assembled with AI assistance and curated by human editors; despite the trust-score filter, mistakes happen. We correct publicly — every article links to its revision history. Nothing here is financial, legal, or medical advice. Verify before relying on any claim.

© 2026 Dispatch. No ads. No sponsorships. No paid placement. Reader-supported via Ko-fi.

Built by a person who cares about honest AI news.