Researchers propose BlockLock for reproducible LLM inference with fixed batch geometry
BlockLock maintains shared batched prefill while pinning only the numerical path of protected requests, reducing latency by up to 55% and improving throughput by up to 122% relative to vLLM Batch-Invariant in evaluations across five dense models and three NVIDIA GPUs.
1 source · cross-referenced
- BlockLock introduces two modes—BlockLock-Rows and BlockLock-Pack—to preserve output reproducibility for protected requests in batched LLM inference.
- Evaluated on five dense models (Qwen3-8B/14B, Llama-3.1-8B-Instruct, Gemma-3-4B/12B-IT) and three NVIDIA GPUs (A6000, A100, H100).
- BlockLock-Rows reduces latency by 53.5–55.0% and improves throughput by 115.2–122.2% over vLLM Batch-Invariant at equal delivered work.
- Both modes preserve target sequences and Top20-logit bits under stated per-deployment contracts.
Researchers from an unnamed institution propose BlockLock, a method to achieve reproducible inference for large language models (LLMs) in batched settings. The core idea is to fix the geometry that a protected request sees while allowing shared batched prefill, thereby preserving the numerical path of the target request even as batch companions change.
The approach introduces two modes: BlockLock-Rows, which uses fixed-width rows and retains batched decode for all requests, and BlockLock-Pack, which employs terminal-anchored packing with an isolated dummy segment and extracts the protected request’s KV state for target-only decode. Under a stated per-deployment contract, both modes preserve the target’s output bitwise when its companions change.
The authors evaluate BlockLock across five dense models—Qwen3-8B, Qwen3-14B, Llama-3.1-8B-Instruct, Gemma-3-4B, and Gemma-3-12B-IT—and three single-GPU deployments (NVIDIA A6000, A100, and H100). They report results on tinyBenchmarks and BFCL, including initial tensor-parallel experiments on two NVIDIA A6000 GPUs.
Compared with vLLM Batch-Invariant, BlockLock-Rows reduces latency by 53.5–55.0% and improves throughput by 115.2–122.2% at equal delivered work. Unlike batch-invariant kernels, BlockLock does not require custom kernel implementations and constrains only the protected requests, simplifying deployment.
The paper emphasizes that reproducibility is achieved without sacrificing shared prefill compute or resorting to padding in decode, addressing limitations of prior approaches such as isolating prefill with fixed-shape decode.
- Nov 11, 2026 · The Verge — AI
Google DeepMind unveils AlphaGenome 2 to predict genetic variant effects on cell types
Trust79 - Nov 11, 2026 · Google DeepMind — Blog
Google DeepMind unveils AlphaGenome 2 to predict DNA variant effects in specific cell types
Trust84 - Nov 11, 2026 · arXiv cs.AI
Researchers demonstrate auditable human-agent teams for autonomous synchrotron X-ray experiments
Trust79