Cerebras details how trillion-parameter models are deployed across wafer-scale chips
A technical post explains splitting a 3T-parameter model across 59 Cerebras WSE-3T wafers, enabling up to 2,000+ tokens/sec with 4-bit weights and pipeline parallelism.
1 source · single source
- Cerebras describes a method to split a 3-trillion-parameter model across 59 wafer-scale chips, keeping most weights on-chip and streaming activations between chips.
- The company reports up to 2,000+ tokens/sec generation for trillion-parameter models using 4-bit weight quantization and pipeline parallelism.
- Kimi K3 (≈3T parameters) is used as an example; it achieves 88.3% on Terminal-Bench 2.1 and 91.2% on BrowseComp, per the post.
- Concurrency is achieved by keeping the pipeline full with micro-batches across stages, avoiding batch-size-one bottlenecks.
Cerebras published a technical explanation of how it deploys multi-trillion-parameter models on its wafer-scale chips, using the open-source Kimi K3 model (~3 trillion parameters) as a concrete example.
The company says a single Cerebras WSE-3T wafer has 44 GB of on-chip SRAM, which is insufficient to store all model weights in full precision. To fit the model, Cerebras applies 4-bit weight quantization and splits the model across multiple wafers. For Kimi K3, this results in 59 wafers: 35 wafers for the 69 linear-attention layers (two layers per wafer) and 24 wafers for the 24 dense layers (one layer per wafer). Dense layers also require on-chip KV cache space, further constraining wafer allocation.
Cerebras reports that once mapped, the model runs as a pipeline where only activations move between wafers via RDMA over Ethernet, with roughly 2 microseconds per hop. Because each layer is about 16 GB and wafers have 44 GB of SRAM, most layers fit entirely on a single wafer, minimizing weight movement.
To serve many requests concurrently without falling to a batch-size-of-one, Cerebras pipelines micro-batches across stages: while one stage processes a token for request A, the previous stage processes a token for request B, and the next stage processes a token for request C. Cerebras notes that in production, stages are finer-grained than single layers, enabling higher concurrency and throughput.
The post states that the maximum number of concurrent requests is determined by the on-chip memory left for KV cache on dense-layer wafers. For a 256k-token average request size, the company calculates that each dense-layer wafer can support a specific number of concurrent requests based on remaining SRAM after placing one dense layer.
Cerebras claims up to 1,000–2,000+ tokens per second generation for trillion-parameter models under these configurations, attributing the speed to on-chip weight residency and pipeline parallelism rather than off-chip memory movement.
- Nov 11, 2026 · Google AI — Blog
Google launches Gemini Decisions API with new Gemini Decision 1 model for fast, calibrated multimodal decisions
Trust78 - Nov 11, 2026 · OpenAI — News
OpenAI offers free ChatGPT Plus to more than 20,000 veterans and servicemembers
Trust75 - Nov 11, 2026 · Latent Space — swyx
Anthropic releases Claude Fable 5.2 and Mythos 5.2, claiming top LLM spot on Anthropic’s IPO roadshow day
Trust76