Cerebras Details Trillion-Parameter AI Model Deployment on Wafer-Scale Chips

Cerebras Details Trillion-Parameter AI Model Deployment

On October 22, 2026, Cerebras published a detailed technical breakdown of how it deploys trillion-parameter AI models on its wafer-scale chips. The company, which recently went public on the Nasdaq, claims its WSE-3T wafers can run models with 3 trillion parameters at speeds of 1,000 to 2,000+ tokens per second—far outpacing GPU-based systems.

The blog post, authored by Sarah Chieng, uses the open-source Kimi K3 model as a working example. With 3 trillion parameters, 69 linear-attention layers, and 24 dense layers, Kimi K3 serves as an ideal illustration of Cerebras's approach to handling models that require terabytes of memory for weights alone.

The Challenge: Memory and Scale

A single Cerebras WSE-3T wafer packs 44 GB of on-chip SRAM. While impressive for a single chip, trillion-parameter models need far more. With 4-bit weight quantization, Kimi K3's 3 trillion parameters translate to roughly 1.5 TB of raw weight data—requiring approximately 35 wafers just to store the model's weights.

Cerebras's solution is to distribute the model across multiple wafers, connecting them into a high-speed pipeline. The company's compiler maps model layers to specific wafer regions, and the runtime establishes paths for tokens to flow between wafers via RDMA over Ethernet, with latency of roughly 2 microseconds per hop.

Breaking Down the Deployment

For Kimi K3, each layer requires about 16 GB of storage. Dense layers—which use full attention and require key-value (KV) caches—are placed one per wafer, leaving 28 GB for cache. Linear-attention layers, which maintain small fixed-size states, can fit two per wafer (32 GB total). This results in 35 wafers for the 69 linear layers and 24 wafers for the dense layers, totaling 59 wafers.

Critically, because each layer fits entirely on a single wafer, model weights never need to move during inference. Only activations cross between wafers, minimizing latency and maximizing throughput.

Pipelining for High Throughput

Cerebras addresses a common misconception about its architecture: that a token traveling through 59 sequential wafers would leave most of the system idle. In reality, the system operates like an assembly line, with each wafer processing a different request simultaneously.

As Sarah Chieng explains, each wafer can handle a micro-batch of requests, and the number of requests in flight is roughly the number of pipeline stages multiplied by the micro-batch size. This pipelining approach enables sustained high throughput without the batch-size limitations of traditional GPU inference.

Concurrency and KV Cache Limitations

The maximum number of concurrent requests is determined by the KV cache memory on dense-layer wafers. With an 8-bit KV cache and a 256,000-token context window, each request consumes significant memory. After storing one dense layer per wafer, the remaining 28 GB of SRAM limits how many requests can be processed simultaneously.

While concurrency is important, Cerebras emphasizes that the more meaningful metric is token throughput per unit of power. The company claims its systems deliver 6.7 times faster performance than the next-fastest GPU cloud provider, based on independent benchmarking by Artificial Analysis.

Market Context and Validation

Cerebras's approach has gained significant traction. In January 2026, OpenAI signed a multiyear agreement worth over $20 billion to deploy 750 megawatts of Cerebras wafer-scale systems from 2026 through 2028. The two companies collaborated on Codex-Spark, a low-latency coding model that exceeds 1,000 tokens per second.

Independent testing has validated these claims. Artificial Analysis measured Cerebras running Moonshot AI's Kimi K2.6 at 981 output tokens per second—23 times faster than the median inference provider. On a representative coding request, Cerebras completed the task in 5.6 seconds versus 163.7 seconds on Kimi's official endpoint.

Scaling Beyond 3 Trillion Parameters

Cerebras's approach scales linearly: each additional wafer adds compute and on-chip memory. The same mathematical framework—calculating weight memory, distributing layers, and connecting wafers—applies to models beyond 3 trillion parameters.

This scalability, combined with the company's direct hardware sales model, positions Cerebras as a viable alternative to GPU-based inference, particularly as GPU availability tightens and costs rise. With its Nasdaq listing and OpenAI partnership, Cerebras is demonstrating that wafer-scale architecture can compete at the frontier of AI inference.