Advanced

Cerebras, Groq and Dataflow Accelerators

Cerebras and Groq attack the same wall from the same direction: keep the working set in on-chip SRAM, which is tens of times faster than HBM, so that memory-bound decode stops being memory-bound. Cerebras does it with a wafer-scale engine holding 44 GB of SRAM; Groq with a deterministic, compiler-scheduled dataflow chip holding a few hundred megabytes per part and many parts per model. Both win single-stream decode by large factors and pay in capacity, cost per token at high batch, and flexibility. Knowing where each fits is the interview question.

Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew

RELATED CONCEPTS

GPU Memory Hierarchy Memory-Bound vs Compute-Bound Kernels Bandwidth-Bound Decode Throughput TPU Architecture and Systolic Arrays Speculative Decoding Cost per Million Tokens

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator ArchitectureWhy are Groq and Cerebras so fast at batch 1, and what does that speed cost at scale?→GPU & Accelerator ArchitectureExplain how a systolic array computes a matrix multiply. Why is it efficient, and why does it want big square tiles?→Napkin Math, Cost & CapacityRank the H100, MI300X and Trainium2 by cost per token for decode→Hardware, Cabling & Cluster Build-OutA vendor claims their accelerator beats an H100 at half the price. How do you evaluate that?→GPU & Accelerator ArchitectureWalk me through the CUDA execution model: what are grids, blocks and warps, and what does the hardware actually schedule?→GPU & Accelerator ArchitectureDescribe the GPU memory hierarchy. Where can a byte live on an H100, and what does each level cost?→