Advanced

Cerebras, Groq and Dataflow Accelerators

Dataflow accelerators organize computation and memory differently from GPUs. On-chip storage can help latency, but capacity, communication and batching determine the wider tradeoff.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

GPU Memory Hierarchy Memory-Bound vs Compute-Bound Kernels Bandwidth-Bound Decode Throughput TPU Architecture and Systolic Arrays Speculative Decoding Cost per Million Tokens

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator ArchitectureWhy are Groq and Cerebras so fast at batch 1, and what does that speed cost at scale?→GPU & Accelerator ArchitectureExplain how a systolic array computes a matrix multiply. Why is it efficient, and why does it want big square tiles?→Napkin Math, Cost & CapacityRank the H100, MI300X and Trainium2 by cost per token for decode→Hardware, Cabling & Cluster Build-OutA vendor claims their accelerator beats an H100 at half the price. How do you evaluate that?→GPU & Accelerator ArchitectureWalk me through the CUDA execution model: what are grids, blocks and warps, and what does the hardware actually schedule?→GPU & Accelerator ArchitectureDescribe the GPU memory hierarchy. Where can a byte live on an H100, and what does each level cost?→