mediumTogether AIBaseten

Why does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.

Derive how reusing weights changes arithmetic intensity, then separate the roofline ceiling from tile-wave effects and measured GEMM throughput.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Derive how reusing weights changes arithmetic intensity, then separate the roofline ceiling from tile-wave effects and measured GEMM throughput.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Foundational

🧩 GPU & Accelerator Architecture

Roofline ModelDerive a throughput ceiling from work, traffic and hardware limits. Read a measured point below the roof, distinguish memory boundaries, and choose an optimization without promising peak performance.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

Memory-Bound vs Compute-Bound KernelsUse a worked timing example to distinguish a bandwidth ceiling from a measured bottleneck. Learn which profiler evidence to inspect and why decode, prefill and a whole serving step need separate diagnoses.

Advanced

⚡ Kernels & Compilers🔒 Premium

GEMM Work Partitioning and Wave QuantizationOutput tiles determine how much independent matrix work is available. Split-K and work-centric schedules can reduce an underfilled final wave, while adding partial-result reduction and workspace costs.

Core

🧩 GPU & Accelerator ArchitectureSign in

Tensor Cores and Matrix UnitsTensor cores accelerate supported matrix multiply-accumulate operations. Their performance depends on precision, matrix shape and how efficiently the kernel supplies data.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhat can you rely on from the L2 cache on a GPU? When does it save a kernel, and when does it mislead you?Next in this trackWhat does ECC on a GPU cost you, and why do you keep it on across a fleet?Next in this trackExplain NUMA and CPU affinity on a GPU node. Why can the wrong socket halve host-to-device bandwidth?

FEDITOR'S NOTE

Explain weight reuse and whole-tile scheduling separately. Neither a roofline bound nor a wave-efficiency estimate is a measured throughput curve.

DISCUSSION · 0

No comments yet — be the first to share your approach.