hardNVIDIAFireworksTogether AI

An Nsight summary reads DRAM 90%, SM 30%, occupancy 45%, L2 hit rate 20%. What is the bottleneck?

Use a profiler summary to form a memory-traffic hypothesis, then check counters, timing and useful work before choosing an optimization.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Use a profiler summary to form a memory-traffic hypothesis, then check counters, timing and useful work before choosing an optimization.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

⚡ Kernels & Compilers🔒 Premium

Profiling with NsightNsight Systems shows where an application spends time; Nsight Compute investigates individual kernels. Use the timeline to choose what deserves detailed profiling.

Foundational

🧩 GPU & Accelerator Architecture

Roofline ModelDerive a throughput ceiling from work, traffic and hardware limits. Read a measured point below the roof, distinguish memory boundaries, and choose an optimization without promising peak performance.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

Memory-Bound vs Compute-Bound KernelsUse a worked timing example to distinguish a bandwidth ceiling from a measured bottleneck. Learn which profiler evidence to inspect and why decode, prefill and a whole serving step need separate diagnoses.

Foundational

🚀 Inference & Serving

Prefill vs DecodeFollow one prompt through its first prediction and later decode steps, then separate GPU work from the wait a user sees. Learn why batch size and context change the likely bottleneck.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.Next in this trackWhat can you rely on from the L2 cache on a GPU? When does it save a kernel, and when does it mislead you?Next in this trackWhat does ECC on a GPU cost you, and why do you keep it on across a fleet?

FEDITOR'S NOTE

Explain what each metric actually measures. Four percentages can suggest a bottleneck, but they cannot supply FLOP counts or prove which change will improve runtime.

DISCUSSION · 0

No comments yet — be the first to share your approach.