TL;DR: Four trillion FLOPs over 40 billion HBM bytes gives 100 FLOPs/byte. Under SXM dense BF16 peaks, the HBM floor is 11.94 ms and the compute floor is 4.04 ms, so memory supplies the tighter ideal bound. Actual runtime is still needed to establish what limits the kernel.
How to approach it
Check operation precision and the memory boundary first. Calculate both time floors and the intensity, then compare an actual timing. Use the gap to choose a discriminating experiment instead of treating a roofline position as a complete diagnosis.
A strong answer
A profiler reports a large amount of arithmetic. That alone says little about whether arithmetic is the limiting resource. The Roofline Model compares useful work with the bytes needed to supply it; Memory-Bound vs Compute-Bound Kernels connects that bound to the next measurement.
Assume four trillion floating-point operations in a mode appropriately compared with the H100 SXM dense BF16 peak of 989 TFLOP/s and 3.35 TB/s bandwidth. Assume the 40 GB count is actual traffic across the HBM boundary for that same invocation, using decimal units. Arithmetic intensity is work per byte transferred. The ridge is the intensity where the ideal compute and memory time floors are equal; below it, memory gives the tighter bound.
F = 4e12 FLOPs; D = 40e9 bytes
intensity = F/D = 100 FLOPs/byte
ridge = 989e12/3.35e12 = 295.22 FLOPs/byte
compute floor = F/989e12 = 4.0445 ms
HBM floor = D/3.35e12 = 11.9403 ms
ideal throughput ceiling = min(989, 100×3.35) = 335 TFLOP/s
sanity: F/335e12 = 11.9403 ms, matching the tighter time floor.
The duration cannot beat that tighter bound under these assumptions. It is not guaranteed to attain it. The synthetic 20 ms observation shown in the chart would give 4e12/0.02 = 200 TFLOP/s and 40e9/0.02 = 2 TB/s. Both are below the declared peaks. Latency, issue dependencies, limited parallelism, access patterns or another resource may account for the gap; the ratio alone cannot separate them.
For real inputs, inspect Nsight Compute's roofline and profiling guidance. HBM read and write byte counters describe traffic, while counting FLOPs requires weighting executed instructions by their operation and active work. A tensor instruction count is not directly a FLOP count. Check whether reported work includes padded or sparse operations and whether profiling replay changes cache behavior.
| Possible finding | Experiment |
|---|---|
| Repeated HBM traffic | Test reuse or fusion; measure any register/spill cost |
| Dependent memory accesses | Change access pattern or available independent work |
| Tiny grid or launch gap | Compare complete timing and work distribution |
| Arithmetic path differs | Recompute the peak reference for the actual mode |
If a valid transformation cuts D to 20 GB at fixed F, the HBM floor becomes 5.9701 ms and intensity becomes 200. If D falls to roughly 13.55 GB, the two ideal floors meet. This does not promise the corresponding runtime: fusion might also increase F, reduce occupancy, introduce synchronization or move traffic to a different memory boundary. Recount and remeasure the transformed kernel.
Changing to FP8 also changes more than the peak. Its payload may fall, scaling and conversion may add work, and quality needs validation. Moving only the ridge on a chart while holding an incompatible byte count fixed describes no real transformation. The same caution applies when changing hardware: preserve the workload and use the actual precision, memory specification and measured clocks.
I would start with the largest measured traffic source if the trace supports that hypothesis. Reverse a fusion or precision change if complete latency or numerical quality worsens. Arithmetic optimization can still help a kernel below the ideal memory ridge when its implementation has significant arithmetic or issue overhead; the roofline rules out impossible results, not every useful engineering change.
What interviewers probe next
- “Why do tensors total 4 GB but traffic totals 40 GB?” Repeated accesses, writes, spills or the measurement scope may account for it. A low cache hit rate alone does not identify the source.
- “What if 40 GB includes L2-served bytes?” Use a consistent L2 boundary and rate, or obtain actual HBM bytes. Mixing boundaries invalidates the HBM floor.
- “Can the two floors be added?” Not as a general model. Compute and transfer can overlap; a trace determines the serial critical path.
Common mistakes
- Calling 335 TFLOP/s achieved throughput without a timing.
- Converting tensor instructions to FLOPs with no shape or dtype weighting.
- Declaring every left-of-ridge optimization except traffic reduction pointless.
Key takeaways
- 100 FLOPs/byte gives a tighter HBM bound under the stated H100 BF16 model.
- 11.94 ms is a floor; a 20 ms observation needs further diagnosis.
- Recount work and bytes after changing precision, layout or fusion.