TL;DR: Explain what was slow, which evidence identified the cause, and what your change removed. Compare equivalent workloads, report variation and distinguish the local improvement from the end-to-end result. Use your real measurements; a modest, well-supported gain is a complete story.
How to approach it
State the user or system problem and your part in solving it. Describe the profile, the hypothesis and the change. Then give the baseline, measurement protocol, result and limits. Include a regression if one occurred, but do not imply every optimization must make something worse.
A strong answer
Suppose a profile shows three elementwise operations reading and writing the same large tensor. Combining them may avoid intermediate memory traffic and kernel launches. That mechanism is worth testing, but the profile must establish which cost matters for the workload.
A low DRAM-bandwidth percentage (how busy the GPU's main memory was) alone does not prove wasteful fetching or a bandwidth bottleneck. The kernel might be launch-bound, limited by dependencies, serving data from cache or processing too little work to fill the device. Profiling with Nsight explains how counters and timelines narrow those possibilities. Kernel Fusion explains what combining operations can remove.
Make the comparison answer the intended question
Record the input distribution, dtype, implementation versions, device, power settings and relevant runtime configuration. If the claim concerns steady-state service, warm both candidates consistently. If startup matters to users, measure cold start separately rather than discarding it. Alternating old and new runs can help expose temperature or background-load drift.
Report repeated measurements with a useful summary and variation. For an illustrative kernel result, median latency drops from 2.40 ms to 1.26 ms: 2.40 / 1.26 ≈ 1.90×. State the number of independent runs and their spread if you actually measured them. Do not invent “50 runs” or an interquartile range to make an example sound rigorous.
Reusing one tensor can produce a cache-resident benchmark. That may be representative for some workloads and misleading for others. Compare the working set and reuse pattern with production, then design the benchmark accordingly. Clearing caches is not automatically a fairer test if real requests benefit from reuse.
Translate the local result into a useful outcome
Amdahl's calculation makes the distinction concrete. Suppose the optimized component previously occupied 8% of a serial step, and its execution becomes three times as fast while everything else remains unchanged.
Normalize the old step time to 1
new time = 0.92 + 0.08/3 = 0.9466667
latency reduction = 1 - 0.9466667 = 5.33%
throughput speedup = 1 / 0.9466667 ≈ 1.0563×, or 5.63% higher
sanity: even eliminating the component entirely saves at most 8% of step time.
Those two percentages describe different quantities. A 5.33% reduction in latency is not exactly a 5.33% increase in throughput. Real pipelines can overlap work, so verify whether this serial decomposition applies before using it to predict the whole system.
Use the Roofline Model to estimate a relevant limit from operation count, data movement and an appropriate compute/bandwidth basis. The ridge point depends on hardware, precision and the rates chosen. It is not a universal “295 FLOP/byte on ” certificate that a particular kernel is memory-bound.
Measure the integrated result as well: completed work per second, user latency, memory fit or cost. A local latency gain can still be valuable if it frees enough memory for a larger batch or removes a bottleneck expected at the next scale. Name that benefit and test it rather than describing it as an end-to-end speedup.
Keep the limitations in the same story
Test supported shapes, numerical accuracy, memory use and compile/startup cost. Explain where the fast path applies and what happens outside it. If a result only holds above a particular sequence length, give the measured boundary and avoid extrapolating to unrelated traffic.
Close with what shipped, how you monitored it and what would justify reverting it. For example, retain a fusion when the integrated workload improves without unacceptable numerical or memory regressions; reconsider it if shape changes make compilation or fallback cost dominate. If no material regression was found, say which checks passed rather than inventing a tradeoff.
What interviewers probe next
- “How did you know the baseline was fair?” Show equivalent configuration, workload and timing boundaries.
- “What is left to improve?” Use the measured bottleneck and a relevant bound, acknowledging uncertainty.
- “Who did what?” Attribute diagnosis, implementation, validation and rollout accurately.
Common mistakes
- Inferring a bottleneck from one utilization counter.
- Comparing cold baseline runs with warmed candidate runs without saying so.
- Reporting the best input shape as a general result.
- Confusing latency reduction with throughput increase.
Key takeaways
- The toy kernel improvement is 1.90×, but only an integrated measurement establishes its user benefit.
- A 3× improvement to 8% of a serial step cuts latency 5.33% and raises throughput about 5.63% under fixed-work assumptions.
- A defensible claim includes the workload, timing boundary, variation and limits.