GPU Architecture Interview Questions

Trace a grid into blocks and warps, then distinguish resident work from ready work. Derive an occupancy limit without treating it as a speed target.

Separate storage scope from physical backing, then follow a warp through adjacent and strided loads. Count useful bytes and requested sectors before inferring HBM traffic or speed.

Compare useful parallel work, data movement and reuse. A synthetic CPU/GPU crossover shows why a faster kernel can still lose the end-to-end request.

Follow operand tiles into a matrix-multiply-accumulate instruction. Explain why odd dimensions and low tensor activity do not prove a fallback.

Count two matrix shapes to derive the H100 ridge, throughput ceilings and time floors. Compare an illustrative measurement with its bound before deciding what to optimize.

Derive the weight-only shortcut, then add KV traffic and feasible eight-GPU placement. A worked latency floor shows why batch size alone cannot establish the decode bottleneck.

Use proportional bit layouts and adjacent values around one to distinguish range from precision. Then choose formats from numerical tests and actual kernel support.

Separate H200 bandwidth and capacity gains from measured speedup, then test realistic memory headroom and cost per useful completed workload.

Normalize link directions, count collective bytes, and see why a gradient reduction can outlast its real overlap window.

Compare the same 1,024 items under mixed and grouped lane assignments. Derive issue-work waste, then include regrouping and memory costs before claiming a speedup.

Derive a block-level occupancy limit, then compare the extra residency with spill traffic and lost reuse. Use a matched experiment rather than a target percentage.

Derive HBM bandwidth from interface width and pin rate, then connect it to a model that actually fits. Separate raw interface capacity from measured decode time.

Separate Blackwell chip gains from the 72-GPU domain, follow a token to its expert owners, and test placement instead of dividing model bytes by 72.

Carry compute and memory reasoning to TPU v6e, correct all-gather ownership arithmetic, and learn the compilation, sharding and custom-kernel workflow.

Trace a two-by-two output-stationary array cycle by cycle. Derive its fill and drain cost without turning a toy schedule into a TPU benchmark.

Compare actual useful work and memory units, validate Neuron support, and calculate when rental savings repay porting and operations.

See how 192 GB changes replica shape, calculate KV headroom with exact units, and compare feasible deployments at equal quality and latency.

Follow a dependent read path to understand SRAM locality, aggregate bandwidth and the difference between one fast stream and total serving throughput.

See the GPU-instance and compute-instance boundaries, budget a small slice, and distinguish memory capacity controls from bandwidth isolation.

Quantize a sixteen-value block, count the scale bytes, and use paired quality tests to decide which tensors need higher precision.

Derive a 16-leaf, 8-spine fabric from 512 endpoints, trace local and cross-leaf routes, and check what rail placement and oversubscription actually guarantee.

Compare host-staged and direct GPU-to-NIC transfers, count traffic at each boundary, and check transport selection before blaming a slow collective on missing GPUDirect.

Use a profiler summary to form a memory-traffic hypothesis, then check counters, timing and useful work before choosing an optimization.

Derive how reusing weights changes arithmetic intensity, then separate the roofline ceiling from tile-wave effects and measured GEMM throughput.

Calculate the data a kernel could reuse, then test whether it stays in L2. Distinguish capacity, cache-policy hints and measured traffic from a guarantee of residency.

Trace a changed BF16 bit, distinguish corrected and uncorrectable errors, and calculate recovery cost without assuming every device has the same ECC overhead.

Follow a host buffer across a NUMA boundary, calculate the cost with declared bandwidths, and align CPU workers and memory with the GPU that consumes the data.

Compare the cost of serving the same workload at the same latency and quality. A memory-traffic estimate explains what each GPU might improve; measured throughput and your actual rental price determine the purchase.

A proposal for more memory bandwidth needs a workload that benefits, a budget for what it costs, and a reason to abandon it. Work through a serving-specific design without assuming all LLM work has the same bottleneck.

Compare the same benchmark on a healthy node, then follow the evidence through NCCL transport choices, device topology and fabric health. Half the measured bandwidth is a symptom, not a unique diagnosis.

Separate valid 2:4 structure, packed representation and sparse dispatch from model quality and the fraction of request time that can improve.