Amazon Web Services GPU & Accelerator Architecture interview questions

Compare actual useful work and memory units, validate Neuron support, and calculate when rental savings repay porting and operations.

Trace a grid into blocks and warps, then distinguish resident work from ready work. Derive an occupancy limit without treating it as a speed target.

Count two matrix shapes to derive the H100 ridge, throughput ceilings and time floors. Compare an illustrative measurement with its bound before deciding what to optimize.

Separate storage scope from physical backing, then follow a warp through adjacent and strided loads. Count useful bytes and requested sectors before inferring HBM traffic or speed.

Derive the weight-only shortcut, then add KV traffic and feasible eight-GPU placement. A worked latency floor shows why batch size alone cannot establish the decode bottleneck.

Normalize link directions, count collective bytes, and see why a gradient reduction can outlast its real overlap window.

Compare useful parallel work, data movement and reuse. A synthetic CPU/GPU crossover shows why a faster kernel can still lose the end-to-end request.

Follow operand tiles into a matrix-multiply-accumulate instruction. Explain why odd dimensions and low tensor activity do not prove a fallback.

Use proportional bit layouts and adjacent values around one to distinguish range from precision. Then choose formats from numerical tests and actual kernel support.

Carry compute and memory reasoning to TPU v6e, correct all-gather ownership arithmetic, and learn the compilation, sharding and custom-kernel workflow.

Compare the cost of serving the same workload at the same latency and quality. A memory-traffic estimate explains what each GPU might improve; measured throughput and your actual rental price determine the purchase.

Compare the same 1,024 items under mixed and grouped lane assignments. Derive issue-work waste, then include regrouping and memory costs before claiming a speedup.

Derive a block-level occupancy limit, then compare the extra residency with spill traffic and lost reuse. Use a matched experiment rather than a target percentage.

See the GPU-instance and compute-instance boundaries, budget a small slice, and distinguish memory capacity controls from bandwidth isolation.

Derive HBM bandwidth from interface width and pin rate, then connect it to a model that actually fits. Separate raw interface capacity from measured decode time.

Trace a changed BF16 bit, distinguish corrected and uncorrectable errors, and calculate recovery cost without assuming every device has the same ECC overhead.