Amazon Web Services AI Infrastructure Engineer interview questions

Compare actual useful work and memory units, validate Neuron support, and calculate when rental savings repay porting and operations.

Validate the exact model, accelerator generation and serving features. Separate packed storage from kernel execution, derive the real memory fallback, and compare qualified throughput and total migration cost.

Use bandwidth per dollar as a first comparison, then check feasible memory, supported execution and complete service cost before ranking accelerators.

Compare the cost of committed capacity with the demand each additional GPU would actually serve. Keep billing discounts separate from capacity guarantees, and account for bursts without counting the same work twice.

Explain a career move using the actual role, team and your priorities. Test assumptions about ownership, hardware access and support.

Trace a grid into blocks and warps, then distinguish resident work from ready work. Derive an occupancy limit without treating it as a speed target.

Follow one request from API admission to streamed output, then zoom into the token-generation loop. See where the cache lives, who schedules work, and how workload measurements turn the drawing into a capacity plan.

Count gradient traffic from a named dtype, separate initialization and buffer broadcasts, and measure the reduction tail instead of assuming it is hidden.

Count two matrix shapes to derive the H100 ridge, throughput ceilings and time floors. Compare an illustrative measurement with its bound before deciding what to optimize.

Trace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.

Separate storage scope from physical backing, then follow a warp through adjacent and strided loads. Count useful bytes and requested sectors before inferring HBM traffic or speed.

Derive the weight-only shortcut, then add KV traffic and feasible eight-GPU placement. A worked latency floor shows why batch size alone cannot establish the decode bottleneck.

Normalize link directions, count collective bytes, and see why a gradient reduction can outlast its real overlap window.

Track request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.

Derive a 10,240-GPU layout from model work, then account for collective traffic, durable checkpoints, power and replacements. A nested diagram connects tensor, pipeline and data parallel groups to the fleet.

Compare useful parallel work, data movement and reuse. A synthetic CPU/GPU crossover shows why a faster kernel can still lose the end-to-end request.

Compute MFU from a declared global token count and time interval. Check the accounting before judging the percentage or choosing an optimization.

Use a flexible 45-minute plan to connect requirements, a request path, capacity arithmetic, selected deep dives and a clear closing decision.

Follow preemption through checkpoint success or timeout, a safe stop barrier and requeue. See why an expired lease is not enough to free a GPU, and how observed checkpoint age changes victim cost.

Follow operand tiles into a matrix-multiply-accumulate instruction. Explain why odd dimensions and low tensor activity do not prove a fallback.

Trace a stalled stream without unbounded telemetry, understand what sampling loses, and reconcile GPU allocation with a stated cost policy.