RunPod AI Infrastructure Engineer interview questions

Trace a model startup through storage, runtime setup and device loading. Transfer floors and dependencies determine whether a minute is plausible.

Calculate the fleet, duration and rental budget for a 7B training run. Check the deadline, actual sharding scheme and paid reservation period before placing the order.

Define warm, cached and truly cold paths, then size transfer budgets and warm capacity without promising universal sub-second restores.

Balance durable checkpoint cost against replay and restart loss, with explicit assumptions and a warning sign when the approximation breaks down.

Eight ready slots meet a twelve-request burst. See why starting instances do not count as ready capacity, then design caches, isolation, admission and a measurable startup contract.

Follow one request from API admission to streamed output, then zoom into the token-generation loop. See where the cache lives, who schedules work, and how workload measurements turn the drawing into a capacity plan.

Trace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.

Track request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.

Derive a 10,240-GPU layout from model work, then account for collective traffic, durable checkpoints, power and replacements. A nested diagram connects tensor, pipeline and data parallel groups to the fleet.

Compare raw weight sizes at 16, 8 and 4 bits, then budget KV cache and runtime memory for Llama 3.1 70B. Follow the worked per-GPU example.

Separate quota, borrowing, reclaim latency and fairness, with a demand-conserving example of when pooling improves utilization.

Work from training FLOPs to fleet throughput, days and GPU-hours. Then see how utilization and downtime change the estimate.

Follow a four-GPU request from advertised capacity to concrete devices, and see why free GPUs across nodes cannot satisfy one pod.

DRA gives the scheduler device claims and attributes to work with. Compare that with extended-resource counts, then check the deployed driver and controller support before promising sharing, partitioning or gang placement.

Use the Llama 3 interruption record without turning job interruptions into device lifetime, predictable failures or a universal spare percentage.

Use a flexible 45-minute plan to connect requirements, a request path, capacity arithmetic, selected deep dives and a clear closing decision.

Handle an inaccessible GPU with bounded recovery and preserved evidence, without treating a reboot or repeat count as proof of root cause.

Follow preemption through checkpoint success or timeout, a safe stop barrier and requeue. See why an expired lease is not enough to free a GPU, and how observed checkpoint age changes victim cost.

Solve for useful throughput, round to a supported parallel layout, and account for calendar time once. The right GPU count need not be a power of two.

Use DCGM fields with their actual semantics, combine device health with workload progress, and calibrate alerts on correlated fleet data.

Trace a stalled stream without unbounded telemetry, understand what sampling loses, and reconcile GPU allocation with a stated cost policy.