Microsoft AI Infrastructure Engineer interview questions

Separate Blackwell chip gains from the 72-GPU domain, follow a token to its expert owners, and test placement instead of dividing model bytes by 72.

See how 192 GB changes replica shape, calculate KV headroom with exact units, and compare feasible deployments at equal quality and latency.

Trace a changed BF16 bit, distinguish corrected and uncorrectable errors, and calculate recovery cost without assuming every device has the same ECC overhead.

Compare rack-level power, cooling, structural and connectivity requirements with the offered site, then evaluate quoted deployment alternatives.

Build a cluster power budget with clear equipment boundaries, facility overhead and failover requirements. Distinguish energy use from reserved electrical capacity.

Derive cooling flow for a 4.8 MW hall, separate liquid and air loads, and test CDU capacity and redundancy at the actual design conditions.

Define acceptance with component and system tests, absolute targets, peer comparisons and documented retests. Interpret burn-in exposure without treating it as a reliability guarantee.

Investigate rack power events at the correct electrical boundary and time scale. Test load reductions without assuming GPU caps control every watt or transient.

Design a management path that survives specified production failures. Count all endpoint classes, restrict device authority and test the complete recovery route.

Compare cooling methods from the supported hardware and heat path. Calculate airflow or liquid flow, then verify capacity, residual air load and failure response.

Build a deployment schedule from dependencies and committed lead times. Calculate float and price the delay only when an action moves usable capacity earlier.

Investigate widespread acceptance failures using matched controls, valid thresholds and cohort denominators. Preserve hardware and shared causes as testable hypotheses.

Count endpoints, allocate every switch port once, and compare a common fabric with independent rails before adding a third tier.

Compare 64 or 72 useful GPUs, identify the communication that could improve, and test whether a larger NVLink domain justifies the operational requirements.

Budget the full megawatt, fit whole servers, and compare useful training throughput under measured power limits. A worked cap example shows when more GPUs help.

Reproduce the baseline, compare equivalent work, and trace the missing time. Acceptance, production and the original plan may differ in more than one way.

Protect running work, validate a supported upgrade bundle, and expand through measured cohorts with a tested recovery path.

Plan a GPU-generation migration around usable workload capacity, site readiness, compatibility and a tested rollback path.

Gate admission with fresh, platform-aware evidence. Use absolute and peer baselines, budget test overhead and keep automated cordons under control.

Keep the full failure history and investigate what changed. One passing retry is evidence, but it is not a repair or an acceptance policy.

Survey real facility headroom, phase work around shared dependencies, and compare retrofit with relocation using useful capacity and total transition cost.

Collect stable device identities, versions and freshness, then compare them with approved configurations and retain the history for investigations.

Compare complete fabric proposals on workload performance, support and total cost. Translate any measured regression into application impact before pricing the discount.

Diagnose a rack outage through independent evidence, preserve safety interlocks, and restore capacity only after its failed paths pass acceptance checks.

Track which state remains replicated at each ZeRO stage. Compute a declared 70B memory recipe, then add live gathers, activations and buffers before deciding fit.

Compare FSDP2 and ZeRO-3 through one parameter group’s lifetime, a scoped traffic model and a save-resume experiment, rather than a universal speed ranking.

Balance checkpoint pauses against lost work using job-level interruption data, while accounting for asynchronous durability lag.

Describe logical tensors independently of writer ranks, publish only a complete checkpoint, and reconstruct the target layout from stored chunks. Asynchronous saving reduces blocking but still needs a consistent snapshot and a memory budget.

Inventory a resumable checkpoint instead of rounding its bytes per parameter. Trace snapshot stall, durable drain and publication, then calculate the storage floor.

Build a power budget from complete servers to the facility meter, then check rack limits and distinguish a design maximum from an annual electricity bill.

Count endpoint, leaf, aggregation and core ports for an explicit 4,096-GPU design. Compare 1:1 and 2:1 without confusing switch ports, links or optics.

Design snapshot ownership, background writes and manifest publication separately, with explicit checkpoint bytes and recovery scope.

Derive the two-tier endpoint limit, audit explicit 8,192-endpoint constructions and compare switches, links, optical modules and failure paths.

Use DCGM fields with their actual semantics, combine device health with workload progress, and calibrate alerts on correlated fleet data.

Derive a checkpoint interval from measured job interruptions and exposed save cost. Trace durability, lost work and recovery before treating an estimate as goodput.

Measure throughput at several power limits, translate the facility target into a GPU budget, and account for slower ranks before changing the fleet.

Loss of cooling can outpace human response. Use a simple heat calculation to explain why protection must be automatic, then distinguish that illustration from the installed system’s validated behavior.

Size a spare pool by separating nodes in repair from ready replacements. Work through a 2,048-node example, then check placement, recovery time and shared failure domains.

Determine what the replacement count measures, compare equivalent device exposure and investigate differences by cohort, location, age, software and policy. Job-interruption statistics cannot supply a hardware replacement baseline.

Plan a useful first month by tracing a real job, reviewing incidents, helping under supervision and shipping a bounded improvement with the team.

Explain what grew, which constraints appeared and how the system stayed usable. Work through metric-cardinality and device-event examples without assuming a universal failure order.

Describe senior and staff responsibility through scope, technical judgment, influence and durable results, while recognizing that company rubrics differ.

Report a capacity shortfall with assumptions, consequences, feasible options and a decision deadline. Show how one uncertain project changes the forecast.

Follow one request from API admission to streamed output, then zoom into the token-generation loop. See where the cache lives, who schedules work, and how workload measurements turn the drawing into a capacity plan.

Count gradient traffic from a named dtype, separate initialization and buffer broadcasts, and measure the reduction tail instead of assuming it is hidden.

Trace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.

Choose between supported InfiniBand and RoCE designs using workload tests, congestion behavior and a complete cost model.

Track request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.

Derive a 10,240-GPU layout from model work, then account for collective traffic, durable checkpoints, power and replacements. A nested diagram connects tensor, pipeline and data parallel groups to the fleet.

Separate quota, borrowing, reclaim latency and fairness, with a demand-conserving example of when pooling improves utilization.

Compute MFU from a declared global token count and time interval. Check the accounting before judging the percentage or choosing an optimization.