xAI AI Infrastructure Engineer interview questions

Separate the 100B memory floor from the training deadline, then compare two 1,024-GPU layouts with their actual state ownership and communication schedules.

Distinguish stale telemetry, skipped updates and a real learning plateau using counters, parameter changes and a fixed evaluation batch.

Turn the daily loss into recovery and replay costs, then identify which part is measured and which is assumed. Compare checkpoint cadence, recovery work and failure prevention without promising one universal floor.

Build a training schedule from explicit model, token, state and recovery assumptions. Keep active compute time separate from availability and contingency.

Solve for useful throughput, round to a supported parallel layout, and account for calendar time once. The right GPU count need not be a power of two.

Choose between supported InfiniBand and RoCE designs using workload tests, congestion behavior and a complete cost model.

Calculate queue growth during incast, then distinguish endpoint rate feedback from link-level priority pause and shared-resource contention.

Derive bandwidth from actual per-rank traffic and readiness deadlines, showing why bucketing cannot erase a capacity shortfall.

Follow pause requests against the direction of data, estimate queue growth and distinguish cascade evidence from its possible causes.

Derive a 10,240-GPU layout from model work, then account for collective traffic, durable checkpoints, power and replacements. A nested diagram connects tensor, pipeline and data parallel groups to the fleet.

Separate total and active parameters, then size persistent state, approximate compute and token dispatch. Compare supported parallel layouts using per-rank memory, measured communication and recovery.

Plan a useful first month by tracing a real job, reviewing incidents, helping under supervision and shipping a bounded improvement with the team.

Follow one request from API admission to streamed output, then zoom into the token-generation loop. See where the cache lives, who schedules work, and how workload measurements turn the drawing into a capacity plan.

Follow the weight bytes through a small decode batch and a long prefill. A scoped roofline calculation explains the difference without pretending to predict actual latency.

Trace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.

Follow a token into retained attention state, derive bytes per position for a 70B model, and distinguish present cache usage from the room needed to finish a response.

Track request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.

Work from each GPU's cache budget, reserve room for generated tokens, then check latency separately. A limiting-rank example shows why fitting today's prompts is not enough to admit tomorrow's work.

Separate quota, borrowing, reclaim latency and fairness, with a demand-conserving example of when pooling improves utilization.

Work from token timestamps to first-token wait, average pace and individual stalls. Then count requests meeting both latency limits, keeping failures and measurement boundaries explicit.