Hugging Face AI Infrastructure Engineer interview questions

Separate packed base weights, quantization metadata and trainable adapter state. Count the real projection shapes, then leave room for activations and temporary work before calling a one-card configuration feasible.

Follow a shard index to the tensors a loader actually receives. Diagnose a missing file and a shape mismatch, then check architecture, quantization and request compatibility before changing hardware.

Bind weights, tokenizer, configuration and evaluation to one immutable release. Follow verification, prefetch, canary readiness and rollback, with bandwidth floors that do not pretend to be rollout measurements.

Follow the weight bytes through a small decode batch and a long prefill. A scoped roofline calculation explains the difference without pretending to predict actual latency.

Follow the running maximum and weighted sum across attention tiles, with executable arithmetic and an honest memory-traffic comparison.

Derive resident-block limits and distinguish resident warps from ready work before trading registers for more occupancy.

Choose between supported InfiniBand and RoCE designs using workload tests, congestion behavior and a complete cost model.

Follow a token into retained attention state, derive bytes per position for a 70B model, and distinguish present cache usage from the room needed to finish a response.

Trace stable row normalization, handle independent strides and fully masked inputs, then separate fusion traffic from measured speed.

Work from each GPU's cache budget, reserve room for generated tokens, then check latency separately. A limiting-rank example shows why fitting today's prompts is not enough to admit tomorrow's work.

Work from token timestamps to first-token wait, average pace and individual stalls. Then count requests meeting both latency limits, keeping failures and measurement boundaries explicit.

Draw query-head sharing, count actual stored state and compare payloads with explicit model geometry and binary units.

Map a non-divisible array onto blocks, guard the final 189 threads, and separate kernel traffic from host transfers without inventing measured timings.

Trace one cache position through a block table, calculate the memory recovered from oversized reservations, and explain why sharing still needs careful writes and matching.

Separate weight storage, matrix arithmetic and attention-cache precision. Choose a supported candidate using task quality, memory and service measurements.

Follow intermediate tensors through separate and fused stages, calculate the traffic and whole-block bounds, and check spills and numerical behavior.

Compare the mechanisms in FlashAttention versions 1, 2 and 3: IO savings, work partitioning, asynchronous dependencies and FP8 accuracy.

Separate the all-reduce result from NCCL execution choices, and compare latency models without treating them as universal selection rules.

Implement priority admission and completion, preserve a preempted job’s sequence, and distinguish a sufficient greedy victim set from an optimal one.