NVIDIA Networking, Interconnects & Storage interview questions

Separate the all-reduce result from NCCL execution choices, and compare latency models without treating them as universal selection rules.

Track actual values across ranks, then count bytes with an explicit buffer definition. Reduction combines values; gathering and all-to-all change ownership.

Choose between supported InfiniBand and RoCE designs using workload tests, congestion behavior and a complete cost model.

Trace registered-memory access and GPU staging, then compare RDMA with sockets using measured CPU and application performance.

Follow corresponding GPU positions through a rail, including the leaf and spine links needed when one rail outgrows a switch.

Calculate queue growth during incast, then distinguish endpoint rate feedback from link-level priority pause and shared-resource contention.

Compare serialized and overlapped storage paths, then test whether direct GPU I/O improves the actual pipeline stage.

Construct concrete TP and DP groups, then verify physical devices and estimate traffic with per-rank rather than whole-model bytes.

Separate NCCL logging, device selection and execution overrides from PyTorch failure handling. Work through a faster collective that makes the job slower, and learn which timeout actually fired.

Derive bandwidth from actual per-rank traffic and readiness deadlines, showing why bucketing cannot erase a capacity shortfall.

Follow expert assignments from logical payload to off-rank and cross-node traffic. Compare dispatch and combine dtypes, receiver skew and local-domain placement.

Distinguish GPU-memory capability, registration and selected transport. Compare direct and staged paths without relying on a half-speed rule or one log substring.

Count endpoint, leaf, aggregation and core ports for an explicit 4,096-GPU design. Compare 1:1 and 2:1 without confusing switch ports, links or optics.

Design a durable dataset and optional working-set cache around access semantics, request overhead and versioned readiness.

A bandwidth regression narrows the search only after the experiment is defined. Work through rank-count normalization, controlled path tests and the difference between collective time and training progress.

Design snapshot ownership, background writes and manifest publication separately, with explicit checkpoint bytes and recovery scope.

Compare tensor, pipeline and expert traffic with per-request KV transfer. Work through declared payload and latency assumptions before choosing a serving network.

Two tenants can fit the average storage budget and still miss both checkpoint deadlines. Work through reservations, borrowing and bounded admission while protecting training reads, then test the design under failure.