Which NCCL environment variables are worth knowing, what does each change, and which should you almost never set?

Separate NCCL logging, device selection and execution overrides from PyTorch failure handling. Work through a faster collective that makes the job slower, and learn which timeout actually fired.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Separate NCCL logging, device selection and execution overrides from PyTorch failure handling. Work through a faster collective that makes the job slower, and learn which timeout actually fired.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Foundational

🔌 Networking & Storage

Debugging a Slow All-ReduceInterpret NCCL algbw and busbw, then isolate a slow collective with controlled tests. Distinguish rank delays, GPU-memory paths and shared-fabric limits without guessing from bandwidth ratios.

Advanced

🔌 Networking & Storage🔒 Premium

GPUDirect RDMA and GPUDirect StorageGPUDirect enables supported devices to transfer data to or from GPU memory without a host staging buffer. Its benefit depends on topology, software support and the transfer path.

Foundational

🧮 Open Weights & Serving Engines

vLLM Server Arguments That MatterSeparate executor memory, allocated cache pools and logical occupancy. Map context, sequence and token-batch limits to the constraints they control, then tune a pinned deployment against per-rank memory and latency.

Foundational

🧮 Open Weights & Serving Engines

SGLang Server Arguments That MatterTune static memory, prefill chunks and admission against measured workload behavior. Separate offline throughput heuristics from interactive queue budgets, and calculate the part of request work that prefix caching can save.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackHow do you establish that the link between two GPU nodes is healthy, and what numbers should each test return?Next in this trackWhy do large training clusters provision roughly 400 gigabits per second per GPU rather than more or less?Next in this trackWhat is Ultra Ethernet trying to fix, and does it change how you would choose a fabric today?

FEDITOR'S NOTE

Explain who owns each setting and what evidence it provides. Compare complete application steps and distinguish transport retries from framework timeouts.

DISCUSSION · 0

No comments yet — be the first to share your approach.