TL;DR: InfiniBand and RoCEv2 can both support GPU communication. Choose between concrete, supported designs by testing the intended workload, congestion and recovery behavior, then comparing total cost and operational ownership. Neither a port-speed label nor a universal cluster-size threshold decides the answer.
How to approach it
State the job mix, scale and available operations team. Compare full designs at the same useful capacity and failure tolerance. Explain flow control without promising a congestion-free network, and name a result that would reverse the recommendation.
A strong answer
RDMA (remote direct memory access) lets one machine's network card read or write another machine's memory without the remote CPU copying the data. InfiniBand is a dedicated network built around it; RoCEv2 (RDMA over Converged Ethernet, version 2) carries it over ordinary Ethernet and IP. A two-node transfer can pass while a large job stalls under contention. That gap is why fabric selection needs more than a headline throughput result. RDMA, InfiniBand and RoCEv2 introduces the transports; acceptance should exercise the collective and storage patterns the cluster will actually run.
InfiniBand uses link-level credit flow control and managed fabric routing. Credits prevent a sender from overrunning the receiving link's advertised buffers; they do not make congestion, faulty links or configuration errors disappear. Backpressure can still affect progress, and operations still need ownership of topology, firmware and routing.
RoCEv2 carries RDMA over an IP/Ethernet network. Supported deployments vary. PFC and ECN are common in lossless configurations, while vendors also document other modes. NVIDIA's RoCE configuration guide describes distinct supported configurations; do not assume PFC is an unconditional protocol requirement or copy thresholds between platforms.
ECN tells endpoints about congestion so their rate-control mechanism can respond. PFC acts on a link instead, pausing traffic in a priority class; other flows in that class may be paused too. Read these signals alongside queue occupancy and useful job progress, as Congestion Control for AI Fabrics explains. Some pauses can be part of the intended control behavior. Conversely, a network with no dropped packets can still delay work badly.
Compare costs only after constructing a valid topology. An endpoint count is not the total switch-port count: inter-switch links use ports at both ends. Include transceivers, cables, spares, power, support, testing, staff coverage and the value of lost useful training time. Use quotations for the actual equipment and service term.
A purely hypothetical break-even example makes the tradeoff visible:
quoted capital difference between complete designs = 600,000 currency units
extra annual operating cost for the cheaper design = 180,000
simple three-year difference = 600,000 − 3 × 180,000 = 60,000
sanity: this excludes financing, residual value and performance differences.
That calculation does not assert that RoCE is cheaper or that one engineer can cover the service. If recurring cost were 250,000 instead, the same three-year comparison would favor the other design by 150,000. Useful training performance can dominate either accounting result, so compare cost per successful workload rather than cost per port alone.
For a team with an established, tested InfiniBand operating model and no supported RoCE deployment process, I would begin with its supported InfiniBand design. Reverse that choice if a RoCE candidate passes the same contention, recovery and workload tests with a credible ownership and cost advantage. For a team already operating a proven RoCE stack, the starting point can reasonably be the opposite.
The acceptance plan should include small and large messages, all intended rails, concurrent jobs, incast, degraded links and recovery after configuration or firmware changes. Compare p99 step time and useful throughput as well as transport bandwidth. Sharing Ethernet with other traffic is an option to test, not an automatic saving: QoS classes still share finite links and buffers.
What interviewers probe next
- "Does one fabric guarantee the best NCCL performance?" No. Check the installed library, transport plugin, topology and workload.
- "Can I decide from a vendor reference design?" It establishes a supported starting point; validate your placement and operating conditions.
- "What changes at larger scale?" More paths, failure domains and contention patterns require broader testing; there is no universal GPU-count crossover.
Common mistakes
- Calling InfiniBand self-managing or RoCE inherently unusable without one fixed configuration.
- Comparing endpoint prices while omitting inter-switch ports and operations.
- Accepting a cluster from an isolated pair test.
Key takeaways
- Compare supported designs at equal useful capacity and failure tolerance.
- The hypothetical 600,000 saving becomes only 60,000 after three years of added operations.
- Keep the recommendation conditional on measured workload and recovery performance.