03InfiniBand or RoCE version 2 for a new GPU training cluster. Make the call and say what would reverse it.▼medium★ EssentialNewNVIDIACrusoexAI4 repliesunlockedBoth carry RDMA at the same line rate. One arrives lossless because the transport was designed that way, the other becomes lossless only if a set of switch settings is correct on every port. What that difference costs in operations, what it saves in money and hiring, and the fleet size where the answer flips.Open full answer →
08What is incast, why does it break an RDMA fabric, and what do ECN and priority flow control each do about it?▼hardNewxAINVIDIAMeta4 repliesunlockedThirty-two senders aimed at one port fill a switch buffer in microseconds, and what happens next depends on which of two mechanisms fires first. The buffer arithmetic, why the slow mechanism is the safe one and the fast one is the dangerous one, and the counters that tell you which is happening.Open full answer →
18What is Ultra Ethernet trying to fix, and does it change how you would choose a fabric today?▼hardNewBroadcomMetaAMD4 replies○ sign inRoCE inherited a transport that assumes one ordered path per connection, which is why it needs a lossless network built by hand underneath it. What a redesigned transport changes about that, which of today's operational burdens it removes, and why the honest answer to a purchase decision is still about what ships.Open full answer →
26Several unrelated jobs slow down at once and nothing is dropping packets. How do you confirm a pause cascade and find where it started?▼hardNewMetaxAI4 replies◆ premiumThe signature is unusual: many jobs degrade together, throughput collapses, and the drop counters stay at zero. Reading pause counters backward along the paths to find the origin, the three conditions that produce one, and the configuration change that stops it happening again.Open full answer →