TL;DR: Incast concentrates simultaneous traffic on an output that cannot drain it as fast as it arrives. ECN provides congestion feedback for sender rate control; PFC pauses a priority class on a link. Their thresholds, feedback delay and available headroom must match the supported fabric design.
How to approach it
Name the congested output and its arrival and drain rates. Compute the burst headroom in time units. Explain what each control can influence, then use queue, pause, mark and endpoint counters alongside application latency to test the diagnosis.
A strong answer
A receiver can be the bottleneck even when the fabric has many alternative routes. Thirty-two storage servers replying to one client are an incast pattern; thirty-two clients reading from one server describe a different direction of pressure. Congestion Control for AI Fabrics starts with where bytes actually meet.
Assume, for a worst-case synthetic burst, that 32 inputs can simultaneously offer their full 400 Gb/s nominal rates to one 400 Gb/s output. Ignore framing and other bottlenecks for this calculation:
per-port nominal rate = 400/8 = 50 GB/s
arrival = 32 × 50 = 1,600 GB/s
queue growth = arrival − drain = 1,550 GB/s
available headroom = 16 MB
fill time = 16e6 / 1,550e9 ≈ 10.32 microseconds
sanity: 1,550 GB/s × 10 microseconds already adds 15.5 MB.
This is available headroom for the traffic in question, not a claim that every switch has a 16 MB per-port buffer. Shared-buffer allocation, existing occupancy, packet sizes and admission thresholds change the usable amount. Upstream bottlenecks may reduce the offered rate before this output.
ECN marks packets to signal congestion. In a supported RoCE control loop, the destination generates congestion feedback and the sender adjusts its rate. Marking does not instantly stop the bytes already in flight. If excess arrival is 100 GB/s for a 20-microsecond feedback interval, another 2 MB arrives before a reaction; use the actual rate and delay when budgeting headroom.
PFC acts hop by hop on a priority class. A pause can stop other traffic in that class on the affected link, even if it has a different destination. Backpressure can spread as upstream queues fill. PFC is not instantaneous either: its reaction and in-flight bytes also require headroom. In configurations that use both, rate control aims to manage sustained congestion while pause protects against bursts beyond that response.
NVIDIA's RoCE guide documents supported operating modes and configuration. Keep thresholds within the vendor-supported design and test the installed NIC/switch combination. Do not infer that a pause proves ECN malfunction; a short burst can outrun a functioning feedback loop.
Collect time-aligned queue occupancy, ECN marks, congestion notifications, pause duration, discards and endpoint retry counters. Pair them with p99 useful-step latency. High pause counts and poor throughput support investigation of backpressure, but counters alone cannot locate the initiating queue. Inspect the path and compare with an uncontended baseline.
Separate traffic classes can prevent one class's PFC pause from directly pausing another class. They still share physical capacity, buffers and scheduling resources. Likewise, adaptive routing can spread path hot spots but cannot increase the final receiver's link rate. RDMA, InfiniBand and RoCEv2 connects these controls to transport behavior.
Start by reducing avoidable synchronized fan-in or providing the required receiving capacity. Tune supported feedback and buffer settings against representative bursts. Reverse a change if it reduces marks while increasing drops, pause duration or application tails. On InfiniBand, use its credit and congestion-control model; backpressure and shared-resource contention still require analysis.
What interviewers probe next
- "Will a larger buffer fix it?" It absorbs a longer burst, but sustained arrival above drain still grows a queue.
- "Does zero packet loss prove success?" No. A lossless stalled queue can have terrible latency and throughput.
- "Can I copy ECN thresholds?" Only within a validated configuration; link rate, headroom and feedback behavior matter.
Common mistakes
- Treating a hypothetical maximum fan-in as a measured switch rate.
- Calling rate feedback or pause instantaneous.
- Promising that separate priorities eliminate all interference.
Key takeaways
- The synthetic 32-to-one burst consumes 16 MB in about 10.3 microseconds.
- ECN influences sender rates; PFC pauses a link priority class.
- Validate controls with queue growth and useful application progress together.