You have budget for roughly 1,000 GPUs and a greenfield site. Specify the cluster.

TL;DR: Work from the facility inward. A thousand Blackwell-class GPUs is roughly 125 nodes, and at NVIDIA's published figure of over 50 kW for a rack of four DGX B300 that is about 32 racks and 1.6 MW of compute before networking, storage or cooling overhead. If the site cannot deliver and reject that, the design is already fixed at whatever it can. Then use a published scalable unit rather than inventing a topology: NVIDIA's DGX SuperPOD B300 reference architecture defines a unit of 64 nodes and 512 GPUs with 8 leaf switches, so a thousand GPUs is two of those. Build four networks, not one: an 800 Gb/s rail-optimized compute fabric, a storage fabric sized by checkpoint burst, in-band management, and out-of-band reaching every BMC, PDU, switch and CDU. Size storage from the checkpoint write rather than the dataset. And order power, cooling and facility work first, because they take months to years while GPUs take weeks to months.

How to approach it

Ask what the site can power and cool before touching the GPU count, because that answer usually overrides the budget. Then pick the unit of replication from a published reference architecture and say why. Then walk the four networks. Then storage, from the checkpoint. Then the schedule, longest lead time first. Close by naming what you would cut if the power envelope came in lower than hoped.

A strong answer

A typical situation: a team has approval for a thousand GPUs, a colocation hall offered at 15 kW per rack, and a delivery date six months out. The hall is the problem and nobody has noticed, because every conversation so far has been about which GPU.

The power envelope first, since it sets the count:

what 1,000 Blackwell-class GPUs costs in facility terms
  nodes            1,000 / 8 = 125 nodes
  racks            at 4 nodes per rack (NVIDIA's B300 reference figure): 32 racks
  compute power    32 racks x >50 kW = over 1.6 MW
  plus networking, storage, management: add roughly 10 to 15%
  plus cooling overhead at a PUE of 1.2 to 1.3 for a liquid-cooled hall
  total facility draw ≈ 1.6 x 1.13 x 1.25 ≈ 2.3 MW

now check the offered hall
  15 kW per rack x 32 racks = 480 kW, which is 30% of what the compute alone needs
  to fit 1.6 MW at 15 kW per rack would take 107 racks, and the hall does not have them
sanity: the hall supports roughly 300 GPUs of this class, not 1,000, so the real decision is
        a different site or a different part, and that decision has to happen before anything
        is ordered

Then the unit of replication, chosen rather than invented:

Using the published unit is worth stating out loud as a decision. It means the rack layout, the cable lengths, the switch placement and the power distribution are already worked out by someone who built it, and a custom topology means redoing all of that for no gain at this scale. Scale-Out Fabric Choice: InfiniBand XDR vs Spectrum-X covers the radix arithmetic that says a 144-port switch reaches 10,368 endpoints in two tiers, so a thousand GPUs is nowhere near needing a third tier.

The four networks, and the storage number:

1. compute fabric      8 x 800 Gb/s per node, rail-optimized, 16 leaves plus spines
2. storage fabric      separate ports, sized by the checkpoint burst
3. in-band management  one port per node: provisioning, telemetry, package installs
4. out-of-band         one port per BMC, per PDU, per switch, per CDU
                       more endpoints than nodes, and the network that must work when
                       nothing else does

storage sizing, from the checkpoint rather than the dataset
  a 70B model's training state at 16 bytes per parameter = 1.12 TB
  target write window so the job barely stalls: 60 s
  required sustained write = 1.12e12 / 60 = 18.7 GB/s
  the dataset read for 1,000 GPUs on pre-tokenized text is a few megabytes per second
sanity: the burst is thousands of times the steady read, so a tier specified from the dataset
        rate is wrong by three orders of magnitude for the one operation that matters

The acceptance criteria belong in the specification rather than being invented at handover, and they are the same commands the deployment will live on:

what "delivered" means, written into the purchase
  ibstat                    every port Active at 800 Gb/s, on every node
  nvidia-smi topo -m        NVLink between all 8 GPUs on every node, which also proves the
                            fabric manager is running
  nvidia-smi -q -d POWER    enforced power limit matching the specification on every GPU
  nccl-tests all_reduce_perf, 8 GPUs, one node
                            bus bandwidth within 3% of the fleet median
  nccl-tests all_reduce_perf, 2 nodes
                            a result close to the single-node number proves GPUDirect RDMA is
                            active; roughly half of it means it is not
  nccl-tests, one full scalable unit, 48 to 72 h
                            sustained, with DCGM_FI_DEV_GPU_TEMP and clocks recorded
sanity: every one of these is a command a technician can run and a number the vendor can be
        held to, which is what turns an acceptance clause into something enforceable

The Bill of Materials for a Training Cluster covers the full list, and Rack Power Delivery and Busways covers the electrical side, where the number to have ready is the current: a 132 kW rack at 415 V three-phase is about 185 A, and redundant feeds reserve roughly twice that.

The reversal condition: if the workload is inference rather than training, most of this changes. Inference replicas do not need a non-blocking all-to-all fabric between every node, because a replica that fits in one NVLink domain never communicates with another replica. That removes the spine layer's cost, allows a much cheaper fabric with real oversubscription, and shifts the storage requirement from checkpoint burst to model-weight distribution. A design that carries a training fabric into an inference deployment spends a large amount of money on links that will never carry a collective.

What interviewers probe next

• "Why 4 nodes per rack?" Power, not space. At over 50 kW for four B300 nodes, a rack with more of them exceeds what the busway tap and the cooling loop deliver.
• "What if the site only offers air cooling?" Then Blackwell-class SXM parts are out and the design moves to lower-power parts or a rear-door heat exchanger, and the GPU count falls.
• "How much oversubscription is acceptable on the fabric?" For training, none between nodes in the same job. For inference, substantial, which is why the two designs differ.
• "What do you order first?" Power and cooling. GPUs last, because storing them earns nothing and they improve while you wait.

Common mistakes

• Choosing the GPU count from the budget and discovering the power envelope afterwards.
• Inventing a fabric topology when a published scalable unit exists for the exact hardware.
• Counting one network and then discovering the storage, in-band and out-of-band networks during cabling.
• Sizing storage by dataset capacity rather than by checkpoint write bandwidth.
• Ordering compute first and having it arrive to a hall that is not ready.

Key takeaways

• 1,000 Blackwell-class GPUs is about 125 nodes, 32 racks, over 1.6 MW of compute and roughly 2.3 MW of facility draw.
• Use a published scalable unit: 64 nodes and 512 GPUs with 8 leaf switches, so a thousand GPUs is two of them.
• Four networks: compute fabric, storage fabric, in-band management, and out-of-band with more endpoints than nodes.
• Storage sizing comes from the checkpoint burst: 1.12 TB in 60 seconds is 18.7 GB/s, thousands of times the dataset read.
• Order facility, power and cooling first; compute has the shortest lead time and the highest idle cost.