TL;DR: GPU power is one part of the cluster budget. Add the rest of the servers, network, storage and management equipment without counting any component twice. Apply an explicit facility-overhead assumption for an energy estimate, then separately check peak demand, failover capacity and the contract. In the worked example, 716.8 kW of GPUs becomes 1,058.56 kW of IT load and about 1.27 MW of facility demand. Redundant feeds do not automatically double consumption or the bill.
How to approach it
Ask whether the requested number is expected consumption, maximum supported load, cooling duty or contracted capacity. Build a component inventory with clear measurement boundaries. Calculate an example, then explain which assumptions require equipment data and commissioning measurements.
A strong answer
Suppose a team wants power for 512 GPUs arranged in 64 eight-GPU servers. Multiplying the GPU count by a power limit gives a useful starting estimate, but it cannot size the hall. CPUs, memory, fans, NICs and power supplies draw power too, as do switches and storage. The cooling plant and upstream electrical losses sit outside the IT equipment boundary.
Build an explicit example
The following values are planning assumptions, not specifications for a particular Blackwell system or a completed network design. Assume each GPU draws 1.4 kW at the workload of interest, and the remaining server equipment adds 30% of that total, including server power-supply losses but excluding optical modules.
| Term | Assumed quantity and power | Subtotal |
|---|---|---|
| GPUs | 512 × 1.4 kW | 716.80 kW |
| Remaining server load | 30% × 716.80 kW | 215.04 kW |
| Switch chassis, excluding optics | 12 × 3 kW | 36.00 kW |
| Optical modules across all endpoints | 2,048 × 15 W | 30.72 kW |
| Storage and management | Separate assumed inventory total | 60.00 kW |
| IT total | Sum of the five terms | 1,058.56 kW |
The server subtotal is 931.84 kW, or 14.56 kW per eight-GPU server before the separately counted optics. If a vendor's whole-server or whole-switch power figure already includes those modules, remove them from the separate optics term. Host-side modules belong at their host endpoints; they are not all in the switch row.
The illustrative optics count assumes 1,024 point-to-point optical links with one module at each end. Derive a real count from the port map: breakouts, twin-port modules, passive cables, active cables and fabric oversubscription change the inventory. Twelve switch chassis are an assumed budget line here, not proof that they provide the required ports or topology. Cables, Transceivers and the Optics Power Budget covers this accounting in more detail.
Add facility overhead at the right boundary
The Department of Energy's PUE explanation defines power usage effectiveness using total facility energy divided by IT energy over the same period. A planning PUE of 1.20 therefore gives:
IT energy for one hour at the assumed load = 1,058.56 kWh
Estimated facility energy = 1,058.56 × 1.20 = 1,270.272 kWh
Equivalent average facility demand = 1,270.272 kW ≈ 1.27 MW
Facility overhead in this estimate = 211.712 kW
sanity: 1.27 MW is about 1.772× the GPU-only 716.8 kW, not a universal multiplier.
An annual PUE is not a guarantee of overhead at the hottest hour or at low utilization. Use site-specific seasonal and load-dependent data for an energy model. Peak electrical capacity needs the actual equipment limits and power-path design. Likewise, allocate cooling loads to the loops and rooms that receive them rather than treating the PUE multiplier as a rack cooling requirement.
Illustrative 512-GPU budget, with components counted once.
- GPU boards: 716.80 kW
Add 215.04 kW of other server load, excluding separately listed optics.
- IT equipment: 1,058.56 kW
Includes assumed switches, modules, storage and management.
- Facility estimate: 1,270.272 kW
Apply an assumed PUE of 1.20 for the same operating period.
Keep redundancy separate from consumption
For a simplified dual-feed load requiring 100 kW, normal sharing might put 50 kW on A and 50 kW on B. If either feed must carry the entire load after failure, each path needs sufficient usable capacity for 100 kW. The equipment still consumes about 100 kW in the idealized example, not 200 kW. Losses and failover behavior need their own measurements.
A provider may bill committed IT capacity, circuits, measured energy, demand or a combination. Confirm the contract's boundary and usable capacity; do not infer a 2× reserved-capacity charge merely from two feeds. Rack Power Delivery and Busways explains the per-rack electrical path.
| Recipient | Useful output |
|---|---|
| Electrical design team | Equipment peaks, phase currents, usable circuit ratings and single-failure load cases |
| Cooling team | Heat by rack and loop, liquid capture fraction, residual air load and design conditions |
| Finance or procurement | Metering boundary, expected kWh, demand and capacity charges under the actual contract |
| Capacity planner | Remaining usable power and cooling capacity after the required failure or maintenance event |
Validate the load shape, not just its average
Measure server input and rack/row demand during representative training, inference, startup and recovery activity. Synchronized training phases can cause correlated changes, but inference can also produce bursts or synchronized replica startup. Neither workload has a universal 1.4× peak factor. Telemetry sampling must be fast enough for the transient being studied; average GPU readings cannot characterize every electrical event.
Compare GPU telemetry with server and PDU meters using aligned timestamps and boundaries. Test proposed power caps against application throughput and latency. A 10% cap reduction does not promise a 10% reduction at the rack or only a small performance loss. Use the equipment vendor and facility engineer's validated limits for protection, reserve and failover design.
If measurements show lower demand, revise the energy forecast. Reduce provisioned capacity only when the peak, failure and growth requirements still pass. Conversely, a larger optical inventory, less efficient cooling or a higher-power workload can invalidate the initial budget even when GPU count stays fixed.
What interviewers probe next
- “Why not use nameplate power everywhere?” It is useful for a supported maximum, but may overstate expected energy. State which question each estimate answers.
- “How do you avoid double counting?” Record whether each chassis measurement includes fans, power-supply losses, NICs and modules.
- “Does A/B mean twice the draw?” No. Failover capacity and consumed energy are different quantities.
Common mistakes
- Using GPU limits as measured server power.
- Adding modules again after using a chassis figure that includes them.
- Using annual PUE to establish an instantaneous electrical peak.
- Assuming an inference fleet cannot produce correlated demand.
Key takeaways
- The example totals 1,058.56 kW of IT load and about 1.27 MW at an assumed PUE of 1.20.
- Its 2,048 assumed optical modules contribute 30.72 kW; the real count comes from the topology.
- Report consumption, peak capacity and contractual commitments separately.