Talking About Cost and Capacity with Leadership
A capacity discussion should connect technical measurements to cost, delivery dates and risk. Present the assumptions, feasible options and a recommendation, and distinguish committed capacity from actual consumption under the relevant contract.
TL;DR: Explain what the capacity decision changes for the organization. Show the forecast’s assumptions and uncertainty, compare spending with scheduling or prioritization options, and state the decision needed. Check how the contract charges for reserved and consumed capacity before calculating savings.
Converting into units the listener controls
| You measure | They act on | The conversion |
|---|---|---|
| GPU-hours idle | Dollars, monthly | Idle hours times the hourly rate, stated per month |
| Utilization percentage | Cost per unit of output | Cost per million tokens, or cost per training run |
| Bandwidth per rail | Delivery dates | The runs that slip and by how long |
| Failure rate per GPU-hour | Spares budget and replacements per month | Rate times fleet times repair turnaround |
| Kilowatts | Reserved capacity charges and lead times | Contracted kW, and the months to add more |
A utilization percentage needs context before it supports a spending decision. Explain how much capacity is committed, what useful work it produces and whether idle capacity can actually be reassigned. Include the engineering effort and disruption required to reclaim it.
The shape of the conversation
Start with demand and supply as ranges, naming the assumption that most changes the result.
- Explain the consequence
Connect the shortfall to delivery dates, affected users or blocked work.
- Compare feasible options
Include cost, lead time, risk and any scheduling or reprioritization option.
- Recommend an option
State what evidence would change your recommendation.
- Name the decision and deadline
State what you are authorized to do while waiting.
Two elements of that shape are the ones most often missing. The reprioritization option, which costs nothing infrastructurally and is the one leadership is best placed to choose, is routinely omitted by engineers who present only options that cost money, and its absence frames the whole conversation as a request rather than as a decision. And the default, which makes silence safe and stops a passing date from becoming a decision nobody made.
Reserved against consumed
A cloud commitment or colocation agreement may charge for reserved capacity even when the workload uses less. Other contracts charge partly or entirely for measured consumption. Read the billing basis before translating lower utilization into a saving.
Redundant power also needs careful accounting. If either feed must support a 132 kW rack alone, each feed needs that capacity. That does not by itself establish a bill for 264 kW: the contracted quantity and charging method determine the cost. Distinguish physical redundancy from the billing line item.
This distinction helps explain which changes reduce a bill immediately, which improve useful work within an existing commitment, and which affect the next renewal.
Uncertainty, stated rather than hidden
A confident single line is dismantled by the first person who knows one of the inputs. A range with the sensitive assumption named survives, and it invites the listener to supply the information you lack, which is often exactly what they have. "The crossing is between month three and month six depending on whether team A's project ships on their October plan" is both more honest and more useful than a date, and it tells the listener which question to go and answer.
What interviewers are listening for
Show how you connected the engineering measurement to the decision the audience could make. Explain the assumptions, options and tradeoffs, including whether scheduling or reprioritization could meet the need without additional capacity.
If the forecast changed, say which input changed and how that affected your recommendation. That makes the reasoning easier to assess than presenting a single confident number without its basis.
Key takeaways
- Convert every figure into money, dates or risk before presenting it, because those are what the listener controls.
- Bring a forecast with sourced assumptions as a range, the consequence in their terms, two or three costed options, a recommendation and a date.
- Always include reprioritization among the options; omitting it frames the conversation as a request rather than a decision.
- State a default for what happens if no decision arrives, so a passing date is not a decision by accident.
- Check the contract: reserved capacity, consumption and redundant power can have different charging rules.