Talking About Cost and Capacity with Leadership
Infrastructure engineers are asked to justify large numbers to people who do not share their vocabulary, and the conversations go wrong in predictable ways: a technical objection with no alternative, a forecast with no assumptions, or a cost quoted in a unit the listener cannot act on. What works is a small number of costed options, a stated recommendation, the decision needed by a date, and every figure expressed in whatever the listener actually controls.
TL;DR: Convert before you present. Leadership acts on money, dates and risk, so a figure in GPU-hours or gigabytes per second has to be translated into one of those before it means anything. Bring a forecast with its assumptions listed and sourced, a consequence stated in their terms rather than yours, two or three options with costs and lead times, a recommendation, and the decision you need by a date. Give a range rather than a point and name the assumption the forecast is most sensitive to, because the first question is always about the assumptions and answering it in advance is what makes the rest credible. And distinguish reserved capacity from consumed capacity, since a colocation or cloud commitment bills the reservation whether or not it is used, and that gap is the most common surprise in a first infrastructure budget.
Converting into units the listener controls
| You measure | They act on | The conversion |
|---|---|---|
| GPU-hours idle | Dollars, monthly | Idle hours times the hourly rate, stated per month |
| Utilization percentage | Cost per unit of output | Cost per million tokens, or cost per training run |
| Bandwidth per rail | Delivery dates | The runs that slip and by how long |
| Failure rate per GPU-hour | Spares budget and replacements per month | Rate times fleet times repair turnaround |
| Kilowatts | Reserved capacity charges and lead times | Contracted kW, and the months to add more |
The conversion is not decoration. A director cannot act on "the research pool runs at 19 percent"; they can act on "the research pool holds a sixth of the fleet and is idling about 4.4 million dollars a year, and reclaiming half of it delivers more capacity than the purchase we are discussing, faster and with no capital".
The shape of the conversation
Two elements of that shape are the ones most often missing. The reprioritization option, which costs nothing infrastructurally and is the one leadership is best placed to choose, is routinely omitted by engineers who present only options that cost money, and its absence frames the whole conversation as a request rather than as a decision. And the default, which makes silence safe and stops a passing date from becoming a decision nobody made.
Reserved against consumed
The distinction that surprises people in their first infrastructure budget is that capacity is usually billed on what is reserved rather than on what is drawn, and redundancy means reserving roughly twice the load because both feeds must carry it alone. A rack drawing 132 kW therefore appears on a contract at something closer to 264 kW, and the monthly line does not fall when the GPUs idle.
Saying this early builds credibility, because it is counterintuitive and it is the number the finance side already knows. It also reframes utilization from an engineering metric into the main lever on the bill, which is the connection that makes capacity planning and cost control the same activity rather than two.
Uncertainty, stated rather than hidden
A confident single line is dismantled by the first person who knows one of the inputs. A range with the sensitive assumption named survives, and it invites the listener to supply the information you lack, which is often exactly what they have. "The crossing is between month three and month six depending on whether team A's project ships on their October plan" is both more honest and more useful than a date, and it tells the listener which question to go and answer.
What interviewers are listening for
Whether the numbers were converted. A candidate who reports utilization percentages to a director is describing a conversation that did not land. After that, whether options were costed and whether reprioritization was among them, because presenting only spend options is the most common way an infrastructure request is refused. Then the assumptions, since offering them before being asked is what distinguishes a forecast from a guess.
Key takeaways
- Convert every figure into money, dates or risk before presenting it, because those are what the listener controls.
- Bring a forecast with sourced assumptions as a range, the consequence in their terms, two or three costed options, a recommendation and a date.
- Always include reprioritization among the options; omitting it frames the conversation as a request rather than a decision.
- State a default for what happens if no decision arrives, so a passing date is not a decision by accident.
- Reserved capacity is billed whether or not it is drawn, and redundancy roughly doubles it, which is the most common budget surprise.