AI Infra Interviews logo
← Napkin Math, Cost & Capacity / 13
hard★ EssentialOpenAIBasetenTogether AI

Size an inference fleet for a 70B model serving 1,000 concurrent users

Translate active streams into replicas using growth reservations and matched latency tests. Show peak and failure headroom separately, with no invented benchmark.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Translate active streams into replicas using growth reservations and matched latency tests. Show peak and failure headroom separately, with no invented benchmark.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

UP NEXT ON YOUR JOURNEY
FEDITOR'S NOTE

Ask which failure N−1 covers. Session placement or a simultaneous rollout can invalidate a fleet-average spare-capacity calculation.

DISCUSSION · 0

No comments yet — be the first to share your approach.