AI Infra Interviews logo
← Open-Weights Models & Serving Engines / 24
hardBasetenModalTogether AI

Design a platform that serves twenty open-weights models of varying size to internal teams.

Design a shared catalogue with explicit service tiers, reproducible deployments and bounded cold starts. Residency depends on burst behavior and resource reuse; placement depends on GPU groups and interference as well as memory.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Design a shared catalogue with explicit service tiers, reproducible deployments and bounded cold starts. Residency depends on burst behavior and resource reuse; placement depends on GPU groups and interference as well as memory.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

UP NEXT ON YOUR JOURNEY
FEDITOR'S NOTE

Scored on lifecycle and admission semantics, measured residency economics, per-device placement, cold-start critical paths and failure capacity.

DISCUSSION · 0

No comments yet — be the first to share your approach.