AI Infra Interviews logo
← AI Infrastructure System Design / 11
mediumAnyscaleFireworksDatabricks

Design serving for 100 fine-tuned variants of one 70B base. Multi-LoRA on shared replicas or a replica per variant?

Derive adapter bytes from the target matrices, share compatible base weights, and keep each request's cache identity separate. A worked pool design avoids applying the mixed-adapter cost twice.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Derive adapter bytes from the target matrices, share compatible base weights, and keep each request's cache identity separate. A worked pool design avoids applying the mixed-adapter cost twice.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed
UP NEXT ON YOUR JOURNEY
FEDITOR'S NOTE

Scored on explicit capacity assumptions, correct state and identity boundaries, failure handling and a justified design choice.

DISCUSSION · 0

No comments yet — be the first to share your approach.