hardAnyscaleFireworksDatabricks

How would you serve hundreds of LoRA adapters on one base model, and what does it cost in throughput?

Follow adapter-specific token rows through a shared base, calculate stored adapter bytes, and separate the resident set from the active batch.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Follow adapter-specific token rows through a shared base, calculate stored adapter bytes, and separate the resident set from the active batch.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🚀 Inference & Serving🔒 Premium

Multi-LoRA ServingMulti-LoRA serving applies different adapters to a shared base model. It saves weight memory while adding adapter loading, scheduling and isolation requirements.

Advanced

📐 AI Systems Design🔒 Premium

Multi-Tenant Fine-Tuning ServiceA fine-tuning service turns a tenant’s dataset into an evaluated model or adapter. Its pipeline must protect data while scheduling training and serving the result efficiently.

Core

🚀 Inference & ServingSign in

Continuous BatchingContinuous batching changes which requests run between model iterations. Learn how a waiting request joins, why an empty slot is not enough, and how to balance throughput against each user’s token latency.

Advanced

🧮 Open Weights & Serving Engines🔒 Premium

Pretraining, Fine-Tuning and DistillationUnderstand how a base model becomes a useful application model. Separate training objectives from LoRA and quantization, compare distillation with retrieval, and choose the smallest intervention supported by your evaluation.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDesign an autoscaler for GPU inference replicas that reacts to load without thrashing.Next in this trackA new replica has to load a 70B model and serve traffic in under a minute. Where do the seconds go, and how do you get there?Next in this trackWhat changes when you serve DeepSeek-V3 compared with a dense 70B model?

FEDITOR'S NOTE

Calculate adapter memory from rank and matrix dimensions, then distinguish the stored collection, resident adapters and adapters used by the current batch.

DISCUSSION · 0

No comments yet — be the first to share your approach.