Fine-tune a 70B on one 80 GB card. What do NF4 and double quantization actually buy, and where does DoRA change the arithmetic?

Separate packed base weights, quantization metadata and trainable adapter state. Count the real projection shapes, then leave room for activations and temporary work before calling a one-card configuration feasible.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

GPUs and time-to-train calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🧮 Open Weights & Serving Engines🔒 Premium

Pretraining, Fine-Tuning and DistillationUnderstand how a base model becomes a useful application model. Separate training objectives from LoRA and quantization, compare distillation with retrieval, and choose the smallest intervention supported by your evaluation.

Advanced

🚀 Inference & Serving🔒 Premium

Multi-LoRA ServingMulti-LoRA serving applies different adapters to a shared base model. It saves weight memory while adding adapter loading, scheduling and isolation requirements.

Advanced

📐 AI Systems Design🔒 Premium

Multi-Tenant Fine-Tuning ServiceA fine-tuning service turns a tenant’s dataset into an evaluated model or adapter. Its pipeline must protect data while scheduling training and serving the result efficiently.

Foundational

🧮 Open Weights & Serving Engines

Weight Formats: FP8 Blocks, MXFP4 and AWQDerive the storage cost of low-precision tensors, including scales, zero points and higher-precision exceptions. Distinguish formats from quantization methods, understand fused dequantization, and test whether a smaller checkpoint actually improves serving capacity without unacceptable quality loss.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDesign the infrastructure for RLHF on a 70B: where do rollouts and the learner run, and how do weights move between them every step?Next in this trackYour team wants to replace a PPO-style RLHF pipeline with DPO. What leaves the cluster, and what does the training job look like afterwards?Next in this trackWe are running GRPO at scale. What does the infrastructure have to do that plain RLHF did not, and where do the GPUs sit idle?

FEDITOR'S NOTE

Derive the frozen-base and adapter budgets from explicit shapes and dtypes. Keep double-quantization savings in the metadata term, and treat paging, DoRA and memory fit as measured implementation choices.

DISCUSSION · 0

No comments yet — be the first to share your approach.