hardFireworksTogether AIDeepSeek

What changes when you serve DeepSeek-V3 compared with a dense 70B model?

Separate checkpoint memory, expert assignments and retained MLA state. The active parameter count alone cannot predict per-rank fit or token latency.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Separate checkpoint memory, expert assignments and retained MLA state. The active parameter count alone cannot predict per-rank fit or token latency.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Core

🚀 Inference & ServingSign in

Attention Variants: MHA, GQA, MQA and MLACompare MHA, GQA, MQA and compressed MLA by the state each retains. Derive cache payload in consistent units, distinguish storage capacity from serving throughput, and account for hybrid layers and rank replication.

Foundational

🧮 Napkin Math & Capacity

KV Cache SizingDerive cache payload from the stored attention state, then account for mixed sequence lengths, block allocation and per-device limits. Worked examples distinguish exact bytes from deployable capacity.

Foundational

🧮 Open Weights & Serving Engines

Expert Parallel and All-to-All BackendsCompare tensor and expert placement without confusing either with sparsity. Derive logical dispatch traffic and redundant-expert memory, then choose backends and balancing from per-rank measurements.

Advanced

🕸️ Distributed Training🔒 Premium

Expert Parallelism for MoEExpert parallelism places different MoE experts on different devices. Routing tokens between them adds communication and makes uneven expert demand a performance concern.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackHow do you route requests across replicas to maximize prefix-cache hits without unbalancing the fleet?Next in this trackYou stream tokens to clients over HTTP. What can go wrong, and how does the server handle each case?Next in this trackYou need to serve 128k-token contexts. What breaks first, and what do you change?

FEDITOR'S NOTE

Account separately for loaded weights, active expert computation, communication and cache placement. Active parameters alone cannot establish memory fit or token throughput.

DISCUSSION · 0

No comments yet — be the first to share your approach.