hardAMDMicrosoftOracle

The MI300X has 192 GB per card. When does AMD actually win against an H100 for LLM serving, and what would you check before betting on it?

See how 192 GB changes replica shape, calculate KV headroom with exact units, and compare feasible deployments at equal quality and latency.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

See how 192 GB changes replica shape, calculate KV headroom with exact units, and compare feasible deployments at equal quality and latency.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

AMD Instinct and ROCmAMD Instinct GPUs offer substantial memory capacity and bandwidth. Workload performance also depends on ROCm, the model implementation and the kernels available for the chosen device.

Foundational

🧭 Ownership & Judgment

Talking About Cost and Capacity with LeadershipA capacity discussion should connect technical measurements to cost, delivery dates and risk. Present the assumptions, feasible options and a recommendation, and distinguish committed capacity from actual consumption under the relevant contract.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

TCO: Buy vs RentBuying and renting GPU capacity have different cost and availability risks. Compare them over a stated lifetime and demand pattern, including idle time and operating costs.

Advanced

🚀 Inference & Serving🔒 Premium

Disaggregated Prefill and DecodeSeparate prefill and decode to control phase interference, then budget the KV handoff. A short isolated transfer can still overload a shared path; cancellation stops generation before it necessarily releases transfer buffers.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhy are Groq and Cerebras so fast at batch 1, and what does that speed cost at scale?Next in this trackMIG versus MPS: what isolation does each give you when sharing a GPU, and which would you pick for a multi-tenant inference node?Next in this trackYou are moving a model to fp4 inference on Blackwell. What breaks first, and how would you measure whether the result is acceptable?

FEDITOR'S NOTE

Compare deployments that both fit the model. Budget KV growth on every rank, then test whether the memory advantage improves useful throughput at the required latency and quality.

DISCUSSION · 0

No comments yet — be the first to share your approach.