You must serve a frontier open-weights model on non-NVIDIA accelerators. Plan it.

The weights are portable and almost nothing else is. Four layers have to exist before the model runs at all, the quantization format is the one most likely to be missing, and the honest plan states what will not work in the first quarter rather than promising parity.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

AMD Instinct and ROCmAMD's Instinct line competes on memory: 192 GB on the MI300X, 256 on the MI325X, 288 on the MI355X, with dense fp8 peaks at or above NVIDIA's at each generation. The catch is software: ROCm and HIP run most PyTorch and the main serving engines, but the attention, MoE and quantization kernels arrive on CUDA first and measured throughput has trailed the spec sheet. Interviewers want the numbers, the wavefront-of-64 difference, and a clear-eyed statement of when the memory argument wins.

Foundational

🧮 Open Weights & Serving Engines

Weight Formats: FP8 Blocks, MXFP4 and AWQOpen-weights models now ship pre-quantized, and the format is part of the release rather than something you choose afterwards. Block-scaled FP8 gives one byte per parameter with a scale per tile. MXFP4 gives about 0.53 bytes by pairing four-bit values with a shared exponent every 32 elements. Integer schemes like AWQ reach similar sizes with a different error profile. What decides a deployment is not which is most accurate in the abstract but which one the model was released and evaluated in, and which one your engine and hardware can execute natively.

Advanced

⚡ Kernels & Compilers🔒 Premium

FlashAttention InternalsStandard attention writes the N x N score matrix to HBM and reads it back, which makes it memory-bound and quadratic in memory. FlashAttention tiles Q, K and V through shared memory, keeps a running max and sum so the softmax never needs the full row, and recomputes scores in the backward pass. Knowing the online-softmax rescale, why FlashAttention-2 flipped the loop order, and what FlashAttention-3 overlaps on Hopper is the difference between naming the paper and being able to write the kernel.

Foundational

⚡ Kernels & Compilers

CUDA Programming ModelCUDA splits a program into a host that allocates, copies and enqueues work, and a device that runs thousands of identical threads organized as a grid of blocks. Getting the split right, and knowing that a launch returns before the kernel runs, decides whether your first live-coding kernel produces a correct number or a silent zero.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Closest to this topicThe MI300X has 192 GB per card. When does AMD actually win against an H100 for LLM serving, and what would you check before betting on it?Closest to this topicPort a hand-written CUDA kernel to MI300X. What translates mechanically, and what silently computes the wrong answer?Closest to this topicYou upgraded the serving engine and throughput fell 15 percent. Find the cause.

FEDITOR'S NOTE

Scored on the four-layer dependency, on quantization format support being the usual blocker, and on stating what will not work rather than promising parity.

DISCUSSION · 0

No comments yet — be the first to share your approach.