← Open-Weights Models & Serving Engines / 43

hard

Explain what ONNX Runtime and TensorRT optimize, then diagnose a slower compiled model

Trace graph rewrites, provider placement and shape profiles. Use a fallback counterexample to show why fewer operators can still produce higher latency.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Trace graph rewrites, provider placement and shape profiles. Use a fallback counterexample to show why fewer operators can still produce higher latency.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 318 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

⚡ Kernels & Compilers🔒 Premium

CUDA Compilation: PTX, Cubins and SASSCUDA source, PTX and native GPU instructions describe different stages of execution. Inspect the selected binary and its resource use before explaining a performance change.

🕸️ Distributed TrainingSign in

MFU and HFUMeasure useful model work, separate recomputation, and align token counts, timing and hardware peaks. Learn what activity counters can and cannot tell you.

🧩 GPU & Accelerator Architecture🔒 Premium

The TPU Software Stack: JAX, XLA, PyTorch and vLLM on TPUOn a GPU you run eager PyTorch and call tuned kernels. On a TPU a compiler sees the whole program, so shapes, sharding and compile time become your main tools. This maps JAX, MaxText, PyTorch/XLA, TorchTPU, Pallas and vLLM TPU onto the CUDA stack for training and serving.

⚡ Kernels & Compilers🔒 Premium

Profiling with NsightNsight Systems shows where an application spends time; Nsight Compute investigates individual kernels. Use the timeline to choose what deserves detailed profiling.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhich GPUs can run an FP8 or AWQ checkpoint, and what do you actually get on an older card?Next in this trackYour model now serves 6,568 tokens per second. What does a million tokens cost, and what do you hand the gateway and eval teams?Next in this trackYou need product copy for a 150-item catalog by tomorrow. Batch job or the online endpoint, and how do you deliver it?

DISCUSSION · 0

No comments yet — be the first to share your approach.