← Open-Weights Models & Serving Engines / 43

hardNew

Explain what ONNX Runtime and TensorRT optimize, then diagnose a slower compiled model

Trace graph rewrites, provider placement and shape profiles. Use a fallback counterexample to show why fewer operators can still produce higher latency.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Trace graph rewrites, provider placement and shape profiles. Use a fallback counterexample to show why fewer operators can still produce higher latency.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 303 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🕸️ Distributed TrainingSign in

MFU and HFUMeasure useful model work, separate recomputation, and align token counts, timing and hardware peaks. Learn what activity counters can and cannot tell you.

⚡ Kernels & Compilers🔒 Premium

Profiling with NsightNsight Systems shows where an application spends time; Nsight Compute investigates individual kernels. Use the timeline to choose what deserves detailed profiling.

🧮 Open Weights & Serving Engines

Open-Weights Models of 2026Compare current Chinese open-weight models by the artifact you can deploy: weight bytes, attention state, supported execution paths, and evaluation conditions. A dated guide to GLM, Kimi, DeepSeek, Qwen, and MiniMax, with worked sizing checks that separate parameter counts from GPU capacity.

🧮 Open Weights & Serving Engines

Reading config.json to Size a Model You Have Never RunRead a model configuration as an architecture description, then combine it with tensor metadata and engine allocation rules. Learn which fields determine attention state, why mixed precision needs tensor-level accounting, and how to turn a first estimate into a testable per-GPU capacity plan.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Closest to this topicA kernel does 4 TFLOP and moves 40 GB in one call. On an H100, is it memory-bound or compute-bound?Closest to this topicAn Nsight summary reads DRAM 90%, SM 30%, occupancy 45%, L2 hit rate 20%. What is the bottleneck?Closest to this topicTell me about a performance win you are proud of. How did you measure it?

DISCUSSION · 0

No comments yet — be the first to share your approach.