6nd
AI infra interview questions tagged 6nd, across every topic.
2 questions · 1 unlocked for you
Concepts behind "6nd"
The curriculum that explains the ideas these questions test.
Foundational
Training FLOPs: 6NDThe compute needed to train a language model is six floating-point operations per parameter per token: two for the forward pass and four for the backward. Multiply by the parameter count and the token count and you have the whole run's compute, which is the number every fleet-sizing, time-to-train and cost question starts from. This page derives the 6, states the attention correction and when it matters, and shows where the 2N of inference comes from, so the reader can rebuild the formula rather than recall it.🧮 Napkin Math & Capacity
Core
MFU and HFUModel FLOPs utilization is the fraction of a GPU's peak that goes into the model's own forward and backward math, computed from 6ND and the step time; hardware FLOPs utilization also counts recomputation. Production LLM training lands at 35 to 45% MFU, and knowing where the other 55% goes is the job.🕸️ Distributed TrainingSign in