The AI infra concept map
178 concepts across 13 tracks, drawn with the 820 links between them. 427 of those links (52%) cross tracks, which is the part worth looking at: the concepts that decide AI infra interviews are rarely the ones that sit neatly inside one topic. Hover a concept to see only what it touches. Click to read it.
same trackcrosses tracks free membersbigger dot = more connections178 shown
Where the map is dense
The most-connected concepts are the ones the rest of the library keeps reaching for, which makes them the highest-leverage things to be solid on: Model Memory Footprint (19), Tensor Cores and Matrix Units (18), Roofline Model (18), NVLink, NVSwitch and PCIe (18), and Latency Metrics: TTFT, TPOT and Goodput (18). If you are deciding where to spend a week, start with a hub rather than a leaf.
Every concept, by track
GPU & Accelerator Architecture16
- Tensor Cores and Matrix Units18
- Roofline Model18
- NVLink, NVSwitch and PCIe18
- GPU Memory Hierarchy17
- Memory-Bound vs Compute-Bound Kernels15
- GPU Generations: A100 to Blackwell14
- Numerics: FP32, BF16, FP8 and FP410
- GPU Execution Model9
- TPU Architecture and Systolic Arrays9
- Tensors, Shapes and Dtypes7
- TPU Generations: v4 to TPU 87
- Trainium and Inferentia6
- AMD Instinct and ROCm6
- Cerebras, Groq and Dataflow Accelerators6
- The TPU Software Stack: JAX, XLA, PyTorch and vLLM on TPU6
- Structured Weight Sparsity5
Hardware & Cluster Build-Out11
- NVLink Domains and the NVL72 Rack10
- The Bill of Materials for a Training Cluster10
- Scale-Out Fabric Choice: InfiniBand XDR vs Spectrum-X8
- Accelerator Selection: H100 to B300 and RTX PRO 60007
- Rack Power Delivery and Busways7
- Direct-to-Chip Liquid Cooling and CDUs7
- Cluster Bring-Up: Firmware, Drivers and the Stack7
- Burn-In and Acceptance Testing7
- SXM, PCIe and Rack-Scale Form Factors6
- Cables, Transceivers and the Optics Power Budget6
- Colocation, Power Contracts and Site Selection6
Kernels & Compilers15
- Profiling with Nsight12
- Kernel Fusion11
- FlashAttention Internals11
- CUDA Programming Model10
- Tiled Matrix Multiplication10
- Memory Coalescing9
- Occupancy and Register Pressure9
- torch.compile and CUDA Graphs8
- Shared Memory and Bank Conflicts7
- Triton Programming Model7
- CUTLASS and Tensor Core Kernels6
- CUDA Compilation: PTX, Cubins and SASS6
- Parallel Prefix Scan and Stream Compaction6
- GEMM Work Partitioning and Wave Quantization5
- Kernel Correctness Contracts5
Distributed Training14
- Collective Communication Primitives17
- Expert Parallelism for MoE14
- Tensor Parallelism13
- ZeRO and FSDP10
- Elastic and Fault-Tolerant Training10
- MFU and HFU9
- Checkpointing and Resumption at Scale9
- Ring vs Tree All-Reduce8
- Activation Checkpointing8
- Data Parallelism and DDP7
- Pipeline Parallelism and the Bubble7
- Context and Sequence Parallelism6
- RL Post-Training Infrastructure6
- Forward Pass, Backpropagation and Optimizer Steps6
Inference & Serving23
- Latency Metrics: TTFT, TPOT and Goodput18
- Inference Autoscaling and Cold Starts17
- Serving Engines: vLLM, SGLang and TensorRT-LLM15
- The KV Cache13
- Continuous Batching13
- Prefix Caching and KV Reuse12
- Quantization for Inference12
- Prefill vs Decode11
- PagedAttention11
- Chunked Prefill11
- Speculative Decoding10
- Attention Variants: MHA, GQA, MQA and MLA10
- Disaggregated Prefill and Decode9
- Tokens, Embeddings and Logits8
- Multi-LoRA Serving7
- Inside a Transformer Block6
- Embedding Services and Vector Retrieval6
- Structured Outputs and Constrained Decoding6
- The Serving Optimization Waterfall6
- Reranking and Retrieval Budgets5
- Hybrid State Checkpoints and Rollback5
- Voice Turns, Barge-In and Playback Buffers5
- Reasoning Budgets and Answer Latency5
Open Weights & Serving Engines12
- Serving Benchmarks That Do Not Lie14
- Reading config.json to Size a Model You Have Never Run11
- Model Onboarding: From Hugging Face to Production11
- Open-Weights Models of 20269
- vLLM Server Arguments That Matter9
- Weight Formats: FP8 Blocks, MXFP4 and AWQ8
- Multi-Head Latent Attention and Sparse Indexers7
- SGLang Server Arguments That Matter7
- Pretraining, Fine-Tuning and Distillation7
- Expert Parallel and All-to-All Backends6
- Multi-Node Serving Topologies6
- Capacity Planning for Open-Weights Fleets6
Napkin Math & Capacity11
- Model Memory Footprint19
- Capacity Planning and Utilization17
- Cost per Million Tokens16
- Bandwidth-Bound Decode Throughput15
- TCO: Buy vs Rent14
- KV Cache Sizing12
- Power and Datacenter Constraints9
- GPU-Hours and Time to Train8
- Arithmetic Intensity by Operation8
- Communication Volume Estimates8
- Training FLOPs: 6ND7
Networking & Storage12
- Parallel Filesystems vs Object Storage12
- NCCL and Collective Algorithms11
- RDMA, InfiniBand and RoCEv211
- Checkpoint I/O11
- Rail-Optimized and Fat-Tree Fabrics10
- Topology-Aware Communication10
- Data Loading Pipelines for Training10
- Debugging a Slow All-Reduce10
- GPUDirect RDMA and GPUDirect Storage9
- Dataset Lifecycle: Ingest, Shard and Retain8
- Congestion Control for AI Fabrics7
- Request Networking: DNS, TCP and TLS4
Scheduling & Orchestration13
- Multi-Tenancy, Quotas and Fair Share15
- Containers, Images and GPU Cold Starts13
- Spot, Preemption and Capacity Strategies12
- Topology-Aware Scheduling10
- Gang Scheduling with Kueue and Volcano9
- Kubernetes GPU Scheduling8
- MIG, MPS and Time-Slicing8
- Slurm for AI Clusters7
- Slurm vs Kubernetes7
- Node Lifecycle: Drain, Upgrade and Return7
- Ray on Kubernetes6
- Workload Identity and Secrets5
- GPU Tenant Isolation Boundaries5
Fleet Reliability & Observability13
- Incident Response for GPU Fleets15
- DCGM and GPU Telemetry13
- Node Health Checks and Burn-In13
- SLOs for AI Systems11
- Stragglers and Hangs10
- Training Uptime and Interruption Statistics10
- GPU Failure Modes and XID Errors9
- NVLink and Fabric Faults9
- Thermal, Power and Cooling Events9
- ECC, Row Remapping and Memory Errors7
- Evaluation Metrics and Data Splits7
- Alert Design and On-Call Load6
- Tracing Inference Requests5
AI Systems Design15
- Capacity and Backpressure18
- GPU Job Scheduler Design15
- Inference Platform Architecture14
- Designing for Latency SLOs13
- Request Routing and Load Balancing for LLMs12
- Training Cluster Design at 10k GPUs11
- Evaluation and Data Pipeline Infrastructure11
- Control Plane and API Design for GPU Platforms9
- The AI Infra Design Round Playbook8
- Model Metadata and Artifact Storage7
- Serverless GPU Platforms6
- Multi-Tenant Fine-Tuning Service6
- Multi-Region Serving and Failover6
- Offline Batch Inference vs Online Serving6
- Model Routing and Cascades5
Coding for Infra12
- The Practical Coding Screen Playbook11
- Producer-Consumer Pipelines10
- Rate-Limiting Algorithms9
- Batching Queues and Backpressure9
- The GPU Credit Scheduler Pattern8
- Concurrency in Python, Go and C++8
- Asynchronous GPU Buffer Ownership8
- Retry, Backoff and Idempotency7
- Interval Merging and Utilization Logs7
- Parsing Kernel Traces and Logs7
- Cache-Friendly Data Structures7
- Consistent Hashing and Sharding6
Ownership & Judgment11
- Leveling Signals: Senior vs Staff11
- The Reliability Pushback Story10
- On-Call Narratives That Land10
- Working with Researchers10
- Deciding Under Incomplete Information8
- Escalation That Works8
- Migrations and Deprecations7
- Safety and Mission Rounds at the Labs6
- Your View on Where AI Infrastructure Is Going6
- Mentoring and Growing Engineers6
- Talking About Cost and Capacity with Leadership6
The map is for orientation. If you would rather be told what to do in order, the start-here path sequences the same material by background and stage, and the courses walk it front to back.