Advanced

The TPU Software Stack: JAX, XLA, PyTorch and vLLM on TPU

On a GPU you run eager PyTorch and call tuned kernels. On a TPU a compiler sees the whole program, so shapes, sharding and compile time become your main tools. This maps JAX, MaxText, PyTorch/XLA, TorchTPU, Pallas and vLLM TPU onto the CUDA stack for training and serving.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

TPU Architecture and Systolic Arrays TPU Generations: v4 to TPU 8 torch.compile and CUDA Graphs Triton Programming Model Collective Communication Primitives Serving Engines: vLLM, SGLang and TensorRT-LLM

LESSONS THAT TEACH THIS

Kernels and the MachineOn a compiled accelerator, the program is the unit, not the kernel22 min→Kernels and the MachinePort a training step and prove it trains the same model26 min→Kernels and the MachineServe on a compiled accelerator: choose buckets, budget warmup, size per device26 min→

PRACTICE THIS IN REAL QUESTIONS

Distributed Training & ParallelismA PyTorch team wants to train on TPUs. Do they rewrite in JAX with MaxText, use PyTorch/XLA, or wait for TorchTPU?→GPU & Accelerator ArchitectureYou have trained on GPUs. What transfers to training on TPUs, and what do you have to relearn?→Open-Weights Models & Serving EnginesYou serve a model with vLLM on GPUs. What changes when you move the same model to TPUs?→Distributed Training & ParallelismHow is training on TPUs with JAX different from training on GPUs with PyTorch? What do you stop doing by hand?→CUDA, Triton & Kernel EngineeringA CUDA upgrade improves warm runs but slows the first request. Would you ship it?→Open-Weights Models & Serving EnginesExplain what ONNX Runtime and TensorRT optimize, then diagnose a slower compiled model→