← Open-Weights Models & Serving Engines / 48

hardNewGoogleGoogle DeepMindAnthropic

You serve a model with vLLM on GPUs. What changes when you move the same model to TPUs?

Same model, same engine name, different execution model. Trace what moves into compilation, how shape buckets trade compile time for padding, and how per-device memory decides the slice.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Same model, same engine name, different execution model. Trace what moves into compilation, how shape buckets trade compile time for padding, and how per-device memory decides the slice.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🧩 GPU & Accelerator Architecture🔒 Premium

The TPU Software Stack: JAX, XLA, PyTorch and vLLM on TPUOn a GPU you run eager PyTorch and call tuned kernels. On a TPU a compiler sees the whole program, so shapes, sharding and compile time become your main tools. This maps JAX, MaxText, PyTorch/XLA, TorchTPU, Pallas and vLLM TPU onto the CUDA stack for training and serving.

🧩 GPU & Accelerator Architecture🔒 Premium

TPU Architecture and Systolic ArraysTPUs use matrix units and compiler-managed execution to run machine-learning workloads. Memory, shapes and inter-chip communication still determine how efficiently a model runs.

⚡ Kernels & Compilers🔒 Premium

CUDA Compilation: PTX, Cubins and SASSCUDA source, PTX and native GPU instructions describe different stages of execution. Inspect the selected binary and its resource use before explaining a performance change.

🧮 Open Weights & Serving Engines

vLLM Server Arguments That MatterSeparate executor memory, allocated cache pools and logical occupancy. Map context, sequence and token-batch limits to the constraints they control, then tune a pinned deployment against per-rank memory and latency.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Closest to this topicYou have trained on GPUs. What transfers to training on TPUs, and what do you have to relearn?Closest to this topicHow is training on TPUs with JAX different from training on GPUs with PyTorch? What do you stop doing by hand?Closest to this topicA PyTorch team wants to train on TPUs. Do they rewrite in JAX with MaxText, use PyTorch/XLA, or wait for TorchTPU?

DISCUSSION · 0

No comments yet — be the first to share your approach.