← GPU & Accelerator Architecture / 14

hard★ EssentialGoogleGoogle DeepMind

You have trained on GPUs. What transfers to training on TPUs, and what do you have to relearn?

Carry compute and memory reasoning to TPU v6e, correct all-gather ownership arithmetic, and learn the compilation, sharding and custom-kernel workflow.

Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Carry compute and memory reasoning to TPU v6e, correct all-gather ownership arithmetic, and learn the compilation, sharding and custom-kernel workflow.

This answer is included with a free account. Sign in to read it and save your progress.

free account · no card needed

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🧩 GPU & Accelerator Architecture🔒 Premium

TPU Architecture and Systolic ArraysTPUs use matrix units and compiler-managed execution to run machine-learning workloads. Memory, shapes and inter-chip communication still determine how efficiently a model runs.

🧩 GPU & Accelerator Architecture🔒 Premium

The TPU Software Stack: JAX, XLA, PyTorch and vLLM on TPUOn a GPU you run eager PyTorch and call tuned kernels. On a TPU a compiler sees the whole program, so shapes, sharding and compile time become your main tools. This maps JAX, MaxText, PyTorch/XLA, TorchTPU, Pallas and vLLM TPU onto the CUDA stack for training and serving.

🧩 GPU & Accelerator Architecture🔒 Premium

TPU Generations: v4 to TPU 8TPU generations differ along three axes that decide a deployment: memory per chip, the ratio of compute to memory bandwidth, and how chips connect. Small-memory 2D-torus parts, large 3D-torus pods and the training/inference split in TPU 8 each fit different workloads.

🔌 Networking & Storage🔒 Premium

Data Loading Pipelines for TrainingTrace batches from storage to GPU readiness, budget workers and prefetch memory, and resume from the input position saved with training state. An executable example shows how a read-ahead cursor can silently skip samples.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackExplain how a systolic array computes a matrix multiply. Why is it efficient, and why does it want big square tiles?Next in this trackWould you move a cost-sensitive training and serving fleet from H100 to Trainium2? What do you gain, and what do you have to plan for?Next in this trackThe MI300X has 192 GB per card. When does AMD actually win against an H100 for LLM serving, and what would you check before betting on it?

FEDITOR'S NOTE

Name the accelerator generation and the execution layout. A familiar sharding formula does not prove the compiler uses that layout.

DISCUSSION · 0

No comments yet — be the first to share your approach.