Advanced

Inside a Transformer Block

Trace one token through attention, a residual connection, normalization and a feed-forward network. See which operations mix token positions, which transform features, and why model layers and GPU execution units are different things.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Tokens, Embeddings and Logits Tensors, Shapes and Dtypes Attention Variants: MHA, GQA, MQA and MLA The KV Cache Tensor Cores and Matrix Units Expert Parallelism for MoE

LESSONS THAT TEACH THIS

Foundations of AI InfrastructureA request becomes tokens, and tokens become two different workloads30 min→Inference EngineeringAttention architecture determines which state the serving engine must retain29 min→

PRACTICE THIS IN REAL QUESTIONS

LLM Inference & ServingWhat is the KV cache, and why does it keep growing while a request is being served?→Hardware, Cabling & Cluster Build-OutA GPU cluster has four separate networks. Name them and say what each one carries.→LLM Inference & ServingYou need to serve 128k-token contexts. What breaks first, and what do you change?→Coding for InfraImplement sliding-window attention without confusing token positions with ring-buffer slots→LLM Inference & ServingCompare the KV cache footprint of multi-head, grouped-query and multi-head latent attention with numbers.→Open-Weights Models & Serving EnginesYour deployment is limited by KV cache rather than by compute. What are the options?→