Advanced

Tensors, Shapes and Dtypes

A tensor is an array of numbers with a shape, a representation and a storage location. Learn to read batch and sequence dimensions, calculate bytes, and distinguish a reshape from a transpose before sizing GPU work.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Tensor Cores and Matrix Units Model Memory Footprint Reading config.json to Size a Model You Have Never Run Arithmetic Intensity by Operation Tokens, Embeddings and Logits

LESSONS THAT TEACH THIS

Foundations of AI InfrastructureA request becomes tokens, and tokens become two different workloads30 min→

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator ArchitectureExplain how a systolic array computes a matrix multiply. Why is it efficient, and why does it want big square tiles?→LLM Inference & ServingWhat is the KV cache, and why does it keep growing while a request is being served?→Napkin Math, Cost & CapacityHow much GPU memory does it take to run Llama 3.1 70B?→Napkin Math, Cost & CapacityHow big is the KV cache for Llama 3.1 70B at a 128k context?→Distributed Training & ParallelismYou need to train a 100B dense model and it does not fit on one node. Walk me through how you would lay it out.→Napkin Math, Cost & CapacityHow much memory does it take to fine-tune a 70B model, full fine-tuning versus LoRA?→