accelerators
AI infra interview questions tagged accelerators, across every topic.
2 questions · 0 unlocked for you
Concepts behind "accelerators"
The curriculum that explains the ideas these questions test.
Advanced
Trainium and InferentiaAWS's accelerators trade the GPU's general-purpose flexibility for a compiler-driven design with separate tensor, vector, scalar and GPSIMD engines, software-managed on-chip SRAM, and a proprietary NeuronLink fabric. Trainium2 delivers 667 dense bf16 TFLOPS with 96 GB at 2.9 TB/s, Trainium3 about the same bf16 with 2.5 PFLOPS of fp8 and 4.9 TB/s. The pitch is cost per FLOP; the price is a kernel ecosystem you may have to build yourself, which is exactly what the AWS loop probes.🧩 GPU & Accelerator Architecture🔒 Premium
Advanced
AMD Instinct and ROCmAMD's Instinct line competes on memory: 192 GB on the MI300X, 256 on the MI325X, 288 on the MI355X, with dense fp8 peaks at or above NVIDIA's at each generation. The catch is software: ROCm and HIP run most PyTorch and the main serving engines, but the attention, MoE and quantization kernels arrive on CUDA first and measured throughput has trailed the spec sheet. Interviewers want the numbers, the wavefront-of-64 difference, and a clear-eyed statement of when the memory argument wins.🧩 GPU & Accelerator Architecture🔒 Premium
Advanced
Cerebras, Groq and Dataflow AcceleratorsCerebras and Groq attack the same wall from the same direction: keep the working set in on-chip SRAM, which is tens of times faster than HBM, so that memory-bound decode stops being memory-bound. Cerebras does it with a wafer-scale engine holding 44 GB of SRAM; Groq with a deterministic, compiler-scheduled dataflow chip holding a few hundred megabytes per part and many parts per model. Both win single-stream decode by large factors and pay in capacity, cost per token at high batch, and flexibility. Knowing where each fits is the interview question.🧩 GPU & Accelerator Architecture🔒 Premium