prefetch
AI infra interview questions tagged prefetch, across every topic.
2 questions · 0 unlocked for you
Concepts behind "prefetch"
The curriculum that explains the ideas these questions test.
Advanced
Data Loading Pipelines for TrainingThe dataloader is the only part of a training job that runs on the CPU, the disk and the network at once, and it is the part most often found starving the GPUs. A pipeline that keeps 1,024 accelerators fed has to read sharded files sequentially, decode and tokenize in parallel workers, prefetch several batches ahead, pin memory for the PCIe copy, and do it deterministically enough to resume mid-epoch. The symptom of failure is a GPU at 30% utilization with nothing wrong on the GPU.🔌 Networking & Storage🔒 Premium
Advanced
Producer-Consumer PipelinesA data loader, a log shipper, a batch inference job and a checkpoint writer are the same program: stages connected by bounded buffers, each running at its own pace, the slowest setting the throughput and the buffers absorbing the jitter between them. The coding screen asks you to build one (read, decode, batch, feed a consumer) and then pushes on the production questions: buffer sizes, clean stops, failure propagation, and why it runs at a third of the expected speed. This page derives throughput from stage times, implements the pipeline in threads and asyncio, and works the stop and failure semantics.💻 Coding for Infra🔒 Premium