Why does object storage behave badly for random reads, and what does a training pipeline do about it?

Calculate how many reads a training loader needs in flight, then see when larger requests waste bandwidth. Worked examples separate useful samples, fetched bytes, prefetch and the limits of Little's Law.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🔌 Networking & Storage🔒 Premium

Data Loading Pipelines for TrainingTrace batches from storage to GPU readiness, budget workers and prefetch memory, and resume from the input position saved with training state. An executable example shows how a read-ahead cursor can silently skip samples.

Core

🔌 Networking & StorageSign in

Parallel Filesystems vs Object StorageChoose storage by access semantics, working-set reuse and failure protection. Trace a shared read/checkpoint budget, distinguish local completion from durable export, and learn when a second storage tier earns its cost.

Foundational

🧮 Napkin Math & Capacity

KV Cache SizingDerive cache payload from the stored attention state, then account for mixed sequence lengths, block allocation and per-device limits. Worked examples distinguish exact bytes from deployable capacity.

Advanced

💻 Coding for Infra🔒 Premium

Producer-Consumer PipelinesProducer-consumer pipelines connect stages that run at different rates. Bounded buffers absorb temporary variation, while the slowest sustained stage limits throughput.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackDeduplicate and quality-filter a multi-petabyte web corpus. What does that pipeline cost and where does it bottleneck?Next in this trackA mixture-of-experts model does an all-to-all twice per layer. What does that demand of the fabric, and what changes on a rack-scale system?Next in this trackCheckpoints and datasets both live on storage. Why does one system sized for both usually serve neither well?

FEDITOR'S NOTE

Count complete requests in flight and useful samples delivered. Explain read amplification and test layout changes without silently changing the sampling plan.

DISCUSSION · 0

No comments yet — be the first to share your approach.