TL;DR: GPUDirect Storage can avoid a host bounce for supported storage-to-GPU transfers. It helps when that path materially limits the workload. CPU transforms, storage bandwidth, caching, metadata and checkpoint layout may still dominate; neither a direct-path label nor a fixed two-copy model proves a speedup.
How to approach it
Draw the consumer of each byte and the transforms between storage and compute. Separate first-batch latency from steady-state throughput. Verify path support and actual selection, then compare complete workload time under matched cache and concurrency conditions.
A strong answer
A read finishing sooner does not help training if the next CPU transform takes longer than either read path. Conversely, moving a transform to the GPU can make direct input useful even when an older CPU-based pipeline did not benefit. GPUDirect RDMA and GPUDirect Storage explains the two direct-access families.
NVIDIA's GDS overview describes the direct DMA path (direct memory access: the storage or network hardware copies data into GPU memory itself, without a CPU-staged copy) and the cuFile interface. Control-plane work still runs on the CPU. Supported memory, filesystem, driver and topology combinations determine whether a particular request uses direct access or a compatible path; ordinary success is not proof of direct transfer.
Use an explicit synthetic example with a 100 GB payload. Assume storage delivers 10 GB/s, a staged host-to-device copy delivers 25 GB/s and the direct path also delivers 10 GB/s:
serialized staged latency = 100/10 + 100/25 = 10 + 4 = 14 s
ideal direct payload latency = 100/10 = 10 s
large-stream staged throughput with perfect overlap:
min(10, 25) = 10 GB/s, before startup/drain and contention
sanity: removing a stage can improve latency without raising steady-state throughput.
Now assume the staged path's measured sustainable rate is only 6 GB/s because of a shared resource. Moving to a verified 10 GB/s direct path could improve the payload phase from 16.7 to 10 seconds. That is a hypothetical workload result to test, not a general 1.67× GDS guarantee. PCIe switches, roots, links and DMA engines determine the actual resource sharing.
If an unchanged CPU stage must decompress or transform the payload before the GPU uses it, its host access remains necessary for that stage. This does not mean any CPU involvement erases every possible GDS benefit elsewhere. A pipeline can split stages or use GPU decoding; evaluate each supported arrangement, including the compute capacity consumed by the transform.
Checkpoint restore is also more than a bulk read. The file representation may require deserialization, dtype conversion, tensor reordering or resharding, and optimizer state may belong on the host. Inventory destination tensors and format support before asserting that all checkpoint bytes can land directly in their final GPU locations. Checkpoint I/O covers completion as a recovery concern rather than a raw transfer metric.
Keep cache state explicit. Comparing a cold direct read with a warm page-cache read tests both caching and the I/O path. Repeated small accesses may favor a different design than large streaming reads; request concurrency, alignment and overhead need measurement. Avoid universal claims that small transfers never benefit.
Measure storage read completion, transform duration, GPU input readiness, CPU time and whole-step or restore time. Check the library's path and fallback diagnostics for the installed release. A lower host-memory traffic count supports the direct-path hypothesis but does not prove every request used it.
Start with GDS where the payload is GPU-ready or transformed on the GPU, the stack supports the path and transfer limits useful progress. Reverse the choice when the storage service itself is limiting, CPU transformation remains dominant or measured end-to-end gain is too small to justify complexity. Keep the simpler path as a tested fallback when the application supports it.
What interviewers probe next
- "Does avoiding staging mean one physical bus hop?" Not necessarily. Topology and intermediate device paths vary.
- "Can writes benefit?" Potentially, but checkpoint consistency and durability remain separate requirements.
- "What proves success?" Correct output and improved complete-workload performance under matched conditions.
Common mistakes
- Adding serial transfer times to model an already overlapped pipeline.
- Assuming every checkpoint format is directly GPU-loadable.
- Concluding that API success proves direct-path engagement.
Key takeaways
- The synthetic serialized path takes 14 seconds versus 10 direct, yet both can stream at 10 GB/s with ideal overlap.
- Map transforms and destinations before choosing the I/O path.
- Accept the change on useful pipeline performance, not copy count alone.