Glossary
GPU storage
GPU storage describes storage and its network sized so that data reaches GPUs as fast as they consume it. Its measure is GPU utilization during real work. The slowest hop between drives and GPU memory sets that figure, and the slowest hop is often not the drives.
Utilization is the measure
A GPU waiting for data is still billed, powered and depreciating. Utilization recorded during a full training or inference run, set beside the read rate storage delivers, shows whether the data path is the limit. If utilization falls whenever reads stall, adding GPUs adds cost without adding work. Two limits are checked separately: storage throughput for the steady streams of training, and storage latency for the small requests of inference.
Where does the data path break?
Data moves from drives, across the network, into host memory and then into GPU memory. Any hop can starve the accelerator. Common culprits are a saturated network link, a protocol that adds overhead to every small request, and extra copies through system memory that keep CPUs busy moving bytes. Widening a link or removing a copy can recover more utilization than replacing media.
Checkpoints add a second trap. Average write throughput can look healthy while the moment every GPU saves state at once exceeds what the system absorbs, so training pauses on every save. Over a run lasting weeks, those pauses add up to days.
Flash for active data only
Flash earns its cost for hot training sets, checkpoints and inference lookups. NVMe drives are the usual choice where latency matters most, and NVMe over Fabrics extends that access across the network. Source data and archives stay on high-capacity disk, which handles rare reads and long sequential scans well.
GPU storage versus AI storage
AI storage covers the whole role of storage in AI work, including capacity, tiers and retention. GPU storage is the narrower question of delivery: whether data arrives fast enough. Both sit inside AI data infrastructure, and a system with ample capacity can still fail this test.
Frequently asked questions
Is GPU storage a separate type of hardware?
No. It names a requirement placed on a storage system and its network, not a distinct product category.