File systems
The Denvr Cloud is pre-integrated with network-attached file storage systems to preserve data sets, model checkpoints, and system output.
High-Performance Storage
Features:
Petabyte-scale network-attached NVMe SSD with replication for data protection
Up to 10 GB/s read throughput
Encrypted at rest and delivered to GPUs over private tenant networking
Primary use:
Working space for very large datasets that are copied in and out for local Cache
Application file systems, databases, and large capacity/high IOPS requirements
Faster version of User Storage but is typically used for pre-processing and streaming large datasets in and out of Cache
General Purpose Storage
Features:
Petabyte-scale RAID-protected storage, optimized for low cost, parallel I/O, and persistent storage
Encrypted at rest and delivered to GPUs over tenant-private networking
Up to 1 GB/s of read throughput
Primary use:
Used for home directory files, application code and configuration, and datasets
Clients may optionally prefer to keep data in their own on-prem storage systems and leverage Denvr for caching
GPUs can process data directly from User Storage, but may benefit from Cache storage depending on your use
Local NVMe Storage
Features:
Up to 30 TB of direct-attached NVMe SSD per instance depending on configuration
Lowest latency and up to 30 GB/s read throughput
Cache is non-persistent and is freed when applications are stopped
Primary uses:
Volumes provide local storage for model training data, checkpoint files, and intermediate results data
Data loaders should copy training data into the Cache Storage to ensure full GPU utilization and not bottleneck performance on slower network-attached storage
Last updated