Hammerspace Introduces an All-New Storage Architecture | Read the Press Release

The Need for a New Storage Architecture: Hyperscale NAS

Part 1 of 2: Introducing Hyperscale NAS

By Eric Bassier, Senior Director Solution Marketing

Today Hammerspace unveiled its Hyperscale NAS architecture – a fundamentally different NAS architecture to power GPU computing at scale and move AI innovation forward. LLMs and GenAI models are trained on large unstructured datasets, where file storage is the preferred format, and HPC-class performance is required to keep GPUs fully utilized at scale. 

The Hyperscale NAS architecture is the first to bring the benefits of HPC parallel file systems to enterprise AI and other GPU- and CPU-intensive workflows, using standards-based NFS and enterprise NAS features.

In the first of this two-part blog series, I’ll explain the architecture and what makes it so unique. 

Next week, I’ll dig into its advantages over a scale-out NAS architecture.

Let’s go!

Hyperscale NAS is a New NAS Architecture, Built for the Next Data Cycle Powered by AI and GPU Computing 

Hyperscale NAS is a new category of NAS architecture. It is the first that combines the extreme performance and linear scale of HPC parallel file systems with standards-based connectivity, simplicity and enterprise NAS features. It is ideally suited for powering GPU computing at scale – for use cases such as generative AI training – and is also well suited for any use case that requires parallel processing and high throughput, low-latency data access.

Whereas previous NAS architectures were about scaling capacity, Hyperscale NAS is really about scaling performance and capacity – to feed GPU clusters, CPU clusters, and other use cases that require high speed parallel processing of data at scale. Where traditional scale-out NAS architectures start to hit a performance plateau as data volumes scale, Hyperscale NAS performance scales linearly to thousands of storage nodes. 

For the First Time Ever, Hyperscale NAS Brings Together the Performance of HPC Parallel File Systems with Standards-Based NAS Simplicity

The Hyperscale NAS architecture is the first to use Parallel NFS with the Flexible Files layout type – or “pNFS with FlexFiles” – which are part of the NFSv4.2 spec and part of every modern Linux kernel. These optional capabilities within the NFS client define a standards-based parallel file system architecture using NFS. In the diagram below, any Linux client connecting to the Hammerspace Global File System is effectively a parallel file system client, providing direct data path access from compute to storage without overhead from the file system.

The difference is that all of that magic is built into Linux – everything is standards-based

Metadata and the data path are separated to provide for maximum performance and data placement flexibility, and any client can directly read and write data to multiple storage volumes for high throughput and low-latency performance, all via the Hammerspace Global File System.

Performance and capacity can be linearly scaled simply by adding more clients and storage to the cluster. If you would like to learn more about the Hyperscale NAS architecture, this Technical Brief is a great resource.

Stay tuned for part 2 of this blog series where I’ll talk about the advantages of this architecture over scale-out NAS.

About the Author

Eric is the Senior Director, Solution Marketing and Sales Enablement, at Hammerspace. He is an innovative product leader with extensive experience launching and evangelizing products, driving go-to-market strategies, and creating compelling content to drive customer engagement and growth.