Hammerspace Introduces an All-New Storage Architecture | Read the Press Release

Five Reasons Customers Choose Hammerspace for GPU Computing, GPU-as-a-Service, and GenAI and LLM training

By Eric Bassier, Senior Director, Solution Marketing

The Hammerspace Global Data Platform is ideal for GPU computing, and is used at one of the largest AI training environments in the world, as well as other HPC environments. Our customers running compute-intensive workloads select Hammerspace for five main reasons. 

Reason # 1: Performance that Scales Linearly, and Can Feed Thousands of GPUs Concurrently

Hammerspace is based on a parallel file system architecture which is presented as a NAS for applications and users to connect over standard NFS. The software leverages Parallel NFS and Flex Files to create a direct data path between Linux clients (in this case the GPU servers) and the underlying storage nodes (which can be off-the-shelf or OCP commodity servers). 

Metadata is written out of band.

This extremely efficient data path reduces the number of network transmissions between GPU servers and storage by 2x versus scale-out NAS architectures. As the number of nodes in the storage cluster increase, performance scales linearly without a plateau. 

Reason # 2: Everything is Standards-Based

The Hammerspace Hyperscale NAS architecture is unique because it provides HPC-class performance (like HPC file systems) using standards-based NFS 4.2; a standard client that is part of the Linux kernel. In other words, no proprietary file system clients, no complex NFS client customizations. Existing client servers on any modern build of Linux already can serve as a parallel file system client and achieve both massive and efficient data path performance that is not possible with scale-out NAS systems.

Hammerspace achieves this by being the first and only file system that uses Parallel NFS and Flex Files – optional capabilities of the NFS 4.2 specification that were engineered by Hammerspace and made part of the NFS 4.2 specification in 2018. 

Hammerspace has been and will continue to be a major contributor to the Linux community in the NFS protocol.

To learn more about this architecture, visit our Parallel NFS resource center

Reason # 3: Efficient Architecture That Can Save $Millions

Rather than purchase expensive custom or purpose-built hardware from storage vendors, customers can build Hammerspace storage clusters using commodity off-the-shelf servers – and these can be heterogeneous, meaning not all of the servers need to have identical specifications. 

Typically these clusters would be built with NVMe storage servers to get the lowest latency and highest performance per rack unit, but Hammerspace storage clusters can be built with any combination of NVMe, SSD, and HDD storage. 

But the savings are more pronounced than that.

Hammerspace’s efficient “direct data path” architecture outlined above results in:

  • 2x reduction in the number of servers required*
  • 2x reduction in the number of network ports*
  • 2x reduction in power consumption*
  • 2x reduction in rack space*

(*All comparisons are relative to scale-out NAS architectures)

Reason # 4: Hammerspace Makes It Easy to Assemble and Consolidate Large Datasets for AI Training

Hammerspace offers a unique capability called Data-in-Place Assimilation. This feature allows Hammerspace to import metadata from existing file storage systems, while leaving the data itself in place, so that the files stored on those storage volumes are visible to users and applications in minutes. Data orchestration can move data to new storage systems or alternate compute sites automatically using Hammerspace Objectives (our word for data policies), and this happens transparently and non-disruptively to users. These policies can also be set on a file-granular basis, so move only the files you need.

Reason # 5: GPUs Not Co-Located with Your Training Data? No Problem!

Right now, just procuring GPUs is a challenge. And for many companies in the early stages of their AI journey, they may not be willing to make a big capital investment in a dedicated GPU infrastructure. 

So if your GPU infrastructure is not co-located with your training data – no problem!

Hammerspace data orchestration for compute can move directories, and even individual files, between sites and between clouds. This provides a very fast and very efficient data transfer, and even allows for cloud bursting if you want to access compute resources (including GPU resources) with various cloud providers.

This Global Data Platform eliminates data silos and makes data an instantly accessible resource for users, applications, and compute clusters, no matter where they are located.

Want to learn more? Contact us here.

About the Author

Eric is the Senior Director, Solution Marketing and Sales Enablement, at Hammerspace. He is an innovative product leader with extensive experience launching and evangelizing products, driving go-to-market strategies, and creating compelling content to drive customer engagement and growth.