AI feeds off data – massive amounts of unstructured data that is stored in multiple locations and must be efficiently and quickly moved to GPUs as needed to enable today’s business critical AI workloads.
Enterprises and Cloud Service Providers alike struggle with data that is spread across on-premises data centers, edge locations, and diverse public cloud environments. This data fragmentation creates significant hurdles for access, collaboration, governance, and ultimately, innovation. Hammerspace and Scality have partnered to enable our joint customers to unify, protect, and accelerate access to massive unstructured datasets across any environment, unlocking value by simplifying management, lowering costs, and supporting next-generation AI pipelines.
Hammerspace is a Data Platform for AI Anywhere that unifies unstructured data from any storage, location, or cloud into a single global file system. Hammerspace enables seamless, automated access and movement of data across hybrid and multi-cloud environments for high-performance computing, AI, and enterprise workflows.
Hammerspace and Scality have validated Scality’s object storage solutions with the Hammerspace Data Platform enabling data stored in Scality to be dynamically tiered, replicated, or moved by Hammerspace as needed to optimize performance and availability for workloads such as AI, HPC, and analytics.
How The Solution Works
Hammerspace orchestrates data across disparate storage systems by creating a single global namespace that unifies file and object data, including data stored on Scality’s RING and ARTESCA object storage products. Hammerspace uses standards-based protocols (NFS and SMB for files; S3 for objects) allowing applications and users to access the same dataset via any protocol without duplication or data copy sprawl.
Hammerspace can read/write to Scality’s object storage without proprietary gateways or proxies. This seamless layer dynamically manages metadata-driven policies to move, tier, cache, or replicate data between high-speed file tiers and cost-efficient object storage pools.
Key technical features include:
- Global Data Orchestration: Automates transparent movement of unstructured data between Scality object storage and other storage tiers based on workload demands.
- Multi-protocol Access: Enables simultaneous access to data via file protocols and native S3, simplifying application integration across AI, analytics, and HPC environments.
- Standards-Based Parallel File System Architecture: Supports linear scaling in throughput and capacity as underlying infrastructure scales, without the complexity of proprietary clients or agents – ideal for large datasets typical in AI model training and inference.
- Non-disruptive Integration: Works with existing storage hardware and network infrastructure without requiring application or workflow changes.
