Organizations built hybrid cloud strategies largely to enable agility. However, until recently hybrid cloud strategies have been dependent upon the infrastructure constraints of each siloed storage component, making it difficult and expensive to make data available when and where it is needed by applications and users who are remote from where the data was originally created.
Hammerspace provides the “missing link” to make hybrid cloud strategies for unstructured data all that organizations originally hoped for.
- Infrastructure Independant – The original vision of hybrid cloud infrastructure was that the cloud could supplement existing data center environments.
Historic Storage Challenges:
- Data use challenges: In traditional storage architectures, the metadata that users and applications interact with is typically contained within a local file system. This information is specific to each storage cluster or silo, and does not include information about the rest of the organization’s data. Access to data must be set up for each individual data silo and requires knowing quite a few specifics about specifically which data you are looking for.
- Data movement challenges: Devices can replicate or copy data to their own resources within a cluster or to a remote environment with the same proprietary software. But they cannot, for example, manage data or enable file access outside of that vendor silo without the use of gateways, symbolic links, or other point solutions. Moving data to different hybrid cloud infrastructures requires making copies of data. This is expensive as it creates multiple copies of data to store and creates complexity in losing sight of a “gold” or “master” version of data. Copy proliferation and versions become challenging for both using and managing the data.
Hammerspace Solution:
Overcome limitations of data silos by aggregating metadata from all edge, data center and cloud storage resources into a global namespace, giving users, applications, and IT data services the ability to see and manage all resources globally, regardless of which underlying storage type the data lives on today, or needs to move to tomorrow.
- Elastic Scalability – The original vision of hybrid cloud was to provide elastic scalability so that an organization could quickly add capacity to its existing environment to provide on-demand scale. Ideally, when that extra capacity was no longer needed, it could quickly be turned off.
Historic Storage Challenges:
Using historic storage systems, hybrid cloud did not work out as hoped. Storage systems were designed to scale within a single cluster or cloud instance but not across environments. When an organization needed to scale outside the original storage system, for either performance or for capacity, it required creating a new storage silo and isolated data set. This required IT complexity to set up and manage multiple systems, and created user/application complexity when trying to access data. Needing to connect applications and users to a new storage environment and move data sets between environments was neither simple nor fast. It was not elastic, but, instead a permanent addition of a new storage silo.
Hammerspace Solution:
IT administrators only need to set up applications and users to connect to the Hammerspace Global Data Environment once. Once they have access to the Hammerspace namespace, they can use and access all data – independent of the infrastructure it resides in. If IT needs to add additional storage capacity, retire an existing storage cluster, or provide additional performance, this can be done without affecting users or applications. When the new capacity or performance is added, it is automatically available to the entire global data environment without the need to reconfigure user or application access to their data and data shares. When the added capacity and performance are no longer needed, it can be removed, again, without affecting user or application configurations.
- Global Access to Local Data – The original vision for hybrid cloud was that it would help organizations gain access to new applications and services in the cloud that they did not have in their data centers.
Historic Storage Challenges:
Organizations running hybrid cloud environments found that while they wanted to use applications in the cloud, IT staff often took too much time to move files from their data centers to the cloud for the applications to access. The concept of data gravity started to get some industry momentum as a “truth” because legacy storage was designed to replicate entire data sets, and legacy data management needed to move or copy entire data sets. There was often too much data to move across available network bandwidth, and it was also too expensive to store duplicate/triplicate/multiple data copies. Organizations found they really needed to move applications, services, and compute resources to their data instead of moving the data to where it was really needed. This limited organizations’ ability to use the modern cloud applications or the compute resources they desired, and made it difficult for the remote workforce to be effective.
Hammerspace Solution:
Hammerspace creates a metadata control plane with its high-performance Parallel Global File System. This metadata layer unifies the namespace across on-premises and cloud storage for applications and users to interact with. Users everywhere are accessing the same file system metadata, not forked copies of files stored in disconnected storage silos. This provides the ability for all applications and users, regardless of location, to view and read/write data in remote file shares without moving the entire data set or making multiple copies of the actual data. Global data across all storage silos and locations is accessed over standard protocols including SMB and NFS in the same way as if they were mounted on a local NAS. Hammerspace may even be provisioned in containers via a Cloud Storage Interface (CSI) for Kubernetes to make access from anywhere simple, and without leveraging proprietary clients.
The Parallel Global File System ensures that when files do need to be localized for processing, instantiations may be moved at a file granular level without interrupting user access, and without the need to make copies. All users at all sites are accessing the same files via the ubiquitous metadata layer presented by the Parallel Global File System.
- Automate Cross-Platform Storage and Data Services Management – Few people realized the complexity a hybrid cloud storage environment would add to IT teams’ workload. Any new vendor in the storage stack required new management interfaces to be learned and new data management and protection policies to be created. Few IT teams had team members experienced in cloud services and each cloud has vastly different and numerous offerings that had to be learned.
Historic Storage Challenges:
Policies to ensure compliance for data protection, data privacy, ransomware protection, and disaster recovery were all part of each storage or data management vendor’s own software. Each time a new storage silo was added in the data center or in the cloud, IT needed to be sure that the appropriate policies were configured and tested. As cloud was added, users often had their own storage environments in the cloud, and this caused IT to have to identify these shadow-IT environments created by users, to ensure data policy compliance, and to ensure these data sets were all secured and backed up.
Hammerspace Solution:
Hammerspace overcomes the challenge of managing data across the hybrid cloud by enabling IT to set policies globally for all data, no matter which type of storage the data resides in. With Hammerspace, IT can set global service level objectives for data placement and data services that can be aligned with business requirements. This could include keeping a copy of data in at least two geographically isolated storage environments, or always moving data created in an edge location to a specified cloud region, while also taking and retaining snapshots of all data in two locations. All of this can be done with Hammerspace software no matter which storage or cloud vendors are housing the data.
- Burst to Cloud – The idea of using the cloud for burst computing was one of the most promising goals of hybrid cloud environments. Organizations would have virtually unlimited access to CPUs or GPUs without needing to own them.
Historic Storage Challenges:
Most initial hybrid cloud environments struggled to get their data to the cloud compute environments. Users needed to open tickets with IT to migrate whole volumes of data from on-premises storage clusters to cloud storage. This added complexity to users who often had a difficult time working outside their traditional workflow management tools to identify which data to move at the file system level.
Hammerspace Solution:
Because the Hammerspace Parallel Global File System spans both on-premises and cloud, the user experience is the same, whether the resources are in a local data center or in the cloud. Job runs taking advantage of the rapid provisioning of often massive compute clusters in the cloud can have direct access to only the data that is needed, without IT having to migrate whole datasets to the cloud manually. Integration with existing workflow automation tools familiar to end users makes it simple to kick off a job without IT intervention, or needing specialized tools provided by storage and cloud vendors.
Hammerspace provides the “missing link” so you can overcome limitations of existing storage solutions to accomplish your hybrid cloud vision.
Want to learn more? Join our LinkedIn Live webinar, “Why Consider Hammerspace for Your Hybrid Cloud Storage” on Wednesday, August 10, or view the recording on-demand, after the event!