Preface
This is first in a series of blog posts covering Hammerspace Objectives. The first post introduces the various concepts and aspects of Hammerspace Objectives. Also covered is basic structure and syntax. The intent of this initial blog post is to familiarize those who may be new to Hammerspace Objectives and to, hopefully, deepen the knowledge and awareness to readers who may already be working with Objectives. Future blogs in this series will present cookbook recipes designed to address specific use cases and business challenges.
Overview
Hammerspace Service Level Objectives are a new paradigm that empowers organizations and stakeholders to declare the intent of their data. It couples desired outcomes with programmable conditions. That is the essence of declarative control of data. The significance is that data can, then, be organized through metadata characteristics, inherent or custom, and further tailored to empower desired business outcomes. Declarative statements, such as Hammerspace Objectives, are a far more effective and modern method of creating business outcomes from data than the more archaic imperative policy construct.
In the imperative policy world you have to be exceedingly detailed about exactly how to accomplish a desired end-state. This often translates to very lengthy and complex instructions that outline exactly how something should be accomplished. This can translate to a hundred lines or more of instructions. The problem only gets worse when trying to scale, upgrade, extend, or troubleshoot imperative policies. What if your workflow and production requirements need ten, twenty, or more imperative policies. It’s easy to imagine that this quickly becomes untenable.
Declarative policies, on the other hand, remove the complexity and inefficiency of the imperative and skip directly to the desired end-state. Telling a system WHAT you want to have happen is infinitely simpler and far more elegant than the error-prone, tedious, and time-consuming procedural instructions necessary within the old imperative paradigm. Declarative policies accelerate workflows, operations, and business outcomes.
In the following sections, we will demonstrate why Hammerspace Objectives are one of the most powerful aspects of our innovative Global Data Environment.
Objectives Explained
Hammerspace Objectives are highly granular and can, thus, be applied to a wide variety of data sets. Some examples that can be targeted are SMB shares and NFS exports, directories (folders), individual files as well as file types and custom criteria through metadata tags and descriptors. This places the control of your data directly in your hands through simple yet powerful declarative statements.
Service Level Objectives
This is the umbrella term for Hammerspace Objectives. The Hammerspace Admin CLI aka CLISH exposes three Objective commands: basic, share, and smart. These should not be thought of as separate entities but rather as building blocks of a larger construct, namely the Service Level Objectives used to declare a desired end-state of your data. This could be archiving, tiering, data protection or preservation, collaboration across global geographies, disaster recovery, workflow, or any number of use cases from simple to highly complex.
Basic Objectives
This is an elemental building block of Service Level Objectives. Basic Objectives has a simple set of “asks”. In its simplest form it defines an action, such as the availability, durability, placement, or protection mechanism of a user-specified data set. This can be one or more files, folders, shares, MIME types, custom metadata criteria, or any combination thereof. Hammerspace systems already come with a set of predefined Basic Objectives. To view what already exists on the system, let’s login via SSH and take a look:
ssh admin@10.41.1.12
NOTE: The above IP address is only used to illustrate an example. Replace with the IP address or hostname of your Hammerspace metadata controller (Anvil).
To view already existing Objectives execute the following command:
basic-objective-list --full
You should see something like the image below. Note that the full output is nearly 300 lines listing over 50 Objectives so the output is abbreviated to improve readability.
Share Objectives
The share-objective command is used to apply an Objective to a target data set, such as a file, folder, share, or custom criteria. The following examples illustrate how to apply an Objective to a share, folder, or individual file, respectively:
share-objective-add --objective "<name of the objective>" --name <sharename>
share-objective-add --objective "<name of the objective>" --path /<folder>
share-objective-add --objective "<name of the objective>" --path /<folder> --name filename.txt
There is an additional option, --applicability, that makes a share objective “smart” and thus can slightly muddle Share and Smart Objectives. For that reason, we will set it aside for now so that we can give it full attention in an upcoming blog.
Smart Objectives
A Smart Objective carries conditions and references to other Objectives. At the risk of being overly redundant, it does not create actions of its own but, instead, leverages actions from previously created Basic Objectives. Smart Objectives are merely containers or macro of other Objectives.
To summarize the various aspects of Objectives, a Basic Objective has a simple set of “asks” while a Smart Objective has a filter expression that predicates a condition and the Objective(s) that should be active under the right condition.
Defining Business Outcomes with Objectives
Let’s take a quick look at how to solve a real-world business challenge with Hammerspace Objectives. We will use a common and frequently occuring problem, namely older, stale files that consume the most expensive storage. If a file hasn’t been touched in, say, more than 90 days it is highly likely that it will not be worked on again. Deleting the file is not an option as it may be an asset with inherent value or contain content that can be reused, repurposed, or extrapolated from to create new data assets. Thus, it makes sense to move older data to lower cost storage. In other words, archival. Keep in mind that 90 days is just an example. It can be any number you find suitable to your particular use case.
Using Objectives follows a very simple and straightforward workflow.
Step 1: Define a desired outcome
Step 2: Create an Objective
Step 3: Apply the Objective
NOTE: The target data set can be one or more files, folders, shares, MIME types, or/and rich custom metadata criteria.
A Simple Archival Example
Our desired end-state in this example is to move files that haven’t been actively worked on in more than 90 days to lower cost storage. This is a very rudimentary example. How data is tiered, moved, placed, protected, or archived can be even simpler than this example or supporting the most complex scenarios and requirements imaginable. You can easily create Objectives that don’t evacuate data based on time but rather on pressure. Creating an Objective that states, for example, don’t tier data to a lower cost tier until there is capacity pressure, is very straightforward. Unlike archaic and complex imperative or procedural policies, Hammerspace Objectives provides choices and simplicity that can only be provided through a declarative statement.
Creating an Objective
The syntax and structure for creating a Hammerspace Objective is simple and straightforward. The sections below illustrate the basic syntax and provide examples from the Admin CLI (CLISH). We will also show what it looks like in the Admin GUI.
Syntax
The basic syntax of a our example Service Level Objective would look like the following:
IF LAST_USE_AGE<90*DAYS
THEN {SLO('place-on-NAS')}
ELSE {SLO('place-on-Amazon Web Services')}
By deconstructing this Objective is easily identified how a condition, such as IF LAST_USE_AGE<90*DAYS can be coupled with one or more actions, such as {('place-on-NAS')} and {SLO('place-on-Amazon Web Services')}.
Admin CLI (CLISH)
The Hammerspace Admin CLI or CLISH provides a very straightforward syntax for delivering this Service Level Objective, as shown below. The –name operator is anything we want to call it that makes sense to us. Choose a good and easily understood description that will make sense to anyone else that may work on the system in the future. The –expression operator contains the desired actions based on a condition, i.e. the Service Level Objective. The CLISH command below creates the Objective.
smart-objective-create --name "Older than 90 days" --expression "IF LAST_USE_AGE<90*DAYS THEN {SLO('place-on-NAS')} ELSE {SLO('place-on-Amazon Web Services')}"
But wait, we are not done yet! We still need to apply the Objective which we will do next.
Admin GUI
From the Hammerspace Admin GUI we can view our newly created Objective. We can also choose to further edit, change, or refine the Objective through the GUI as Hammerspace provides feature parity between GUI and CLISH.
Applying the Objective
The final step in the process of solving a business challenge is to apply the Objective we created in the previous section. The three examples below show how to apply the Objective to a single file, a directory, and an entire share. Upcoming blogs will dive progressively deeper to illustrate the immense power of Hammerspace Objectives.
Admin CLI (CLISH)
The following example commands show how to apply an Objective to a single file, to a directory, to a share and, then, how to list them.
Applying an Objective to a file
share-objective-add --objective "Older than 90 days" --name Engineering --path test.txt
Applying an Objective to a directory
share-objective-add --objective "Older than 90 days" --name Marketing --path /Projects
Applying an Objective to a share
share-objective-add --objective "Older than 90 days" --name Home
Listing Applied Objectives
share-objective-list --name Engineering --path test.txt
share-objective-list --name Marketing --path /Projects
share-objective-list --name Home
GUI
The images below illustrate how to apply Objectives using the GUI.
Applying an Objective to a file
Applying an Objective to a directory
Applying an Objective to a share
Observability
A key aspect of managing data is the ability to measure the internal states of a system by examining its outputs. Data placement and movement is easy to track with Hammerspace. It can be observed from multiple perspectives.
File Browser
The two screen captures below show how data movement can be observed immediately after applying an Objective until a state of alignment has been achieved.
Alignment in Progress
- Navigate to “Shares” from the left navigation pane.
- Click on the share you wish to observe.
- Note the information in the Storage Volume and Alignment columns.
Green color denotes that the data is aligned with its Objective. In other words, the Objective we applied to the Share containing the data has been moved to the volume we targeted.
Orange color denotes that the data is not yet aligned with its Objective. While data is in transit you will see the orange traffic light.
Red color denotes failed data placement or movement and warrants further investigation. This may happen, for example, if the target is not available or not writeable.
Alignment Completed
Within a brief time, we can observe how data has been successfully placed on our target volume in alignment with our Objective.
Mobility
We can also observe from the data mobility graph and matrix.
Graphical Presentation
- Navigate to the Mobility icon from the left navigation pane.
- The mobility chord is a graphical representation of live data movement from one storage volume to another. NOTE: This is an example and could just as easily represent multiple data movements, one to one, one to many, many to one, or many to many.
View as List
This view provides highly granular detail on data movement and alignment. Among other things, the list shows:
- File Name and Share
- Source Volume
- Destination Volume
- Status
- Throughput
- Start Time
- End Time
Benefits Summary
Data has been managed through storage systems for a generation. Progress over the past few decades has been evolutionary but not revolutionary. The pain of silos, sprawl, disruptive hardware forklifts and data migration is significant. It causes business disruption, introduces lost productivity, and adds unpredictable and runaway costs. This cannot be sustained in an increasingly data-driven world where everything from medical systems to stock exchanges to defense systems must be able to rely on the availability, durability, and integrity of data.
Hammerspace is a revolution in terms of how data is managed. Service Level Objectives make data programmable. Business outcomes can, thus, be directly controlled through a single paradigm instead of multiple incompatible storage silos that require separate care and feeding. When data is managed directly through its constituent metadata instead of through the limitations of aging storage infrastructure then data-driven workloads and business outcomes are front and center.
