You’re ready to take Tier 1 apps into production on Kubernetes, but storage is holding you back. Here’s how to configure Datera to optimize for production and scale.

You’ve containerized apps in Kubernetes and are ready to move to production but accessing storage dynamically has been a roadblock. This is where your choice of underlying storage fabric can make or break your ability to scale.

Your infrastructure is a wide assortment of server and storage classes, generations, and media—each one with a different personality. With the right storage fabric, neither the app developer nor the storage admin needs to be prescriptive about which resources are the best fit—that should be determined in human language terms (e.g. super performant, highly secure) and based on Storage Classes defined in Kubernetes.

Defining Storage Classes

Optimizing storage in Kubernetes is achieved by managing a class of storage against application intent. Just like a Terraform script that defines application needs, the storage platform should supply storage needs using templates whether you are looking for IOPs, latency, scale, efficiency (compression and dedupe), security (encryption).

Dynamic storage provisioning in Kubernetes is based on the Storage Classes. A persistent volume uses a given Storage Class specified into its definition file. A claim can request a particular class by specifying the name of a Storage Class in its definition file. Only volumes of the requested class can be bound to the claim requesting that class.

Multiple Storage Classes can be defined to match the diverse requirements on storage resources across multiple applications. This allows the cluster administrator to define multiple types of storage within a cluster, each with a custom set of parameters.

Your storage fabric should then have an intelligent optimizer that analyzes the user request, matches to resources, and places them appropriately onto a server that matches the personality of the data. It should also be policy-driven and use telemetry to continuously inventory and optimize for application and microservice needs without human involvement.

Let’s say you want to create a Storage Class to access your MySQL data, add extra protection by making 3 replicas, and place it in a higher performance tier to serve reads/writes better. You can set this up in Datera with just a few steps.

Create a Storage Class for the Datera storage backend as in the following datera-storage-class.yaml file example here:

$ cat datera-storage-class.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
     metadata:
          labels:
          name: datera-storage-class
     namespace:
provisioner: io.datera.csi.dsp
reclaimPolicy: Delete
parameters:
     template: “basic”
     secretName: “mysecret”
     secretNamespace: “default”
     fsType: “xfs”

$ kubectl create -f datera-storage-class.yaml

Datera allows the Kubernetes cluster administrator to configure a wide range of parameters in the Storage Class definition. Another way to define a Storage Class is with labels (policies), e.g. gold, silver, bronze. These options convey the storage personality and define necessary policies at Datera Storage System level.

It’s very easy to compose three or 300 Storage Classes on the fly in Datera. To manage data services such as dedupe, compression and encryption for high velocity apps and microservices, you can attach volumes to a storage pool that can be further configured alongside policies for data protection and efficiency. Where typically this is done in a silo, Datera can achieve this level of velocity and scale for Tier 1 workloads.

If you have absolutely no idea what the app requirements will be, that’s ok—Datera uses AI/ML to find the best placement, and the resources and will automatically adjust based on inputs. For mature apps, you can graduate those to policies and templates.

No Scale Without Automation

Kubernetes lets applications scale along with a Storage Class without deviating from the base definition/requirements of resources. Datera keeps that promise intact by binding Storage Classes to templates, and homogeneously extend resources (volumes) to match the intent of applications (consumers).

As application needs change, the storage should adapt alongside it and not rely on human intervention. This is done by defining policies and templates. The storage fabric should also recognize and adapt to new nodes or hardware and automatically adjust to enhance performance, capacity, and resilience of a cluster, as well as making the resources available to all the underlying PVs.

Create the StatefulSet:

$ kubectl create -f consul-sts.yaml

List the pods:

$ kubectl get pods -o wide
NAME         READY   STATUS            RESTARTS    AGE  IP            NODE
consul-0     1/1     Running           0           32s  10.38.3.152   kubew03
consul-1     0/1     PodInitializing   0           15s  10.38.4.4     kubew04

The first pod has been created. The second pod will be created only after the first one is up and running, and so on. StatefulSets behave this way because some stateful applications can fail if two or more cluster members come up at the same time. For a StatefulSet with N replicas, when pods are deployed, they are created sequentially, in order from {0…N-1} with a sticky, unique identity in the form $(statefulset name)-$(ordinal). The (i)th pod is not created until the (i-1)th is running. This ensures a predictable order of pod creation. However, if the order of pod creation is not strictly required, it is possible to create pods in parallel by setting the podManagementPolicy: Parallel option in the StatefulSet template.

List the pods again to see how the pod creation is progressing:

$ kubectl get pods -o wide
NAME	     	READY    STATUS        RESTARTS  AGE    IP             NODE
consul-0      	1/1      Running       0      	 29m    10.38.3.152    kubew03
consul-1      	1/1      Running       0      	 28m    10.38.4.4      kubew04
consul-2      	1/1      Running       0      	 27m    10.38.5.128    kubew05

Now all pods are running and forming the initial cluster.

Scaling the Cluster

Scaling down a StatefulSet and then scaling it up is similar to deleting a pod and waiting for the StatefulSet to recreate it. Please, remember that scaling down a StatefulSet only deletes the pods, but leaves the Persistent Volume Claims. Also note that scaling down and scaling up is performed similar to how pods are created when the StatefulSet is created. When scaling down, the pod with the highest index is deleted first: only after that pod gets deleted, the pod with the second highest index is deleted, and so on.

What is the expected behavior scaling up the Consul cluster? Since the Consul cluster is based on the Raft algorithm, we have to scale up our 3 nodes cluster by 2 nodes at the same time because an odd number of nodes is always required to form a healthy Consul cluster. We also expect a new Persistent Volume Claim is created for each new pod.

Scale the StatefulSet:

$ kubectl scale sts consul --replicas=5

By listing the pods, we see our Consul cluster gets scaled up:

$ kubectl get pods -o wide
NAME	     	READY    STATUS          RESTARTS    AGE IP           	NODE
consul-0      	1/1      Running         0           5m  10.38.3.160	kubew03
consul-1      	1/1      Running         0           5m  10.38.4.10     kubew04
consul-2      	1/1      Running         0           4m  10.38.5.132    kubew05
consul-3      	1/1      Running         0           1m  10.38.3.161    kubew03
consul-4      	1/1      Running         0           1m  10.38.4.11     kubew04

Check the membership of the scaled cluster:

$ kubectl exec -it consul-0 — consul members
NAME	        ADDRESS           	STATUS        TYPE      BUILD   PROTOCOL    	DC SEGMENT
consul-0      	10.38.3.160:8301    	alive         server    1.0.2   2	        kubernetes
consul-1      	10.38.4.10:8301      	alive         server    1.0.2   2               kubernetes
consul-2      	10.38.5.132:8301    	alive         server    1.0.2   2               kubernetes
consul-3      	10.38.3.161:8301    	alive         server    1.0.2   2               kubernetes
consul-4      	10.38.4.11:8301      	alive         server    1.0.2   2               kubernetes

Also check the dynamic Datera storage provisioner created the additional volumes:

$ kubectl get pvc -o wide 
NAME              STATUS VOLUME                                   CAPACITY  ACCESS MODES   STORAGECLASS             AGE 
data-consul-0     Bound  pvc-91df35af-0123-11e8-86d2-000c29f8a512 1Gi       RWO            datera-storage-class     7m 
data-consul-1     Bound  pvc-a010257c-0123-11e8-86d2-000c29f8a512 1Gi       RWO            datera-storage-class     6m 
data-consul-2     Bound  pvc-adaa4d2d-0123-11e8-86d2-000c29f8a512 1Gi       RWO            datera-storage-class     6m 
data-consul-3     Bound  pvc-1b1b9bd6-0124-11e8-86d2-000c29f8a512 1Gi       RWO            datera-storage-class     3m 
data-consul-4     Bound  pvc-28feff1c-0124-11e8-86d2-000c29f8a512 1Gi       RWO            datera-storage-class     2m

Today, companies may be deploying apps by Storage Class. But according to IDC 90% of new applications will be built with a micro-services architecture this year. Data velocity must match micro-services with the right Storage Class, and continually and granularly enhance the storage environment with additional performance and capabilities—not via legacy arrays or monolithic and silo-ed major migrations, but via discrete racks. In other words, autonomous storage is a necessity for any business that needs to scale.

If you are planning to deploy Kubernetes, and experiencing any storage challenges with your legacy infrastructure, please click here to watch how Datera overcomes those challenges.

According to a recent Forbes article, container adoption is growing rapidly in the enterprise—much faster than expected. And according to a recent Gartner report, “By 2023, more than 70% of global organizations will be running more than two containerized applications in production, up from less than 20% in 2020.

Moving to containers, however, presents new challenges for storage provisioning. In particular, using traditional storage solutions for distributed containerized workloads turns out to be too complex and costly. Manual storage provisioning does not address the dynamic storage requirements efficiently at scale. Containers are dynamically created, destroyed, and moved across hosts in the availability zone. The stateful applications running inside these pods would require storage solutions that can automatically attach or detach and provision storage capacities across data-centers where these pods land. In addition the storage infrastructure should facilitate creating back-up and replicas of data to ensure high availability.

Containerized applications require diverse heterogeneous storage solutions such as HDD, SSD, NVME, Optane, block, or object. Thus the storage solution needs to be flexible, hardware-type agnostic, and support tight integration with popular container orchestration such as Kubernetes(K8s). This leads us to a software-defined architecture that abstracts storage software from its hardware, such as Datera.

Datera is a fully dis-aggregated scale-out storage platform, that runs over multiple standard protocols (iSCSI, Object/S3), combining both heterogeneous compute platform/framework flexibility (HPE, Dell, Fujitsu, Intel, and others) with rapid deployment velocity and access to data from anywhere. Datera’s Container Storage Interface (CSI) driver deeply integrates with the K8s runtime. It allows deploying entire stateful multi-site K8s clusters with a single K8s command and pushing application-specific telemetry to the Datera policy engine so that it can intelligently adapt the data fabric. Datera’s powerful storage classes and policy-driven workloads are a natural fit with Kubernetes.

Datera, organically extends Kubernetes concepts , admins do not need to look and match resources for the workloads, instead they can just define the application and type of resources it needs <admin_state> and Datera’s resource optimizer automatically picks the most suitable resources (including media, Optane, NVMe, SAS/SATA Flash or even spinning) <op_state>.

• Scale: Kubernetes has deployment primitives defined for scale, like StatefulSet which defines pod templates that ensure applications can scale up and down dynamically driven by the defined pod template. As a result of this, Kubernetes defines Storage Class, consisting of dynamic resources such as Persistent Volume Claims(PVC) and Persistent Volumes(PV). Datera extends this by defining Policies and Application-Templates. Templates provide consistency across Instances defined using the template, hence mapping the K8s applications to scale up using the consistent behavior of instance provisioning using Datera’s scale-out templates. While scaling the K8s statefulset, Datera can enable transparent data migrations for the provisioned containers on the fly.
• Resource Isolation: Kubernetes provides resource allocation by having hierarchical namespace structure to provide appropriate access control for the workloads. Datera is built on support for multi-tenancy making all the policies, templates and instances tenant-aware allowing for greater granular access control,consistency and resource isolation.
• On-demand Provisioning: Applications on Kubernetes can scale up really quick, using primitives such as StatefulSets, ReplicaSet. This introduces resource bindings of PVC’s , expecting the storage vendor to bind them to a matching resource .Datera uses thin-provisioned clones to achieve meeting the scaling requirements. Thus only reacting to scale up and down on dynamic demands, rather than to pre-provision the resources while fore-casting requirements.
• Keep it Stupid Simple: It is no co-incidence that the Kubernetes compute concepts naturally map into the Datera Data-fabric concepts since both were designed to dynamically grow and shrink, as application demands change over period of time. Some Key extensions of Kubernetes concepts that have organically infused into Datera’s design concepts:

Thus by creating a Just-in-Time Service defined private/hybrid cloud defined platform utilizing Kubernetes and Datera, enables application owners and storage admins to independently optimize the platforms for their needs. This helps achieve unmatched simplicity, efficiency, and business value to enterprise customers while helping reduce overall TCO.

Background reading:

• Datera CSI deployment guide
• 10 Principles for Evaluating Enterprise Storage for Kubernetes & Cloud Native Applications
• Kubernetes for Data
• Datera OpenShift 3.11 Deployment Guide

Traditional enterprise storage architecture had very limited autonomy. The autonomy on these systems was limited to providing spare drives, to automatically replace failed drives. Controller failures required manual intervention and high cost maintenance contracts to maintain storage SLO. Data migrations done to improve performance or provide resiliency were typically disruptive operations which meant a hit for application performance or data unavailability scenarios in extreme conditions.

Datera was designed with one single mantra in mind “The only Constant is Change”.

How did we design Datera to be so dynamic and adaptive? The Datera distributed control plane resolves your objectives into a set of policies that are given to a high-performance data plane as key value stores for execution of the policy. The data plane then maps key value pairs onto the current physical hardware to deliver performance, reliability and accessibility.

Software on the individual nodes, built from commodity infrastructure, utilize resources-specific capabilities depending on the type of storage, CPU, memory and networking that optimize for:

• Transformation – protection, compression, encryption, deduplication…
• Placement – NVMe, SSD, HDD, Cloud…
• Functionality – snapshot, replication, copy data management…

Datera continuously monitors how the cluster is performing relative to the specified application intent, i.e. compares admin_state and operation_state. Application requirements in the form of policies are specified by the application admin, and the control plane works to apply them constantly to a completely programmable data plane based on the availability of physical resources. A policy change to improve performance of a subset of data would involve that data migrating to a node supporting media-types to better fit the policy autonomously with absolute transparency.

The autonomous characteristics of Datera Storage systems include but are not restricted to the following underlying behavior:

• Recovery: A Datera system will autonomously recover and adjust data in a way to meet the policy intent during failure and restoration of a variety of physical and software components.
• Policy Changes: Policies can be changed on the fly and the system will autonomously adjust data placement in an entirely transparent and non-disruptive manner to configure the data plane to meet the policy intent.
• Autonomous Redistribution: Datera allows creation of application intent to be created via AppInstance, even if the capabilities are not currently available on the cluster. When resources such as new storage media, memory are added, as part of closed loop autonomous optimization, the data will be redistributed in a non-disruptive manner to meet intent. Datera allows admins to decide the end-goal and the system strives to meet the goal when resources are made available.
• Scaling: Datera automatically incorporates scale-out a storage node to the cluster, and this would involve adding network configuration and upgrading of the software version on the node to meet the cluster versioning while adding the node. Conversely Datera also allows a cluster to scale-in by decommissioning of storage nodes and data from the nodes will be migrated out and cleansed on the retired node.
• Data Placement: Datera provides an outcome based data placement mapping driven by application intent.
• Re-balance: As part of continuous optimization, as additional capacity is added to the cluster in terms of nodes or additional media-types, Datera will move data workloads over to the added capacity in-order to balance the capacity and targets across the cluster.
• Rolling-Upgrades: When a new software version is available, the cluster will autonomously deploy new software, in a non-disruptive, per node, rolling basis, including managing any networking changes to ensure continuous availability.

Datera’s forward thinking and holistic design approach is to envision these capabilities to be operating in concert to create autonomous operations. It strives to make data movement transparent to all host environments across generations of “x86” server & storage media-types, driven autonomously by “current+future” application intent state. Datera is Built For Change, and is helping the largest Global 1000 enterprises future proof and modernize their storage infrastructure.

If you would like to learn more about Datera’s flexibility and future-proofing capabilities, please read our Built For Change White Paper.

Vikram Desai heads up Datera’s Technical Program Management team, and has been with the company for nearly three years. He has been instrumental in supporting our largest OEM’s and other Partners.  Prior to Datera he worked across IaaS and Cloud Storage products at Cisco & Akamai.