According to a recent Forbes article, container adoption is growing rapidly in the enterprise—much faster than expected. And according to a recent Gartner report, “By 2023, more than 70% of global organizations will be running more than two containerized applications in production, up from less than 20% in 2020.
Moving to containers, however, presents new challenges for storage provisioning. In particular, using traditional storage solutions for distributed containerized workloads turns out to be too complex and costly. Manual storage provisioning does not address the dynamic storage requirements efficiently at scale. Containers are dynamically created, destroyed, and moved across hosts in the availability zone. The stateful applications running inside these pods would require storage solutions that can automatically attach or detach and provision storage capacities across data-centers where these pods land. In addition the storage infrastructure should facilitate creating back-up and replicas of data to ensure high availability.
Containerized applications require diverse heterogeneous storage solutions such as HDD, SSD, NVME, Optane, block, or object. Thus the storage solution needs to be flexible, hardware-type agnostic, and support tight integration with popular container orchestration such as Kubernetes(K8s). This leads us to a software-defined architecture that abstracts storage software from its hardware, such as Datera.
Datera is a fully dis-aggregated scale-out storage platform, that runs over multiple standard protocols (iSCSI, Object/S3), combining both heterogeneous compute platform/framework flexibility (HPE, Dell, Fujitsu, Intel, and others) with rapid deployment velocity and access to data from anywhere. Datera’s Container Storage Interface (CSI) driver deeply integrates with the K8s runtime. It allows deploying entire stateful multi-site K8s clusters with a single K8s command and pushing application-specific telemetry to the Datera policy engine so that it can intelligently adapt the data fabric. Datera’s powerful storage classes and policy-driven workloads are a natural fit with Kubernetes.
Datera, organically extends Kubernetes concepts , admins do not need to look and match resources for the workloads, instead they can just define the application and type of resources it needs <admin_state> and Datera’s resource optimizer automatically picks the most suitable resources (including media, Optane, NVMe, SAS/SATA Flash or even spinning) <op_state>.
- Scale: Kubernetes has deployment primitives defined for scale, like StatefulSet which defines pod templates that ensure applications can scale up and down dynamically driven by the defined pod template. As a result of this, Kubernetes defines Storage Class, consisting of dynamic resources such as Persistent Volume Claims(PVC) and Persistent Volumes(PV). Datera extends this by defining Policies and Application-Templates. Templates provide consistency across Instances defined using the template, hence mapping the K8s applications to scale up using the consistent behavior of instance provisioning using Datera’s scale-out templates. While scaling the K8s statefulset, Datera can enable transparent data migrations for the provisioned containers on the fly.
- Resource Isolation: Kubernetes provides resource allocation by having hierarchical namespace structure to provide appropriate access control for the workloads. Datera is built on support for multi-tenancy making all the policies, templates and instances tenant-aware allowing for greater granular access control,consistency and resource isolation.
- On-demand Provisioning: Applications on Kubernetes can scale up really quick, using primitives such as StatefulSets, ReplicaSet. This introduces resource bindings of PVC’s , expecting the storage vendor to bind them to a matching resource .Datera uses thin-provisioned clones to achieve meeting the scaling requirements. Thus only reacting to scale up and down on dynamic demands, rather than to pre-provision the resources while fore-casting requirements.
- Keep it Stupid Simple: It is no co-incidence that the Kubernetes compute concepts naturally map into the Datera Data-fabric concepts since both were designed to dynamically grow and shrink, as application demands change over period of time. Some Key extensions of Kubernetes concepts that have organically infused into Datera’s design concepts:
Thus by creating a Just-in-Time Service defined private/hybrid cloud defined platform utilizing Kubernetes and Datera, enables application owners and storage admins to independently optimize the platforms for their needs. This helps achieve unmatched simplicity, efficiency, and business value to enterprise customers while helping reduce overall TCO.
- Datera CSI deployment guide
- 10 Principles for Evaluating Enterprise Storage for Kubernetes & Cloud Native Applications
- Kubernetes for Data
- Datera OpenShift 3.11 Deployment Guide