In a prior blog, we shared the high level view of the Datera architecture and the innovation that enables customers the opportunity to radically reduce complexity and costs, while significantly improving operational agility and efficiency.

The end result is a hybrid cloud, software-defined platform implemented for both block and object data enabling a data infrastructure that is heterogeneous, elastic,  dynamic, and autonomous.

The Datera architecture has three distinct layers :

  • A heterogeneous software defined storage layer for block and object data
  • A data management layer to make the infrastructure Dynamic
  • A programmable layer to make the infrastructure Autonomous and easily programmable/extensible

In this blog I want to take you under the Datera hood, and share one of the core technologies that enables Datera’s architectural differences, distinct from any other company in the space. And that is Datera’s patented Lock-less coherence technology…

Customers select software defined scale out solutions for few basic reasons:

  1. They want rapid technology adoption and automation which they cannot achieve with rigid, hard-to-manage enterprise-class arrays that lock them in with a fixed architecture.
  2. They want agility; being able to scale performance and capacity linearly to avoid complex guessing as to what they will need.
  3. They want freedom from hardware lock-ins and better economics with the ability to leverage industry standard servers and media — thus allowing the flexibility to adopt whichever media makes the most sense for their applications

While at a macro level, most software defined solutions may claim the ability to deliver on these requirements, many struggle in the details.  This is where architecture matters. Asking yourself a few basic questions will provide transparency and shed light on the outcomes you require to modernize your data center:

  1. How is the solution able to scale performance linearly as new nodes are added? Or asked a different way, will adding servers have an incremental or diminishing impact on the overall performance?
  2. Can the solution deliver high performance AND data efficiency- with compression and deduplication at the same time? Or will I need to compromise on either dimension?
  3. Can the solution seamlessly handle multiple types, generations, or brands of servers seamlessly? Or will I be locked in; forced to use the same class of server going forward as when I first deployed the platform? Next, if I can add different servers, will that be easy to do?
  4. Can the solution seamlessly and easily handle new types of media that become available over time? Or will the solution need to use the same type of media over the life of the platform? And if I can use different media, how easy will that be?
  5. Does the solution have an architecture that helps increase media endurance? Or will applications that are write-intensive reduce media life?

The Power and Innovation Of Lock-less coherency

We can confidently say– and demonstrate– that the Datera platform can answer yes to all of these questions! And that is accomplished through the Datera Lock-less Coherency Protocol, which is the secret sauce of the Datera platform.

Datera has a shared nothing architecture, meaning there is no centralized master node that stores all of the cluster metadata. That said, it is imperative for a distributed storage system like Datera to support changes initiated across multiple access points of the cluster.

Most scale-out storage systems use some form of Distributed Lock Management (DLM) in order to ensure data coherency across multiple access points of the cluster. The use of DLM requires that all the nodes in a cluster communicate with each other. As the size of the cluster increases, the amount of intra-node communication would adversely affect the foreground I/O bandwidth, impacting performance.

Together, with Datera’s shared nothing architecture, a time-based Lock-less Coherency Protocol is employed which ensures writes are synchronized across participating nodes without the need to increase intra-node communication. Thus the cluster IOPS and throughput scales linearly as the number of nodes added in the cluster increases.

The Lock-less Coherency Protocol provides correctness by ensuring that out of order writes are executed in time-order to the underlying storage media.▪  “Lock-less Coherency” – two-stage write process and distributed metadata maps (current and future maps) enables changes made from any node to be synchronized with other nodes without using distributed lock management.

Locks add repair complexity and latency.  The advantages of using a time-based, Lock-less Coherency Protocol involve the following:

  1. No locks required when repairing to avoid a new overwrite losing to a repair write.
  2. No locks required even if writes from one client follow different paths.
  3. No locks required even if writes from different clients follow different paths.

The multi-benefits provided by using Lock-less Coherency Protocol mechanism:

  • All acknowledgements for the host application are done out of non-volatile memory(NV-DIMM, NVRAM, 3DXPoint). The write latency for the host applications are not affected by the underlying storage media types, and are able to observe consistent, predictable latency.
  • Allows for the ability to perform log-structured writes from NV-DIMM , down to the underlying flash media. This leads to extending performance and endurance of underlying flash media, as large block writes are written to the media instead of a bunch of small block writes.
  • All of the data services for data efficiency purposes, such as Deduplication, Compression, & Encryption are done behind the ACKs sent to the host application. In typical systems where you have to do in-line deduplication there is a noticeable latency at the host application, and that is not the case with the Lock-less Coherency Protocol.

The outcome of the Lock-less Coherency technology enables Datera to deliver a solution that provides not only scalable performance with data efficiency functionality, but also provides the highest flexibility and heterogeneity in terms of servers and media customers can adopt on the fly over time.

And this is just one of the secret sauces that Datera has developed to deliver on broader outcomes that customers are looking for as they architect for the next generation data infrastructure. Stay tuned and will provide more insight on critical technologies that make Datera a truly unique architecture…

For more information, we recommend reading our white papers:

Built for Performance

Built for Constant Change

Built for Autonomous Operations

Built for Continuous Availability

We can schedule a demo at any time. Please reach us at sales@datera.io and share any specific capability you would like to learn more about. We look forward to the opportunity!