There you were, top of the food chain for millenniums and then it happened, clouds rolled in and you were done. On the other side of the world things were happening that changed your climate, it got cloudy. Too big and too slow to adapt, you tried to survive, but the end was at hand. The apex predators ruled for 100 million years, growing larger and stronger, they had evolved into what seemed like the perfect animal for the top of the food chain but the universe was about to throw you a curve ball…

The thinly veiled metaphor is about what change, ironically clouds, did to wipe out the dinosaurs 66 million years ago and what clouds have done to the top-of-the-food-chain legacy enterprise storage systems. In both cases it was not really “clouds” that were problematic but rather the ability to adapt to a changing world. For the dinosaurs it was quick, once the asteroid struck (or the volcanoes erupted – pick your theory), the clouds would have had their effect in days or weeks. In the case of the Legacy Enterprise Storage Systems (LESS) the process is taking years, but the process is inevitable.

The Triassic Storage Era:
The time of big iron and air-breathing apps

So what happened? It used to be that we had enterprise, mid-range, entry-level and direct attached storage (DAS). Those categories were defined by analysts, had market sizing, growth rates… and had generally acceptable properties that distinguished them from other categories, varying levels of cost, availability, performance, scalability. Then it got weird…

The emergence of virtual machines meant that a single server could run multiple applications, potentially with multiple tenants – DAS struggled. Early hyperscalers, Facebook, Google, E-bay etc., tired of paying egregious prices for enterprise storage systems, wrote their applications to solve the data availability problems usually solved by big iron storage arrays but using commodity servers. Then this company named Amazon introduced something called the Simple Storage Service or S3. Not block, not file, something new – and you could not buy the system, you had to buy the service. What is that bright light shooting across the sky? Another asteroid?

The Jurassic Storage Era:
The age of Goodenoughasaurus

We had entered the phase of “Good Enough” storage. Enterprise storage system customers, noticing what hyperscalers had done, began to question the 10x-100x-over-the-hardware price they were paying for enterprise-class storage. Mid-range arrays began to erode the higher end segments with products like NetApp, 3PAR and Engenio experiencing rapid growth. EMC Symmetrix, the dominant enterprise array still had its place, just not everyplace. People started moving their data to AWS. Big boom…bright light…asteroid hitting earth.

Storage innovators started to borrow from the hyperscale approaches using distributed systems techniques dubbed scale-out (as opposed to scale-up). This new storage software promised to run on commodity servers, just like the hyperscalers. No more 10x-100x-over-the-hardware price, the theory was you can use whatever hardware you want. The era of software defined storage (SDS) had begun – the clouds rolled in, temperatures dropped – apex predators in rapid decline.

The Cretaceous Storage Era:
The evolution of 1st Generation SDS begins

The early days of SDS were rough. Lots of over-promise and under-deliver. There was also a huge shift with the rise of flash memory as an alternative to spinning rust (HDD’s), as my friend Don Molaro likes to say. Lots of startups, venture money was easy to come by. Only one problem – storage is hard. Turns out customers like their data. If they can’t access it, or you get just a few bits out of billions wrong, they care – a lot. Most startups did not survive, they got the cool stuff right and the basics wrong. Wrong recipe.

The Cenozoic Storage Era:
Storage arrays dominate the land

Nobody talks about what happened next because it is not exciting – customers kept buying legacy enterprise storage systems. Sure, those that moved their applications to cloud service providers moved their data long with it. But for those for whom “cloud” was not the right answer, EMC (now Dell), NetApp and HP were happy, thrilled in fact, to keep selling them conventional fibre channel arrays.

As we enter 2019, why is there still a multi-billion dollar storage array business built on Y2K technology? Because it works! Quite simply, the requirements for their mission critical applications were not being met by the hype, let alone the products in the market. Product like VMAX and 3PAR have remained the best answer. The new capabilities of SDS were great…scale-out, commodity hardware… but not at the expense of continuous availability, predictable performance and architected scalability.

The problem with requirements is that they never get easier. Nobody ever said that less data integrity is OK, or that more downtime is acceptable. What was needed, and is available from Datera, is a storage architecture that delivers for the rapidly evolving modern data center without compromising on enterprise class storage requirements.

The Tertiary Storage Era:
New life forms emerge

How did Datera go about solving this challenging blend of historical and modern requirements? First and foremost is that the architecture was designed specifically to solve the problem. You can’t retrofit an old architecture to solve the modern data center problems, nor can you retrofit a mid-range SDS architecture to solve the tier 1 enterprise storage requirements. Dinosaurs didn’t become mammals just because the temperature dropped. Solving both the enterprise storage requirements and providing compelling value for the modern data center, while challenging, is exactly what Datera set out to do. It was five years in the making but we figured out the recipe for Software Defined Enterprise Storage.

The primary challenge Datera faced was that you must use the conjunction “and” not the conjunction “or”. Continuous availability, predictable performance, architected scalability, as-code provisioning, heterogeneous commodity hardware, policy driven, forward agility, analytics guided feedback, data center awareness, security AND non-proprietary ecosystem integration. Three letters (and) versus two (or) make all the difference.

The Quaternary Storage Era:
Modern data centers and dramatic changes in climate drive SDS

So how did Datera create an architecture that meets both the legacy enterprise storage systems requirements as well as supporting the demands of the modern data center? Once again it comes down to that pesky conjunction – and. You cannot solve the requirements one at a time. The sum of the parts does not make it whole. Like any complex system, careful balancing and tradeoffs must occur and as good fortune would have it, in 2012 Datera assembled a team that was up to the task.

There were three guiding principles to the architecture:

  • Manage by intent at the application level
  • Pervasive singular mechanism for data agility
  • Performance through reduced overhead

The common theme in these principles is simplicity. Simplicity for the customer in management, simplicity in managing the data via a single elegant mechanism and delivering predictable (and high) performance by minimizing overhead. Why is simplicity in these guiding principles so important? Having gone to the school of hard knocks I know that it is not possible to meet the enterprise storage requirements for modern data centers with fine grained management, multiple data management approaches and hyper optimization for performance, a.k.a. dinosaur storage.

The good news is that these guiding principles served Datera well. The resulting architecture and product provide the availability, performance and scalability required for modern enterprise storage – as well as the agility, velocity and programmability required in the modern software defined data center.

Learn more about how Datera enables customer’s most critical applications to (finally) benefit from a software defined enterprise storage architecture in the 2-Minute Tech Talk Video Series and the latest Geek Out OnDemand Webinar.

Watch for new upcoming series explaining the “secret sauce” of how Datera architecture uniquely delivers the core values of Data Orchestration, Enterprise Performance, Future Ready Choice, Data Center Awareness and Predictive Operations.

Get a Consultation

If you are ready to transition to a software-based data environment with enterprise class performance and continuous availability, and reduce your storage infrastructure and operations total-cost-of-ownership by as much as 70%, Contact Us to schedule a free consultation.