The hype around self-driving cars keeps reaching new heights, with most pundits focusing on their immediate innovation and novelty. Intel’s recent $15 billion acquisition of Mobileye, which makes self-driving sensors and software, illustrates the immense potential of this opportunity. It also highlights, however, what many have yet to grasp: The true transformation isn’t the self-driving car, it’s the underlying digital technology.

The sensors and other necessary systems in a self-driving car will roughly generate and consume five terabytes of data every day. At 65 miles per hour, one second latency can separate life from death, so much of this data will need to be acted upon quickly.

Gravitating Toward Edge Clouds

Even with networking advances, such as 5G connections for each car, that’s a lot of data that needs to be processed in a very short time. Add other services that require flexible real-time data processing, such as airplanes, smart homes, connected cities, industrial IoT, a myriad of increasingly smart objects, and the challenges to interactively serve billions of connected devices with zettabytes of data become strikingly apparent.

To rise to these challenges, many companies are moving their computing into the cloud. Ironically, however, interactive services will gravitate toward the edge because of latency constraints. All those edge clouds need to run automatically – and dynamically recompose themselves to adapt to continuously fluctuating application demands.

Kubernetes is a Game Changer

A handful of companies have been preparing for this eventuality. Google, for example, uses millions of servers in its operations. To dynamically compose them for each application, Google created its own software, known as “Borg.” Borg has spawned another computing management innovation: its open source pendant “Kubernetes,” which is rapidly democratizing Google-style cloud computing.

Kubernetes allows the disaggregation of applications into scalable microservices, which it dynamically maps onto data center resources based on their respective requirements and capabilities. This way, compute can be composed and delivered as a “swarm” of services, resulting in game-changing operational flexibility, utilization and economics.

Datera is to Data as Kubernetes is to Compute

Datera provides sophisticated data orchestration to complement Kubernetes’ compute orchestration. By leveraging rich machine intelligence, the Datera data services platform continuously reshapes itself to optimize performance, resilience and cost.

Datera is the only data services platform that is driven by application service level objectives (“SLOs”), which makes it the ideal self-driving data foundation for any cloud infrastructure:

  • NoOps: Application SLOs automatically drive the Datera data infrastructure instead of humans, thus providing the foundation for 24×7 self-driving infrastructure.
  • Dynamic resource shaping: The Datera platform enables all its elements (data, exports, services, etc.) to float across it, so that SLOs can constantly reshape it, ultimately spanning private and public clouds.
  • Native I/O performance: Datera built its own low latency log-structured data store with a lockless distributed coherence protocol to match the performance of high-end enterprise arrays, and thus allows replacing them with a modern, cloud-age data services platform.
  • Future-proof extensibility: Datera supports live-insertion of new technologies, which offers a game-changing price/performance band that can accommodate a wide spectrum of applications and eliminates data migration forever.
  • Role-based multi-tenancy: The Datera platform uses adaptive micro-segmentation to deliver security, resilience and network virtualization as a service.

Enter Rack-Scale Infrastructure

Resource orchestration platforms, such as Kubernetes and Datera, align well with highly configurable hardware architectures, such as rack-scale, which allows independent pooling and scaling of resources to compose application-defined, tailored data center services.

Some of the key tenets of rack-scale computing are:

  • Commodity hardware: Commodity servers provide disaggregated resources that can independently be pooled and scaled, offering wide price/performance flexibility.
  • High-speed fabrics: Fast networks significantly reduce communication overhead and latency between the disaggregated resources and make them practical.
  • Composability: Intelligent software dynamically composes the disaggregated resources into a cohesive system.
  • API-driven: A single, global REST API makes the resources easy to provision, consume, move and manage.
  • NoOps: The resources are consumed automatically, driven by applications rather than operating systems, hypervisors or humans.

Flat Networks Allow Effective Resource Pooling

In addition to rack-scale architectures, hyperscale data centers are rapidly adopting flat (“leaf/spine”) network topologies to further reduce latency and make resource pooling more effective:

Independent of, but complementary to flat networks, nodes can participate in network route changes to create a single, flat, dynamic IP namespace. This is accomplished by integrating a software router (BGP) on each node to create a virtual end-to-end L3 network that allows applications and the network to seamlessly co-adapt:

In a virtualized L3 network, each node runs its own private subnet, so it can assign a dedicated IP address to each of its services. As a result, services can easily be mapped onto nodes by updating IP routing tables, which simplifies live-migration of nodes and services across the data center. This is similar to hypervisors that integrate software routers to help live-migration of VMs across the data center.

Some of the innate benefits of virtualized L3 networks are:

  • Adaptive networking: Virtual L3 networks are flexible and scalable, can contain any node across the data center, and can be dynamically reshaped, with fast network convergence times after link or node changes. In contrast, L2 networks are restricted to a single subnet, with overlay networks as a clumsy way to mitigate this limitation.
  • Dynamic load balancing: Virtual L3 networks use equal-cost multipathing (ECMP) and dynamic routing to quickly adapt to topology and bandwidth changes.
  • Quality of Service (QoS): Virtual L3 networks can use fine-grained traffic shaping to transparently compensate network and topology flux.

For data management and storage systems, L3 network virtualization is becoming particularly important to seamlessly co-adapt with the data center network, in order to allow efficient load balancing and failover across the data center.

Datera Brings L3 Network Virtualization to Data Services

Datera is the only data services platform that integrates SLO-based L3 network function virtualization (NFV) with SLO-based data virtualization to automatically provide secure and scalable data connectivity as a service for every application. It accomplishes this by running a software router (BGP) on each of its nodes as a managed service, similar to “Project Calico.”

For backward compatibility, Datera also supports traditional L2 networks with overlay networks (such as VLANs and VxLANs).

Some of the innate benefits of Datera’s L3 network virtualization are:

  • Datacenter awareness: Datera understands the L3 network and how it changes. Its policy engine can infer data center properties such as rack boundaries, availability zones, power domains, etc., so over time, it will become increasingly aware of the data center itself.
  • Non-disruptive growth: Datera services map to IP addresses, which float in the virtualized flat IP namespace. Datera can instantly learn L3 network changes, so its data services can live-migrate across the data center without I/O glitches, and thus seamlessly adapt to data center flux.
  • Service advertisement: As Datera service ports map to IP addresses, it can also automatically advertise service updates, and constantly reshape the L3 network to maintain optimal access to all of its data services without affecting I/O.
  • Adaptive security: Datera policies combine L3 network virtualization with data access control lists (ACLs) and whitelists, so they can secure data connectivity automatically, and far better than any other solution.
  • Adaptive resilience: Datera is data center aware, and continuously optimizes the virtualized data and services along with the virtualized L3 network, so it can auto-configure far higher reliability and availability than any other solution.

While a hyperconverged system can only manage itself, Datera increasingly embraces monitoring and managing the entire data center via application SLOs. Because of this, Datera can be conceptualized as “inside-out hyperconvergence” on flat networks:

Driving it Home

As the world continues to accelerate toward self-driving cars and billions of connected compute devices, the ability to keep up with their demands requires data centers to become just as intelligent, autonomous and agile as they are.

Self-driving infrastructure continuously recomposes itself to adapt to fluctuating application demands, with fully autonomic operations, driven by applications – not by humans. Rack-scale architecture aligns well with self-driving infrastructure, as it offers independently composable resource pools. Flat networks reduce latency to make resource pooling effective, and virtual L3 networks let applications and the data center network seamlessly co-adapt.

Datera uniquely blends all these elements seamlessly together for data, creating an operations-free data services platform that transforms infrastructure and obsoletes storage as we know it.

Just as machine intelligence is increasingly enriching everyday human experiences, self-driving infrastructure is transforming the way digital technology is developed, deployed and delivered.

Datera’s self-driving data infrastructure delivers game-changing speed, scale, agility and econo
mics – to always navigate the best road ahead to any destination.