This article is based on analysis originally published by Moor Insights & Strategy via Forbes (November 2018). The arguments remain highly relevant as hybrid and multi-cloud adoption has accelerated.
The Data Gravity Problem
The IT industry is in the midst of a fundamental shift in how data is generated, stored, and consumed. Data is no longer confined to the data center. It is distributed across on-premises infrastructure, private clouds, public cloud platforms, and increasingly the edge. This distribution creates a challenge that is easy to overlook in high-level architecture conversations: data gravity.
Data gravity is the tendency of large data sets to attract applications and services — and to resist being moved. The more data accumulates in one location, the more expensive, slow, and disruptive it becomes to relocate or replicate that data elsewhere. This is not a software problem or a networking problem in isolation; it is fundamentally a physics problem. Bandwidth costs money, latency is real, and data at rest has inertia.
Why Hybrid and Multi-Cloud Make This Harder
Public cloud computing delivers compelling value: dynamic resource provisioning, elastic capacity, and the elimination of large upfront capital expenditures. These benefits explain why most enterprise IT organizations have adopted some form of public cloud strategy.
However, public cloud adoption does not eliminate on-premises infrastructure. It supplements it. The result is a hybrid environment where:
- Some workloads run on-premises for latency, compliance, or cost reasons.
- Other workloads run in one or more public clouds for scalability and flexibility.
- Data must flow consistently and reliably between all of these environments.
This is where data gravity bites hardest. A real-time analytics pipeline that depends on data sitting in an on-premises storage array cannot simply be lifted to the public cloud without addressing how that data gets there — and how quickly, consistently, and affordably it can be accessed from multiple locations simultaneously.
Migration Is Necessary But Not Sufficient
Almost every enterprise storage vendor offers some form of data migration tooling. Cloud tiering, data replication, and snapshot-based offload are standard features. However, migration alone does not solve the data gravity problem for several reasons:
- Availability vs. timeliness: Moving data to the cloud improves availability in principle, but it does not guarantee that the data is current, consistent, or accessible with acceptable latency at the moment it is needed.
- Cross-site consistency: When the same data set is accessed from multiple environments simultaneously — on-premises and cloud — maintaining consistency without significant engineering overhead is non-trivial.
- Vendor lock-in: Many cloud-provider-native storage solutions make it difficult or expensive to move data back on-premises or to a different cloud provider.
The goal for any modern IT architecture should be seamless data mobility: the ability for hybrid applications and their associated data to move between environments without operational compromise or lock-in.
Storage Is the Right Layer to Solve This
Data accessibility in a hybrid environment is not primarily an application-layer concern. Applications consume data; they do not own its placement, replication, or tiering. The storage layer has a unique end-to-end view of data — from ingest through consumption — regardless of where those endpoints are located. This makes storage the correct place to enforce policies around availability, performance, data protection, and compliance.
An effective hybrid storage architecture should provide:
- Tiered availability from flash to cloud: Hot data stays on high-performance flash; colder data tiers to lower-cost cloud storage — with policies that can promote or demote data automatically.
- Unified management: A single management interface that abstracts the physical and geographic boundaries between on-premises and cloud storage.
- No compromise on resiliency or compliance: Data protection, encryption, and compliance controls must apply uniformly regardless of where data physically lives.
- Open APIs: Enabling application-specific integrations and automation without being locked into proprietary toolchains.
- Consistent data reduction: Deduplication and compression should work across the full data lifecycle, not just on-premises.
Where Converged Infrastructure Falls Short
Converged infrastructure (CI) and hyper-converged infrastructure (HCI) approaches — such as those offered by Dell EMC and Hewlett Packard Enterprise — abstract physical compute, networking, and storage into unified platforms. Composable infrastructure and active data fabric architectures extend these concepts further. These are genuinely useful for simplifying on-premises management and extending consistent operational models toward the cloud.
However, converged approaches optimise the compute and control plane. They do not inherently solve the underlying storage data gravity problem. Extending a converged infrastructure to bridge into a public cloud is valuable, but it does not automatically mean that data can move freely, consistently, and at acceptable performance levels between all environments.
Storage problems are best solved by purpose-built storage innovation, not as a by-product of compute or networking abstraction.
The Role of Software Abstraction Vendors
Software-defined and cloud management vendors — including VMware, Nutanix, and others — play a critical role in smoothing the control plane between on-premises environments and public clouds from providers such as AWS, Microsoft Azure, and Google Cloud. This work is essential for workload portability and operational consistency.
But control plane consistency and data plane consistency are not the same thing. You can have a unified management view of a workload running across environments while still facing severe data gravity constraints that make that workload slow, costly, or brittle. The storage industry must innovate in parallel — not wait for the control plane to catch up — if the full promise of hybrid cloud is to be realized.
What Good Looks Like
The end state that modern enterprises should expect from their storage infrastructure in a hybrid world is straightforward to describe, even if it is technically demanding to deliver:
- 24/7 real-time access to all data and applications, regardless of physical location.
- Data that moves freely and transparently between on-premises, private cloud, edge, and public cloud environments.
- No forced trade-off between on-premises performance and cloud flexibility.
- No vendor lock-in that penalizes data mobility.
- Uniform data protection, compliance, and performance guarantees across all environments.
Achieving this requires storage vendors to extend their architectures beyond the data center boundary and treat the cloud as a native storage tier — not an afterthought or a migration destination.
Final Thoughts
Hybrid cloud is the operational reality for most enterprise IT organizations. The economic and practical benefits of blending on-premises infrastructure with public cloud services are well established. However, the data gravity challenge is a genuine bottleneck that limits how fully those benefits can be realized.
The storage layer is uniquely positioned to solve this problem. Vendors with deep storage expertise — and the willingness to architect their platforms natively for hybrid and multi-cloud environments — have a significant opportunity to change how data moves and is accessed at enterprise scale. This is not a problem that public cloud providers will solve unilaterally, and it is not a problem that software abstraction alone will eliminate.
The storage industry must step up. For organizations planning or evolving their hybrid cloud strategy, storage architecture decisions made today will determine how much operational freedom — or data gravity constraint — they live with for years to come.