This article is based on analysis originally published at itproportal.com (February 2017). The core challenges described remain relevant to enterprises undergoing DevOps transformation today.
The Silo Problem Nobody Warned You About
DevOps adoption in large enterprises often begins with a compelling success story: a new customer-facing application ships faster, works better, and delights users. But look beneath the surface of many of these wins and you will frequently find a familiar problem dressed in modern clothing—a brand new data silo sitting beside all the legacy ones, still unintegrated, still creating blind spots.
This is the paradox of enterprise DevOps transformation. The same drive toward agility and speed that produces nimble new applications can, if left unmanaged, multiply the very silos that Chief Transformation Officers (CTOs) are tasked with eliminating. Enterprises are not short of data; they are short of connected data.
Why What Works for Unicorns Does Not Always Work for Enterprises
Start-ups and technology unicorns can adopt a DevOps mindset from day one. They design their data architecture alongside their delivery pipelines, choose modern databases, and have no legacy baggage. Brownfield enterprises face an entirely different reality:
- Core business processes depend on decades-old CRM, billing, contract management, and ERP systems.
- These systems are maintained by teams who have deep domain knowledge but may be unfamiliar with container orchestration or continuous delivery.
- Replacing these systems entirely is prohibitively risky, expensive, and slow—typically measured in years, not sprints.
The result is a two-speed (or bi-modal) IT landscape: a “mode one” world of stable, process-heavy legacy systems and a “mode two” world of fast-moving, DevOps-driven applications. Without a deliberate integration strategy, these two worlds generate data that never meets.
A Practical Example: The Automotive Manufacturer
Consider a high-end car manufacturer facing competitive pressure from newer entrants. To improve the buying experience it launches a modern, multi-platform application—built on DevOps principles—that lets customers browse models, build configurations, compare prices, and book test drives.
The application is a success on its own terms. But the data it collects lives in isolation from the manufacturer’s legacy dealership CRM, billing records, and production-planning databases. As a result:
- A customer who books a test drive via the new app is treated as a stranger, even though he was a loyal lease-hire customer for several years before going silent.
- Personalisation opportunities are missed because the richer historical context—tenure, past models leased, preferred dealership—is locked in the legacy systems.
- Sales and marketing teams cannot build a genuine 360-degree view of any customer.
This scenario is not unusual. It illustrates why new applications cannot live independently from legacy systems if the business is to extract real value from its DevOps investment.
Two Approaches to Bridging the Gap
When it comes to integrating mode-one and mode-two data, organisations typically consider two broad approaches:
1. Process-Centric Integration
Build ETL pipelines or event-driven processes that copy data from new applications into legacy systems (or vice versa), keeping both silos roughly in sync. This is achievable but has significant limitations:
- Data still lives in separate silos; you are just moving copies of it between them.
- Cross-silo queries and transactions are complex and brittle.
- Each new application or data source requires additional integration work.
- Latency is often high, so real-time use cases are difficult to support.
2. Data-Centric Integration via an Operational Data Hub (ODH)
An operational data hub aggregates data from both legacy and modern systems into a single logical layer. New DevOps applications write their data directly to the hub. Legacy system data is virtualised or replicated into the hub on demand. The result is a unified, queryable view of all organisational data.
Benefits of this model include:
- Unified access: Teams query a single hub rather than connecting to multiple disparate systems.
- Flexibility: New data sources can be onboarded without redesigning the integration layer from scratch.
- Transition safety: Legacy systems remain operational while the hub provides a stable API surface for modern applications, easing the eventual retirement of those legacy systems.
- Cross-silo insight: Because all data coexists in the hub, analytical queries can span the old and new domains—enabling the 360-degree customer view that was previously impossible.
The Role of the Operational Data Hub as a “Mode 1.5” Midpoint
One underappreciated benefit of the ODH model is what it does for organisational knowledge transfer. In many large enterprises there is a widening skills gap between:
- Mode-one teams who are expert in COBOL, mainframes, SAP, and similar technologies—and who are approaching retirement.
- Mode-two teams who are proficient in JavaScript, microservices, and agile frameworks but have limited exposure to the complexity of production-grade legacy systems.
An operational data hub acts as a “mode 1.5” layer: it exposes legacy data through modern APIs, so that newer engineering teams can consume it without needing deep expertise in the underlying legacy platforms. This buys the organisation time to manage the transition gracefully rather than being held hostage by institutional knowledge that is walking out the door.
Choosing the Right Database for an Operational Data Hub
An ODH built on an underpowered database will create as many problems as it solves. When evaluating enterprise NoSQL databases for this role, prioritise the following capabilities:
ACID Transactions and High Availability
Not all NoSQL databases support full ACID (Atomicity, Consistency, Isolation, Durability) transactions. For an operational data hub that must reliably coordinate reads and writes across multiple data sources, ACID compliance is essential. Similarly, look for databases that offer strong high-availability guarantees, horizontal scalability, and tested disaster recovery capabilities.
Security and Privacy
Legacy systems were often siloed enough to avoid being directly exposed to external threats. An ODH changes that posture—it brings together sensitive data from multiple sources and exposes it via APIs. Security requirements to verify in any candidate database include:
- Role-based access control at the field and document level.
- Encryption at rest and in transit.
- Audit logging sufficient for regulatory review.
- Support for compliance obligations such as GDPR, including the ability to locate and redact personally identifiable information (PII) on request.
Search, Semantics, and Geospatial Capabilities
A data hub is only valuable if the data within it can be surfaced quickly and meaningfully. Look for databases that provide:
- Full-text search across all stored data types, with relevancy ranking.
- Semantic query support to surface relationships and patterns that are not obvious from structured queries alone.
- Geospatial indexing for applications that need to filter or analyse data by location—increasingly common in customer-facing and logistics use cases.
Bitemporal Data Management
In regulated industries such as financial services, the ability to query data as it existed at a specific point in time—both in the real world and in the system’s own transaction history—is a regulatory requirement. Bitemporal database support addresses this by maintaining two independent time axes for every record, enabling auditors and compliance teams to reconstruct the state of any data at any historical moment without relying on backup restoration.
Key Takeaways for DevOps and Platform Teams
- DevOps adoption in large enterprises does not automatically reduce data silos—without a deliberate data strategy it can increase them.
- Replacing legacy systems wholesale is rarely feasible in the short term; integration is the practical path forward.
- An operational data hub—backed by a capable enterprise database—provides a pragmatic “mode 1.5” integration layer that serves both legacy consumers and modern DevOps applications.
- Database selection should be driven by operational requirements: ACID compliance, security, search, and temporal data support matter as much as raw write throughput.
- The ODH model also helps manage the organisational skills transition as experienced legacy engineers retire and DevOps-native engineers take over.
Two-speed IT is not a temporary inconvenience—it is the operating reality for most large organisations for the foreseeable future. Building the right data integration layer now is what determines whether your DevOps transformation delivers compound value or simply adds another silo to the pile.