This article is based on a talk by Simon Sabin, Principal Consultant at Sabin.io, delivered at WinOps 2017 and summarised on InfoQ.
The Core Problem: Databases in a Continuous Delivery Pipeline
Including database changes in a continuous deployment pipeline is one of the harder unsolved problems in DevOps. Unlike stateless application code, databases carry persistent state, shared schemas, and often serve multiple consuming services simultaneously. Making a schema change without coordinating with every upstream and downstream consumer is a fast path to production incidents.
At WinOps 2017, Simon Sabin proposed a practical mental model for managing this complexity: treat your shared database the same way you treat a public API.
Databases as APIs: The Core Analogy
When a team exposes an HTTP API, they accept an implicit contract with consumers: the interface will remain stable, breaking changes will be versioned, and deprecations will be communicated well in advance. The same discipline should apply to shared databases, from the perspective of the database owners.
Database objects such as views, stored procedures, and triggers are the mechanisms that make this possible. They form the stable public surface of the database, allowing the internal physical structure to change without exposing those changes to consuming applications.
Practical Example: Encrypting a Credit Card Table
Sabin illustrated the approach with a migration from storing credit card numbers in plain text to storing them encrypted. Without the API pattern, every application that reads or writes the table must be updated simultaneously—a risky, tightly coupled release. With the API pattern:
- The underlying table is migrated to store encrypted values.
- A view is created that presents the decrypted value to applications still expecting plain text.
- A trigger on the view handles encryption transparently on writes.
- Applications continue to query the view as before and require no immediate changes.
This is a clean backwards-compatible migration. The security improvement ships independently of the application release cycle, and consuming teams can update their code to work directly with the encrypted layer at their own pace—exactly as they would adapt to a deprecated API endpoint.
Bridging the Gap Between Application Developers and DBAs
Sabin identified a cultural tension that many organisations will recognise. Application developers tend to treat databases as dumb storage—a persistence layer they interact with through an ORM. Database administrators, on the other hand, view the database as a repository of critical, irreplaceable business data that demands careful stewardship.
Neither perspective is wrong, but the friction between them slows delivery. The API model provides a shared vocabulary: the database exposes a stable contract, developers code against that contract, and DBAs can evolve the internals without disrupting applications—as long as the contract is honoured.
The ideal is for each application team to own their own database and include a DBA role within the team. In practice this is rarely achievable, making a clear contract between shared database teams and their consumers even more important.
Managing Schema Changes: Small Batches and the Right Migration Tool
When applications themselves require schema changes, Sabin recommends reducing batch size. Achieving reliability and consistency during database migrations is genuinely difficult; smaller, incremental changes that ship alongside the corresponding application code change are far easier to test, roll back, and diagnose than large batches.
On the question of how to apply migrations, Sabin treats two common approaches as complementary rather than competing:
- Gold schema (desired-state / declarative): You declare the target schema and a diffing tool—such as SQL Compare—calculates and applies the delta between current and desired state. This works well for straightforward structural changes.
- Migration scripts (imperative): Tools such as DBDeploy, Flyway, or Redgate ReadyRoll execute versioned, ordered scripts. These are better suited to complex data transformations, multi-step migrations, or cases where the order of operations matters.
Choosing between them is a function of the change type. Teams operating mature pipelines often use both: declarative diffing for schema drift and migration scripts for complex releases.
Operational Recommendations
Beyond the core API analogy, Sabin offered several additional recommendations for teams integrating database changes into their delivery pipelines:
- Design your database to fit your tooling, not the other way around. Structural decisions made to accommodate a particular migration tool are legitimate engineering trade-offs.
- Embed deployment metadata in the database itself. Recording which version of which service last modified the schema—or storing a deployment timestamp—makes it significantly easier to trace and diagnose production issues.
- Treat database changes as first-class pipeline citizens. Running schema migrations through the same CI/CD pipeline as application and infrastructure code naturally produces an audit trail, which aids compliance and change management.
- Apply a public/private data model for security. Expose small, tightly scoped “public” databases—containing only views into approved data—to application services. Keep the underlying “private” databases unreachable by applications directly. This limits the blast radius of a compromise and mirrors the principle of least privilege.
Data DevOps, DataOps, and Database Lifecycle Management
The idea of incorporating database changes into an integrated, continuous delivery process is not new—it has existed under several names. Sabin uses the term Data DevOps. Others use DataOps or Database Lifecycle Management (DLM). The terminology varies; the intent is consistent.
Eduardo Piairo, a fellow speaker at WinOps 2017, framed the goal well:
It’s not about defining a new process for handling database changes, it’s about integrating them in the service lifecycle along with application and infrastructure code.
Eduardo Piairo, WinOps 2017
The analogy to DevSecOps is apt. Just as security practices took time to be genuinely integrated into delivery pipelines rather than bolted on at the end, database operations are on a similar journey. The teams that make the most progress are those that stop treating the database as a special case and apply the same engineering rigour—version control, automated testing, pipeline deployment, contract design—that they already apply to application code.
Key Takeaways
- Use views, stored procedures, and triggers as a stable public contract between the database and its consumers—analogous to an API surface.
- Communicate breaking changes early and maintain backwards compatibility during transition periods.
- Keep migration batch sizes small and ship database changes alongside the corresponding application code.
- Use declarative (gold schema) and imperative (migration script) tools as complements, selecting the right one for each change type.
- Embed deployment metadata in the database for traceability.
- Treat database changes as part of the CI/CD pipeline to gain auditability, consistency, and compliance benefits.
- Apply least-privilege data access patterns using public/private database separation.