Evolutionary Database Design Retrospective
The team I work with has been using Liquibase to manage database refactoring for just over a year now, this is a summary of my impressions on that experience. Everything I write is just as applicable to Flyway or any other migration tool.
So some background... The idea of our PostgreSQL database migrations living along with the code became a big topic around the office about four years ago. Through some staff changes, working with our DBAs to apply migrations was becoming a deep slog through red tape. So right off the bat our target was to simplify deploys and work around DBA policies, not necessary implement any other tenants of continuous delivery, or necessarily adopt the requisite “best practices” around Evolutionary DB Design. Kick ass right.
So the tooling we implemented was simple. Allow a developer to spin-up a local Postgres instance with appropriate customization's using Docker and apply schema and test data through a DB project on the trunk. The developer can build migrations, apply them locally and write simple unit tests to verify them. The migrations and tests are checked in together. A CI job builds the DB project (just applies migrations and runs tests) and is a prerequisite to the code being promoted to an environment just like any other unit test. Deployments apply migrations through Liquibase and version information is available through the Liquibase table in the public schema.
I believe the implementation is good. Some pluses are:
+ Agile teams write and test DB migrations as the team see fit, without any DBA’s on the team. (some might see this as a minus, in my experience it is a plus)
+ Migrations are applied though the deployment pipeline the same way. Any issues typically show up before anything gets close to production.
+ DB’s have a definite version which can be quantified and labeled/branched/released.
+ Databases roll forward!
- Evolutionary DB Design works as a part of a continuous delivery strategy. To adopt one part of continuous delivery without working toward the goal of constantly pushing software complicates some of the design decisions of the tooling. For example:
- No Rollbacks! We don’t do rollbacks. They are impossibly hard to test and unpredictable, we only roll forward. This is an inconceivable concept to to someone who has not embraced agile development methodology. “You are giving the database a version, then why cant I go back to an older version? I don’t get it.” Its been a year plus and I still have to explain this to managers and other new engineers on a monthly basis.
- Long release cycles mean large numbers of migrations, this means longer test cycles and greater risk during deploys. Again, you should be pushing often.
- Developers don’t like to test. Maybe this is just my team but in general the majority of developers feel like writing tests is a waste of time for them. If we did more pairing and XP I don’t think this would not be an issue.
So overall its been very positive. I would suggest NOT adopting this practice as a first step to continuous delivery otherwise it’s pretty neat.