Automated Database Deployments with Spinnaker

When we’re talking about building mature continuous delivery pipelines, it’s pretty easy when we’re talking about moving application code through a deployment pipeline. However, applications connect to databases can be trickier than updating applications.

Database management has traditionally been in the DBA’s realm of responsibilities. When adopting a DevOps culture, this means a developer will sometimes play the role of DBA, along with managing application lifecycles, which can have high a learning curve.

Database changes contain a lot of risks, and can affect a wide range of downstream dependencies. For one, they are more difficult to rollback (the ability to rollback is key to any deployment process), and a bad database update can lead application failure, along with data loss and service downtime. It’s not confidence-inspiring for a developer after such a failure. It’s also been a challenge to automating database deployments, which leads to having database updates becoming a bottleneck in the pipeline.

Before a database can be updated/migrated we’ll need a plan. First we’ll need preserve the current state of the database, ensure we have a plan on going back if everything fails (rollback). Ensure our changes will only change exactly what we need to change (precision). Understand how long the migration will take, and how our apps behaves with its state of the world changing underneath it (backwards/forwards compatibility). If we forget one step along the way, this could mean downtime, or worse, data loss.

The good news? Spinnaker can help. Here’s how.

Orchestration

Database migrations are complex, multi-step processes. For minimal amount of service downtime, our app and database must be changed and rolled out in a specific order. Managing these steps is easily automatable when using a pipelining tool like Spinnaker

Database as code

Your database changes should be written up and stored as code. By doing this, we have the ability to have database changes to be made using the same tools that you use to update the rest of your application. This unlocks the ability to visualize your database migration and leverage no downtime deployments.

Visualization

Spinnaker gives a “single pane of glass” to look at your entire application deployment, including database migrations. Having a complete and historical view of your database migration in the same view as your application rollout makes it easier and safer to make future database updates.

Blue / Green and Canary Deployments

By leverage Spinnaker’s deployment strategies you can test your old and new database migration performance with your old and new application version. Using Automated Canary Analysis, allows you to safely and slowly, rollout your changes to production. Canaries also give you the ability to perform performance/functionality between different versions. By making your deployments safer, we can now deploy more frequently, limiting the scope of affected customers.

Some best practices

Release incrementally, with one feature updated at a time.
Use a framework to handle database version control.
Never make changes directly on the database.
Be very cautious when dropping columns, unused extra bytes are better than losing the ability to rollback.
Always be backward compatible.
Want to learn more? Check out this talk Armory co-founder Isaac Mosquera gave at the Spinnaker summit.