Google Support for Spinnaker Doubles in the Last Year

Matt Duftler, Software Engineer at Google who works full time on Spinnaker, speaking at the recent GoogleCloudNext in San Francisco, assured the community of Google’s commitment to Spinnaker, which has doubled in size over the last year.  He is one of a dedicated team of 16 focused on supporting customers, OSS build/test/release, the internal usage of Spinnaker and more.  

Duftler was joined on stage with  Andy Glover, Director of Netflix’s Delivery Engineering team and Tom Feiner, Infrastructure Team Lead at Google working on the Waze app.

This team of heavyweights in Google’s CI/CD space let us in on what’s new and what’s next with the Spinnaker open source project.  Feiner covered how Spinnaker is the secret sauce to Waze’s success, and Glover talked about Spinnaker’s origins, how it makes things move faster and what’s next at Netflix.

Google’s involvement with Spinnaker started in 2014 with one engineer adding to the community as his 20% project (each Google engineer is allowed 20% of their time allowed to an outside project of their choosing).  They have been steadily adding more support since then.  

Spinnaker aligns with Google’s internal best practices, said Duftler, noting that  “convergence between internal tools, open-source and cloud offerings is a good thing,”

Not surprisingly, data control is a big issue at Google.  They have over a billion files, tens of millions of commits, and tens of thousands of software developers, all working out the same giant repository.

Spinnaker provides a consistent experience across multiple cloud plates (including the Borg which is the precursor to kubernetes), and it abstracts away details developers should not need to care about.  Another advantage is the faster on-boarding of Nooglers (er, new Google employees).  

The Spinnaker team at Google supports customers and users, and not just Google users.  As a part of the open source community, they answer questions on the Slack channel and have been building projects that are fully open source.

They’ve found the community welcoming and helpful, Duftler said. His team is putting energy into understanding the code base itself.  “It’s been kind of a tough nut to crack,” Duftler said, “so we want to figure out how to make it easier for other folks to get involved.”  

The community as a whole has been writing developer guides and doing workshops for this and Google has put significant energy behind this community effort. They’ve been contributing a lot of administrative tasks, less sexy but critical to the future success of the project. One avenue is formalizing processes and procedures within the Spinnaker community itself, for example, how people can become committers.  His team is also adding best practices and smarts into the platform as they go to share back what they’ve learned.

Other recently released projects:

  • Built a large-release workflow for building, testing and releasing Spinnaker itself, the project is a big, complicated project, now have a process, including version control.
  • Built Halyard, a tool to make Spinnaker easier to install and manage. Spinnaker is not easy, he said. It’s large and complex and requires a sophisticated platform. Halyard helps, which in turn makes the life of Spinnaker users easier.
  • An automated canary analysis service, called Kayenta, which works with all cloud platforms supported by Spinnaker.
  • An extensive set of metrics service plug ins for working with companies working on top of Spinnaker, including Armory.

What’s Next

In addition to continuing to grow their robust community, Spinnaker has some exciting plans to continue to grow. They are:

  • Adding a set of supported cloud platforms next steps
  • Providing comprehensive support for artifacts: instead of imbedding artifacts deep into the code on processes like understanding what comes from Jenkins or from a container registry, externalize that information, making it more consumable and easier to find
  • Adding comprehensive provisioning support for canary stages and orchestration
  • Making it easier to configure the canary release
  • Broadening the capabilities for Spinnaker Command Line Interface (CLI), which currently lets you programmatically publish and interact with pipelines

But wait, there’s more  

There are two exciting new initiatives for providing an alternative SQL backend for the workflow engine and providing support for configuration as code.

Adding support for a SQL backend as an alternative for in-flight workflow representation and provide more durability and other capabilities is a long-term project.  It’s inside Netflix for the time being but is on the roadmap for release.

There are several streams of works underway, Duftler said, involving configuration as code, from manifest deployment to approaches where you define the desired state, and it’s constantly trying to achieve that state on some schedule.

“Think of it as a spectrum from ‘I do everything with a UI and use a wizard to fill in fields’ to ‘I check in a manifest somewhere, whether its a support control system or a system that buckets and then Spinnaker tries to achieve that state,’” he explained.