4 Tips on How to Ship Software Fast Like Facebook
It’s humbling to attend a conference like OSCON. It acts as an immediate reminder of how much you don’t know about software engineering, and offers a fantastic means of broadening your knowledge in unfamiliar areas. On the other hand, it’s also comforting to go deeper into challenges every software engineer faces, regardless of language, framework, or tooling. One of those topics is how to ship software fast.
Andrei Alexandrescu, a well known author and speaker, gave a great presentation on moving and shipping software fast. Specifically, he focused on practices Facebook leverages to serve customers high quality reliable software at a rapid pace. At Facebook, they deploy changes to their front end twice a day, which is pretty remarkable for an application that serves billions of users. Here’s some of the tips he shared on how to ship software fast:
1. Every line of code should be reviewed by another developer
In order to ship software fast, the first practical tip we should implement is code reviews. Kent Beck states in Extreme Programming Explained that often times the code that causes us the most trouble is the code that wasn’t reviewed. Andrei’s point serves an important reminder that our development process should enforce reviews so they don’t take us back a step.
2. Capture and track metrics for everything you can
The second tip we should consider in order to ship software fast is to capture and track metrics. Metrics serve us in so many ways. They tell us who’s using our system and how, enabling us to write software that better serves our customers. They also tell us how our system is performing, which is critical for recognizing when we’ve broken something.
Andrei spoke of a time when after a deployment, CPU usage dropped dramatically. At first, you might think – big win! We just drastically increased performance! Turns out, only half of Facebook’s pages were getting rendered. Not such a good deal after all.
Dramatic changes in metrics after a deployment are a huge warning sign something is wrong, but the only way you’ll know is if you’re watching.
3. Keep metrics constantly visible
The third tip we should implement if we want to ship software fast is to constantly keep metrics visible. Once you’re capturing those metrics, you want them on display all the time. At Facebook, right above the scrum board was a monitor showing the live metrics. If more of your team sees your products current state more often, you’ll find abnormalities that much quicker. Plus, who doesn’t like a reminder of “oh yeah – we built that!”
4. Release to production incrementally
Lastly, in order to ship software fast we should release to production incrementally. Having redundant production environments is a no-brainer for applications that depend on availability. When it comes to deployment of new features however, the best approach is divide and conquer.
Update a small portion of your production servers at a time so that your load balancer will introduce the changes to a small percentage of your users. Then, track the experience of those users with your metrics. Perform A/B testing, comparing the live metrics of the new experience against live metrics of the old, and you’ll have a good sense if anything is horribly broken.
There’s so much more to this topic and to Andrei’s talk than I can fit here. But what do you think?