A colleague asks “in the distributed and continuous development process, how do you handle the tension between stability and agility?”
You decide how much you value stability versus speed, and I can give you a way to dial in the right amount of stability.
The release process is a series of “test layers”. For example, a change might pass through automated unit tests, static code analysis, code review, automated integration tests, manual feature testing, beta testing where it is released but visible only to specific users, and production monitoring.
This layering gives us a dial that we can move to increase stability. You can increase quality by adding more test layers. You can increase stability by holding features longer before switching them on.
Big batch releases tend to have a lot of bugs that remain “unstable” until the next release, so if you are starting from that method, it’s pretty easy to improve stability.
You can also increase speed by removing test layers — as demonstrated by startups that release directly to production, and then monitor for a need to rollback. This is going to 11 on the speed dial with essentially one test layer — the production monitoring.
When Assembla moved from one release every two weeks to ten releases per day, our product became much more stable. We used the extra capacity to hold features longer in a sort of beta test state and improve them. So, we were releasing better features, less frequently. This is a typical result.