F3 Technology Partners | Transform your IT Infrastructure
Accelerate Value Delivery with Trunk-based Development
Continuous Integration Best Practice
Table of Contents
Big bang merge
Throughout the history of computer software development, organizations have often managed independent streams of development on a single code base by using “branches”. Multiple streams are often perceived as being necessary in order to allow different individual developers or teams to work on different sets of changes, simultaneously. The sets of changes could be new features, fixing bugs, or refactoring existing code.
Multiple branches offer the appearance of isolation, especially in the early stages of the development cycle. However, they often cause problems down the road, due to the inevitable necessity to combine, or “merge”, all the different branches together. The different new features may not all be ready for release, even though some of the critical bug fixes are. Attempting to merge different branches together inevitably causes conflicts, and unforeseen problems, due to it being the first time they can be tested together. The longer the branches were isolated, the greater chance there is for incompatibilities between them. The term “big-bang merge” is often used to describe this process. A rush to fix any resulting bugs becomes necessary, often with a “code freeze” on new features. Valuable developer time gets wasted. The release date often slips unpredictably.
From the customers’ point of view, they only see the delays in the release. And when the release is finally received, it often has bugs, sometimes major ones, which were never discovered due to the lack of testing after the big merge. Customers end up constantly dealing with a buggy product, and waiting long periods of time for updates to the software.
Resist the urge to branch
Customers (whether external or internal) want to see improvements in the software, at a regular cadence, and without the introduction of new bugs. This can be facilitated by using a trunk-based strategy when developing the software. In a trunk-based development strategy, all developers work in the same branch simultaneously. All new features or bug fixes are developed in the main trunk branch; there are no other branches.
There are many advantages to this strategy, which prevents big-bang merges. Integration between multiple independent changes happens sooner in the development process. This allows them to be tested thoroughly, in smaller chunks, and find any any bugs early. They can be fixed without requiring a code freeze. It allows more time to evaluate whether any bug or bad feature found should be fixed, or redesigned, or even abandoned (thereby cutting any losses). With an effective trunk-based development process, and a full suite of automated tests, in place, teams can deploy new versions of the software to a production environment on a frequent, regular basis: weekly, daily, or even multiple times per day, allowing for quick, “agile” reaction to customer needs, as well as proactive changes, fixes, and refactoring.
But, in the real world?
- “It’s not ready yet.” Developers could be in the middle of their changes and haven’t thought about how a release at this point would affect the system. To alleviate this, developers must always be in the frame of mind that every change needs to be deployable to customers at any time. This can be achieved by not changing existing (known-good) code, only adding new code. If the new code is not “reachable” by the end user, then it can safely be deployed without causing any regression in the software. The changes can still be made accessible to the testing process in the meantime. Using a technique known as “feature flags”, new features can by turned on or off without restarting the system. This allows the software to be tested and deployed, but not impact the customer. Once the feature is complete, then the old code is switched out and the new code is switched in, which then requires only a minimal amount of additional testing.
- “I can’t do that.” Trunk-based development is a newer technique that may take a while to fully be adopted by the software development community. Other branch-based approaches have been the norm for years. Indeed, SCM tools have helped make branching and merging commonplace. This has led developers into a certain, isolated, way of thinking about any set of changes they need to make. Additional training may be necessary to enable them to see that, in fact, it can be done a different way. They can add feature flags, they can leave old code in place, they can add new unreachable features without disturbing existing functionality. These techniques, and others, can be used to allow a trunk-based development environment.
- “That’s not elegant.” Software developers (rightly so) develop code that is simple, straightforward, and easy to maintain throughout time. The term “elegant” is often used to describe code of this nature. Trunk-based development may require legacy code to be left untouched, and duplicate new code added. It may require the addition of “if-then-else” statements that wouldn’t otherwise be necessary. Many of these types of coding could be perceived as “inelegant”. However, developers should understand that the ultimate purpose of the software is to increase the company’s profits; “elegance” is only a means to that end. This problem can be alleviated by the realization that any inelegant, legacy, or duplicated code will (eventually) be removed. This removal can be scheduled appropriately, and doesn’t need to be forced, so it won’t delay deployment. Software developers may need to take a different approach to their development style, one with a wider, DevOps-focused view. Nothing is more elegant than code that is fully tested, working as the customer expects, and delivered timely into the customer’s hands.
Results
In practice, a trunk-based development strategy prevents most bugs from even being deployed. But in the event a bug is discovered after release, it can usually be easily and quickly fixed. This is a “fix forward” strategy, as opposed to a “roll back” one. The end result is the customers see a more stable system, with a steady stream of smaller changes, happening more often, with less bugs, and quicker response to fixes.
