Branch-Per-Feature Source Control. Part 2: How (Theory)

In the first part of my Branch-Per-Feature (BPF) series, I talked about why you would want to use a source control strategy like BPF – what circumstances would warrant such a strategy, what problems it solves, and a little bit of the cost involved. 

I had originally intended to combine the strategy and theory of how to do Branch-Per-Feature into a single post, for my second entry. However, after fleshing out most of the How (Theory) and half of the What and When (Strategy) sections, I decided that the information represented by these two concepts was distinct enough, and large enough, to warrant separate articles. So, this time around, I’m going to walk through the high level, somewhat source control agnostic theory on how to do branch per feature.

For the remainder of this article, I am going to use the word “feature” to represent any work that warrants it’s own branch. This will include features, user stories, bugs, or any other breakdown of work that you may use. The point is not to say that other breakdowns are not correct or valid, but to simplify the language and explanations throughout the explanations.

The next post will cover the strategy of when to branch and merge what.

Branching Your Feature

Creating the branch is the most basic part of the Branch-Per-Feature process. At the point in time that work starts, you create a branch from your main line of development – typically, the trunk or master (which I will call “source”, for the remainder of the article). For example, if we are starting on Feature #12 (F12) right now, we would create a branch for it. The team working on F12 will do all of the work for it on this branch.

image

Sub-Branching Tasks

If a feature team is made up of several persons, you may end up in a situation where you want to break the feature down into individual tasks that each person can work on. This division can be done in a number of different ways with both vertical and horizontal segmentation. You may have all of the team members working on the branch directly, but you may also have some situations where a feature team member has to introduce breaking changes. In these cases, you need to consider how long the breaking change will take and when the other team members will be able to use the changes made. If the breaking change in question is going to take more than a few hours, or if the team member(s) working on the breaking change want to commit often and not worry about clobbering other developers, then we need to consider a sub-branch for their work.

image

Branching for breaking changes is one of the most common scenarios for branching, that I’ve seen in source control usage. We should apply the same logic and principles to sub-branching, when doing Branch-Per-Feature, and allow our team members to use this technique to further help segment the development effort.

We do have to be careful in how far down this path we go, though. If we start creating sub-sub-sub-sub branches, we can easily run into an administrative nightmare and forget which branches are from where and when. If we try to limit our sub-branches down to breaking changes for a feature branch, though, we should be able manage them fairly easily. Each feature team members knows which feature they are working on, so they should know which branch they came from and need to merge back into. 

The Merge Dance: A Single Branch

Whether you are merging a feature branch into the source, or a sub-branch into a feature branch, the process is the same. Furthermore, this is the standard process for merging any branch, whether you are using a branch per feature strategy or not.

If you have ever done the Check-In Dance, the overall process of merging a branch – the Merge Dance – should be a familiar one.

1. Synchronize From The Source

The first step to take is to bring all of the changes that have occurred on the source, up into your branch. This includes resolving any merge conflicts that you may encounter during this process.

image

The synchronization process, including the conflict resolution, is the responsibility of the team that is working on the branch. They need to ensure that all of the changes from the source are correctly applied to the code that they are working on. This may involve some discussion with the people that made the changes on the source, if there are conflicts or extenuating changes that need to be examined.

2. Smoke Test The Branch

Once the source changes have been synchronized into the active branch, the team working on the branch is then responsible for running a smoke test. If you have a unit test or other automated test suite, it would be a good idea to run all of those test suites against the branch. If your automation suite takes hours to run, you want to have a ‘smoke test’ or ‘fast’ suite of tests that can be executed in lieu of the entire suite.

The specific test suite you run is context dependent, of course. You need to look at the changes that are being brought in, to determine the best course of action. Your team’s overall testing strategy becomes a vital part of your branching and merging strategy, at this point. Understanding how to achieve ‘just enough’ testing for the sync into your branch will likely come with experience, through trial and error.

3. Merge Into The Source

After you have smoke tested the branch with the changes form the source, you’re ready to merge into the source. There’s nothing terribly special about this step. It’s just a merge process from your branch into your source. Since, at this point, you have all of the changes from the source in your branch, you should get a merge change set that reflects the real work done in your branch.

image

4. Test The Source

Once again, the suite of tests that you execute is context dependent. If you are merging a sub-branch into a feature branch, you may only need to run the smoke tests on the feature branch. However, if you are merging into the source, you will likely want to run your entire test suite against the system.  Hopefully you have a Continuous Integration server set up that can execute these tests for you. Whether or not you do, though, there is no substitute for the human factor in testing. I highly recommend that you do at least a modicum of exploratory testing for the areas of the system that were modified in the branch. For a small branch, this may only take a minute or two. For a larger feature branch, though, this may necessitate a much more involved and rigorous test plan.

Merging With Multiple Active Branches

Merging a single branch into the source is easy enough. Anyone that has ever worked on a breaking changes branch should be familiar with this process. If you were not familiar with this process, though, I hope that the preceding section has filled in the missing detail for you. A far more interesting challenge, though, is merging with multiple active branches and keeping them synchronized at the right points in time.

Two Active Branches

If we have Feature #12 (F12) in progress and Feature #13 (F13) begins at a later point in time, we will have a source tree similar to this:

image

If the team for F12 is ready to merge before F13 is done, they will follow the standard Merge Dance as outlined above. Once F12 has been merged into the source, it is now their responsibility to inform the collective team of the changes to the source. The F13 team would then be responsible for synchronizing from the source into their branch, and they would follow step 1 and step 2 of the Merge Dance.

image

Why should F13 synchronize once F12 has merged into the source? Because we expect all future work to contain the feature set and functionality from the F12 branch. We would not want the F13 team to merge into the source at a later point in time and blow away the changes made by the F12 team. Additionally, there is a very real possibility that F12 will be affected by the changes being made by F13, or vice-versa.

The F13 team should synchronize from the source as soon as possible. This will help the F13 team stay on top of the integration between F12 and F13, ensuring that they don’t continue forward with changes that will break between the two features. Once synchronized, F13 would continue working on their branch until they have completed their feature. They can then merge their changes into the source with far more confidence that they are not breaking the system.

More Active Branches

Now lets say that Feature #14 (F14) begins after F12 has merged into the source. F13 then merges into the source after F14 is created. To ensure F14 is up to date with all of the latest features and functionality, the same process that I just outlined is followed. After F13 merges into the source, F14 will need to synchronize. This cycle can repeat itself indefinitely, with as many features as you can imagine.

image

A previous team that I was on, was able to continue this process for more than 4 months with no significant issues. In fact, once we got into the groove and really got good at the Merge Dance, it became second nature to the team. Every team member, including the junior developers, were able to do the Merge Dance practically in their sleep.

Long Life Branches

Of course, we don’t always have such a simple cycle. We often work in larger project teams with many smaller feature teams. We sometimes have branches and features that run into trouble or need more work than originally thought. We have small bug fix batches that get branched and merged more frequently, and we generally end up in situations where a single branch will outlive many other branches. In this scenario, the same principles and process still apply. When a branch is created, it is the responsibility of that branch’s team to keep it up to date with the source.

For example, let’s say Feature #15 (F15) was started shortly after F12. F15 turns into a very long running branch for whatever reason, and continues to live on even after F14 is done.

image

Every time another feature is merged into the source, the F15 team is responsible for synchronizing those changes to their branch. There will be three points at which the F15 team needs to synchronize: after F12 merges, after F13 merges, and after F14 merges.

image

In spite of the long running nature of this branch, the process is the same. It only introduces more points at which the F15 team needs to synchronize from the source.

A Complete Picture

The number of active branches that we had in our repository directly dictates the number of merges into the source. With a total of 4 branches, we can see 4 distinct merges. Over the life of those branches, though, we have to account for the number of synchronizations from the source out to the active branches. From this perspective we end up with 5 points in time that we we needed to synchronize to from the source.

image

If it takes an average of 1 hour to perform the Merge Dance (and I hope it doesn’t. That’s a very high estimate, by my own experience), and an average of 30 minutes to synchronize from the source (since the sync process is 2 out of 4 Merge Dance steps), then we have spent a total of 6 1/2 hours managing our branches. If these 4 branches took 1 month (20 business days) total, to complete, then we have spent just over 4% of our time managing the branches.

The advantage that we have gained, with this 4% overhead, though, will hopefully outweigh the cost. I’ve worked on teams where we would spend far more than 1 day a month fighting the broken builds and bad deliveries. In the end, I have seen the cost of Branch-Per-Feature as a reduction in the overall cost of developing new features. (For more discussion on the cost, benefits, and why of Branch-Per-Feature, check out the first article in the series.)

Coming Up Next

I’m hoping that I’ve given a more complete picture of Branch-Per-Feature, at this point. It is by no means a silver bullet, and there will be times when it just doesn’t make sense to add the administrative tasks of branching and merging to a small fix or change. However, the general process is fairly straightforward once you have practiced the Merge Dance a few times.

If you are using a sensible source control system, such as Subversion or Git, then there should not be a need for a ‘source control admin’ position on your team, either. I’ve trained even the most junior of developers – interns from a local college – on branching and merging, with great success. It is not rocket science or black magic. It only requires that you follow the steps that are appropriate for your task at hand.

A future entry in the Branch-Per-Feature series will cover all of the steps of the Merge Dance and the overall process of managing branches, with Subversion specifically. The next entry, though, will be some additional theory and strategy for handling Branch-Per-Feature. I will try to outline the basic pieces of information that you need, to create a branching and merging strategy (what and when) for your specific situations, by giving examples of circumstances in which I have used BPF.


Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Derick Bailey

Derick Bailey is an entrepreneur, problem solver (and creator? :P ), software developer, screecaster, writer, blogger, speaker and technology leader in central Texas (north of Austin). He runs SignalLeaf.com - the amazingly awesome podcast audio hosting service that everyone should be using, and WatchMeCode.net where he throws down the JavaScript gauntlets to get you up to speed. He has been a professional software developer since the late 90's, and has been writing code since the late 80's. Find me on twitter: @derickbailey, @mutedsolutions, @backbonejsclass Find me on the web: SignalLeaf, WatchMeCode, Kendo UI blog, MarionetteJS, My Github profile, On Google+.
This entry was posted in Branch-Per-Feature, Continuous Integration, Git, Management, Principles and Patterns, Smoke Test, Source Control, Subversion, Workflow. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Eyston

    Really enjoyable articles so far, very well done. Thanks.

  • Robert Lewis

    I like this idea. We have 16 developers in our group, and I think we could manage this without much trouble. For bigger groups, you would need to coordinate merging branches into the source branch to avoid conflicts. Apparently at Microsoft, product teams do this sort of thing on a bigger scale, and they take turns merging so than only one team is merging back into the main line at any one time. (http://blogs.msdn.com/bharry/archive/2005/11/12/492198.aspx)

    Thanks. I’m really enjoying this series.

    - Robert

  • https://twitter.com/vanillawafer Jim Martin

    Derick,
    How are you guys handling CI with each branch. Do you have to manually set up a new CI each time you branch?

  • http://twitter.com/damonmorgan Damon

    Derick

    This is a great series. One thing I am interested in – have you tried never synchronizing from the source? If you keep your features small I suspect that this may be waste. If you have tried it, what problems did you find that led you to decide to do regular synchs?

    Also, as Jim asked, how are you handling CI for each branch?

  • Niall Connaughton

    Interesting articles, I’ll be reading the rest of the series as it comes out.

    I find this way of working to be pretty intuitive – ie: dev teams I’ve worked with in the past have just ended up working this way, there was no formal education process required.

    What I find disturbing and frustrating is that it is such a manual process, and that it requires such long articles to explain to people that have not found it the obvious or intuitive way to work with source control.

    Why do we need such lengthy articles to try to explain to us how to use a tool properly? I think the problem here is that modern source control is still just a glorified (and only slightly glorified) text diffing engine not much better than it was 20 years ago.

    If this is the clear and obvious and most productive way of using source control, why doesn’t source control help more? Source control tries to remain agnostic of the other parts of development – managing tasks, organising iterations, releases etc. This is why we’re stuck doing stupid merges and trying to keep things up to date on a manual, text based level.

    I haven’t used Git, but it sounds interesting. I also haven’t used TFS, I’ve heard plenty of complaints about it, but it’s the only source control package I’ve heard of that attempts to bring source control to the rest of the development party.

    Until we have source control that actually has an understanding of the content it’s managing and can use that understanding to guide the user, we’ll be stuck doing this manual stuff and writing lengthy articles to help people find ways of using the tools that works better.

    If you’ve heard of Rico Mariani’s “Pit of Success” (see http://blogs.msdn.com/brada/archive/2003/10/02/50420.aspx),I think current source control is a very long way away from this.

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Jim, Damon

    We do branch-per-feature, and we use Hudson for our build. It’s literally 5 minutes to set up a new build. yes, Hudson is Java, building our .NET app, but it’s so easy to create a new build.

    All of our other mainline trunk builds and deployments are still CCNET for historical reasons.

  • Ike Casteleyn

    I’ve been thinking about this too.
    However I know my collegues and they don’t like to branch. Mainly because you still need to checkout the branch and it takes time (and probably a littel fear of merging).

    Is there any added value for having a branch per feature (the only thing I can think of now is, you can easily see what changes were needed for a feature)

    instead of having

    a branch per team (only works of course if the teams don’t change often). Then you could setup your CI also to this.

    Best regards,
    Ike

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Ike

    SVN switch is your friend. I switch from trunk to branches on one local working copy as I move back and forth.

  • http://kevin-berridge.blogspot.com Kevin Berridge

    I’m curious you’ve encountered either of these issues, and if so, how you’ve handled it.

    What do you do if you land a feature in the trunk and then realize that its not ready? (Saying “that shouldn’t happen” isn’t an answer, it happens, despite the best intentions) Do you just let it hold up your release, or do you try to roll it back (if other people haven’t also landed in trunk), or disable it manually?

    What do you do if you want to merge two feature branches? Suppose you originally thought these two “features” would be independent but you now need to work on them together for any number of reasons. Does SVN force you to merge along the branch lines (like TFS), so your only choice is to merge to trunk, or can you merge between “siblings” (ex: from Feature A’s branch into Feature B’s branch w/o going through the trunk)?

    Thanks,
    Kevin

  • http://www.lostechies.com/members/derick.bailey/default.aspx derick.bailey

    @Kevin,

    Good question – and not an easy one to answer. I’ve been having conversations around that exact issue with my current team leads, and a few other people via email, recently. My Part 3 Strategy post will cover this topic in some detail. I’ll provide different scenarios and issues surrounding the scenarios.

    this issue really is a question of ‘what is your merge strategy?” what do you merge, where and when…

    I’m trying to find time to finish that post and am hoping to have it done this week.

  • charl v

    Very informative!

    This has answered some of my previous questions in part 1. The key is the sync from the trunk. I usually merge into my trunk from the branch without syncing first and this caused some major headaches.

    Can’t wait for the next one :)

  • Nathan Prather

    Great article!

    We use VisualSVN to manage our code. How do you manage SQL Schema changes in the database? Do you branch these off with each feature as well?

    Thanks for the great articles!
    Nathan

  • dandy

    pro screecaster