Mercurial workflows: local development work

The nice thing about distributed version control systems (DVCS) such as Git and Hg is that they both allow me to basically decide how my source control should fit with my short and long-term development workflows.  A while back, I wrote what was basically a stream of consciousness post on getting my Git workflow working in Hg.  Well, a teammate tried to follow those steps…and found that I missed a few important nuggets.

My local workflow revolves around local branches and rebasing.  There are plenty of good articles out there on why this is an interesting and valuable workflow to know and understand, so I won’t rehash all the arguments.  I will say that I like this workflow because it:

  • Gives me a clean, linear, understandable timeline
  • Allows me to keep lines of work separate from each other, until it’s ready to push back upstream
  • Works well in the face of unpredictable work
  • Is light, quick, and does not leak into the public, mainline work

I tried a few other options, such as real branches, patch queues, and so on, but none had the same flavor that I was looking for with local topic branches.  With local topic branches, I don’t use different commands for committing (as I would with patch queues), nor do my branches leak metadata into the public timeline, as it would with normal Hg branches.

First, let’s get our local environment set up appropriately.

Prepping our environment

The cornerstone of my local development workflow include the Rebase and Bookmarks extension.  To enable these extensions, just modify your hgrc file.  You’ll also want to enable tracking the current commit for bookmarks.  This ensures that our bookmark gets moved up with each commit, instead of getting just stuck on one.  Your hgrc file would now include:

[extensions]
rebase = 
bookmarks =

[bookmarks]
track.current = True

I’ve enabled the Rebase and Bookmarks extension, and configured bookmarks to track the current commit.  Tools like TortoiseHg have features that light up when bookmarks are enabled, so you’ll have all sorts of things help you in those tools.

Now that we have our extensions enabled, we need to create our local marker for a master branch.  This bookmark represents the last pushed commit, so you can execute the “outgoing” command to make sure that you have nothing to push:

image

If everything’s good, we’ll create the “master” bookmark:

image

Git will create a “master” branch by default when you clone, but we’ll need to do this manually.  You can think of “master” as trunk.  It represents the mainline of the code we’re working on, and everything will be pulled into this line, both from upstream and from our local branches.  Our repository explorer, as seen in TortoiseHg, now looks like:

image

Note here that “master” is orange.  This indicates that “master” is the current bookmark being tracked.  Now that we have our local repository set up, we can walk through making local changes.

Scenario 1: One local branch, no upstream changes

First, let’s create a local topic branch and make some changes:

  • hg bookmark SomeTopic
  • –work work work
  • hg commit –Am “Simple Change”
  • – work work work
  • hg commit –Am “Some other change”

At this point, we decide we want to push our changes up.  We first want to synchronize our local repository with upstream, but we want to do this on master.  So we:

  • hg checkout master
  • hg pull –rebase

We switch back to the master branch and synchronize with upstream.  But since there aren’t any upstream changes, we want to now fold our SomeTopic branch back to master.  Here’s what the picture looks like right now:

image

Since there are no other local branches, we follow a special workflow as Mercurial’s rebase extension does not do a fast-forward merge by default.  That is, if I tell Hg to rebase or merge SomeTopic, I really just want to move master up to SomeTopic, and not perform some merge.  So I:

  • hg checkout SomeTopic
  • hg bookmark -f master
  • hg bookmark -d SomeTopic
  • hg push -b master

I switch back to the SomeTopic branch, and move the master bookmark up to SomeTopic.  I could have done this with “hg bookmark –f master –r SomeTopic”, but I want to switch to SomeTopic instead.  I delete the SomeTopic bookmark, as “master” is now moved up to SomeTopic.  Finally, I push master, and ONLY master up.  I don’t want any other local topic branches to get pushed until they’re integrated with my mainline master branch.  When we’re finished, this is what our local repository looks like:

Scenario 2: Need to work on unrelated items but not push unfinished work

aka, the whole reason for topic branches.  In this case, we’ve created our local topic branch, but now some other work comes in that we need to do.  We need to work on something else unrelated to our feature work, and don’t want to push the feature work until it’s done.  Our local repository first starts out like this:

image

Just to review what we’re seeing here, the “master” bookmark points to the last pulled commit from upstream.  We’re currently on the TopicOne bookmark, indicated here because it’s orange.  The two up arrows indicate that I have not yet integrated and pushed my TopicOne branch.  You can also execute “hg bookmark” at the command line to view the bookmarks and your current tracked one:

image

So we’re working on TopicOne, which might represent some feature we’re working on.  Some other work comes up, maybe it’s to fix some CSS or a batch script that has a higher priority than this feature.  But we don’t want to push our TopicOne changes yet, it’s not ready to deploy, the tests are broken, it’s just not finished.  So, we’ll start a new topic by:

  • hg checkout master
  • hg bookmark TopicTwo

At this point we can start committing as need be:

  • work work work
  • hg ci –Am “Critical work”
  • work work work
  • hg ci –Am “More critical work”

Once we’ve done that first commit, Hg will tell you that a new head was created.  This is because we first switched back to master, created a new bookmark, and started committing.  Here’s what our repository looks like right now:

image

Again, master sits back as our last pulled commit.  It’s the critical placeholder that helps us know where to start new topic branches from.  It’s not required, as I could look at these arrows to know what the last pulled commit was to start from, but it’s a lot easier when doing rebases and merges.

Now that TopicTwo is finished, I want to integrate TopicTwo into master and push it back upstream.  Because master is a direct ancestor of TopicTwo, I only need to follow the Scenario #1 workflow:

  • hg co master
  • hg pull –rebase
  • hg co TopicTwo
  • hg bookmark –f master
  • hg bookmark –d TopicTwo
  • hg push –b master

It’s very important that I only push master, as that lets my local repository now look like:

image

Now that my critical work is done, I can go back to working on TopicOne:

  • hg co TopicOne
  • work work work
  • hg ci –Am “Finishing work on a feature”

Once I do this, my local repository looks a little changed now:

image

Here we see the master branch hanging off to the side, and my un-pushed changes in the TopicOne timeline.  But now master is no longer in the ancestry of TopicOne.  Now that I want to integrate TopicOne into master, I can either merge this branch, or rebase it.  I prefer rebase, so I’ll follow the normal rebase workflow.  But first, whenever we’re about to push changes, we ALWAYS:

  • hg co master
  • hg pull –rebase

Now that we’re sure we have the latest and greatest, we can rebase our TopicOne onto master:

  • hg co TopicOne
  • hg rebase –b TopicOne –d master

We switch to the TopicOne branch, then rebase from the base of TopicOne to the destination of master.  This command replays the commits from TopicOne onto master, then deletes the TopicOne commits.  Because the timeline changes, these are entirely new commits with new hashes, but containing the exact same changes/commit messages/commit times:

image

Even though I committed two of my changes before the original TopicTwo branch, after a rebase, these commits show up after the TopicTwo.  This is because a rebase replays the commits one at a time on top of the destination (master).  At this point, I can run the build to make sure everything works, and then follow the normal workflow when master is a direct ancestor of my topic branch, skipping the steps of pulling (we already did that):

  • hg bookmark –f master
  • hg bookmark –d TopicOne
  • hg push –b master

Finally, here’s what my repository looks like:

image

Because I’ve always rebased, no one ever needs to know about my topic branches until I integrate.  The pushed timeline is always a clean, linear progression for the mainline master branch (in this case, it’s “dev” as the actual Hg branch).  With topic branches, each topic is independent of each other, and I decide when that topic is ready to be integrated into the mainline.  I might never integrate back, and switching topic branches is a VERY VERY fast “hg checkout” command away.  This workflow is fast, cheap, flexible, and allows me to have one working directory and one repository that contains all the work I’m doing, no matter what its state.

When working with other people, it’s inevitable that file conflicts arise.  In the next post, I’ll dive in to how to deal with conflicts along each step of the way and how this workflow functions in a team environment.

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Jimmy Bogard

I'm a technical architect with Headspring in Austin, TX. I focus on DDD, distributed systems, and any other acronym-centric design/architecture/methodology. I created AutoMapper and am a co-author of the ASP.NET MVC in Action books.
This entry was posted in Mercurial. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://kevin-berridge.blogspot.com Kevin Berridge

    This seems like this is more work than doing the standard hg practice of creating local clones to represent your topic branches. What do you prefer about having one local working directory with manually tracked bookmarks over having multiple local working directories?

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Kevin

    Local clones I think are a kludge, as they don’t allow a singular view into the entire repository. A single repository is easier to manage than trying to remember what folder is in what state. I can look at the repository browser and know exactly what state every topic branch is in, from exactly one view! This is the standard git way of working, and as fast as local cloning is, hg checkout and hg bookmark are much, much faster.

    Seriously, one of the main purposes of a local repository is to enable this exact scenario. If that’s the standard Hg practice, it’s a workaround. I shouldn’t have to manage folders AND repositories. Just one repository is fine with me.

    I think if you worked out the actual steps for folder-based topic branches, you’ll find that it’s more work and less flexible. However, some folks prefer a more physical than logical separation of topic branches, that’s more of a preference thing.

  • http://www.sullivansoftdev.com/blog Brian Sullivan

    This post clarified the benefits of this workflow for me a bit. Thanks for (re)explaining it!

    I did have a question, though. What does the ‘force’ option on the bookmark command do? I couldn’t find anything about it in the documentation of the extension.

  • Ike

    For clarity:
    Bookmarks always move on commit

    “Track.Current = True”
    makes that only the current bookmark moves forward (if you would have several bookmarks pointing at the same changeset)

  • ike

    @Brian:

    -f is used because to force the creation of a “new” bookmark.
    If you don’t use the “-f” hg will complain that it can’t create the bookmark because it allready exists.

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Ike

    Thanks for the clarifications!

  • DaRage

    I disagree that local repositories are a “workaround”. I think, and I could be wrong, that they’re essential part of Hg design and DVCS in general.

    Local repositories and the utilization of file system are the big advantage that DCSV have over central ones because by using simple, known and ubiquitous notions of file system (copy, rename, del) and adding a couple of notions of version controls systems (like commit, merge) you can accomplish all you need without the complexities of a central version control system. It’s a powerful and genius unity. Breaking this unity by discarding the file system is disadvantage and misunderstanding. The argument of having a central repository contradicts the usage of a distributed version control, literal. You simply don’t need a central repository and you shouldn’t care. Understanding this will save you a lot of key strokes and brain cycles.

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @DaRage

    Right now I have 4 topic branches locally that have not been integrated back to trunk. Why would I want the additional complexity and maintenance overhead of putting these in separate folders?

    With git, doing folder-based topic branches is pretty much unheard of. This is because local branches are a first-class citizen, making the need for separate folders moot. I do have to work in a couple of extra steps in Hg for local topic branches versus Git, but I can now manage exactly one repository, versus one repository and folder per branch.

  • DaRage

    @bogardj

    It depends how you measure complexity, but for me copying folders and giving them names is way simpler than the workflow described above, in orders of magnitude.

    It is strange to know that folder-based branches are unheard with git. I have never used git because of its complexity compared to hg, so i can’t speak to that.

    Maybe you should work the hg way instead of forcing a git workflow that is foreign to it.

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @DaRage

    Git’s workflow is simpler because this is all built in. Local branches are “the way” to work, and are not an afterthought as it is in Hg.

    Maybe I have your workflow wrong. To start new work, the first thing I do is create a new bookmark. This would mean in Hg land, clone into a new folder. Bookmark = 1 very cheap step; new folder+clone = two steps, cheap because it’s local, but not close to as cheap as bookmark.

    Doing normal work is the same. OK, now additional work comes in, and I have to abandon that folder for now. Do the same, create new folder, clone from the “master/trunk” folder. With topic branches, it’s checkout master, bookmark. Because it’s a checkout, it’s cheaper than a clone.

    Now I want to fold changes from the additional work back in. So you pull and merge/rebase not from local trunk, but from server trunk, make sure everything works, and push it back out. Here local branches in Hg is more complicated, as the extensions don’t support a fast-forward merge as Git does. Bookmarks adds 1-2 steps here.

    This is the exact workflow I used with SVN. We had feature branches, and created a folder per feature, and had one folder for trunk. Switching between branches meant loading up a new solution from a different folder. With Git and Hg, I can now accomplish the exact same workflow, but now all in one local repository (very, very fast) and one local folder.

    Still, the main disadvantage I see with multiple repositories is the conceptual overhead of multiple repositories. With bookmarks, I’m working with commits. With folders, I’m working with file operations. When a bookmark is completed, I just need to delete the bookmark. To delete a folder, it’s more expensive and I have to make sure that everything is closed. With hg checkout, I can move between trunk/topic branches with ease, and not have the extra burden of managing folders.

    There are definitely benefits and drawbacks to both approaches (and others). Going from SVN -> Git -> Hg, I saw the benefits of local topic branches in a single repository in the SVN -> Git change. Although it’s more steps in Hg vs Git, I really like the benefits that a single local repository gives me, from very very cheap local branches to fast switching to a single, unified view of all my local work.

  • DaRage

    It’s good that you brought up a concrete scenario. To start a new work clone the repository. Locally, this is a simple folder copy. Work comes -> another folder copy and give the folder the name of the topic. When finished working on the feature pull from original folder and merge then push back to original folder. super simple. In addition, I don’t have to touch the command line and I can do it all from TortoiseHg interface.

    You mentioned moving back and forth between features and branches very fast. Well this is complicated, why do you like to complicate your life? plus from visual studio there is no way to tell which feature you’re working on when you come from the coffee break. With local folders you can. finally, KISS:)

  • Arnis L.

    Jimmy, sorry for highly subjective question, but is there anything You like in Hg that Git lacks?

  • Ike

    could be that you did this for informational purposes, but

    # hg checkout SomeTopic1
    # hg bookmark -f master
    # hg bookmark -d SomeTopic1

    could be shorter if you don’t really need the checkout (because you’re going to work on SomeTopic2 next) you can also specify “-r” together with “-f”

    hg bookmark -f master -r SomeTopic1
    hg bookmark -d SomeTopic1

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @DaRage

    I’ve done the folder thing for about a year with SVN, I just prefer having a single repository. There are advantages to having one repository over multiple folders, and conceptually I find it easier to manage. Otherwise, the actions are pretty much the same.

    I tend to stay away from TortoiseHg, I work faster from the command line. My mouse is lonely ;)

    @Arnis

    - Better integration with Windows
    - Extensibility model
    - Configuration

    @Ike

    Funny, a teammate just brought that up to me the other day, thanks for the tip!

  • http://blog.caraulean.com Valeriu Caraulean

    > hg push –b master
    Shouldn’t it be “-r” option here?
    master is a bookmark. When trying to run “hg push –b master” Mercurial is saying: “abort: unknown branch master”.

    I’ve tried do this operation with “-r” and it seems to work correctly…
    PS: hg –version: 1.6.1023

    • James

      Would be nice if the blog post could be updated with this. It’s a bit confusing… ;-)

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @DM

    Don’t push them, then. Even when I use named branches, I only push/pull my current branch/bookmark. I have right now a dozen or so branches that have never made it up to the central repo, that no one’s ever seen. Pretty much like how I’d work with git branching really.