Git/Workflow/Topic

Introduction

This workflow is based on the branchy development workflow documented by git help workflows.

Motivation

The primary goal of this workflow is to make release preparation and maintenance easier. We set the following requirements, of which some are themselves worthwhile goals:

• Amortize the effort of release preparation throughout development
• Support granular selection of features for release
• Allow immature features to be published without delaying release
• Keep unrelated development paths (topics) independent of one another
• Maintain a clean shape of history

Design

The design of this workflow is based on the observation that meeting the highlighted goal makes the other goals easy. It is based on branchy development in which each branch has a well-defined purpose.

We define two branch types:

• Topic Branch
  • Commits represent changes (real work)
  • Distinguished by feature (one topic per feature or fix)
  • Named locally by each developer (describe purpose of work)
  • Heads not published (no named branch on server)
• Integration Branch
  • Commits represent merges (merge topics together)
  • Distinguished by stability (release maintenance, release preparation, development edge)
  • Named everywhere
  • Heads published on server

Notation

This document uses ascii-art to depict commit history:

Branch name	master
Current branch	*master
Commit with parent in same branch	----o
Commit with parent in another branch	\ o
Commit with two parents (merge)	----o /
Commit with arbitrary ancestry	...o

Topic branches generally consist of a linear sequence of commits forked off an integration branch:

...o  master
    \
     o----o----o----o  topic

Integration branches generally consist of a sequence of merge commits:

        ...o      ...o
            \         \
...o----o----o----o----o  master
       /         /
   ...o      ...o

Integration Branches

We use an integration branch for each stage of development:

• maint: Release maintenance (high stability). Only bug fixes should be published here. Only the release manager can push here.
• master: Release preparation (medium stability). Only mature features and bug fixes should be published here.
• next: Development (low stability). New features should be published here.

Development

We cover below the steps to take during each phase of development.

New Topic

Create a new topic branch for each separate feature or bug fix. Always start the topic from a stable integration branch, usually master. If the topic fixes a bug in the current release, use maint. Never start a topic from next! In the following table we review the steps and commands to create, develop, and publish a topic branch based on master.

Actions	Results	Troubleshooting
Update master to base work on the most recently integrated features.
$ git checkout master	...o *master
$ git pull	...o----o *master
Create the local topic branch. Use a meaningful name for "topic". Name topics like you might name functions: concise but precise. A reader should have a general idea of the change to be made given just the branch name.
$ git checkout -b topic	...o----o master ^ *topic
This is where the real work happens. Edit, stage, and commit files repeatedly as needed for your work. During this step, avoid the "urge to merge" from an integration branch. Your changes are safely committed; you can merge your work with others when you are finished.
$ edit files $ git add -- files $ git commit	...o----o master \ o *topic
$ edit files $ git add -- files $ git commit	...o----o master \ o----o *topic
When the topic is ready for publication it must be merged into next. Do not merge from next! It should be the current local branch when you merge. Switch to next and update it. Use "git checkout -b next origin/next" to create a local next the first time.
$ git checkout next $ git pull	...o----o master . \ . o----o topic . ........o *next
Merge the topic and test it.
$ git merge topic	...o----o master . \ . o----o topic . \ ........o----o *next
Finally, publish the change.
$ git push	...o----o master . \ . o----o topic . \ ........o----o *next (and origin/next)	non-fast-forward?

Mature Topic

When a topic is ready for inclusion in the next release, we merge it into master.

Actions	Results	Troubleshooting
Update master to get the latest work by others. We will merge the topic into it.
$ git checkout master	...o----o *master . \ . o----o topic . \ ........o----o next
$ git pull	.......... . \ ...o----o----o *master . \ . o----o topic . \ ........o----o next
Merge the topic and test it.
$ git merge topic	.......... . \ ...o----o----o---o *master . \ / . o----o topic . \ ........o----o next
Delete the local branch.
$ git branch -d topic	.......... . \ ...o----o----o---o *master . \ / . o----o . \ ........o----o next
Finally, publish the change.
$ git push	.......... . \ ...o----o----o---o *master, origin/master \ / o----o	non-fast-forward?

Note that master sees only the topics that have been merged into it. It cannot reach any of the merges into next:

  ..........
 .          \
...o----o----o---o *master
         \      /
          o----o (topic)

Old Topic

Sometimes we need to continue work on an old topic that has already been merged to an integration branch and for which we no longer have a local topic branch. To revive an old topic, we create a local branch based on the last commit from the topic (this is not one of the merges into an integration branch).

First we need to identify the commit by its hash. It is an ancestor of the integration branch into which it was once merged, say next. Run git log with the --first-parent option to view the integration history:

$ git log --first-parent next
commit 9057863...
Merge: 2948732 a348901
...
    Merge branch topicA

commit 2948732...
Merge: 1094687 b235725
...
    Merge branch topicB

commit 1094687...
Merge: 8267263 c715789
...
    Merge branch topicC

Locate the merge commit for the topic of interest, say topicB. It's second parent is the commit from which we will restart work (b235725 in this example).

Actions	Results	Troubleshooting
Create a local topic branch starting from the commit identified above.
$ git checkout -b topic b235725	.......... . \ ...o----o----o---o master . \ / . o----o *topic (b235725) . \ ........o----o----o next / / ...o ...o (c715789) (a348901)
Continue development on the topic.
$ edit files $ git add -- files $ git commit	.......... . \ ...o----o----o---o master . \ / . o----o----o *topic . \ ........o----o----o next / / ...o ...o
$ edit files $ git add -- files $ git commit	.......... . \ ...o----o----o---o master . \ / . o----o----o----o *topic . \ ........o----o----o next / / ...o ...o
When the new portion of the topic is ready, merge it into next and test.
$ git checkout next $ git pull $ git merge topic	.......... . \ ...o----o----o---o master . \ / . o----o----o----o topic . \ \ ........o----o----o----o *next / / ...o ...o
Publish next.
$ git push
Optionally repeat the above, publishing to next, to continue development based on feedback from initial publication. Finally, when the new changes are mature, merge to master and publish.
$ git checkout master $ git pull $ git merge topic	.......... . \ ...o----o----o---o---------o *master . \ / / . o----o----o----o topic . \ \ ........o----o----o----o next / / ...o ...o
$ git branch -d topic $ git push	.......... . \ ...o----o----o---o---------o *master . \ / / . o----o----o----o . \ \ ........o----o----o----o next / / ...o ...o

Again, note that master sees only the topics that have been merged into it. It cannot reach any of the merges into next:

  ..........
 .          \
...o----o----o---o---------o *master
         \      /         /
          o----o----o----o (topic)

Discussion

History Shape

The history graphs produced by this workflow may look complex compared to the fully linear history produced by a rebase workflow (used by CVS and Subversion):

  ..........
 .          \
...o----o----o---o---------o  master
 .       \      /         /
  .       o----o----o----o  topic
   .            \         \
    ........o----o----o----o  next
           /         /
       ...o      ...o

However, consider the shape of history along each branch. We can view it using Git's --first-parent option. It traverses history by following only the first parent of each merge. The first parent is the commit that was currently checked out when the git merge command was invoked to create the merge commit. By following only the first parent, we see commits that logically belong to a specific branch.

Command	View
$ git log --first-parent topic	... \ o----o----o----o topic
$ git log --first-parent master	\ ...o----o----o---o---------o master / /
$ git log --first-parent next	\ \ ...o----o----o----o next / /

Each branch by itself looks linear and has only commits with a specific purpose. The history behind each commit is unique to that purpose. Topic branches are independent, containing only commits for their specific feature or fix. Integration branches consist of merge commits that integrate topics together.

Note that achieving the nice separation of branches requires understanding of the above development procedure and strict adherence to it.

Troubleshooting

Here we document problems one might encounter while following the workflow instructions above. This is not a general Git troubleshooting page.

Trouble Merging

TODO: Write this sub-section and link sub-sub-sections from git merge commands above.

Trouble Pushing

non-fast-forward

When trying to publish new merge commits on an integration branch, perhaps next, the final push may fail:

$ git push
To ...
 ! [rejected]        next -> next (non-fast-forward)
error: failed to push some refs to '...'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes before pushing again.  See the 'Note about
fast-forwards' section of 'git push --help' for details.

This means that the server's next refers to a commit that is not reachable from the next you are trying to push:

...o----o  master
 .       \
  .       o----o  topic
   .            \
    .       .----o *next
     .     .
      ....o----o  origin/next
              /
          ...o  (other-topic)

This is the Git equivalent to when cvs commit complains that your file is not up-to-date, but now it applies to the whole project and not just one file. Git is telling you that it cannot update next on the server to point at your merge commit because that would throw away someone else's work (such as other-topic). There are a few possible causes, all of which mean you have not yet integrated your work with the latest from upstream:

• You forgot to run git pull before git merge so you didn't have everything from upstream
• Someone else managed to merge and push something into next since you last ran git pull

Some Git guides may tell you to just git pull again to merge upstream work into yours. That approach is not compatible with the goals of this workflow. We want to preserve a clean shape of history.

The solution is to throw away your previous merge and try again, but this time start from the latest upstream work:

Actions	Results
$ git reset --hard origin/next	...o----o master . \ . o----o topic . ...o----o *next, origin/next / ...o (other-topic)
$ git merge topic	...o----o master . \ . o----o topic . \ ...o----o----o *next / ...o (other-topic)
Now your next can reach the upstream work as well as yours. Publish it.
$ git push	...o----o master . \ . o----o topic . \ ...o----o----o *next, origin/next / ...o (other-topic)

See git rerere to help avoid resolving the same conflicts on each merge attempt.

first-parent sequence not preserved

One goal of this workflow is to preserve a clean shape of history. This means that a --first-parent traversal of an integration branch, such as master, should see only the merge commits that integrate topics into the branch:

  \         \
...o----o----o----o  master
       /         /

The commits on the individual topic branches are not included in the traversal. This provides a medium-level overview of the development of the project.

We enforce the shape of history on the server's integration branches using an update hook at push-time. Each update must point its branch at a new commit from which a first-parent traversal reaches the old head of the branch:

                       master@{1}
  \         \         \v
...o----D----C----B----A----M  master
       /         / \       /
                    U-----T  (topic)

A first-parent traversal of master from before the update (master@{1}) sees A B C D:

  \         \         \
...o----D----C----B----A  master@{1}
       /         /

A first-parent traversal of master from after the update sees M A B C D:

  \         \         \
...o----D----C----B----A----M  master
       /         /         /

The above assumes correct history shape. Now, consider what happens if merge M is incorrectly made on the topic branch:

                            master@{1}
  \         \              \v
...o----D----C----B---------A
       /         / \         \
                    U----T----M  master
                         ^ (topic)

Now a first-parent traversal of master from after the update sees M T U B C D:

  \         \
...o----D----C----B
       /         / \         \
                    U----T----M  master
                         ^ (topic)

This not only shows details of the topic branch, but skips over A altogether! Our update hooks will reject the push in this case because the new master cannot see the old one in a first-parent traversal.