Svnmerge.py

svnmerge.py is a tool for automatic branch management. It allows branch maintainers to merge changes from and to their branch very easily, and automatically records which changes were already merged. This allows displaying an always updated list of changes yet to be merged, and totally prevents merge mistakes (such as merging the same change twice).

You can think of svnmerge as the equivalent of yellow sticky notes on a Subversion branch that record which revisions have been merged in already from one or more other branches, plus some useful functionality for viewing, managing and updating that information.

See our feature list for a more detailed description.

Features

• Merge tracking support: the tool remembers which revisions have been already merged, and always does the right thing by default (only merges new revisions).
• Specific support for development branches with a very simple "merge everything" command (does everything automatically).
• Specific support for release branches through cherry-picking: name and merge single revisions or revision ranges.
• List revisions available for merging (revision numbers, logs, or diffs).
• Bidirectional merges: merges changes forth and back between a branch and its head.
• Multiple heads support: merge changes from multiple sources. This is useful, for instance, for the trunk, which usually needs to merge changes from multiple branches.
• Revision blocking: mark some revisions as unwanted in the branch, so that they will never get merged and you can forget about them.
• Merge rollbacks: freely revert merges in case you changed your mind.
• Absolutely commit-free: the user will always have to do the commit by himself, and thus will always have a chance to review svnmerge.py's modifications.
• Commit message suggestions: an informative commit message listing all the logs of the merged revisions is generated in a text file, as a suggestion for a good commit message.
• Manual merges support: if you merged some changes manually, you can inform svnmerge.py to update its merge tracking info.

FAQ

What problem does svnmerge solve? If we have to apply the same change to another branch, subversion already has a method for doing this.

Traditional subversion will let you merge changes, but it doesn't "remember" what you've already merged. It also doesn't provide a convenient way to exclude a change set from being merged. svnmerge.py automates some of the work, and simplifies it. Svnmerge also creates a commit message with the log messages from all of the things it merged.

It sounds like maybe everyone has to use svnmerge if someone is going to get any use out of it... In other words, do manual merges wreck whatever it is svnmerge is trying to keep track of?

Manual merges don't wreck it, but it means that svnmerge doesn't give you quite as much benefit. However, once you've done an svnmerge on already manually propogated changes, the changes that were already propogated will be recorded as "merged" in its state tracking. From then onward, svnmerge won't try to re-merge those changes.

Why is merging the same thing twice a problem? Subversion doesn't seem to care -- the working copy doesn't appear to have any other changes than when I merged it once.

Merging the same change twice isn't usually a problem. It's a problem in two situations:

1. if you don't want a certain change. Then you have to exclude it every single time you do a merge.
2. If your trunk and your branch are both diverging. Developer A makes a change to line 10 in the trunk, and Developer B makes a change to line 10 in the branch. They're conflicting changes. If you do repeated merging, you will have to manually resolve the conflict every time.

When I merge changes from my branch into the trunk and then later merge trunk into the branch, I get the error "Trying to add new property 'svnmerge-integrated' with value 'xxx' but property already exists with value 'yyy'.

Apply the patch discussed here to avoid the problem, and use "svn resolved ." to fix it. Note that there is some controversy around the patch - see this discussion for the details.

How do I migrate from svnmerge.py to Subversion 1.5's Merge Tracking?

Use svnmerge-migrate-history.py to convert the merge history written by svnmerge.py into Subversion 1.5's format.

System requirements

System requirements are down to a bare minimum, to lower as much as possible the bar for using the tool. Currently, you need:

• SVN command line client, version 1.1 or later.
  It might actually work with an older version but it has never been tested. Notice that the command line client is required: so, noticeably, if you use TortoiseSVN, you need to go install the official SVN distribution to use svnmerge.py (and no, this causes no conflicts whatsoever!)
  To verify, open a command prompt and type

  svn --version

  If this does not work, you need to install Subversion before running svnmerge.py.
• Python 2.0 or later.
  This is used to run svnmerge.py directly from its source; another option is to download the binary Windows distribution which does not require an existing Python installation.

Downloads

Link	Description
svnmerge.py trunk version	This is the default development version, which is usually very stable. Get it without worrying too much!
svnmerge.py 1.4-branch version	This is the version shipped with SVN 1.4. It's less updated than the trunk version so it will usually have less features but maybe be a little more stable. Get this if the current trunk version is temporarily broken.
svnmerge.exe Windows executable	Self-contained Windows executable, does not require a Python installation (but still requires SVN command line client). This is updated every once in a while, use --version to compare it with the trunk version. Ask Giovanni Bajo if you need a newer version.

svnmerge.py is maintained within the Subversion repository. There is not a proper release plan or development map, so there are no official releases. svnmerge.py gets shipped with Subversion mostly 'the way it is' when Subversion itself is shipped. Thus, the trunk version is recommended: we believe it to be mostly stable (there is a quite extensive testsuite).

Mailing List

A mailing list has been established. Details are at: http://www.orcaware.com/mailman/listinfo/svnmerge. The mailing list is the place to go for information about active development and new features.

List archives are available at:

• OrcaWare
• GMANE
• marc.info
• Nabble

Development

Patches, bugfixes, and development of svnmerge are all discussed on the mailing list. Patches to svnmerge are also tracked using the 'svnmerge' subcomponent on the Subversion bug tracker.

To submit a patch or bug report, please follow the buddy system as described here, but use the svnmerge mailing list.

Quick Usage Overview

1. Use svnmerge init to intialize merge tracking on a branch directory.
2. Use svnmerge avail to review the revisions available for merging.
3. Use svnmerge merge to merge in some or all available revisions from other branches.
4. Commit the merge changes using svn commit.
5. Return to step 2 and repeat.

Quick tutorials

What follows are two quick tutorials for two common usage cases.

Development branches

This tutorial assumes that you are working on a recently-created development branch, made off the trunk. It also assumes that the development branch has never merged changes from the trunk.

The svnmerge.py command works as a pull operation, so all commands are run from the development branch working copy. This example pulls changes made on the trunk to your development branch.

• Go to the top-level directory of a pristine working copy of the branch (if you're currently in a working copy directory containing trunk, you can switch to the branch using: svn switch url://path/to/branch). This is your development branch. svnmerge.py is meant to always operate in this condition (so let me repeat: top-level directory, no local modifications).
• Initialize merge tracking support:

$ svnmerge.py init

This command will scan the branch history to find out when the branch was created, so to initialize merge tracking support. This needs to be done only once for each branch you want to use svnmerge.py on.

• The svnmerge.py init command added a property to the top level directory. Since svnmerge.py never does a commit, so it's your turn to commit that change to the repository. You can use the handy automatically generated file:

$ svn ci -F svnmerge-commit-message.txt
$ rm svnmerge-commit-message.txt

Or use your favourite commit message.

• It's time to do a merge. To merge everything from the trunk into the branch, it's sufficient to do:

$ svnmerge.py merge

and that's it! Then, review the merge, fix any eventual conflict, and commit. There is a handy commit message listing the logs of all the merged revisions (can be quite long), which many people find useful.

• Repeat the last step whenever you want to merge new changes from the trunk.
• If you want to have a look at what new changes are available to be merged from the trunk, do this:

$ svnmerge.py avail               # show only the revision numbers
$ svnmerge.py avail --log         # show logs of the revisions on the trunk available for merging
$ svnmerge.py avail --diff        # show diffs of the revisions on the trunk available for merging

Release branches

This tutorial assumes that you are working on a recently-created release branch, made off the trunk, in which no changes were previously merged. With release branch, we mean a branch commonly used to stabilize a release: only a few selected changes must be merged from the trunk, the others must be ignored.

The svnmerge.py command works as a pull operation, so all commands are run from the release branch working copy. This example pulls selected changes made on the trunk to your release branch.

• Go to the top-level directory of a pristine working copy of the branch (svn copy trunk release; cd release). svnmerge.py is meant to always operate in this condition (so let me repeat: top-level directory, no local modifications).
• Initialize merge tracking support:

$ svnmerge.py init

This command will scan the branch history to find out when the branch was created, so to initialize merge tracking support. This needs to be done only once for each branch you want to use svnmerge.py on.

• svnmerge.py never does a commit, so it's your turn. You can use the handy automatically generated file:

$ svn ci -F svnmerge-commit-message.txt
$ rm svnmerge-commit-message.txt

Or use your favourite commit message.

Note, however, that svnmerge.py generates the commit message directly from svn client's output, so, on Windows, it'll have an OEM encoding. You need to tell that to svn if your log messages contains non-ASCII characters, otherwise it'll assume an ANSI encoding and the message may be written incorrectly to the repository. Then, instead of the command above you should use:

$ svn ci --encoding IBM850 -F svnmerge-commit-message.txt
$ rm svnmerge-commit-message.txt

To know the current encoding used in the console one can use the chcp command.

• Review changes available on the trunk to be integrated into the branch:

$ svnmerge.py avail               # show only the revision numbers
$ svnmerge.py avail --log         # show logs of the revisions
$ svnmerge.py avail --diff        # show diffs of the revisions

• When you identify specific revision(s) that you want to merge into the release branch, use the following command to do the merge:

$ svnmerge.py merge -r4500,4671,4812

Then review the changes and commit them. The automatically generated commit message (svnmerge-commit-message.txt) quotes the logs of the revisions that you have merged. These revisions will of course disappear from the list shown by svnmerge.py avail.

• When you identify specific revision(s) which are obviously not going to be integrated into the branch, mark them as blocked:

$ svnmerge.py block -r6456-6460,6881

and commit. These revisions will disappear from the list shown by svnmerge.py avail, so that you don't have to re-review them every time.

• Repeat the last two steps ad libitum, until you reduce the avail list to the bare minimum. A well-maintained release branch always has a small avail list of revisions which are still to be reviewed and/or discussed for backport into the branch: all the other revisions are either already merged or blocked.

Merging development branches back to trunk

This tutorial assumes that you have an active development branch, made off the trunk, in which you made changes and want to merge back to the trunk.

The svnmerge.py command works as a pull operation, so all commands are run from the trunk working copy. This example pulls changes made on the development branch to your trunk.

• Go to the top-level directory of a pristine working copy of the trunk (svn co trunk trunk; cd trunk). Again, svnmerge.py is meant to always operate in this condition, top-level directory, with no local modifications.
• Initialize the merge tracking support on the trunk, related to the given branch, using:

  $ svnmerge.py init BRANCH_URL 

  where BRANCH_URL is the full URL of the branch, such as https://server/svn/project/branches/1.0. This needs to be done once per branch you want to merge back. Note: if you are using a version of svnmerge.py older than r22788, you must also specify a revision range (or just upgrade svnmerge.py!). In that case, you need to follow this procedure:

  1. Find out the revision at which the branch was created by executing the following command:

      svn log --stop-on-copy BRANCH_URL
     

     The last revision that appears on the screen should be the one at which the branch was created.
  2. Initialize the merge tracking support on the trunk:

     svnmerge.py init BRANCH_URL -r1-NNNNN

     where NNNNN is the revision found out at the previous step.
• Commit the merge tracking initialization, using:

  $ svn ci -F svnmerge-commit-message.txt
  $ rm svnmerge-commit-message.txt
  

• If you just want to merge the whole branch into the trunk, you can simply do:

  $ svnmerge.py merge --bidirectional

  This command might report an error and tell you to use the -S (aka --head) option to disambiguate; this happens if you are tracking several branches at the same time in the trunk. In that case, just use something like:

  $ svnmerge.py merge --bidirectional -S BRANCH_NAME

  where BRANCH_NAME is a string useful to disambiguate (or the full URL if you prefer).
  --bidirectional is useful if the branch is managed with svnmerge.py as well: in fact, it tells svnmerge.py to perform an additional analysys to automatically ignore changes in the branch which are just merges from the trunk itself (so-called reflected revisions).
• If you want to cherry-pick some changes (or just merge some revision ranges), you can follow the same advices as in the section Release branches. Make sure to always use --bidirectional to all avail and merge command you issue to ignore reflected revisions.
• When you are done with the branch and you want to close it, you can uninitialize merge support from the trunk:

  $ svnmerge.py uninit -S BRANCH_NAME

  . This is useful because it keeps the house clean, reduces the number of available branches for merge, and keep the need of using the -S option at the bare minimum.
• It's also good practice to really close the branch:

  $ svn rm BRANCH_URL -m "branch is now closed"

  Some people don't like to rm the branch for weird reasons, so they to move it to a specific directory holding closed branches.

Handy Usage Tips

Revision lists are specified as comma separated ranges, e.g., 1412-1419,1423,1427,1433-1440. Ranges may overlap or be out of order; svnmerge will automaticaly normalize them. So 1413-1417,1410-1414,1402,1401 is equivalent to 1401-1402,1410-1417.

svnmerge never commits anything; it always leaves that final step to you. Use svn revert -R . if you need to start over. Note however that "svn revert" will not delete any files added by the merge, so you may have to remove some files yourself to get a complete reversion.

svnmerge merge requires that there be no outstanding changes in the branch directory (otherwise you could accidentally "merge" in a change that never existed in the head directory); use the --force flag to override this check.

Merges of discontinuous revision ranges can produce conflicts or omissions; this is an inherent fact of discontinuous merges, for example, if revision X modifies a file that was created in a previous revision Y < X that you haven't merged in yet. Pay special attention to the output from svnmerge (which comes directly from the "svn merge") for any ignored files or other complaints from Subversion; fixing these cases is up to you. Note that an "ignoring" message indicates a real conflict; it's just that the conflicting file doesn't exist in the target branch so Subversion can't mark it with a "C". In such cases, it's probably a good idea to include a comment in the commit message, e.g., "Note: patch to foo/bar.c in r1234 not included".