Mercurial: The Definitive Guide

There are a number of reasons why you or your team might want to use an automated revision control tool for a project.

Most of these reasons are equally valid—at least in theory—whether you're working on a project by yourself, or with a hundred other people.

A key question about the practicality of revision control at these two different scales (“lone hacker” and “huge team”) is how its benefits compare to its costs. A revision control tool that's difficult to understand or use is going to impose a high cost.

A five-hundred-person project is likely to collapse under its own weight almost immediately without a revision control tool and process. In this case, the cost of using revision control might hardly seem worth considering, since without it, failure is almost guaranteed.

On the other hand, a one-person “quick hack” might seem like a poor place to use a revision control tool, because surely the cost of using one must be close to the overall cost of the project. Right?

Mercurial uniquely supports both of these scales of development. You can learn the basics in just a few minutes, and due to its low overhead, you can apply revision control to the smallest of projects with ease. Its simplicity means you won't have a lot of abstruse concepts or command sequences competing for mental space with whatever you're really trying to do. At the same time, Mercurial's high performance and peer-to-peer nature let you scale painlessly to handle large projects.

No revision control tool can rescue a poorly run project, but a good choice of tools can make a huge difference to the fluidity with which you can work on a project.

Revision control is a diverse field, so much so that it is referred to by many names and acronyms. Here are a few of the more common variations you'll encounter:

Some people claim that these terms actually have different meanings, but in practice they overlap so much that there's no agreed or even useful way to tease them apart.

If you take a shine to an open source project and decide that you would like to start hacking on it, and that project uses a distributed revision control tool, you are at once a peer with the people who consider themselves the “core” of that project. If they publish their repositories, you can immediately copy their project history, start making changes, and record your work, using the same tools in the same ways as insiders. By contrast, with a centralised tool, you must use the software in a “read only” mode unless someone grants you permission to commit changes to their central server. Until then, you won't be able to record changes, and your local modifications will be at risk of corruption any time you try to update your client's view of the repository.

Many commercial projects are undertaken by teams that are scattered across the globe. Contributors who are far from a central server will see slower command execution and perhaps less reliability. Commercial revision control systems attempt to ameliorate these problems with remote-site replication add-ons that are typically expensive to buy and cantankerous to administer. A distributed system doesn't suffer from these problems in the first place. Better yet, you can easily set up multiple authoritative servers, say one per site, so that there's no redundant communication between repositories over expensive long-haul network links.

Centralised revision control systems tend to have relatively low scalability. It's not unusual for an expensive centralised system to fall over under the combined load of just a few dozen concurrent users. Once again, the typical response tends to be an expensive and clunky replication facility. Since the load on a central server—if you have one at all—is many times lower with a distributed tool (because all of the data is replicated everywhere), a single cheap server can handle the needs of a much larger team, and replication to balance load becomes a simple matter of scripting.

If you have an employee in the field, troubleshooting a problem at a customer's site, they'll benefit from distributed revision control. The tool will let them generate custom builds, try different fixes in isolation from each other, and search efficiently through history for the sources of bugs and regressions in the customer's environment, all without needing to connect to your company's network.

Subversion is a popular revision control tool, developed to replace CVS. It has a centralised client/server architecture.

Subversion and Mercurial have similarly named commands for performing the same operations, so if you're familiar with one, it is easy to learn to use the other. Both tools are portable to all popular operating systems.

Prior to version 1.5, Subversion had no useful support for merges. At the time of writing, its merge tracking capability is new, and known to be complicated and buggy.

Mercurial has a substantial performance advantage over Subversion on every revision control operation I have benchmarked. I have measured its advantage as ranging from a factor of two to a factor of six when compared with Subversion 1.4.3's ra_local file store, which is the fastest access method available. In more realistic deployments involving a network-based store, Subversion will be at a substantially larger disadvantage. Because many Subversion commands must talk to the server and Subversion does not have useful replication facilities, server capacity and network bandwidth become bottlenecks for modestly large projects.

Additionally, Subversion incurs substantial storage overhead to avoid network transactions for a few common operations, such as finding modified files (status) and displaying modifications against the current revision (diff). As a result, a Subversion working copy is often the same size as, or larger than, a Mercurial repository and working directory, even though the Mercurial repository contains a complete history of the project.

Subversion is widely supported by third party tools. Mercurial currently lags considerably in this area. This gap is closing, however, and indeed some of Mercurial's GUI tools now outshine their Subversion equivalents. Like Mercurial, Subversion has an excellent user manual.

Because Subversion doesn't store revision history on the client, it is well suited to managing projects that deal with lots of large, opaque binary files. If you check in fifty revisions to an incompressible 10MB file, Subversion's client-side space usage stays constant The space used by any distributed SCM will grow rapidly in proportion to the number of revisions, because the differences between each revision are large.

In addition, it's often difficult or, more usually, impossible to merge different versions of a binary file. Subversion's ability to let a user lock a file, so that they temporarily have the exclusive right to commit changes to it, can be a significant advantage to a project where binary files are widely used.

Mercurial can import revision history from a Subversion repository. It can also export revision history to a Subversion repository. This makes it easy to “test the waters” and use Mercurial and Subversion in parallel before deciding to switch. History conversion is incremental, so you can perform an initial conversion, then small additional conversions afterwards to bring in new changes.

Git is a distributed revision control tool that was developed for managing the Linux kernel source tree. Like Mercurial, its early design was somewhat influenced by Monotone.

Git has a very large command set, with version 1.5.0 providing 139 individual commands. It has something of a reputation for being difficult to learn. Compared to Git, Mercurial has a strong focus on simplicity.

In terms of performance, Git is extremely fast. In several cases, it is faster than Mercurial, at least on Linux, while Mercurial performs better on other operations. However, on Windows, the performance and general level of support that Git provides is, at the time of writing, far behind that of Mercurial.

While a Mercurial repository needs no maintenance, a Git repository requires frequent manual “repacks” of its metadata. Without these, performance degrades, while space usage grows rapidly. A server that contains many Git repositories that are not rigorously and frequently repacked will become heavily disk-bound during backups, and there have been instances of daily backups taking far longer than 24 hours as a result. A freshly packed Git repository is slightly smaller than a Mercurial repository, but an unpacked repository is several orders of magnitude larger.

The core of Git is written in C. Many Git commands are implemented as shell or Perl scripts, and the quality of these scripts varies widely. I have encountered several instances where scripts charged along blindly in the presence of errors that should have been fatal.

Mercurial can import revision history from a Git repository.

CVS is probably the most widely used revision control tool in the world. Due to its age and internal untidiness, it has been only lightly maintained for many years.

It has a centralised client/server architecture. It does not group related file changes into atomic commits, making it easy for people to “break the build”: one person can successfully commit part of a change and then be blocked by the need for a merge, causing other people to see only a portion of the work they intended to do. This also affects how you work with project history. If you want to see all of the modifications someone made as part of a task, you will need to manually inspect the descriptions and timestamps of the changes made to each file involved (if you even know what those files were).

CVS has a muddled notion of tags and branches that I will not attempt to even describe. It does not support renaming of files or directories well, making it easy to corrupt a repository. It has almost no internal consistency checking capabilities, so it is usually not even possible to tell whether or how a repository is corrupt. I would not recommend CVS for any project, existing or new.

Mercurial can import CVS revision history. However, there are a few caveats that apply; these are true of every other revision control tool's CVS importer, too. Due to CVS's lack of atomic changes and unversioned filesystem hierarchy, it is not possible to reconstruct CVS history completely accurately; some guesswork is involved, and renames will usually not show up. Because a lot of advanced CVS administration has to be done by hand and is hence error-prone, it's common for CVS importers to run into multiple problems with corrupted repositories (completely bogus revision timestamps and files that have remained locked for over a decade are just two of the less interesting problems I can recall from personal experience).

Mercurial can import revision history from a CVS repository.

Perforce has a centralised client/server architecture, with no client-side caching of any data. Unlike modern revision control tools, Perforce requires that a user run a command to inform the server about every file they intend to edit.

The performance of Perforce is quite good for small teams, but it falls off rapidly as the number of users grows beyond a few dozen. Modestly large Perforce installations require the deployment of proxies to cope with the load their users generate.

With the exception of CVS, all of the tools listed above have unique strengths that suit them to particular styles of work. There is no single revision control tool that is best in all situations.

As an example, Subversion is a good choice for working with frequently edited binary files, due to its centralised nature and support for file locking.

I personally find Mercurial's properties of simplicity, performance, and good merge support to be a compelling combination that has served me well for several years.

The best version of Mercurial for Windows is TortoiseHg, which can be found at http://bitbucket.org/tortoisehg/stable/wiki/Home. This package has no external dependencies; it “just works”. It provides both command line and graphical user interfaces.

Lee Cantey publishes an installer of Mercurial for Mac OS X at http://mercurial.berkwood.com.

Because each Linux distribution has its own packaging tools, policies, and rate of development, it's difficult to give a comprehensive set of instructions on how to install Mercurial binaries. The version of Mercurial that you will end up with can vary depending on how active the person is who maintains the package for your distribution.

To keep things simple, I will focus on installing Mercurial from the command line under the most popular Linux distributions. Most of these distributions provide graphical package managers that will let you install Mercurial with a single click; the package name to look for is mercurial.

SunFreeWare, at http://www.sunfreeware.com, provides prebuilt packages of Mercurial.

Mercurial provides a built-in help system. This is invaluable for those times when you find yourself stuck trying to remember how to run a command. If you are completely stuck, simply run hg help; it will print a brief list of commands, along with a description of what each does. If you ask for help on a specific command (as below), it prints more detailed information.

$ hg help init
hg init [-e CMD] [--remotecmd CMD] [DEST]

create a new repository in the given directory

    Initialize a new repository in the given directory. If the given
    directory does not exist, it will be created.

    If no directory is given, the current directory is used.

    It is possible to specify an ssh:// URL as the destination.
    See 'hg help urls' for more information.

options:

 -e --ssh        specify ssh command to use
    --remotecmd  specify hg command to run on the remote side

use "hg -v help init" to show global options

For a more impressive level of detail (which you won't usually need) run hg help -v. The -v option is short for --verbose, and tells Mercurial to print more information than it usually would.

Copying a repository is just a little bit special. While you could use a normal file copying command to make a copy of a repository, it's best to use a built-in command that Mercurial provides. This command is called hg clone, because it makes an identical copy of an existing repository.

$ hg clone http://hg.serpentine.com/tutorial/hello
destination directory: hello
requesting all changes
adding changesets
adding manifests
adding file changes
added 5 changesets with 5 changes to 2 files
updating working directory
2 files updated, 0 files merged, 0 files removed, 0 files unresolved

One advantage of using hg clone is that, as we can see above, it lets us clone repositories over the network. Another is that it remembers where we cloned from, which we'll find useful soon when we want to fetch new changes from another repository.

If our clone succeeded, we should now have a local directory called hello. This directory will contain some files.

$ ls -l
total 0
drwxr-xr-x 3 dongsheng g11n 45 Oct 22 03:28 hello
$ ls hello
Makefile  hello.c

These files have the same contents and history in our repository as they do in the repository we cloned.

Every Mercurial repository is complete, self-contained, and independent. It contains its own private copy of a project's files and history. As we just mentioned, a cloned repository remembers the location of the repository it was cloned from, but Mercurial will not communicate with that repository, or any other, unless you tell it to.

What this means for now is that we're free to experiment with our repository, safe in the knowledge that it's a private “sandbox” that won't affect anyone else.

When we take a more detailed look inside a repository, we can see that it contains a directory named .hg. This is where Mercurial keeps all of its metadata for the repository.

$ cd hello
$ ls -a
.  ..  .hg  Makefile  hello.c

The contents of the .hg directory and its subdirectories are private to Mercurial. Every other file and directory in the repository is yours to do with as you please.

To introduce a little terminology, the .hg directory is the “real” repository, and all of the files and directories that coexist with it are said to live in the working directory. An easy way to remember the distinction is that the repository contains the history of your project, while the working directory contains a snapshot of your project at a particular point in history.

As English is a notoriously sloppy language, and computer science has a hallowed history of terminological confusion (why use one term when four will do?), revision control has a variety of words and phrases that mean the same thing. If you are talking about Mercurial history with other people, you will find that the word “changeset” is often compressed to “change” or (when written) “cset”, and sometimes a changeset is referred to as a “revision” or a “rev”.

While it doesn't matter what word you use to refer to the concept of “a changeset”, the identifier that you use to refer to “a specific changeset” is of great importance. Recall that the changeset field in the output from hg log identifies a changeset using both a number and a hexadecimal string.

This distinction is important. If you send someone an email talking about “revision 33”, there's a high likelihood that their revision 33 will not be the same as yours. The reason for this is that a revision number depends on the order in which changes arrived in a repository, and there is no guarantee that the same changes will happen in the same order in different repositories. Three changes a,b,c can easily appear in one repository as 0,1,2, while in another as 0,2,1.

Mercurial uses revision numbers purely as a convenient shorthand. If you need to discuss a changeset with someone, or make a record of a changeset for some other reason (for example, in a bug report), use the hexadecimal identifier.

To narrow the output of hg log down to a single revision, use the -r (or --rev) option. You can use either a revision number or a hexadecimal identifier, and you can provide as many revisions as you want.

$ hg log -r 3
changeset:   3:0272e0d5a517
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:08:02 2008 +0200
summary:     Get make to generate the final binary from a .o file.

$ hg log -r 0272e0d5a517
changeset:   3:0272e0d5a517
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:08:02 2008 +0200
summary:     Get make to generate the final binary from a .o file.

$ hg log -r 1 -r 4
changeset:   1:82e55d328c8c
user:        mpm@selenic.com
date:        Fri Aug 26 01:21:28 2005 -0700
summary:     Create a makefile

changeset:   4:2278160e78d4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:16:53 2008 +0200
summary:     Trim comments.

If you want to see the history of several revisions without having to list each one, you can use range notation; this lets you express the idea “I want all revisions between abc and def, inclusive”.

$ hg log -r 2:4
changeset:   2:fef857204a0c
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:05:04 2008 +0200
summary:     Introduce a typo into hello.c.

changeset:   3:0272e0d5a517
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:08:02 2008 +0200
summary:     Get make to generate the final binary from a .o file.

changeset:   4:2278160e78d4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:16:53 2008 +0200
summary:     Trim comments.

Mercurial also honours the order in which you specify revisions, so hg log -r 2:4 prints 2, 3, and 4. while hg log -r 4:2 prints 4, 3, and 2.

While the summary information printed by hg log is useful if you already know what you're looking for, you may need to see a complete description of the change, or a list of the files changed, if you're trying to decide whether a changeset is the one you're looking for. The hg log command's -v (or --verbose) option gives you this extra detail.

$ hg log -v -r 3
changeset:   3:0272e0d5a517
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:08:02 2008 +0200
files:       Makefile
description:
Get make to generate the final binary from a .o file.

If you want to see both the description and content of a change, add the -p (or --patch) option. This displays the content of a change as a unified diff (if you've never seen a unified diff before, see Section 12.4, “Understanding patches” for an overview).

$ hg log -v -p -r 2
changeset:   2:fef857204a0c
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:05:04 2008 +0200
files:       hello.c
description:
Introduce a typo into hello.c.


diff -r 82e55d328c8c -r fef857204a0c hello.c
--- a/hello.c	Fri Aug 26 01:21:28 2005 -0700
+++ b/hello.c	Sat Aug 16 22:05:04 2008 +0200
@@ -11,6 +11,6 @@
 
 int main(int argc, char **argv)
 {
-	printf("hello, world!\n");
+	printf("hello, world!\");
 	return 0;
 }

The -p option is tremendously useful, so it's well worth remembering.

When you try to run hg commit for the first time, it is not guaranteed to succeed. Mercurial records your name and address with each change that you commit, so that you and others will later be able to tell who made each change. Mercurial tries to automatically figure out a sensible username to commit the change with. It will attempt each of the following methods, in order:

If all of these mechanisms fail, Mercurial will fail, printing an error message. In this case, it will not let you commit until you set up a username.

You should think of the HGUSER environment variable and the -u option to the hg commit command as ways to override Mercurial's default selection of username. For normal use, the simplest and most robust way to set a username for yourself is by creating a .hgrc file; see below for details.

2.7.1.1. Creating a Mercurial configuration file

To set a user name, use your favorite editor to create a file called .hgrc in your home directory. Mercurial will use this file to look up your personalised configuration settings. The initial contents of your .hgrc should look like this.

	“Home directory” on Windows
	When we refer to your home directory, on an English language installation of Windows this will usually be a folder named after your user name in C:\Documents and Settings. You can find out the exact name of your home directory by opening a command prompt window and running the following command. C:\> echo %UserProfile%

# This is a Mercurial configuration file.
[ui]
username = Firstname Lastname <email.address@example.net>

The “[ui]” line begins a section of the config file, so you can read the “username = ...” line as meaning “set the value of the username item in the ui section”. A section continues until a new section begins, or the end of the file. Mercurial ignores empty lines and treats any text from “#” to the end of a line as a comment.

2.7.1.2. Choosing a user name

You can use any text you like as the value of the username config item, since this information is for reading by other people, but will not be interpreted by Mercurial. The convention that most people follow is to use their name and email address, as in the example above.

	Note
	Mercurial's built-in web server obfuscates email addresses, to make it more difficult for the email harvesting tools that spammers use. This reduces the likelihood that you'll start receiving more junk email if you publish a Mercurial repository on the web.

When we commit a change, Mercurial drops us into a text editor, to enter a message that will describe the modifications we've made in this changeset. This is called the commit message. It will be a record for readers of what we did and why, and it will be printed by hg log after we've finished committing.

$ hg commit

The editor that the hg commit command drops us into will contain an empty line or two, followed by a number of lines starting with “HG:”.

This is where I type my commit comment.

HG: Enter commit message.  Lines beginning with 'HG:' are removed.
HG: --
HG: user: Bryan O'Sullivan <bos@serpentine.com>
HG: branch 'default'
HG: changed hello.c

Mercurial ignores the lines that start with “HG:”; it uses them only to tell us which files it's recording changes to. Modifying or deleting these lines has no effect.

Since hg log only prints the first line of a commit message by default, it's best to write a commit message whose first line stands alone. Here's a real example of a commit message that doesn't follow this guideline, and hence has a summary that is not readable.

changeset:   73:584af0e231be
user:        Censored Person <censored.person@example.org>
date:        Tue Sep 26 21:37:07 2006 -0700
summary:     include buildmeister/commondefs. Add exports.

As far as the remainder of the contents of the commit message are concerned, there are no hard-and-fast rules. Mercurial itself doesn't interpret or care about the contents of the commit message, though your project may have policies that dictate a certain kind of formatting.

My personal preference is for short, but informative, commit messages that tell me something that I can't figure out with a quick glance at the output of hg log --patch.

If we run the hg commit command without any arguments, it records all of the changes we've made, as reported by hg status and hg diff.

	A surprise for Subversion users
	Like other Mercurial commands, if we don't supply explicit names to commit to the hg commit, it will operate across a repository's entire working directory. Be wary of this if you're coming from the Subversion or CVS world, since you might expect it to operate only on the current directory that you happen to be visiting and its subdirectories.

If you decide that you don't want to commit while in the middle of editing a commit message, simply exit from your editor without saving the file that it's editing. This will cause nothing to happen to either the repository or the working directory.

Once we've finished the commit, we can use the hg tip command to display the changeset we just created. This command produces output that is identical to hg log, but it only displays the newest revision in the repository.

$ hg tip -vp
changeset:   5:c7a9f12460e4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
files:       hello.c
description:
Added an extra line of output


diff -r 2278160e78d4 -r c7a9f12460e4 hello.c
--- a/hello.c	Sat Aug 16 22:16:53 2008 +0200
+++ b/hello.c	Thu Oct 22 03:28:02 2009 +0000
@@ -8,5 +8,6 @@
 int main(int argc, char **argv)
 {
 	printf("hello, world!\");
+	printf("hello again!\n");
 	return 0;
 }

We refer to the newest revision in the repository as the tip revision, or simply the tip.

By the way, the hg tip command accepts many of the same options as hg log, so -v above indicates “be verbose”, -p specifies “print a patch”. The use of -p to print patches is another example of the consistent naming we mentioned earlier.

To get started, let's clone our original hello repository, which does not contain the change we just committed. We'll call our temporary repository hello-pull.

$ cd ..
$ hg clone hello hello-pull
updating working directory
2 files updated, 0 files merged, 0 files removed, 0 files unresolved

We'll use the hg pull command to bring changes from my-hello into hello-pull. However, blindly pulling unknown changes into a repository is a somewhat scary prospect. Mercurial provides the hg incoming command to tell us what changes the hg pull command would pull into the repository, without actually pulling the changes in.

$ cd hello-pull
$ hg incoming ../my-hello
comparing with ../my-hello
searching for changes
changeset:   5:c7a9f12460e4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

Bringing changes into a repository is a simple matter of running the hg pull command, and optionally telling it which repository to pull from.

$ hg tip
changeset:   4:2278160e78d4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:16:53 2008 +0200
summary:     Trim comments.

$ hg pull ../my-hello
pulling from ../my-hello
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
(run 'hg update' to get a working copy)
$ hg tip
changeset:   5:c7a9f12460e4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

As you can see from the before-and-after output of hg tip, we have successfully pulled changes into our repository. However, Mercurial separates pulling changes in from updating the working directory. There remains one step before we will see the changes that we just pulled appear in the working directory.

Pulling specific changes

It is possible that due to the delay between running hg incoming and hg pull, you may not see all changesets that will be brought from the other repository. Suppose you're pulling changes from a repository on the network somewhere. While you are looking at the hg incoming output, and before you pull those changes, someone might have committed something in the remote repository. This means that it's possible to pull more changes than you saw when using hg incoming. If you only want to pull precisely the changes that were listed by hg incoming, or you have some other reason to pull a subset of changes, simply identify the change that you want to pull by its changeset ID, e.g. hg pull -r7e95bb.

We have so far glossed over the relationship between a repository and its working directory. The hg pull command that we ran in Section 2.8.1, “Pulling changes from another repository” brought changes into the repository, but if we check, there's no sign of those changes in the working directory. This is because hg pull does not (by default) touch the working directory. Instead, we use the hg update command to do this.

$ grep printf hello.c
	printf("hello, world!\");
$ hg update tip
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ grep printf hello.c
	printf("hello, world!\");
	printf("hello again!\n");

It might seem a bit strange that hg pull doesn't update the working directory automatically. There's actually a good reason for this: you can use hg update to update the working directory to the state it was in at any revision in the history of the repository. If you had the working directory updated to an old revision—to hunt down the origin of a bug, say—and ran a hg pull which automatically updated the working directory to a new revision, you might not be terribly happy.

Since pull-then-update is such a common sequence of operations, Mercurial lets you combine the two by passing the -u option to hg pull.

If you look back at the output of hg pull in Section 2.8.1, “Pulling changes from another repository” when we ran it without -u, you can see that it printed a helpful reminder that we'd have to take an explicit step to update the working directory.

To find out what revision the working directory is at, use the hg parents command.

$ hg parents
changeset:   5:c7a9f12460e4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

If you look back at Figure 2.1, “Graphical history of the hello repository”, you'll see arrows connecting each changeset. The node that the arrow leads from in each case is a parent, and the node that the arrow leads to is its child. The working directory has a parent in just the same way; this is the changeset that the working directory currently contains.

To update the working directory to a particular revision, give a revision number or changeset ID to the hg update command.

$ hg update 2
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg parents
changeset:   2:fef857204a0c
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Sat Aug 16 22:05:04 2008 +0200
summary:     Introduce a typo into hello.c.

$ hg update
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg parents
changeset:   5:c7a9f12460e4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

If you omit an explicit revision, hg update will update to the tip revision, as shown by the second call to hg update in the example above.

Mercurial lets us push changes to another repository, from the repository we're currently visiting. As with the example of hg pull above, we'll create a temporary repository to push our changes into.

$ cd ..
$ hg clone hello hello-push
updating working directory
2 files updated, 0 files merged, 0 files removed, 0 files unresolved

The hg outgoing command tells us what changes would be pushed into another repository.

$ cd my-hello
$ hg outgoing ../hello-push
comparing with ../hello-push
searching for changes
changeset:   5:c7a9f12460e4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

And the hg push command does the actual push.

$ hg push ../hello-push
pushing to ../hello-push
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files

As with hg pull, the hg push command does not update the working directory in the repository that it's pushing changes into. Unlike hg pull, hg push does not provide a -u option that updates the other repository's working directory. This asymmetry is deliberate: the repository we're pushing to might be on a remote server and shared between several people. If we were to update its working directory while someone was working in it, their work would be disrupted.

What happens if we try to pull or push changes and the receiving repository already has those changes? Nothing too exciting.

$ hg push ../hello-push
pushing to ../hello-push
searching for changes
no changes found

When we clone a repository, Mercurial records the location of the repository we cloned in the .hg/hgrc file of the new repository. If we don't supply a location to hg pull from or hg push to, those commands will use this location as a default. The hg incoming and hg outgoing commands do so too.

If you open a repository's .hg/hgrc file in a text editor, you will see contents like the following.

[paths]
default = http://www.selenic.com/repo/hg

It is possible—and often useful—to have the default location for hg push and hg outgoing be different from those for hg pull and hg incoming. We can do this by adding a default-push entry to the [paths] section of the .hg/hgrc file, as follows.

[paths]
default = http://www.selenic.com/repo/hg
default-push = http://hg.example.com/hg

The commands we have covered in the previous few sections are not limited to working with local repositories. Each works in exactly the same fashion over a network connection; simply pass in a URL instead of a local path.

$ hg outgoing http://hg.serpentine.com/tutorial/hello
comparing with http://hg.serpentine.com/tutorial/hello
searching for changes
changeset:   5:c7a9f12460e4
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

In this example, we can see what changes we could push to the remote repository, but the repository is understandably not set up to let anonymous users push to it.

$ hg push http://hg.serpentine.com/tutorial/hello
pushing to http://hg.serpentine.com/tutorial/hello
searching for changes
ssl required

Remember that Mercurial records what the parent of each change is. If a change has a parent, we call it a child or descendant of the parent. A head is a change that has no children. The tip revision is thus a head, because the newest revision in a repository doesn't have any children. There are times when a repository can contain more than one head.

Figure 3.2. Repository contents after pulling from my-hello into my-new-hello

In Figure 3.2, “Repository contents after pulling from my-hello into my-new-hello”, you can see the effect of the pull from my-hello into my-new-hello. The history that was already present in my-new-hello is untouched, but a new revision has been added. By referring to Figure 3.1, “Divergent recent histories of the my-hello and my-new-hello repositories”, we can see that the changeset ID remains the same in the new repository, but the revision number has changed. (This, incidentally, is a fine example of why it's not safe to use revision numbers when discussing changesets.) We can view the heads in a repository using the hg heads command.

$ hg heads
changeset:   6:c7a9f12460e4
tag:         tip
parent:      4:2278160e78d4
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

changeset:   5:719bfe3a5ee1
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:08 2009 +0000
summary:     A new hello for a new day.

What happens if we try to use the normal hg update command to update to the new tip?

$ hg update
abort: crosses branches (use 'hg merge' or 'hg update -C')

Mercurial is telling us that the hg update command won't do a merge; it won't update the working directory when it thinks we might want to do a merge, unless we force it to do so. (Incidentally, forcing the update with hg update -C would revert any uncommitted changes in the working directory.)

To start a merge between the two heads, we use the hg merge command.

$ hg merge
merging hello.c
0 files updated, 1 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)

We resolve the contents of hello.c This updates the working directory so that it contains changes from both heads, which is reflected in both the output of hg parents and the contents of hello.c.

$ hg parents
changeset:   5:719bfe3a5ee1
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:08 2009 +0000
summary:     A new hello for a new day.

changeset:   6:c7a9f12460e4
tag:         tip
parent:      4:2278160e78d4
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:02 2009 +0000
summary:     Added an extra line of output

$ cat hello.c
/*
 * Placed in the public domain by Bryan O'Sullivan.  This program is
 * not covered by patents in the United States or other countries.
 */

#include <stdio.h>

int main(int argc, char **argv)
{
	printf("once more, hello.\n");
	printf("hello, world!\");
	printf("hello again!\n");
	return 0;
}

Whenever we've done a merge, hg parents will display two parents until we hg commit the results of the merge.

$ hg commit -m 'Merged changes'

We now have a new tip revision; notice that it has both of our former heads as its parents. These are the same revisions that were previously displayed by hg parents.

$ hg tip
changeset:   7:7c72be0a93ec
tag:         tip
parent:      5:719bfe3a5ee1
parent:      6:c7a9f12460e4
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:08 2009 +0000
summary:     Merged changes

In Figure 3.3, “Working directory and repository during merge, and following commit”, you can see a representation of what happens to the working directory during the merge, and how this affects the repository when the commit happens. During the merge, the working directory has two parent changesets, and these become the parents of the new changeset.

Figure 3.3. Working directory and repository during merge, and following commit

We sometimes talk about a merge having sides: the left side is the first parent in the output of hg parents, and the right side is the second. If the working directory was at e.g. revision 5 before we began a merge, that revision will become the left side of the merge.

My preferred graphical merge tool is kdiff3, which I'll use to describe the features that are common to graphical file merging tools. You can see a screenshot of kdiff3 in action in Figure 3.5, “Using kdiff3 to merge versions of a file”. The kind of merge it is performing is called a three-way merge, because there are three different versions of the file of interest to us. The tool thus splits the upper portion of the window into three panes:

In the pane below these is the current result of the merge. Our task is to replace all of the red text, which indicates unresolved conflicts, with some sensible merger of the “ours” and “theirs” versions of the file.

All four of these panes are locked together; if we scroll vertically or horizontally in any of them, the others are updated to display the corresponding sections of their respective files.

Figure 3.5. Using kdiff3 to merge versions of a file

For each conflicting portion of the file, we can choose to resolve the conflict using some combination of text from the base version, ours, or theirs. We can also manually edit the merged file at any time, in case we need to make further modifications.

There are many file merging tools available, too many to cover here. They vary in which platforms they are available for, and in their particular strengths and weaknesses. Most are tuned for merging files containing plain text, while a few are aimed at specialised file formats (generally XML).

In this example, we will reproduce the file modification history of Figure 3.4, “Conflicting changes to a document” above. Let's begin by creating a repository with a base version of our document.

$ cat > letter.txt <<EOF
> Greetings!
> I am Mariam Abacha, the wife of former
> Nigerian dictator Sani Abacha.
> EOF
$ hg add letter.txt
$ hg commit -m '419 scam, first draft'

We'll clone the repository and make a change to the file.

$ cd ..
$ hg clone scam scam-cousin
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd scam-cousin
$ cat > letter.txt <<EOF
> Greetings!
> I am Shehu Musa Abacha, cousin to the former
> Nigerian dictator Sani Abacha.
> EOF
$ hg commit -m '419 scam, with cousin'

And another clone, to simulate someone else making a change to the file. (This hints at the idea that it's not all that unusual to merge with yourself when you isolate tasks in separate repositories, and indeed to find and resolve conflicts while doing so.)

$ cd ..
$ hg clone scam scam-son
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd scam-son
$ cat > letter.txt <<EOF
> Greetings!
> I am Alhaji Abba Abacha, son of the former
> Nigerian dictator Sani Abacha.
> EOF
$ hg commit -m '419 scam, with son'

Having created two different versions of the file, we'll set up an environment suitable for running our merge.

$ cd ..
$ hg clone scam-cousin scam-merge
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd scam-merge
$ hg pull -u ../scam-son
pulling from ../scam-son
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
not updating, since new heads added
(run 'hg heads' to see heads, 'hg merge' to merge)

In this example, I'll set HGMERGE to tell Mercurial to use the non-interactive merge command. This is bundled with many Unix-like systems. (If you're following this example on your computer, don't bother setting HGMERGE. You'll get dropped into a GUI file merge tool instead, which is much preferable.)

$ export HGMERGE=merge
$ hg merge
merging letter.txt
sh: merge: command not found
merging letter.txt failed!
0 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg up --clean' to abandon
$ cat letter.txt
Greetings!
I am Shehu Musa Abacha, cousin to the former
Nigerian dictator Sani Abacha.

Because merge can't resolve the conflicting changes, it leaves merge markers inside the file that has conflicts, indicating which lines have conflicts, and whether they came from our version of the file or theirs.

Mercurial can tell from the way merge exits that it wasn't able to merge successfully, so it tells us what commands we'll need to run if we want to redo the merging operation. This could be useful if, for example, we were running a graphical merge tool and quit because we were confused or realised we had made a mistake.

If automatic or manual merges fail, there's nothing to prevent us from “fixing up” the affected files ourselves, and committing the results of our merge:

$ cat > letter.txt <<EOF
> Greetings!
> I am Bryan O'Sullivan, no relation of the former
> Nigerian dictator Sani Abacha.
> EOF
$ hg resolve -m letter.txt
$ hg commit -m 'Send me your money'
$ hg tip
changeset:   3:c7d1c2022d8f
tag:         tip
parent:      1:2d888eb87411
parent:      2:b1e21f18a5f5
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:28:09 2009 +0000
summary:     Send me your money

	Where is the hg resolve command?
	The hg resolve command was introduced in Mercurial 1.1, which was released in December 2008. If you are using an older version of Mercurial (run hg version to see), this command will not be present. If your version of Mercurial is older than 1.1, you should strongly consider upgrading to a newer version before trying to tackle complicated merges.

When Mercurial tracks modifications to a file, it stores the history of that file in a metadata object called a filelog. Each entry in the filelog contains enough information to reconstruct one revision of the file that is being tracked. Filelogs are stored as files in the .hg/store/data directory. A filelog contains two kinds of information: revision data, and an index to help Mercurial to find a revision efficiently.

A file that is large, or has a lot of history, has its filelog stored in separate data (“.d” suffix) and index (“.i” suffix) files. For small files without much history, the revision data and index are combined in a single “.i” file. The correspondence between a file in the working directory and the filelog that tracks its history in the repository is illustrated in Figure 4.1, “Relationships between files in working directory and filelogs in repository”.

Figure 4.1. Relationships between files in working directory and filelogs in repository

Mercurial uses a structure called a manifest to collect together information about the files that it tracks. Each entry in the manifest contains information about the files present in a single changeset. An entry records which files are present in the changeset, the revision of each file, and a few other pieces of file metadata.

The changelog contains information about each changeset. Each revision records who committed a change, the changeset comment, other pieces of changeset-related information, and the revision of the manifest to use.

Within a changelog, a manifest, or a filelog, each revision stores a pointer to its immediate parent (or to its two parents, if it's a merge revision). As I mentioned above, there are also relationships between revisions across these structures, and they are hierarchical in nature.

For every changeset in a repository, there is exactly one revision stored in the changelog. Each revision of the changelog contains a pointer to a single revision of the manifest. A revision of the manifest stores a pointer to a single revision of each filelog tracked when that changeset was created. These relationships are illustrated in Figure 4.2, “Metadata relationships”.

Figure 4.2. Metadata relationships

As the illustration shows, there is not a “one to one” relationship between revisions in the changelog, manifest, or filelog. If a file that Mercurial tracks hasn't changed between two changesets, the entry for that file in the two revisions of the manifest will point to the same revision of its filelog^[3].

The revlog provides efficient storage of revisions using a delta mechanism. Instead of storing a complete copy of a file for each revision, it stores the changes needed to transform an older revision into the new revision. For many kinds of file data, these deltas are typically a fraction of a percent of the size of a full copy of a file.

Some obsolete revision control systems can only work with deltas of text files. They must either store binary files as complete snapshots or encoded into a text representation, both of which are wasteful approaches. Mercurial can efficiently handle deltas of files with arbitrary binary contents; it doesn't need to treat text as special.

Mercurial only ever appends data to the end of a revlog file. It never modifies a section of a file after it has written it. This is both more robust and efficient than schemes that need to modify or rewrite data.

In addition, Mercurial treats every write as part of a transaction that can span a number of files. A transaction is atomic: either the entire transaction succeeds and its effects are all visible to readers in one go, or the whole thing is undone. This guarantee of atomicity means that if you're running two copies of Mercurial, where one is reading data and one is writing it, the reader will never see a partially written result that might confuse it.

The fact that Mercurial only appends to files makes it easier to provide this transactional guarantee. The easier it is to do stuff like this, the more confident you should be that it's done correctly.

Mercurial cleverly avoids a pitfall common to all earlier revision control systems: the problem of inefficient retrieval. Most revision control systems store the contents of a revision as an incremental series of modifications against a “snapshot”. (Some base the snapshot on the oldest revision, others on the newest.) To reconstruct a specific revision, you must first read the snapshot, and then every one of the revisions between the snapshot and your target revision. The more history that a file accumulates, the more revisions you must read, hence the longer it takes to reconstruct a particular revision.

Figure 4.3. Snapshot of a revlog, with incremental deltas

The innovation that Mercurial applies to this problem is simple but effective. Once the cumulative amount of delta information stored since the last snapshot exceeds a fixed threshold, it stores a new snapshot (compressed, of course), instead of another delta. This makes it possible to reconstruct any revision of a file quickly. This approach works so well that it has since been copied by several other revision control systems.

Figure 4.3, “Snapshot of a revlog, with incremental deltas” illustrates the idea. In an entry in a revlog's index file, Mercurial stores the range of entries from the data file that it must read to reconstruct a particular revision.

Along with delta or snapshot information, a revlog entry contains a cryptographic hash of the data that it represents. This makes it difficult to forge the contents of a revision, and easy to detect accidental corruption.

Hashes provide more than a mere check against corruption; they are used as the identifiers for revisions. The changeset identification hashes that you see as an end user are from revisions of the changelog. Although filelogs and the manifest also use hashes, Mercurial only uses these behind the scenes.

Mercurial verifies that hashes are correct when it retrieves file revisions and when it pulls changes from another repository. If it encounters an integrity problem, it will complain and stop whatever it's doing.

In addition to the effect it has on retrieval efficiency, Mercurial's use of periodic snapshots makes it more robust against partial data corruption. If a revlog becomes partly corrupted due to a hardware error or system bug, it's often possible to reconstruct some or most revisions from the uncorrupted sections of the revlog, both before and after the corrupted section. This would not be possible with a delta-only storage model.

The dirstate stores parent information for more than just book-keeping purposes. Mercurial uses the parents of the dirstate as the parents of a new changeset when you perform a commit.

Figure 4.5. The working directory can have two parents

Figure 4.5, “The working directory can have two parents” shows the normal state of the working directory, where it has a single changeset as parent. That changeset is the tip, the newest changeset in the repository that has no children.

Figure 4.6. The working directory gains new parents after a commit

It's useful to think of the working directory as “the changeset I'm about to commit”. Any files that you tell Mercurial that you've added, removed, renamed, or copied will be reflected in that changeset, as will modifications to any files that Mercurial is already tracking; the new changeset will have the parents of the working directory as its parents.

After a commit, Mercurial will update the parents of the working directory, so that the first parent is the ID of the new changeset, and the second is the null ID. This is shown in Figure 4.6, “The working directory gains new parents after a commit”. Mercurial doesn't touch any of the files in the working directory when you commit; it just modifies the dirstate to note its new parents.

It's perfectly normal to update the working directory to a changeset other than the current tip. For example, you might want to know what your project looked like last Tuesday, or you could be looking through changesets to see which one introduced a bug. In cases like this, the natural thing to do is update the working directory to the changeset you're interested in, and then examine the files in the working directory directly to see their contents as they were when you committed that changeset. The effect of this is shown in Figure 4.7, “The working directory, updated to an older changeset”.

Figure 4.7. The working directory, updated to an older changeset

Having updated the working directory to an older changeset, what happens if you make some changes, and then commit? Mercurial behaves in the same way as I outlined above. The parents of the working directory become the parents of the new changeset. This new changeset has no children, so it becomes the new tip. And the repository now contains two changesets that have no children; we call these heads. You can see the structure that this creates in Figure 4.8, “After a commit made while synced to an older changeset”.

Figure 4.8. After a commit made while synced to an older changeset

Note

If you're new to Mercurial, you should keep in mind a common “error”, which is to use the hg pull command without any options. By default, the hg pull command does not update the working directory, so you'll bring new changesets into your repository, but the working directory will stay synced at the same changeset as before the pull. If you make some changes and commit afterwards, you'll thus create a new head, because your working directory isn't synced to whatever the current tip is. To combine the operation of a pull, followed by an update, run hg pull -u. I put the word “error” in quotes because all that you need to do to rectify the situation where you created a new head by accident is hg merge, then hg commit. In other words, this almost never has negative consequences; it's just something of a surprise for newcomers. I'll discuss other ways to avoid this behavior, and why Mercurial behaves in this initially surprising way, later on.

When you run the hg merge command, Mercurial leaves the first parent of the working directory unchanged, and sets the second parent to the changeset you're merging with, as shown in Figure 4.9, “Merging two heads”.

Figure 4.9. Merging two heads

Mercurial also has to modify the working directory, to merge the files managed in the two changesets. Simplified a little, the merging process goes like this, for every file in the manifests of both changesets.

There are more details—merging has plenty of corner cases—but these are the most common choices that are involved in a merge. As you can see, most cases are completely automatic, and indeed most merges finish automatically, without requiring your input to resolve any conflicts.

When you're thinking about what happens when you commit after a merge, once again the working directory is “the changeset I'm about to commit”. After the hg merge command completes, the working directory has two parents; these will become the parents of the new changeset.

Mercurial lets you perform multiple merges, but you must commit the results of each individual merge as you go. This is necessary because Mercurial only tracks two parents for both revisions and the working directory. While it would be technically feasible to merge multiple changesets at once, Mercurial avoids this for simplicity. With multi-way merges, the risks of user confusion, nasty conflict resolution, and making a terrible mess of a merge would grow intolerable.

A surprising number of revision control systems pay little or no attention to a file's name over time. For instance, it used to be common that if a file got renamed on one side of a merge, the changes from the other side would be silently dropped.

Mercurial records metadata when you tell it to perform a rename or copy. It uses this metadata during a merge to do the right thing in the case of a merge. For instance, if I rename a file, and you edit it without renaming it, when we merge our work the file will be renamed and have your edits applied.

When appropriate, Mercurial will store both snapshots and deltas in compressed form. It does this by always trying to compress a snapshot or delta, but only storing the compressed version if it's smaller than the uncompressed version.

This means that Mercurial does “the right thing” when storing a file whose native form is compressed, such as a zip archive or a JPEG image. When these types of files are compressed a second time, the resulting file is usually bigger than the once-compressed form, and so Mercurial will store the plain zip or JPEG.

Deltas between revisions of a compressed file are usually larger than snapshots of the file, and Mercurial again does “the right thing” in these cases. It finds that such a delta exceeds the threshold at which it should store a complete snapshot of the file, so it stores the snapshot, again saving space compared to a naive delta-only approach.

[ui]
ssh = ssh -C

Appending to files isn't the whole story when it comes to guaranteeing that a reader won't see a partial write. If you recall Figure 4.2, “Metadata relationships”, revisions in the changelog point to revisions in the manifest, and revisions in the manifest point to revisions in filelogs. This hierarchy is deliberate.

A writer starts a transaction by writing filelog and manifest data, and doesn't write any changelog data until those are finished. A reader starts by reading changelog data, then manifest data, followed by filelog data.

Since the writer has always finished writing filelog and manifest data before it writes to the changelog, a reader will never read a pointer to a partially written manifest revision from the changelog, and it will never read a pointer to a partially written filelog revision from the manifest.

The read/write ordering and atomicity guarantees mean that Mercurial never needs to lock a repository when it's reading data, even if the repository is being written to while the read is occurring. This has a big effect on scalability; you can have an arbitrary number of Mercurial processes safely reading data from a repository all at once, no matter whether it's being written to or not.

The lockless nature of reading means that if you're sharing a repository on a multi-user system, you don't need to grant other local users permission to write to your repository in order for them to be able to clone it or pull changes from it; they only need read permission. (This is not a common feature among revision control systems, so don't take it for granted! Most require readers to be able to lock a repository to access it safely, and this requires write permission on at least one directory, which of course makes for all kinds of nasty and annoying security and administrative problems.)

Mercurial uses locks to ensure that only one process can write to a repository at a time (the locking mechanism is safe even over filesystems that are notoriously hostile to locking, such as NFS). If a repository is locked, a writer will wait for a while to retry if the repository becomes unlocked, but if the repository remains locked for too long, the process attempting to write will time out after a while. This means that your daily automated scripts won't get stuck forever and pile up if a system crashes unnoticed, for example. (Yes, the timeout is configurable, from zero to infinity.)

Critical to Mercurial's performance is the avoidance of seeks of the disk head, since any seek is far more expensive than even a comparatively large read operation.

This is why, for example, the dirstate is stored in a single file. If there were a dirstate file per directory that Mercurial tracked, the disk would seek once per directory. Instead, Mercurial reads the entire single dirstate file in one step.

Mercurial also uses a “copy on write” scheme when cloning a repository on local storage. Instead of copying every revlog file from the old repository into the new repository, it makes a “hard link”, which is a shorthand way to say “these two names point to the same file”. When Mercurial is about to write to one of a revlog's files, it checks to see if the number of names pointing at the file is greater than one. If it is, more than one repository is using the file, so Mercurial makes a new copy of the file that is private to this repository.

A few revision control developers have pointed out that this idea of making a complete private copy of a file is not very efficient in its use of storage. While this is true, storage is cheap, and this method gives the highest performance while deferring most book-keeping to the operating system. An alternative scheme would most likely reduce performance and increase the complexity of the software, but speed and simplicity are key to the “feel” of day-to-day use.

Because Mercurial doesn't force you to tell it when you're modifying a file, it uses the dirstate to store some extra information so it can determine efficiently whether you have modified a file. For each file in the working directory, it stores the time that it last modified the file itself, and the size of the file at that time.

When you explicitly hg add, hg remove, hg rename or hg copy files, Mercurial updates the dirstate so that it knows what to do with those files when you commit.

The dirstate helps Mercurial to efficiently check the status of files in a repository.

Storing the modification time and size dramatically reduces the number of read operations that Mercurial needs to perform when we run commands like hg status. This results in large performance improvements.

A useful behavior that Mercurial has is that if you pass the name of a directory to a command, every Mercurial command will treat this as “I want to operate on every file in this directory and its subdirectories”.

$ mkdir b
$ echo b > b/somefile.txt
$ echo c > b/source.cpp
$ mkdir b/d
$ echo d > b/d/test.h
$ hg add b
adding b/d/test.h
adding b/somefile.txt
adding b/source.cpp
$ hg commit -m 'Added all files in subdirectory'

Notice in this example that Mercurial printed the names of the files it added, whereas it didn't do so when we added the file named myfile.txt in the earlier example.

What's going on is that in the former case, we explicitly named the file to add on the command line. The assumption that Mercurial makes in such cases is that we know what we are doing, and it doesn't print any output.

However, when we imply the names of files by giving the name of a directory, Mercurial takes the extra step of printing the name of each file that it does something with. This makes it more clear what is happening, and reduces the likelihood of a silent and nasty surprise. This behavior is common to most Mercurial commands.

Mercurial does not track directory information. Instead, it tracks the path to a file. Before creating a file, it first creates any missing directory components of the path. After it deletes a file, it then deletes any empty directories that were in the deleted file's path. This sounds like a trivial distinction, but it has one minor practical consequence: it is not possible to represent a completely empty directory in Mercurial.

Empty directories are rarely useful, and there are unintrusive workarounds that you can use to achieve an appropriate effect. The developers of Mercurial thus felt that the complexity that would be required to manage empty directories was not worth the limited benefit this feature would bring.

If you need an empty directory in your repository, there are a few ways to achieve this. One is to create a directory, then hg add a “hidden” file to that directory. On Unix-like systems, any file name that begins with a period (“.”) is treated as hidden by most commands and GUI tools. This approach is illustrated below.

$ hg init hidden-example
$ cd hidden-example
$ mkdir empty
$ touch empty/.hidden
$ hg add empty/.hidden
$ hg commit -m 'Manage an empty-looking directory'
$ ls empty
$ cd ..
$ hg clone hidden-example tmp
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ ls tmp
empty
$ ls tmp/empty

Another way to tackle a need for an empty directory is to simply create one in your automated build scripts before they will need it.

It is important to understand that removing a file has only two effects.

Removing a file does not in any way alter the history of the file.

If you update the working directory to a changeset that was committed when it was still tracking a file that you later removed, the file will reappear in the working directory, with the contents it had when you committed that changeset. If you then update the working directory to a later changeset, in which the file had been removed, Mercurial will once again remove the file from the working directory.

Mercurial considers a file that you have deleted, but not used hg remove to delete, to be missing. A missing file is represented with “!” in the output of hg status. Mercurial commands will not generally do anything with missing files.

$ hg init missing-example
$ cd missing-example
$ echo a > a
$ hg add a
$ hg commit -m 'File about to be missing'
$ rm a
$ hg status
! a

If your repository contains a file that hg status reports as missing, and you want the file to stay gone, you can run hg remove --after at any time later on, to tell Mercurial that you really did mean to remove the file.

$ hg remove --after a
$ hg status
R a

On the other hand, if you deleted the missing file by accident, give hg revert the name of the file to recover. It will reappear, in unmodified form.

$ hg revert a
$ cat a
a
$ hg status

You might wonder why Mercurial requires you to explicitly tell it that you are deleting a file. Early during the development of Mercurial, it let you delete a file however you pleased; Mercurial would notice the absence of the file automatically when you next ran a hg commit, and stop tracking the file. In practice, this made it too easy to accidentally remove a file without noticing.

Mercurial offers a combination command, hg addremove, that adds untracked files and marks missing files as removed.

$ hg init addremove-example
$ cd addremove-example
$ echo a > a
$ echo b > b
$ hg addremove
adding a
adding b

The hg commit command also provides a -A option that performs this same add-and-remove, immediately followed by a commit.

$ echo c > c
$ hg commit -A -m 'Commit with addremove'
adding c

What happens during a merge is that changes “follow” a copy. To best illustrate what this means, let's create an example. We'll start with the usual tiny repository that contains a single file.

$ hg init my-copy
$ cd my-copy
$ echo line > file
$ hg add file
$ hg commit -m 'Added a file'

We need to do some work in parallel, so that we'll have something to merge. So let's clone our repository.

$ cd ..
$ hg clone my-copy your-copy
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Back in our initial repository, let's use the hg copy command to make a copy of the first file we created.

$ cd my-copy
$ hg copy file new-file

If we look at the output of the hg status command afterwards, the copied file looks just like a normal added file.

$ hg status
A new-file

But if we pass the -C option to hg status, it prints another line of output: this is the file that our newly-added file was copied from.

$ hg status -C
A new-file
  file
$ hg commit -m 'Copied file'

Now, back in the repository we cloned, let's make a change in parallel. We'll add a line of content to the original file that we created.

$ cd ../your-copy
$ echo 'new contents' >> file
$ hg commit -m 'Changed file'

Now we have a modified file in this repository. When we pull the changes from the first repository, and merge the two heads, Mercurial will propagate the changes that we made locally to file into its copy, new-file.

$ hg pull ../my-copy
pulling from ../my-copy
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg merge
merging file and new-file to new-file
0 files updated, 1 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ cat new-file
line
new contents

This behavior—of changes to a file propagating out to copies of the file—might seem esoteric, but in most cases it's highly desirable.

First of all, remember that this propagation only happens when you merge. So if you hg copy a file, and subsequently modify the original file during the normal course of your work, nothing will happen.

The second thing to know is that modifications will only propagate across a copy as long as the changeset that you're merging changes from hasn't yet seen the copy.

The reason that Mercurial does this is as follows. Let's say I make an important bug fix in a source file, and commit my changes. Meanwhile, you've decided to hg copy the file in your repository, without knowing about the bug or having seen the fix, and you have started hacking on your copy of the file.

If you pulled and merged my changes, and Mercurial didn't propagate changes across copies, your new source file would now contain the bug, and unless you knew to propagate the bug fix by hand, the bug would remain in your copy of the file.

By automatically propagating the change that fixed the bug from the original file to the copy, Mercurial prevents this class of problem. To my knowledge, Mercurial is the only revision control system that propagates changes across copies like this.

Once your change history has a record that the copy and subsequent merge occurred, there's usually no further need to propagate changes from the original file to the copied file, and that's why Mercurial only propagates changes across copies at the first merge, and not afterwards.

If, for some reason, you decide that this business of automatically propagating changes across copies is not for you, simply use your system's normal file copy command (on Unix-like systems, that's cp) to make a copy of a file, then hg add the new copy by hand. Before you do so, though, please do reread Section 5.3.2, “Why should changes follow copies?”, and make an informed decision that this behavior is not appropriate to your specific case.

When you use the hg copy command, Mercurial makes a copy of each source file as it currently stands in the working directory. This means that if you make some modifications to a file, then hg copy it without first having committed those changes, the new copy will also contain the modifications you have made up until that point. (I find this behavior a little counterintuitive, which is why I mention it here.)

The hg copy command acts similarly to the Unix cp command (you can use the hg cp alias if you prefer). We must supply two or more arguments, of which the last is treated as the destination, and all others are sources.

If you pass hg copy a single file as the source, and the destination does not exist, it creates a new file with that name.

$ mkdir k
$ hg copy a k
$ ls k
a

If the destination is a directory, Mercurial copies its sources into that directory.

$ mkdir d
$ hg copy a b d
$ ls d
a  b

Copying a directory is recursive, and preserves the directory structure of the source.

$ hg copy z e
copying z/a/c to e/a/c

If the source and destination are both directories, the source tree is recreated in the destination directory.

$ hg copy z d
copying z/a/c to d/z/a/c

As with the hg remove command, if you copy a file manually and then want Mercurial to know that you've copied the file, simply use the --after option to hg copy.

$ cp a n
$ hg copy --after a n

Since Mercurial's rename is implemented as copy-and-remove, the same propagation of changes happens when you merge after a rename as after a copy.

If I modify a file, and you rename it to a new name, and then we merge our respective changes, my modifications to the file under its original name will be propagated into the file under its new name. (This is something you might expect to “simply work,” but not all revision control systems actually do this.)

Whereas having changes follow a copy is a feature where you can perhaps nod and say “yes, that might be useful,” it should be clear that having them follow a rename is definitely important. Without this facility, it would simply be too easy for changes to become orphaned when files are renamed.

The case of diverging names occurs when two developers start with a file—let's call it foo—in their respective repositories.

$ hg clone orig anne
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg clone orig bob
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Anne renames the file to bar.

$ cd anne
$ hg rename foo bar
$ hg ci -m 'Rename foo to bar'

Meanwhile, Bob renames it to quux. (Remember that hg mv is an alias for hg rename.)

$ cd ../bob
$ hg mv foo quux
$ hg ci -m 'Rename foo to quux'

I like to think of this as a conflict because each developer has expressed different intentions about what the file ought to be named.

What do you think should happen when they merge their work? Mercurial's actual behavior is that it always preserves both names when it merges changesets that contain divergent renames.

# See http://www.selenic.com/mercurial/bts/issue455
$ cd ../orig
$ hg pull -u ../anne
pulling from ../anne
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg pull ../bob
pulling from ../bob
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg merge
warning: detected divergent renames of foo to:
 bar
 quux
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ ls
bar  quux

Notice that while Mercurial warns about the divergent renames, it leaves it up to you to do something about the divergence after the merge.

Another kind of rename conflict occurs when two people choose to rename different source files to the same destination. In this case, Mercurial runs its normal merge machinery, and lets you guide it to a suitable resolution.

Mercurial has a longstanding bug in which it fails to handle a merge where one side has a file with a given name, while another has a directory with the same name. This is documented as issue 29.

$ hg init issue29
$ cd issue29
$ echo a > a
$ hg ci -Ama
adding a
$ echo b > b
$ hg ci -Amb
adding b
$ hg up 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ mkdir b
$ echo b > b/b
$ hg ci -Amc
adding b/b
created new head
$ hg merge
abort: Is a directory: /tmp/issue29BEL_hR/issue29/b

When a merge occurs, most files will usually remain unmodified. For each file where Mercurial has to do something, it tracks the state of the file.

If Mercurial sees any file in the unresolved state after a merge, it considers the merge to have failed. Fortunately, we do not need to restart the entire merge from scratch.

The --list or -l option to hg resolve prints out the state of each merged file.

$ hg resolve -l
U myfile.txt

In the output from hg resolve, a resolved file is marked with R, while an unresolved file is marked with U. If any files are listed with U, we know that an attempt to commit the results of the merge will fail.

We have several options to move a file from the unresolved into the resolved state. By far the most common is to rerun hg resolve. If we pass the names of individual files or directories, it will retry the merges of any unresolved files present in those locations. We can also pass the --all or -a option, which will retry the merges of all unresolved files.

Mercurial also lets us modify the resolution state of a file directly. We can manually mark a file as resolved using the --mark option, or as unresolved using the --unmark option. This allows us to clean up a particularly messy merge by hand, and to keep track of our progress with each file as we go.

The most important aspect of any model that you must keep in mind is how well it matches the needs and capabilities of the people who will be using it. This might seem self-evident; even so, you still can't afford to forget it for a moment.

I once put together a workflow model that seemed to make perfect sense to me, but that caused a considerable amount of consternation and strife within my development team. In spite of my attempts to explain why we needed a complex set of branches, and how changes ought to flow between them, a few team members revolted. Even though they were smart people, they didn't want to pay attention to the constraints we were operating under, or face the consequences of those constraints in the details of the model that I was advocating.

Don't sweep foreseeable social or technical problems under the rug. Whatever scheme you put into effect, you should plan for mistakes and problem scenarios. Consider adding automated machinery to prevent, or quickly recover from, trouble that you can anticipate. As an example, if you intend to have a branch with not-for-release changes in it, you'd do well to think early about the possibility that someone might accidentally merge those changes into a release branch. You could avoid this particular problem by writing a hook that prevents changes from being merged from an inappropriate branch.

I wouldn't suggest an “anything goes” approach as something sustainable, but it's a model that's easy to grasp, and it works perfectly well in a few unusual situations.

As one example, many projects have a loose-knit group of collaborators who rarely physically meet each other. Some groups like to overcome the isolation of working at a distance by organizing occasional “sprints”. In a sprint, a number of people get together in a single location (a company's conference room, a hotel meeting room, that kind of place) and spend several days more or less locked in there, hacking intensely on a handful of projects.

A sprint or a hacking session in a coffee shop are the perfect places to use the hg serve command, since hg serve does not require any fancy server infrastructure. You can get started with hg serve in moments, by reading Section 6.4, “Informal sharing with hg serve” below. Then simply tell the person next to you that you're running a server, send the URL to them in an instant message, and you immediately have a quick-turnaround way to work together. They can type your URL into their web browser and quickly review your changes; or they can pull a bugfix from you and verify it; or they can clone a branch containing a new feature and try it out.

The charm, and the problem, with doing things in an ad hoc fashion like this is that only people who know about your changes, and where they are, can see them. Such an informal approach simply doesn't scale beyond a handful people, because each individual needs to know about n different repositories to pull from.

For smaller projects migrating from a centralised revision control tool, perhaps the easiest way to get started is to have changes flow through a single shared central repository. This is also the most common “building block” for more ambitious workflow schemes.

Contributors start by cloning a copy of this repository. They can pull changes from it whenever they need to, and some (perhaps all) developers have permission to push a change back when they're ready for other people to see it.

Under this model, it can still often make sense for people to pull changes directly from each other, without going through the central repository. Consider a case in which I have a tentative bug fix, but I am worried that if I were to publish it to the central repository, it might subsequently break everyone else's trees as they pull it. To reduce the potential for damage, I can ask you to clone my repository into a temporary repository of your own and test it. This lets us put off publishing the potentially unsafe change until it has had a little testing.

If a team is hosting its own repository in this kind of scenario, people will usually use the ssh protocol to securely push changes to the central repository, as documented in Section 6.5, “Using the Secure Shell (ssh) protocol”. It's also usual to publish a read-only copy of the repository over HTTP, as in Section 6.6, “Serving over HTTP using CGI”. Publishing over HTTP satisfies the needs of people who don't have push access, and those who want to use web browsers to browse the repository's history.

A wonderful thing about public hosting services like Bitbucket is that not only do they handle the fiddly server configuration details, such as user accounts, authentication, and secure wire protocols, they provide additional infrastructure to make this model work well.

For instance, a well-engineered hosting service will let people clone their own copies of a repository with a single click. This lets people work in separate spaces and share their changes when they're ready.

In addition, a good hosting service will let people communicate with each other, for instance to say “there are changes ready for you to review in this tree”.

Projects of any significant size naturally tend to make progress on several fronts simultaneously. In the case of software, it's common for a project to go through periodic official releases. A release might then go into “maintenance mode” for a while after its first publication; maintenance releases tend to contain only bug fixes, not new features. In parallel with these maintenance releases, one or more future releases may be under development. People normally use the word “branch” to refer to one of these many slightly different directions in which development is proceeding.

Mercurial is particularly well suited to managing a number of simultaneous, but not identical, branches. Each “development direction” can live in its own central repository, and you can merge changes from one to another as the need arises. Because repositories are independent of each other, unstable changes in a development branch will never affect a stable branch unless someone explicitly merges those changes into the stable branch.

Here's an example of how this can work in practice. Let's say you have one “main branch” on a central server.

$ hg init main
$ cd main
$ echo 'This is a boring feature.' > myfile
$ hg commit -A -m 'We have reached an important milestone!'
adding myfile

People clone it, make changes locally, test them, and push them back.

Once the main branch reaches a release milestone, you can use the hg tag command to give a permanent name to the milestone revision.

$ hg tag v1.0
$ hg tip
changeset:   1:03d816125d7d
tag:         tip
user:        Bryan O'Sullivan <bos@serpentine.com>
date:        Thu Oct 22 03:27:39 2009 +0000
summary:     Added tag v1.0 for changeset 46480f306c43

$ hg tags
tip                                1:03d816125d7d
v1.0                               0:46480f306c43

Let's say some ongoing development occurs on the main branch.

$ cd ../main
$ echo 'This is exciting and new!' >> myfile
$ hg commit -m 'Add a new feature'
$ cat myfile
This is a boring feature.
This is exciting and new!

Using the tag that was recorded at the milestone, people who clone that repository at any time in the future can use hg update to get a copy of the working directory exactly as it was when that tagged revision was committed.

$ cd ..
$ hg clone -U main main-old
$ cd main-old
$ hg update v1.0
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat myfile
This is a boring feature.

In addition, immediately after the main branch is tagged, we can then clone the main branch on the server to a new “stable” branch, also on the server.

$ cd ..
$ hg clone -rv1.0 main stable
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

If we need to make a change to the stable branch, we can then clone that repository, make our changes, commit, and push our changes back there.

$ hg clone stable stable-fix
updating working directory
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd stable-fix
$ echo 'This is a fix to a boring feature.' > myfile
$ hg commit -m 'Fix a bug'
$ hg push
pushing to /tmp/branchingNEeB6O/stable
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files

Because Mercurial repositories are independent, and Mercurial doesn't move changes around automatically, the stable and main branches are isolated from each other. The changes that we made on the main branch don't “leak” to the stable branch, and vice versa.

We'll often want all of our bugfixes on the stable branch to show up on the main branch, too. Rather than rewrite a bugfix on the main branch, we can simply pull and merge changes from the stable to the main branch, and Mercurial will bring those bugfixes in for us.

$ cd ../main
$ hg pull ../stable
pulling from ../stable
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg merge
merging myfile
0 files updated, 1 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg commit -m 'Bring in bugfix from stable branch'
$ cat myfile
This is a fix to a boring feature.
This is exciting and new!

The main branch will still contain changes that are not on the stable branch, but it will also contain all of the bugfixes from the stable branch. The stable branch remains unaffected by these changes, since changes are only flowing from the stable to the main branch, and not the other way.

For larger projects, an effective way to manage change is to break up a team into smaller groups. Each group has a shared branch of its own, cloned from a single “master” branch used by the entire project. People working on an individual branch are typically quite isolated from developments on other branches.

Figure 6.1. Feature branches

When a particular feature is deemed to be in suitable shape, someone on that feature team pulls and merges from the master branch into the feature branch, then pushes back up to the master branch.

Some projects are organized on a “train” basis: a release is scheduled to happen every few months, and whatever features are ready when the “train” is ready to leave are allowed in.

This model resembles working with feature branches. The difference is that when a feature branch misses a train, someone on the feature team pulls and merges the changes that went out on that train release into the feature branch, and the team continues its work on top of that release so that their feature can make the next release.

The development of the Linux kernel has a shallow hierarchical structure, surrounded by a cloud of apparent chaos. Because most Linux developers use git, a distributed revision control tool with capabilities similar to Mercurial, it's useful to describe the way work flows in that environment; if you like the ideas, the approach translates well across tools.

At the center of the community sits Linus Torvalds, the creator of Linux. He publishes a single source repository that is considered the “authoritative” current tree by the entire developer community. Anyone can clone Linus's tree, but he is very choosy about whose trees he pulls from.

Linus has a number of “trusted lieutenants”. As a general rule, he pulls whatever changes they publish, in most cases without even reviewing those changes. Some of those lieutenants are generally agreed to be “maintainers”, responsible for specific subsystems within the kernel. If a random kernel hacker wants to make a change to a subsystem that they want to end up in Linus's tree, they must find out who the subsystem's maintainer is, and ask that maintainer to take their change. If the maintainer reviews their changes and agrees to take them, they'll pass them along to Linus in due course.

Individual lieutenants have their own approaches to reviewing, accepting, and publishing changes; and for deciding when to feed them to Linus. In addition, there are several well known branches that people use for different purposes. For example, a few people maintain “stable” repositories of older versions of the kernel, to which they apply critical fixes as needed. Some maintainers publish multiple trees: one for experimental changes; one for changes that they are about to feed upstream; and so on. Others just publish a single tree.

This model has two notable features. The first is that it's “pull only”. You have to ask, convince, or beg another developer to take a change from you, because there are almost no trees to which more than one person can push, and there's no way to push changes into a tree that someone else controls.

The second is that it's based on reputation and acclaim. If you're an unknown, Linus will probably ignore changes from you without even responding. But a subsystem maintainer will probably review them, and will likely take them if they pass their criteria for suitability. The more “good” changes you contribute to a maintainer, the more likely they are to trust your judgment and accept your changes. If you're well-known and maintain a long-lived branch for something Linus hasn't yet accepted, people with similar interests may pull your changes regularly to keep up with your work.

Reputation and acclaim don't necessarily cross subsystem or “people” boundaries. If you're a respected but specialised storage hacker, and you try to fix a networking bug, that change will receive a level of scrutiny from a network maintainer comparable to a change from a complete stranger.

To people who come from more orderly project backgrounds, the comparatively chaotic Linux kernel development process often seems completely insane. It's subject to the whims of individuals; people make sweeping changes whenever they deem it appropriate; and the pace of development is astounding. And yet Linux is a highly successful, well-regarded piece of software.

A perpetual source of heat in the open source community is whether a development model in which people only ever pull changes from others is “better than” one in which multiple people can push changes to a shared repository.

Typically, the backers of the shared-push model use tools that actively enforce this approach. If you're using a centralised revision control tool such as Subversion, there's no way to make a choice over which model you'll use: the tool gives you shared-push, and if you want to do anything else, you'll have to roll your own approach on top (such as applying a patch by hand).

A good distributed revision control tool will support both models. You and your collaborators can then structure how you work together based on your own needs and preferences, not on what contortions your tools force you into.

Once you and your team set up some shared repositories and start propagating changes back and forth between local and shared repos, you begin to face a related, but slightly different challenge: that of managing the multiple directions in which your team may be moving at once. Even though this subject is intimately related to how your team collaborates, it's dense enough to merit treatment of its own, in Chapter 8, Managing releases and branchy development.

Because it provides unauthenticated read access to all clients, you should only use hg serve in an environment where you either don't care, or have complete control over, who can access your network and pull data from your repository.

The hg serve command knows nothing about any firewall software you might have installed on your system or network. It cannot detect or control your firewall software. If other people are unable to talk to a running hg serve instance, the second thing you should do (after you make sure that they're using the correct URL) is check your firewall configuration.

By default, hg serve listens for incoming connections on port 8000. If another process is already listening on the port you want to use, you can specify a different port to listen on using the -p option.

Normally, when hg serve starts, it prints no output, which can be a bit unnerving. If you'd like to confirm that it is indeed running correctly, and find out what URL you should send to your collaborators, start it with the -v option.

An ssh URL tends to look like this:

ssh://bos@hg.serpentine.com:22/hg/hgbook

There's plenty of scope for confusion with the path component of ssh URLs, as there is no standard way for tools to interpret it. Some programs behave differently than others when dealing with these paths. This isn't an ideal situation, but it's unlikely to change. Please read the following paragraphs carefully.

Mercurial treats the path to a repository on the server as relative to the remote user's home directory. For example, if user foo on the server has a home directory of /home/foo, then an ssh URL that contains a path component of bar really refers to the directory /home/foo/bar.

If you want to specify a path relative to another user's home directory, you can use a path that starts with a tilde character followed by the user's name (let's call them otheruser), like this.

ssh://server/~otheruser/hg/repo

And if you really want to specify an absolute path on the server, begin the path component with two slashes, as in this example.

ssh://server//absolute/path

Almost every Unix-like system comes with OpenSSH preinstalled. If you're using such a system, run which ssh to find out if the ssh command is installed (it's usually in /usr/bin). In the unlikely event that it isn't present, take a look at your system documentation to figure out how to install it.

On Windows, the TortoiseHg package is bundled with a version of Simon Tatham's excellent plink command, and you should not need to do any further configuration.

To avoid the need to repetitively type a password every time you need to use your ssh client, I recommend generating a key pair.

	Key pairs are not mandatory
	Mercurial knows nothing about ssh authentication or key pairs. You can, if you like, safely ignore this section and the one that follows until you grow tired of repeatedly typing ssh passwords.

When you generate a key pair, it's usually highly advisable to protect it with a passphrase. (The only time that you might not want to do this is when you're using the ssh protocol for automated tasks on a secure network.)

Simply generating a key pair isn't enough, however. You'll need to add the public key to the set of authorised keys for whatever user you're logging in remotely as. For servers using OpenSSH (the vast majority), this will mean adding the public key to a list in a file called authorized_keys in their .ssh directory.

On a Unix-like system, your public key will have a .pub extension. If you're using puttygen on Windows, you can save the public key to a file of your choosing, or paste it from the window it's displayed in straight into the authorized_keys file.

An authentication agent is a daemon that stores passphrases in memory (so it will forget passphrases if you log out and log back in again). An ssh client will notice if it's running, and query it for a passphrase. If there's no authentication agent running, or the agent doesn't store the necessary passphrase, you'll have to type your passphrase every time Mercurial tries to communicate with a server on your behalf (e.g. whenever you pull or push changes).

The downside of storing passphrases in an agent is that it's possible for a well-prepared attacker to recover the plain text of your passphrases, in some cases even if your system has been power-cycled. You should make your own judgment as to whether this is an acceptable risk. It certainly saves a lot of repeated typing.

Because ssh can be fiddly to set up if you're new to it, a variety of things can go wrong. Add Mercurial on top, and there's plenty more scope for head-scratching. Most of these potential problems occur on the server side, not the client side. The good news is that once you've gotten a configuration working, it will usually continue to work indefinitely.

Before you try using Mercurial to talk to an ssh server, it's best to make sure that you can use the normal ssh or putty command to talk to the server first. If you run into problems with using these commands directly, Mercurial surely won't work. Worse, it will obscure the underlying problem. Any time you want to debug ssh-related Mercurial problems, you should drop back to making sure that plain ssh client commands work first, before you worry about whether there's a problem with Mercurial.

The first thing to be sure of on the server side is that you can actually log in from another machine at all. If you can't use ssh or putty to log in, the error message you get may give you a few hints as to what's wrong. The most common problems are as follows.

In summary, if you're having trouble talking to the server's ssh daemon, first make sure that one is running at all. On many systems it will be installed, but disabled, by default. Once you're done with this step, you should then check that the server's firewall is configured to allow incoming connections on the port the ssh daemon is listening on (usually 22). Don't worry about more exotic possibilities for misconfiguration until you've checked these two first.

If you're using an authentication agent on the client side to store passphrases for your keys, you ought to be able to log into the server without being prompted for a passphrase or a password. If you're prompted for a passphrase, there are a few possible culprits.

If you're being prompted for the remote user's password, there are another few possible problems to check.

In the ideal world, you should be able to run the following command successfully, and it should print exactly one line of output, the current date and time.

ssh myserver date

If, on your server, you have login scripts that print banners or other junk even when running non-interactive commands like this, you should fix them before you continue, so that they only print output if they're run interactively. Otherwise these banners will at least clutter up Mercurial's output. Worse, they could potentially cause problems with running Mercurial commands remotely. Mercurial tries to detect and ignore banners in non-interactive ssh sessions, but it is not foolproof. (If you're editing your login scripts on your server, the usual way to see if a login script is running in an interactive shell is to check the return code from the command tty -s.)

Once you've verified that plain old ssh is working with your server, the next step is to ensure that Mercurial runs on the server. The following command should run successfully:

ssh myserver hg version

If you see an error message instead of normal hg version output, this is usually because you haven't installed Mercurial to /usr/bin. Don't worry if this is the case; you don't need to do that. But you should check for a few possible problems.

If you can run hg version over an ssh connection, well done! You've got the server and client sorted out. You should now be able to use Mercurial to access repositories hosted by that username on that server. If you run into problems with Mercurial and ssh at this point, try using the --debug option to get a clearer picture of what's going on.

Mercurial does not compress data when it uses the ssh protocol, because the ssh protocol can transparently compress data. However, the default behavior of ssh clients is not to request compression.

Over any network other than a fast LAN (even a wireless network), using compression is likely to significantly speed up Mercurial's network operations. For example, over a WAN, someone measured compression as reducing the amount of time required to clone a particularly large repository from 51 minutes to 17 minutes.

Both ssh and plink accept a -C option which turns on compression. You can easily edit your ~/.hgrc to enable compression for all of Mercurial's uses of the ssh protocol. Here is how to do so for regular ssh on Unix-like systems, for example.

[ui]
ssh = ssh -C

If you use ssh on a Unix-like system, you can configure it to always use compression when talking to your server. To do this, edit your .ssh/config file (which may not yet exist), as follows.

Host hg
  Compression yes
  HostName hg.example.com

This defines a hostname alias, hg. When you use that hostname on the ssh command line or in a Mercurial ssh-protocol URL, it will cause ssh to connect to hg.example.com and use compression. This gives you both a shorter name to type and compression, each of which is a good thing in its own right.

Before you continue, do take a few moments to check a few aspects of your system's setup.

If you don't have a web server installed, and don't have substantial experience configuring Apache, you should consider using the lighttpd web server instead of Apache. Apache has a well-deserved reputation for baroque and confusing configuration. While lighttpd is less capable in some ways than Apache, most of these capabilities are not relevant to serving Mercurial repositories. And lighttpd is undeniably much easier to get started with than Apache.

On Unix-like systems, it's common for users to have a subdirectory named something like public_html in their home directory, from which they can serve up web pages. A file named foo in this directory will be accessible at a URL of the form http://www.example.com/username/foo.

To get started, find the hgweb.cgi script that should be present in your Mercurial installation. If you can't quickly find a local copy on your system, simply download one from the master Mercurial repository at http://www.selenic.com/repo/hg/raw-file/tip/hgweb.cgi.

You'll need to copy this script into your public_html directory, and ensure that it's executable.

cp .../hgweb.cgi ~/public_html
chmod 755 ~/public_html/hgweb.cgi

The 755 argument to chmod is a little more general than just making the script executable: it ensures that the script is executable by anyone, and that “group” and “other” write permissions are not set. If you were to leave those write permissions enabled, Apache's suexec subsystem would likely refuse to execute the script. In fact, suexec also insists that the directory in which the script resides must not be writable by others.

chmod 755 ~/public_html

chmod 755 ~
find ~/public_html -type d -print0 | xargs -0r chmod 755
find ~/public_html -type f -print0 | xargs -0r chmod 644

<Directory /home/*/public_html>
  AllowOverride FileInfo AuthConfig Limit
  Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
  <Limit GET POST OPTIONS>
    Order allow,deny
    Allow from all
  </Limit>
  <LimitExcept GET POST OPTIONS>
    Order deny,allow Deny from all
  </LimitExcept>
</Directory>

AddHandler cgi-script .cgi

userdir.path = "public_html"
cgi.assign = (".cgi" => "" )

The hgweb.cgi script only lets you publish a single repository, which is an annoying restriction. If you want to publish more than one without wracking yourself with multiple copies of the same script, each with different names, a better choice is to use the hgwebdir.cgi script.

The procedure to configure hgwebdir.cgi is only a little more involved than for hgweb.cgi. First, you must obtain a copy of the script. If you don't have one handy, you can download a copy from the master Mercurial repository at http://www.selenic.com/repo/hg/raw-file/tip/hgwebdir.cgi.

You'll need to copy this script into your public_html directory, and ensure that it's executable.

cp .../hgwebdir.cgi ~/public_html
chmod 755 ~/public_html ~/public_html/hgwebdir.cgi

With basic configuration out of the way, try to visit http://myhostname/~myuser/hgwebdir.cgi in your browser. It should display an empty list of repositories. If you get a blank window or error message, try walking through the list of potential problems in Section 6.6.2.1, “What could possibly go wrong?”.

The hgwebdir.cgi script relies on an external configuration file. By default, it searches for a file named hgweb.config in the same directory as itself. You'll need to create this file, and make it world-readable. The format of the file is similar to a Windows “ini” file, as understood by Python's ConfigParser [web:configparser] module.

The easiest way to configure hgwebdir.cgi is with a section named collections. This will automatically publish every repository under the directories you name. The section should look like this:

[collections]
/my/root = /my/root

Mercurial interprets this by looking at the directory name on the right hand side of the “=” sign; finding repositories in that directory hierarchy; and using the text on the left to strip off matching text from the names it will actually list in the web interface. The remaining component of a path after this stripping has occurred is called a “virtual path”.

Given the example above, if we have a repository whose local path is /my/root/this/repo, the CGI script will strip the leading /my/root from the name, and publish the repository with a virtual path of this/repo. If the base URL for our CGI script is http://myhostname/~myuser/hgwebdir.cgi, the complete URL for that repository will be http://myhostname/~myuser/hgwebdir.cgi/this/repo.

If we replace /my/root on the left hand side of this example with /my, then hgwebdir.cgi will only strip off /my from the repository name, and will give us a virtual path of root/this/repo instead of this/repo.

The hgwebdir.cgi script will recursively search each directory listed in the collections section of its configuration file, but it will not recurse into the repositories it finds.

The collections mechanism makes it easy to publish many repositories in a “fire and forget” manner. You only need to set up the CGI script and configuration file one time. Afterwards, you can publish or unpublish a repository at any time by simply moving it into, or out of, the directory hierarchy in which you've configured hgwebdir.cgi to look.

6.6.3.1. Explicitly specifying which repositories to publish

In addition to the collections mechanism, the hgwebdir.cgi script allows you to publish a specific list of repositories. To do so, create a paths section, with contents of the following form.

[paths]
repo1 = /my/path/to/some/repo
repo2 = /some/path/to/another

In this case, the virtual path (the component that will appear in a URL) is on the left hand side of each definition, while the path to the repository is on the right. Notice that there does not need to be any relationship between the virtual path you choose and the location of a repository in your filesystem.

If you wish, you can use both the collections and paths mechanisms simultaneously in a single configuration file.

	Beware duplicate virtual paths
	If several repositories have the same virtual path, hgwebdir.cgi will not report an error. Instead, it will behave unpredictably.

Mercurial's web interface lets users download an archive of any revision. This archive will contain a snapshot of the working directory as of that revision, but it will not contain a copy of the repository data.

By default, this feature is not enabled. To enable it, you'll need to add an allow_archive item to the web section of your ~/.hgrc; see below for details.

Mercurial's web interfaces (the hg serve command, and the hgweb.cgi and hgwebdir.cgi scripts) have a number of configuration options that you can set. These belong in a section named web.

If you are using hgwebdir.cgi, you can place a few configuration items in a web section of the hgweb.config file instead of a ~/.hgrc file, for convenience. These items are motd and style.

One situation in which a global hgrc can be useful is if users are pulling changes owned by other users. By default, Mercurial will not trust most of the configuration items in a .hg/hgrc file inside a repository that is owned by a different user. If we clone or pull changes from such a repository, Mercurial will print a warning stating that it does not trust their .hg/hgrc.

If everyone in a particular Unix group is on the same team and should trust each other's configuration settings, or we want to trust particular users, we can override Mercurial's skeptical defaults by creating a system-wide hgrc file such as the following:

# Save this as e.g. /etc/mercurial/hgrc.d/trust.rc
[trusted]
# Trust all entries in any hgrc file owned by the "editors" or
# "www-data" groups.
groups = editors, www-data

# Trust entries in hgrc files owned by the following users.
users = apache, bobo

This is an overview of the kinds of patterns you can use when you're matching on glob patterns.

The “*” character matches any string, within a single directory.

$ hg add 'glob:*.py'
adding main.py

The “**” pattern matches any string, and crosses directory boundaries. It's not a standard Unix glob token, but it's accepted by several popular Unix shells, and is very useful.

$ cd ..
$ hg status 'glob:**.py'
A examples/simple.py
A src/main.py
? examples/performant.py
? setup.py
? src/watcher/watcher.py

The “?” pattern matches any single character.

$ hg status 'glob:**.?'
? src/watcher/_watcher.c

The “[” character begins a character class. This matches any single character within the class. The class ends with a “]” character. A class may contain multiple ranges of the form “a-f”, which is shorthand for “abcdef”.

$ hg status 'glob:**[nr-t]'
? MANIFEST.in
? src/xyzzy.txt

If the first character after the “[” in a character class is a “!”, it negates the class, making it match any single character not in the class.

A “{” begins a group of subpatterns, where the whole group matches if any subpattern in the group matches. The “,” character separates subpatterns, and “}” ends the group.

$ hg status 'glob:*.{in,py}'
? MANIFEST.in
? setup.py

$ hg status 'glob:*.py'
? setup.py
$ hg status 'glob:**.py'
A examples/simple.py
A src/main.py
? examples/performant.py
? setup.py
? src/watcher/watcher.py

Mercurial accepts the same regular expression syntax as the Python programming language (it uses Python's regexp engine internally). This is based on the Perl language's regexp syntax, which is the most popular dialect in use (it's also used in Java, for example).

I won't discuss Mercurial's regexp dialect in any detail here, as regexps are not often used. Perl-style regexps are in any case already exhaustively documented on a multitude of web sites, and in many books. Instead, I will focus here on a few things you should know if you find yourself needing to use regexps with Mercurial.

A regexp is matched against an entire file name, relative to the root of the repository. In other words, even if you're already in subbdirectory foo, if you want to match files under this directory, your pattern must start with “foo/”.

One thing to note, if you're familiar with Perl-style regexps, is that Mercurial's are rooted. That is, a regexp starts matching against the beginning of a string; it doesn't look for a match anywhere within the string. To match anywhere in a string, start your pattern with “.*”.

Mercurial's repository storage mechanism is case safe. It translates file names so that they can be safely stored on both case sensitive and case insensitive filesystems. This means that you can use normal file copying tools to transfer a Mercurial repository onto, for example, a USB thumb drive, and safely move that drive and repository back and forth between a Mac, a PC running Windows, and a Linux box.

When operating in the working directory, Mercurial honours the naming policy of the filesystem where the working directory is located. If the filesystem is case preserving, but insensitive, Mercurial will treat names that differ only in case as the same.

An important aspect of this approach is that it is possible to commit a changeset on a case sensitive (typically Linux or Unix) filesystem that will cause trouble for users on case insensitive (usually Windows and MacOS) users. If a Linux user commits changes to two files, one named myfile.c and the other named MyFile.C, they will be stored correctly in the repository. And in the working directories of other Linux users, they will be correctly represented as separate files.

If a Windows or Mac user pulls this change, they will not initially have a problem, because Mercurial's repository storage mechanism is case safe. However, once they try to hg update the working directory to that changeset, or hg merge with that changeset, Mercurial will spot the conflict between the two file names that the filesystem would treat as the same, and forbid the update or merge from occurring.

If you are using Windows or a Mac in a mixed environment where some of your collaborators are using Linux or Unix, and Mercurial reports a case folding conflict when you try to hg update or hg merge, the procedure to fix the problem is simple.

Just find a nearby Linux or Unix box, clone the problem repository onto it, and use Mercurial's hg rename command to change the names of any offending files or directories so that they will no longer cause case folding conflicts. Commit this change, hg pull or hg push it across to your Windows or MacOS system, and hg update to the revision with the non-conflicting names.

The changeset with case-conflicting names will remain in your project's history, and you still won't be able to hg update your working directory to that changeset on a Windows or MacOS system, but you can continue development unimpeded.

You won't often need to care about the .hgtags file, but it sometimes makes its presence known during a merge. The format of the file is simple: it consists of a series of lines. Each line starts with a changeset hash, followed by a space, followed by the name of a tag.

If you're resolving a conflict in the .hgtags file during a merge, there's one twist to modifying the .hgtags file: when Mercurial is parsing the tags in a repository, it never reads the working copy of the .hgtags file. Instead, it reads the most recently committed revision of the file.

An unfortunate consequence of this design is that you can't actually verify that your merged .hgtags file is correct until after you've committed a change. So if you find yourself resolving a conflict on .hgtags during a merge, be sure to run hg tags after you commit. If it finds an error in the .hgtags file, it will report the location of the error, which you can then fix and commit. You should then run hg tags again, just to be sure that your fix is correct.

You may have noticed that the hg clone command has a -r option that lets you clone an exact copy of the repository as of a particular changeset. The new clone will not contain any project history that comes after the revision you specified. This has an interaction with tags that can surprise the unwary.

Recall that a tag is stored as a revision to the .hgtags file. When you create a tag, the changeset in which its recorded refers to an older changeset. When you run hg clone -r foo to clone a repository as of tag foo, the new clone will not contain any revision newer than the one the tag refers to, including the revision where the tag was created. The result is that you'll get exactly the right subset of the project's history in the new repository, but not the tag you might have expected.

Since Mercurial's tags are revision controlled and carried around with a project's history, everyone you work with will see the tags you create. But giving names to revisions has uses beyond simply noting that revision 4237e45506ee is really v2.0.2. If you're trying to track down a subtle bug, you might want a tag to remind you of something like “Anne saw the symptoms with this revision”.

For cases like this, what you might want to use are local tags. You can create a local tag with the -l option to the hg tag command. This will store the tag in a file called .hg/localtags. Unlike .hgtags, .hg/localtags is not revision controlled. Any tags you create using -l remain strictly local to the repository you're currently working in.