Words in Boxes

Nouns, verbs, and occasionally adjectives.

Sunday, February 05, 2012

Migrating repositories from Bitbucket to Github

A few years ago, I decided to use Mercurial & Bitbucket instead of Git & Github for my public repositories, because Mercurial seemed simpler and worked better on Windows. But times have changed. I use OS X now, and Github has become both overwhelmingly better and more popular. So it’s time to pick up and move.

This is a guide on how to do just that. It assumes that your repos have a small number of committers, and that you’re comfortable with the command line.

1. Clean up your Mercurial repository

Chances are your Bitbucket and Github usernames are not the same. Or if you’re like me, you didn’t keep your ~/.hgrc settings the same across machines, so your commit usernames aren’t consistent even if they’re all actually yours. The migration is a chance to tidy up.

To do this, we can use Mercurial’s convert extension, which can create a new, ‘filtered’ repository from an existing repository. In our case, we’ll alter the usernames to match what we want to appear on Github.

The convert extension is enabled by adding these lines to our ~/.hgrc:

[extensions]
hgext.convert=

Next, we’ll create a text file that maps old usernames to our new usernames. This is easy with some bash magic. Go to your repository directory and on the command line type:

hg log | grep user: | sort | uniq | sed ’s/user: *//‘ > users.txt

users.txt contains a sorted and filtered list of all the usernames attached to commits, like this:

username
username@localhost
username <username@gmail.com>

Edit this file so it maps old usernames to new usernames. If you have any existing Github repositories, I recommend running git logto get the exact username / email pair Github expects.

username=Username Fullname <username@gmail.com>
username@localhost=Username Fullname <username@gmail.com>
username <username@gmail.com>=Username Fullname <username@gmail.com>

We’re ready to convert (replace SOURCE_HG_REPO and CLEAN_HG_REPO with real directory names, of course):

cd ..
hg convert --authors SOURCE_HG_REPO/users.txt SOURCE_HG_REPO CLEAN_HG_REPO

And bam, you have a new, tidied mercurial repository in CLEAN_HG_REPO.

2. Convert the repository to Git

Converting a repository from Mercurial to Git is simple. We’ll use a script called fast-export. The following instructions are adapted from Dan Benjamin’s.

The easiest way to get fast-export is to clone it from its Git repository:

cd ~/tmp
git clone git://repo.or.cz/fast-export.git 

Now we’ll create a new, empty Git repository, and use fast-export to populate it.

git init DEST_GIT_REPO
cd DEST_GIT_REPO
~/tmp/fast-export/hg-fast-export.sh -r CLEAN_HG_REPO
git checkout HEAD

And bam, you have a new Git repository, complete with your entire commit history. Don’t forget: you’ll need to convert your .hgignore into .gitignore.

3. Push the new repository to Github

Now it’s time take your shiny new Git repository and slap it up on Github. Go to your Github profile page, and click “New Repository.” Fill out the form, and follow the instructions to import an existing Git repo, which goes something like this:

cd DEST_GIT_REPO
git remote add origin git@github.com:username/REPO_NAME.git
git push -u origin master

4. Don’t forget to migrate other data

I haven’t found a way to automatically migrate them, but don’t forget about:

  • Issues
  • Wiki. I tended to use a one-page Wiki as a landing page. The Github convention is to use a readme.md markdown document. You’ll have to convert the Wiki markup to Markdown.

5. Deleting your Bitbucket repository

On the “Admin” tab of your Bitbucket repository page, you can choose to delete the repository. Make sure to enter the url of your new Github repository in the “Redirect to” field!

I'm James Sulak, a software developer in Houston, Texas.

You can also find me on Twitter, or if you're curious, on my old-fashioned home page. If you want to contact me directly, you can e-mail comments@wordsinboxes.com.