UsingGITCentralSVN

Using GIT with a SVN central master repository (obsolete)

This guide tells you how to use GIT to checkout the source code from the GIT mirror repository and develop on the NG project.

Note:: We are using a Subversion repository as central master repository! This has several important implications. It means that all commits from SVN and GIT need to end up in the Subversion repository. Furthermore the GIT mirror repository gets commit updates from the SVN central master repository.

Why GIT?

Linus explains in depth why GIT rules over Subversion and similar Source Code Management (SCM) Systems:

As you can see, there are good reasons to switch from Subversion to GIT.

An introduction to GIT

On the GIT home page you can find an excelent free book on how to use GIT called Pro Git.

Conceptual design

The SVN/GIT/SVN development cycle is explained in detail in this document with the title “Subversion Master With Git Mirrors”. The document explains how to setup such coupled SVN and GIT repositories and describes how to use these to full effect.

Note:: Using a central remote SVN repository as master repository requires you to comply to certain merge rules as explained in the above document. Essentially it means that you have to provide a linear merge history to Subversion. To assure that this is the case you have to use the commands ‘git svn rebase’, ‘git pull –rebase’ to pull from SVN or GIT and to rebase your own branches to the SVN branches and then fast-forward merge your changes. The details are explained below.

The following picture shows the full development cycle:

The UAVP-NG GIT repositories

There are several UAVP-NG GIT repositories available to the NG developers. On one hand there are the SVN mirroring GIT master repositories containing our master trunk of our code and the other hand there are the GIT developer repositories, one for every NG developer named the same as his NG developer username.

Installing GIT on Windows in order to work with NG-UAVP repositories

1. Download and follow the instructions to install Git for Windows.
2. Extract the contents of https://dev.uavp.ch/download/windows-tools/openssh-6.1-win.zip to “<Path to Git for Windows>\bin” and overwrite the already existing files.

The UAVP-NG GIT master repositories

The UAVP-NG GIT master repositories are currently GIT mirror repositories feed from GIT fetch repositories which in turn pull from our central master Subversion repository as described above.

These are the UAVP-NG SVN mirroring GIT master repositories:

The UAVP-NG GIT developer repositories

These are the UAVP-NG GIT developer repositories:

Note:: You will need a NG Developer account and a registered public key to be able to access the above repositories.

Note:: On some systems you need to use a new URL syntax: ssh://git@dev.uavp.ch/stefan.git

The git2svn-sync.sh tool

To simplify the process of creating and maintaining the above described infrastructure we wrote a script called ‘git2svn-sync.sh’. It allows to create and sync Subversion repositories in a fetch repository which will pull Subversion commits on request and push the as GIT pushes to a mirror repository.

This is the help page of the small script:

Usage:

    Client mode from a user's client host:

      git2svn-sync.sh [git2svn-<repo>.conf]+ --client|-c [--clone <dir>|--clone-bash]

          --clone <dir> ................ clone git final repository remotely
          --clone-bash ................. show commands to clone git final repository

    Server/Admin mode on the git & svn server host:

      git2svn-sync.sh [git2svn-<repo>.conf]+ --server|-s [--sync] 

       --sync ....................... create or synchronize fetch + final git repositories
       --clean ...................... remove synchronized fetch + final git repositories

The server mode of the command is useful to maintainers of SVN/GIT mirrored repository. The client mode of the command can be used by any developer to clone a GIT repository from our GIT mirror which is automatically coupled with the central remote Subversion master.

Use git2svn-sync.sh to clone the central UAVP-NG master GIT mirror

The create a clone of the central remote GIT mirror repository you can call the git2svn-sync.sh script as follows:

$ ./git2svn-sync.sh conf/git2svn-uavp-ng.conf --client --clone /tmp/test

This will use the configuration file conf/git2svn-uavp.conf to clone the UAVP Project’s GIT mirror to /tmp. It will use the directory ‘uavp’ as configured in the configuration file.

The above command will result in something similar to this:

 * cloning uavp-ng at git@dev.uavp.ch:uavp-ng.git to /tmp/test/uavp-ng

Cloning into uavp-ng...

remote: Counting objects: 68358, done.
remote: Compressing objects: 100% (17984/17984), done.
remote: Total 68358 (delta 48126), reused 68358 (delta 48126)
Receiving objects: 100% (68358/68358), 247.24 MiB | 5.42 MiB/s, done.
Resolving deltas: 100% (48126/48126), done.

 * initializing svn to be able to 'git svn rebase' and 'git svn dcommit'
 * git svn rebase

Rebuilding .git/svn/refs/remotes/mirror/trunk/.rev_map.0adbd6a5-702f-0410-8250-c659bc5bf515 ...
r303 = 2e0fff445ee1cbec8c518bd58b39e34db6ef785a
r304 = 7292f73faaf8f61f87e5e356335235f37b06fce0
r408 = 0e3fd1d4a236964e3a532ac8aa590b0a1e94e611
r420 = 195d9711d40acce209ecb3460dbe1c8be40554cf
r426 = b3db1ff96719af77292892ab0e7adb0657f6aa6f
r427 = 3f161577da930b5c1ecd14d766dfac75e884de93
r428 = f402014a7e30ab3ae6b5f4f9f382634497a00732
r430 = 7ec2a06e91ac1a6991e60859d3a4d10095655b73
r432 = 209ea64589f71150ff51a845a2d151623686ef58
r437 = f2593760c5a251d98fcf3d013d9b13e6217f0773
...
r7578 = d8c9c5f3e3b0cfc227114078178bd1a475a2214f
r7610 = 0c6aec51e6b971993d4b1adc53eeef99c6857e01
r7615 = 38b3e348ee9446bfdc6934bec71f77019f57104d
Done rebuilding .git/svn/refs/remotes/mirror/trunk/.rev_map.0adbd6a5-702f-0410-8250-c659bc5bf515
Current branch trunk is up to date.

 * git svn pull

Current branch trunk is up to date.

You can also have git2svn-sync.sh print you the needed shell commands to clone the repository:

$ ./git2svn-sync.sh ./git2svn-uavp.conf --client --clone-bash /tmp/test

##
## commands to clone uavp-ng at git@dev.uavp.ch:uavp-ng.git to ./uavp-ng
##

# clone final repository
git clone -o mirror git@dev.uavp.ch:uavp-ng.git uavp-ng
cd uavp-ng
# initialize svn to be able to 'git svn rebase' and 'git svn dcommit'
git svn init --prefix=mirror/ \
  -Twolferl-ng/trunk -twolferl-ng/tags -bwolferl-ng/branches -bwolferl-ng/releases \
   https://dev.uavp.ch/pw.testing
# rebase with svn source repository
git svn rebase
# rebase with git final repository
git pull --rebase

Bash aliases and git2svn-sync.sh

You can easily alias the git2svn-sync.sh command for ease of use:

alias g2s="~bin/git2svn-sync.sh ~/conf/git2svn-uavp-ng.conf --client"

This way you can clone the remote central GIT mirror with the command:

g2s --clone /tmp

Updating your GIT clone

From the GIT master mirror repository

To update your local GIT clone with all changes from the master GIT mirror issue the following command:

git pull --rebase

The above command makes sure all changes in the GIT master mirror repository get applied before your own local changes and assures that the merge history stays linear.

/!\ Always use the above command to pull from the GIT master mirror repository! Do not try to merge changes from upstream, always rebase them or else you will get a non-linear history, which Subversion does not like!

Directly from the SVN master repository

To update your local GIT clone with all changes directly from the central SVN master repository issue the following command:

git svn rebase

The above command makes sure all changes in the central SVN master repository get applied before your own local changes and assures that the merge history stays linear.

/!\ Always use the above command to pull from the SVN master repository! Do not try to merge changes from upstream, always rebase them or else you will get a non-linear history, which Subversion does not like!

Committing to the central SVN master repository

You can use GIT however you like, just make sure you have a linear merge history for Subversion. When you are happy with your changes you may use git svn dcommit to commit the changes back to our central subversion master repository.

Here’s a normal workflow:

git pull --rebase               [ensure up to date with mirror]
...                             [modify a files and content]
git add -A                      [add any changes in files to the index]
git commit -m "updated a file"  [commit the changes in the index]
git svn rebase                  [ensure up to date with svn]
git svn dcommit                 [push changes back to Subversion]

Working with branches pushing to a central SVN master repository

When adding features it is good practice to use a feature branch.

The master branch can be kept up to date with subversion using git svn rebase and then the feature branch can be rebased onto master as and when you want to align with the current stream of development. When you are ready to push back to the central Subversion master repository, rebase the feature branch from the master branch and merge the branch using a fast forward merge onto the master branch. You can then dcommit the changes as normal.

A sample workflow would be:

git checkout -b feature         [create a feature branch and checkout]
...                             [modify a file]
git add -A                      [add any changes in files to the index]
git ci -m "updated a file"      [commit the changes in the index]

Occasionally rebase from mirror and rebase branch from master:

                                [update master branch with SVN changes by]
                                [... rebasing from GIT master mirror]
git checkout master             [switch back to the master branch]
git pull --rebase               [ensure up to date with mirror]
                                [update feature branch with SVN changes by]
                                [...rebasing master branch to feature branch]
git checkout feature            [switch back to the feature branch]
git rebase master               [replay branch changes on top of master and fix any problems]
....                            [do some work, until you are ready to commit back to svn]
                                [update master branch with SVN changes by]
                                [...rebasing from GIT master mirror]
git checkout master             [switch back to the master branch]
git svn rebase                  [ensure up to date with svn]
                                [update feature branch with SVN changes by]
                                [...rebasing master branch to feature branch]
git checkout feature            [switch back to the master branch]
git rebase master               [replay branch changes on top of master and fix any problems]
                                [fast forward merge linear merge history on master branch]
                                [--ff-only assures that only linear merge histories are accepted]
git checkout master             [switch back to the master branch]
git merge feature --ff-only     [merge feature branch ensuring fast forward merge]
                                [commit to central Subversion master repository]
git svn dcommit                 [push changes back to subversion]

It is important to be careful with merges due to the way git svn works, see CAVEATS from git svn man page.