Drupal Subversion Repository

Transmachina has built and maintains a number of Drupal-based websites. Due to the active community, updates to both the core Drupal code and the modules come frequently and can take a lot of time to keep a Drupal installation up-to-date and secure. As a result, Transmachina has developed a subversion repository for tracking Drupal releases and have decided to make this open to the open source community. The repository is available at http://svn-drupal.transmachina.net/ and can be read with anonymous access. Currently all updates are performed by Transmachina staff, but we would be interested in volunteers in the community to help out with this process.

What we do

Every time a new release of Drupal is made, we download the source and use subversion's svn_load_dirs script to update our 5.x "current" release code to this new version. The script automatically adds new files, removes dropped ones and updates any existing files. The "diff" is stored in subversion in an efficient way. Using a tool such as TortoiseSVN you can quickly see an overview of changes made during each release. We also add a copy of the release as a whole in /trunk/drupal/5.10 for example.

How do I use it?

Starting a new Drupal project

  • Export the latest copy from the repository, svn export http://svn-drupal.transmachina.net/trunk/drupal/5.x mydrupal (this will create a local folder "mydrupal" which you edit locally.
  • Add the mydrupal folder into your version control system (any should work), for example svn add mydrupal then svn commit mydrupal -m "Adding new fresh drupal install for myproject"
  • You can now install new modules, and even make changes to core drupal files (you should really know what you are doing when you change core files)
  • You should regularly commit changes back to your source control system, so that you can track changes you make and "roll back" those changes if you break something.
  • When a new version of Drupal is released, we will update our repository. You can then update your local drupal project (without losing your local tweaks to the drupal core) by running: 
    svn merge http://svn-drupal.transmachina.net/trunk/drupal/ http://svn-drupal.transmachina.net/trunk/drupal/5.x mydrupal
    You should replace with the version of drupal that you are currently using say, "5.10". This procedure check the Transmachina repository for all changes from your version of drupal to the latest release, and patches your working copy with those diffs.
  • You should test the upgrade locally and then commit the changes back into your version control system.
  • You have successfully upgraded the drupal core without losing any local changes!

Upgrading an existing Drupal project

If you are already running Drupal and want to simply upgrade to the latest release without losing your local changes, that is easy too! You should probably commit all local changes back to your repository and tag the release so that you can easily rollback in case of a mistake in the following steps.
  • Simply run: 
    svn merge http://svn-drupal.transmachina.net/trunk/drupal/ http://svn-drupal.transmachina.net/trunk/drupal/5.x mydrupal
    You should replace with the version of drupal that you are currently using say, "5.10".
Note: You can find the current version of your drupal installation by either navigating to http:///admin/logs/status or by looking in /modules/drupal/drupal.info on the line beginning "version = "

More details

Updating the repository

For those who are interested, we have developed a simple script to update our subversion repository quickly after each drupal release. 
#!/bin/sh
# $1 drupal version
# the base of the WC
repobase=/home//drupal/trunk

Echo downloading to tmp
# remove file so that it doesn't concat
rm -f /tmp/drupal-$1.tar.gz
# download the relase
wget -O /tmp/drupal-$1.tar.gz http://ftp.drupal.org/files/projects/drupal-$1.tar.gz
# extract contents to tmp folder
tar -C /tmp -xvzf /tmp/drupal-$1.tar.gz

echo Loading diff changes into repos
svn_load_dirs -svn_username  -svn_password  http://svn-drupal.transmachina.net/trunk/drupal/5.x . /tmp/drupal-$1

# and now add that relase as-is into subversion too
echo Copying new release from $1 to location drupal/$1
cp -R -v /tmp/$1 $repobase/drupal/$1

echo Queuing new release files to svn
svn add $repobase/drupal/$1

echo Commiting new release
svn commit $repobase/drupal -m "Adding new drupal release $1"

Updating a working copy

We have a script to update each of our working copies also: 
#!/bin/sh
# $1 drupal version of WC
# $2 location of WC drupal root
#svn merge --dry-run http://svn-drupal.transmachina.net/trunk/drupal/$1 http://svn-drupal.transmachina.net/trunk/drupal/5.x $2
svn merge http://svn-drupal.transmachina.net/trunk/drupal/$1 http://svn-drupal.transmachina.net/trunk/drupal/5.x $2
When you are first getting this going, it may be worth running with the --dry-run line instead. This option will print out all the changes that will be made to your working copy without actually changing anything. If there are more than about 40 updates and 3-4 conflicts something probably have gone wrong. If the merge makes a mess of your working copy, you can try 
svn revert -R mydrupal
to recursively revert all local changes back to the last time you commited to the repository. In you worst case, you can use TortoiseSVN (only on Windows unfortunately) to rollback an entire installation to a previous revision - it basically does the last commited changes in reverse.

Thanks

Many thanks go to David Grant to working out how this could be achieved and for explaining the process on his site http://www.davidgrant.ca/maintaining_vendor_sources_with_subversion

Feedback

We hope that the community will find this useful, and welcome any feedback you have, especially suggestions on improving our scripts and service to the community. We plan to add drupal 6.x releases and modules in the near future.

Comments

David Grant (not verified)

Tue, 2008-09-16 23:49

I've always thought of doing this, and actually just recently, I also put my drupal repositories on an svn server that is available for everyone via http (previously, my repository was only available over ssh). I won't bother publishing the url here though, since that would water-down your effort.

I recently published a script for updating drupal modules in a subversion working directory. It's sort of like svn_load_dirs. It basically just deletes all the files in a module directory (except .svn dirs) and then untars the new module. Then it looks at the "svn status" output and looks for the "M", "?" or "!" and commits, add-commit, or remove-commits, respectively, thus bringing a module up to a new version.

The problem with this is that I lose my local changes, however, I can always do an "svn log" and merge some old changes back into the repository. I may make a script to do this as well. Because the commit message from the "update module" step I described above is always the same auto-generated type of message, I can search for them in the svn log output and detect which changes were my own. I can then auto-merge those back on top of HEAD. :-)