> Home > News Index

2008-04-25 - Distributed SCM

I've become a recent convert to distributed SCM systems, particularly Mercurial and Git.

Use of a distributed SCM is a much more sensible way to release open source software than the traditional tarfile, or even having a public CVS or SVN repository. In particular:

Having a public CVS or SVN repository solves only the problem of giving users quick access to the latest version of the software. CVS and SVN don't help with feedback or maintenance of local changes. They can help with upgrades - but it's of little practical benefit when local changes can't be committed into the repository.

I was in the process of converting my CVS repositories to SVN (it takes a long time) when I discovered the virtues of Distributed SCMs. I'm using both Git and Mercurial. Git is perhaps more powerful but Mercurial is more user-friendly. So I have halted all conversion to SVN and I'll be converting remaining repositories to Mercurial. I'll be converting most of my SVN repositories over to Mercurial too. Ultimately the only code/data remaining in SVN will be that which I particularly need to be centralised (it might end up being none at all).

I have started to put some repositories of my public source code online already. The full list is available at: http://www.nick-andrew.net/hg and individual repositories which are currently online are:

develooper.com-src-qpsmtpd I use qpsmtpd on the mail servers here primarily for spam entrapment; for example to refuse emails addressed to nonexistent users at my domains. I get quite a lot of spam due to my long history with USENET. This repository contains the qpsmptd 0.26, 0.32 and 0.40 releases as downloaded from tarballs plus my local changes (mostly against 0.26 so far).
nick-pub-src-minix-ar To extract my old stuff (circa 1990) successfully, I need a V7-compatible 'ar' program. This is the original 'ar' program from Minix 1.x, modified to work on a modern machine (32-bit, with prototypes, etc).
nick-src-apt-cacher This is the original apt-cacher program (versions 0.1 through 0.4) before being picked up by Jon Oxer and packaged into Debian. Don't use this code; the repository is here merely for historical purposes.
nick-src-backup-cdr I wrote this system to do encrypted incremental backups of modified files - first to CDR, then to DVD. I need something to do size limited incremental backups since the total amount of backup is many times the capacity of a DVD. This system regulates the quantity of backup per day.
nick-src-direct This is a perl implementation of the 'direct' program I used in the TRS-80 days, to pack several files together into 2 (!) structured files, a 'DIR' and a 'DAT' file. Think of it as like a zipfile without any compression. Anyway when I restored my old TRS-80 software I needed to extract data from some of these structured file pairs so I wrote it in perl - a language which was not available back in the mid-1980s. It was trivial to do in perl, heh.
nick-src-distributed-filesystem This repository contains my 2003 ruminations on distributed filesystems. Useful if nothing else for the pointers to various designs people may have implemented.
nick-src-incremental-tar This C program replicates the function of Gnu TAR and optionally compresses its output on the fly. The point is that you can feed it with a (possibly infinite) list of files to be archived, and an output size limit, and it will ensure that its output size does not exceed the limit. It is useful for incremental backup - you give it a list of all your modified files, and a size (say 699 MBytes) and it will cram as many of those files in compressed form as can fit in 699 megs.
nick-src-misc-https-proxy This perl program listens on localhost port 80 and when it receives a connection, makes an SSL connection to a specified server on port 443 and forwards the data in both directions. This allows logging or debugging of an HTTPS data stream by using HTTP at the browser end.
nick-src-misc-logrun This cute little program reopens STDOUT and STDERR to files in a logging directory and runs the program. If the program exits with a non-zero code, the log files are moved to a 'bad' directory, otherwise they are moved to an 'ok' directory.
nick-src-misc-showdns This program does several DNS lookups on a specified domain name, and possibly filters the output. The aim is to quickly find the nameservers, web servers etc for a domain name and particularly to find those which are _not_ served by a specified nameserver.
nick-src-trs80 This is all my old TRS-80 source code in a single repository, rather than having to download many tarfiles. Again, it's for historical purposes. Unless you like Z80 assembler a lot.