Moving to Subversion

CVS has a number of limitations that have been eliminated bySubversion. For the development of Python, the most notableimprovements are:

• the ability to rename files and directories, and to removedirectories, while keeping the history of these files.
• support for change sets (sets of correlated changes to multiplefiles) through global revision numbers. Change sets aretransactional.
• atomic, fast tagging: a cvs tag might take many minutes; aSubversion tag (svn cp) will complete quickly, and atomically.Likewise, branches are very efficient.
• support for offline diffs, which is useful when creating patches.

Moving to python.org

SourceForge has kindly provided an important infrastructure for thepast years. Unfortunately, the attention that SF received has alsocaused repeated overload situations in the past, to which the SFoperators could not always respond in a timely manner. In particular,for CVS, they had to reduce the load on the primary CVS server byintroducing a second, read-only CVS server for anonymous access. Thisserver is regularly synchronized, but lags behind the read-write CVSrepository between synchronizations. As a result, users withoutcommit access can see recent changes to the repository only after adelay.

On python.org, it would be possible to make the repository accessiblefor anonymous access.

Collect SSH keys

After some discussion, svn+ssh was selected as the best methodfor write access to the repository. Developers can continue touse their SSH keys, but they must be installed on python.org.

In order to avoid having to create a new Unix user for eachdeveloper, a single account should be used, with command=attributes in the authorized_keys files.

The lines in the authorized_keys file should read like this(wrapped for better readability):

command="/usr/bin/svnserve --root=/svnroot -t--tunnel-user='<username>'",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-ptyssh-dss<key><comment>

As the usernames, the real names should be used instead ofthe SF account names, so that people can be better identifiedin log messages.

Administrator Access

Administrator access to the pythondev account should be grantedto all current admins of the Python SF project. To distinguishbetween shell login and svnserve login, admins need to maintaintwo keys. Using OpenSSH, the following procedure can beused to create a second key:

cd.sshssh-keygen-tDSA-fpythondev-C<user>@pythondevviconfig

In the config file, the following lines need to be added:

HostpythondevHostnamedinsdale.python.orgUserpythondevIdentityFile~/.ssh/pythondev

Then, shell login becomes possible through “ssh pythondev”.

Downloading the CVS Repository

The CVS repository can be downloaded from

http://cvs.sourceforge.net/cvstarballs/python-cvsroot.tar.bz2

Since this tarball is generated only once a day, some time must passafter the repository freeze before the tarball can be picked up. Itshould be verified that the last commit, as recorded on thepython-commits mailing list, is indeed included in the tarball.

After the conversion, the converted CVS tarball should be keptforever on www.python.org/archive/python-cvsroot-<date>.tar.bz2

Converting the CVS Repository

The Python CVS repository contains two modules: distutils and python.The python module is further structured into dist and nondist,where dist only contains src (the python code proper). nondistcontains various subdirectories.

These should be reorganized in the Subversion repository to getshorter URLs, following the <project>/{trunk,tags,branches}structure. A project will be created for each nondist directory,plus for src (called python), plus distutils. Reorganizing therepository is best done in the CVS tree, as shown below.

The fsfs backend should be used as the repository format (whichrequires Subversion 1.1). The fsfs backend has the advantage of beingmore backup-friendly, as it allows incremental repository backups,without requiring any dump commands to be run.

The conversion should be done using the cvs2svn utility, availablee.g. in the cvs2svn Debian package. As cvs2svn does not currentlysupport the project/trunk structure, each project needs to beconverted separately. To get each conversion result into a separatedirectory in the target repository, svnadmin load must be used.

Subversion has a different view on binary-vs-text files than CVS.To correctly carry the CVS semantics forward, svn:eol-style shouldbe set to native on all files that are not marked binary in theCVS.

In summary, the conversion script is:

#!/bin/shrm cvs2svn-*rm -rf python py.newtar xjf python-cvsroot.tar.bz2rm -rf python/CVSROOTsvnadmin create --fs-type fsfs py.newmv python/python python/origmv python/orig/dist/src python/pythonmv python/orig/nondist/* python# nondist/nondist is emptyrmdir python/nondistrm -rf python/origfor a in python/*do  b=`basename $a`  cvs2svn -q --dump-only --encoding=latin1 --force-branch=cnri-16-start \  --force-branch=descr-branch --force-branch=release152p1-patches \  --force-tag=r16b1 $a  svn mkdir -m"Conversion to SVN" file:///`pwd`/py.new/$b  svnadmin load -q --parent-dir $b py.new < cvs2svn-dump  rm cvs2svn-dumpdone

Sample results of this conversion are available at

http://www.dcl.hpi.uni-potsdam.de/pysvn/

Publish the Repository

The repository should be published athttp://svn.python.org/projects.Read-write access should be granted to all current SF committersthrough svn+ssh://pythondev@svn.python.org/;read-only anonymous access through WebDAV should also begranted.

As an option, websvn (available e.g. from the Debian websvn package)could be provided. Unfortunately, in the test installation, websvnbreaks because it runs out of memory.

The current SF project admins should get write access to theauthorized_keys2 file of the pythondev account.

Disable CVS

It appears that CVS cannot be disabled entirely. Only the userinterface can be removed from the project page; the repository itselfremains available. If desired, write access to the python anddistutils modules can be disabled through a CVS commitinfo entry.