[Twisted-Python] Twisted Project Jobs Volunteer

[Tim Allen]

On Wed, Nov 09, 2011 at 12:50:23AM -0800, Glyph Lefkowitz wrote:
> On Nov 8, 2011, at 7:39 PM, Tim Allen wrote:
> > As far as I know (having written most of the documentation in the linked
> > wiki page, and from a brief skim through the git-svn manpage) it's
> > impossible to make a "shallow" clone with git-svn (something like an
> > ordinary "svn checkout", or "git clone --depth N"), so anyone who wants
> > to contribute to Twisted via git needs to clone the repository from
> > scratch (potentially overloading the SVN server, although nobody seemed
> > to notice or complain when I was doing my git-svn clone), or just copy
> > a tarball of somebody's comprehensive, elaborate, automatic mirroring
> > setup.
> 
> This is the part I don't understand.  Why doesn't 'git clone' work
> right in the face of svn metadata?

Git has a ".git" directory in each repository, and expects certain files
to be present within it. If there's more stuff that it doesn't know
about, it just ignores it. git-svn keeps its metadata in other files
that git-clone doesn't know or care about, and hence they don't get
cloned.

I guess there's an argument for not cloning them - just because person
X has particular access rights to an SVN server doesn't mean that person
Y should have them just because they cloned a repository from X.

> bzr-svn has some metadata it caches about the svn repository which
> doesn't stick around in the bzr repo or branch, but (from the
> performance of using it, at least) it doesn't need to stop the world
> and grab all of that data for most operations.

In a git-svn-cloned repository, ordinary git operations are as speedy as
you'd expect. "git svn fetch" is slow because (I think) it has to
separately request each changed file from each SVN revision, and I think
there's some other operations that are slow because they involve talking
to the SVN server (like 'tell me what SVN properties are attached to
this file').

> It would be better (for most users) to point to a canonical way to get
> access to a git-svn clone than to document how to make one, if making
> one takes 24 hours :).

Don't worry, it doesn't take 24 hours! It's *much* longer than that! ;)

(as the wiki page states, in early 2010 and from the other side of the
world it took about a week; I don't have a feel for how it changes over
time)

> > I'm not sure why that would be, except that possibly they found
> > a tarball of somebody else's git-svn clone and forgot to update it, or
> > they're confused about the best way to get cloned from some unofficial,
> > no-longer updated mirror. Fixing this probably depends on having the
> > canonical, correct, convenient instructions and advertising them widely.
> 
> You edited GitMirror before, you can do it again :).

I'm editing it to include what we've discussed here, as well as a few
other things that I researched this afternoon and put into a reply that
my MUA ate before it was sent. 

I believe the next step should be that somebody with the required
permissions should connect to a machine on the same physical network as
the SVN server and run:

    time git svn clone --stdlayout --prefix="svn/" \
	    svn+ssh://svn.twistedmatrix.com/svn/Twisted

...then check the load on the SVN server and see if it's going to be an
issue to let the clone complete. If the clone completes successfully,
then (a) we know about how long it takes, and (b) we have a "seed"
repository we can potentially put up for people to download. I'd be
happy to download it, check it, and write up some documentation about
how people should update it. If it doesn't complete successfully, we
should have some helpful error messages, adjust the clone command line
and try again.