[Twisted-Python] git clone with spawnProcess() can prevent socket data flow, any clue why?
enselic at gmail.com
Sat Aug 28 03:34:24 EDT 2010
I am getting a weird problem where git cloning with
reactor.spawnProcess() fails for some repositories. It appears as if
Twisted blocks reading from the server socket that git packs should come on.
I originally got the problem with buildbot, but I have has isolated the
problem further to Twisted. The following program can be used to
reproduce the problem:
from twisted.internet import reactor, protocol
def outReceived(self, data):
def errReceived(self, data):
def processEnded(self, status_object):
print "exit code %d" % status_object.value.exitCode
argv = ['/usr/bin/git', 'clone', 'git://git.gnome.org/gimp']
reactor.spawnProcess(ProcessPrinter(), argv, argv)
If a debugger is attached to the spawned git process, it can be seen
that the process blocks on read() on the socket that is supposed to be
fed with git packs from the server.
I have done a tcpdump of the traffic between my computer and the git
server, and there *is* successful communication going on. More
specifically, the git client receives the refs the server has, and sends
the refs it wants, which is acknowledged by the server, but for some
reason it still eventually blocks on read().
Curiously enough, if the program is changed to clone e.g.
'git://git.gnome.org/gegl' instead of 'git://git.gnome.org/gimp', it
works! The GIMP repo is much bigger than the GEGL one, which might be
the trigger for the bug, but I don't understand why it would matter.
I see this problem on Fedora 13 x86_64, both with the distro version
Twisted 0.8.2, and with the latest stable release Twisted 10.1.0. My
personal guess is that it has something to do with how spawnProcess()
manages the fds or the child process.
Does anyone have any guess of what is going on?
Can you reproduce it on other systems?
Thanks in advance for any input.
More information about the Twisted-Python