[Twisted-Python] SSHSessionProcessProtocol.inConnectionLost() behavior and Git over Conch
Bo Shi
bs1984 at gmail.com
Wed Mar 3 21:46:33 MST 2010
Hi All,
I've been struggling with this issue off and on for the better part of
a month now. Having implemented a simple git SSH server using some of
the Conch examples in the official documentation, I encountered an
issue that would cause a git client to throw an error only part of the
time.
The little program to reproduce the issue is found at
http://gist.github.com/321403
(raw version for download)
http://gist.github.com/raw/321403/82ab2111a2709c8fe50a77aabb08565749087408/gitconnbug.py
and can be executed to start the sample server. Run a command like
# The /abspath/to/git/repo should be readable by the server user
$ git clone ssh://user@localhost:2222/abspath/to/git/repo
(password "user")
To test the server. On my workstation, I am able to reproduce the
error at least once every 5 tries. It's definitely not consistent.
The behavior I see is attached at the end of this email. The client
fails due to a premature inConnectionLost() call in the
SSHSessionProcessProtocol that sends an EOF.
As a workaround, when TraceProcessProtocol.inConnectionLost() is
overriden to do nothing, the client error goes away. This is somewhat
foreign territory for me so I'm not sure whether it's a bug in git or
whether it's a bug in the twisted ProcessProtocol implementation. RFC
4254 isn't terribly helpful here:
5.3. Closing a Channel
When a party will no longer send more data to a channel, it SHOULD
send SSH_MSG_CHANNEL_EOF.
So on the surface the current behavior of sending an EOF appears fine,
however, I can't really find any definitive cases of this type of
problem popping up via, say, default OpenSSH/git combinations.
Any advice? Is the gitconnbug.py implementation flawed? Should I
open a ticket? Any git experts know of a case where git-upload-pack
might close it's stdout pipe?
Thanks,
Bo
Successful case (server side logging, timestamp cut out):
[SSHChannel session (0) on SSHService ssh-connection
on SSHServerTransport,1,127.0.0.1] executing command
"git-upload-pack '/Users/bshi/sandbox/poop'"
[-] TPP.outReceived(...) 199 bytes
[-] TPP.outReceived(...) 146 bytes
[-] TPP.outReceived(...) 8 bytes
[-] TPP.outReceived(...) 33 bytes
[-] TPP.outReceived(...) 41 bytes
[-] TPP.outReceived(...) 77 bytes
[-] TPP.outReceived(...) 1228 bytes
[-] TPP.outReceived(...) 8192 bytes
[-] TPP.outReceived(...) 4567 bytes
[-] TPP.inConnectionLost()
[-] sending eof
[-] exitCode: 0
Successful case (client-side shell session):
$ GIT_TRACE=1 git clone ssh://user@localhost:2222/Users/bshi/sandbox/poop
trace: built-in: git 'clone'
'ssh://user@localhost:2222/Users/bshi/sandbox/poop'
Initialized empty Git repository in /tmp/poop/.git/
trace: run_command: 'ssh' '-p' '2222' 'user at localhost' 'git-upload-pack
'\''/Users/bshi/sandbox/poop'\'''
user at localhost's password:
trace: run_command: 'index-pack' '--stdin' '-v' '--fix-thin'
'--keep=fetch-pack 763 on Bo-Shis-MacBook-Pro.local'
trace: exec: 'git' 'index-pack' '--stdin' '-v' '--fix-thin'
'--keep=fetch-pack 763 on Bo-Shis-MacBook-Pro.local'
trace: exec: 'git-index-pack' '--stdin' '-v' '--fix-thin'
'--keep=fetch-pack 763 on Bo-Shis-MacBook-Pro.local'
trace: run_command: 'git-index-pack' '--stdin' '-v' '--fix-thin'
'--keep=fetch-pack 763 on Bo-Shis-MacBook-Pro.local'
remote: Counting objects: 50, done.
remote: Compressing objects: 100% (35/35), done.
remote: Total 50 (delta 16), reused 43 (delta 14)
Receiving objects: 100% (50/50), 12.36 KiB, done.
Resolving deltas: 100% (16/16), done.
Failure case (server side):
[SSHChannel session (0) on SSHService ssh-connection
on SSHServerTransport,2,127.0.0.1] executing command
"git-upload-pack '/Users/bshi/sandbox/poop'"
[-] TPP.outReceived(...) 345 bytes
[-] TPP.outReceived(...) 8 bytes
[-] TPP.outReceived(...) 33 bytes
[-] TPP.outReceived(...) 8192 bytes
[-] TPP.inConnectionLost()
[-] sending eof
[-] TPP.outReceived(...) 5903 bytes
[-] exitCode: 0
[-] sending request exit-status
[-] sending close 0
Failure case ("trace:" messages similar, cut out for readability):
$ GIT_TRACE=1 git clone ssh://user@localhost:2222/Users/bshi/sandbox/poop
user at localhost's password:
...
...
remote: Counting objects: 50, done.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
More information about the Twisted-Python
mailing list