[Twisted-web] Garbage SSL application data

[Joshua Boverhof]

I'm using the twisted.web.getPage for https, but for some reason garbage 
application data is preceeding the data I'm specifying (it completes the 
handshake).   This is causing a java server to bomb because it is 
expecting a POST or GET command:  It does a readLine, gets nothing and 
throws an IOException. 

A twisted.web server digests the message w/o problem, apparently it's 
ignoring the garbage bytes (which change w/each invocation).

I'm also getting a few failures w/ "twisted.test.test_ssl".  I'm 
interpretting the error message below "startTLS doesn't empty buffer 
before starting TLS. :(" as the cause of my problem...

I'm seeing this behavior w/clients running on a Tiger laptop and a Linux 
box w/the following packages:

    Twisted-2.0
    TwistedWeb-0.5
    pyOpenSSL-0.6/openssl-0.9.7

thanks,
-josh

Code

    f = client.getPage(url,
            contextFactory=None,
            postdata=DATA,
            agent=agent,
            method=method,
            headers=headers)


SSLDump after handshake

    ---------------------------------------------------------------
2 13 0.6668 (0.0629)  C>SV3.0(24)  application_data
    ---------------------------------------------------------------
application_data(23)/major(3)/minor(0)/length(24)

17 03 00 00 18
        a7 f3 45 ee 7b e1 19 d6 0c 0f fa 7a 9d a4 eb 98 ce 84 7f e3 1e 
a1 cc 70
        GARBAGE


    ---------------------------------------------------------------
2 14 0.6668 (0.0000)  C>SV3.0(1224)  application_data
    ---------------------------------------------------------------
application_data(23)/major(3)/minor(0)/length(1224)

17 03 00 04 c8
        50 4f 53 54 20 2f 77 73 72 66 2f  .....
        P  O  S  T     /  w  s  r  f  /   .....



bundy:~/Desktop/Python/Twisted-2.0.0/bin boverhof$ ./trial 
twisted.test.test_ssl
....
....

[TODO]: testTLS (twisted.test.test_ssl.SpammyTLSTestCase)

startTLS doesn't empty buffer before starting TLS. :(


  File 
"/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/internet/defer.py", 
line 117, in maybeDeferred
    result = f(*args, **kw)
  File 
"/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/test/test_ssl.py", 
line 285, in testTLS
    TLSTestCase.testTLS(self)
  File 
"/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/test/test_ssl.py", 
line 225, in testTLS
    UnintelligentProtocol.pretext + UnintelligentProtocol.posttext
  File 
"/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/trial/unittest.py", 
line 415, in <lambda>
    failUnlessEqual = lambda self, a, b, msg=None: failUnlessEqual(a, b, 
msg)
  File 
"/Users/boverhof/Desktop/Python/Twisted-2.0.0/twisted/trial/assertions.py", 
line 77, in failUnlessEqual
    raise FailTest, (msg or '%r != %r' % (first, second))
twisted.trial.assertions.FailTest: ['first line', 'last thing before tls 
starts', 'STARTTLS'] != ['first line', 'last thing before tls starts', 
'STARTTLS', 'first thing after tls started', 'last thing ever']
-------------------------------------------------------------------------------
Ran 10 tests in 5.512s

PASSED (expectedFailures=3, successes=7)