RFC 6091: Using OpenPGP keys for TLS authentication

[daniele_athome]

I want to implement a server-to-server XMPP component, authenticating to each other using TLS with OpenPGP keys; but I see TLS API in Twisted are very much anchored to OpenSSL (references to SSL.Context are all over the code). I can code actively on this part, but first I wish to know if it's worth the while to code it on the Twisted side - GnuTLS for python doesn't have bindings for OpenPGP features, shall I develop those instead and use their "integration" with Twisted?

[exarkun]

Hi daniele_athome,

Twisted's SSL support is indeed based entirely on OpenSSL - via pyOpenSSL. To support this feature, pyOpenSSL will first need to support it. Actually, that would probably be most or all of the work necessary - Twisted's SSL support basically inherits all of the features of pyOpenSSL for free.

Since neither pyOpenSSL nor the Python GnuTLS bindings have support for OpenPGP keys, I wonder why you brought up GnuTLS at all. Twisted itself has no support for using GnuTLS for SSL, and the Twisted integration that the Python GnuTLS bindings include are entirely part of that project, not part of Twisted.

Since it sounds like this is either a pyOpenSSL feature or a GnuTLS feature, I'm closing this ticket. That doesn't mean the Twisted project isn't interested in this feature, just that I can't see what work on Twisted is actually necessary. If I've misunderstood, please feel free to re-open the ticket. Thanks.

[glyph]

Hi daniele_athorne,

I basically agree with exarkun's assessment of this ticket; I can't see what Twisted itself needs to do in order to support this (except possibly to add a constructor class method to twisted.internet.ssl.PrivateCertificate; that class needs some additional work in general, though).

But, personally, I'm quite interested in this idea, and I am guessing there are other Twisted users out there who are too. Please feel free to update this ticket with information about our dependencies' progress on this front (like, for example, pyOpenSSL) even if you don't re-open it, just to provide a point of reference; that might also provide more information for later in order to decide whether there is something for Twisted to do.

Thanks.

[daniele_athome]

(after 5 months :D) thanks for your replies. I think the main problem here is supporting GnuTLS. Viable options are:

1. implement RFC 6091 into OpenSSL and consequently in pyOpenSSL 2. make bindings for RFC 6091 for python-gnutls (but it would require support for GnuTLS by twisted #2706) 3. like point (2), but using my own custom implementation only for my own uses (that is, a XMPP server)

What do you suggest? With my skills I might be able to do part of (1) (that is, python bindings and integration with twisted), part of (2) (python bindings for gnutls) and possibly all of (3). What do you suggest?

[exarkun]

Hi daniele_athome,

If you wanted to implement RFC 6091 for OpenSSL, I could lend guidance on exposing that functionality in pyOpenSSL. I have no idea how extensive an undertaking adding this feature to OpenSSL would be, so I won't comment on whether I think that's a good direction to take or not.

And I don't know how the Python bindings for GnuTLS work, so I can't comment on whether that's a good direction to take either.

I can point out that with the introduction of endpoints, introducing an eminently usable GnuTLS-based SSL transport is extremely straightforward, though. An endpoint which set up such a connection would effectively be "first-class" (that is, applications could use it just as easily as they can use Twisted's built-in, OpenSSL-based SSL transport) so I don't think there's any reason not to consider this. The endpoint can also be developed and distributed independent of Twisted. This should be a good thing for you, since it keeps you decoupled from Twisted's release process.

This isn't to say that such functionality shouldn't be included in Twisted, but it would probably be easier to see why it's a good idea to include it once it works and has features Twisted's own SSL support lacks.

I'm also eager to see someone attempt this (third-party SSL implementation) and figure out whether it's really a good idea or not. There's some interest from other people in having SSL implementations based on SecureTransport (OS X) and SChannel (Windows). So you'd be doing those people a favor by breaking ground in this area. :)

So I think I might recommend pursing your option (3) - but in a way which makes the functionality useful to everyone, and potentially in a way that it could ultimately be integrated with Twisted.

Also, as an aside, I don't really agree with #2706 being closed. I think that was an overeager contributor trying to help get the open ticket count down. However, in the absence of any real effort to develop GnuTLS-based SSL to completely replace Twisted's pyOpenSSL-based SSL, the point is academic.

[daniele_athome]

Thanks exarkun. I guess we will talk again when I will have something that can be used on python-gnutls :)

[daniele_athome]

PyGnuTLS OpenPGP bindings

[daniele_athome]

I wrote a patch to pygnutls 1.2.4 (I can't find any source repository anywhere, and AG Project seems to have dropped the project) which interfaces the GnuTLS parts concerning OpenPGP. Tested using some example scripts. Twisted endpoint coming soon I guess. I will seek for your help on IRC very soon :)

[daniele_athome]

It's hard to find any documentation about endpoints - or even about Twisted internals at all. I'm going crazy trying to understand interfaces, protocols, transports, wrappers... just by reading Twisted source code. I really don't know where to start.
Doing it from socket listening would be relatively easy, but starting TLS handshake mid-connection (e.g. for XMPP) is another thing. Twisted current SSL implementation does some tricks with the memory BIO and protocol wrapper that I don't understand.

[glyph]

Replying to daniele_athome:

It's hard to find any documentation about endpoints

Here's some documentation about endpoints:  http://twistedmatrix.com/documents/current/core/howto/endpoints.html

- or even about Twisted internals at all.

Endpoints aren't really twisted "internals"; they're an important public API.

I'm going crazy trying to understand interfaces, protocols, transports, wrappers...

Interfaces:  http://docs.zope.org/zope.interface/ Protocols and transports:  http://twistedmatrix.com/documents/current/core/howto/servers.html

Wrappers are a more general programming thing, so I'm not sure I can help you there.

just by reading Twisted source code. I really don't know where to start.

Start with the documentation, not the source code. If you need a general introduction, many swear by Dave Peticolas's twisted introduction:  http://krondo.com/?page_id=1327

Doing it from socket listening would be relatively easy, but starting TLS handshake mid-connection (e.g. for XMPP) is another thing.

Doing what, exactly? Implementing this ticket? I will certainly agree it's a tricky ticket :).

Twisted current SSL implementation does some tricks with the memory BIO and protocol wrapper that I don't understand.

Do you have any specific questions about these "tricks"?

[daniele_athome]

Replying to glyph:

Replying to daniele_athome: Here's some documentation about endpoints:  http://twistedmatrix.com/documents/current/core/howto/endpoints.html Endpoints aren't really twisted "internals"; they're an important public API.

I've already gone through that, but I could use that API only for a direct listen/connect TLS implementation - mid-connection is another thing (see below about the memory BIO "tricks").

Interfaces:  http://docs.zope.org/zope.interface/ Protocols and transports:  http://twistedmatrix.com/documents/current/core/howto/servers.html

I was talking about the many interfaces and abstractions Twisted uses to do the job. Actually I really appreciate that as a developer using Twisted, but as Twisted developer I'm going crazy :D

Wrappers are a more general programming thing, so I'm not sure I can help you there.

Also here I'm referring actually to the apparently "twisted" way TLS implementation works in Twisted. As far as I understand, it uses a memory BIO to read/write data from/to the application, then protocol forwards data to the real transport (e.g. tcp) but I still have to fully grasp the concepts and the logic behind that.

Start with the documentation, not the source code. If you need a general introduction, many swear by Dave Peticolas's twisted introduction:  http://krondo.com/?page_id=1327

That seems really good stuff - thanks!

Doing what, exactly? Implementing this ticket? I will certainly agree it's a tricky ticket :).

Yes, some aspects of what I want are part of this ticket. My specific needs are startTLS mid-connection in XMPP. You know, thinking about it, I guess it would be less bandwidth-expensive to make a direct TLS connection instead of starting it in cleartext and then requesting TLS - I mean my objective is authenticate over OpenPGP and encrypt the channel, trying to save as much bandwidth as possible).

Do you have any specific questions about these "tricks"?

I gave a deeper look to the source code, I understand the memory BIO logic itself, but I still don't understand why the protocol is replaced (transport.protocol = TLSMemoryBIOProtocol) and then a "bypass" transport is connected to the protocol just created again. Maybe I need to draw this on paper to follow the actual logic :)

[daniele_athome]

P.S. I had no response for the original mainteiner, so I'm forking the project:  https://gitorious.org/pygnutls/pygnutls It is in usable state (just added OpenPGP bindings), but I want to upgrade it to a more recent GnuTLS version (3.x). Also I would like to improve twisted integration.