id	summary	reporter	owner	description	type	status	priority	milestone	component	resolution	keywords	cc	branch	branch_author	launchpad_bug
4387	API for distributing `ITransport` and `IListeningPort` implementations to different processes	glyph		"One design pattern for high-performance servers is to have a master process which accepts incoming connections, but then dispatches those connections to sub-processes or peer processes which handle the actual content of the connection, either immediately or after some initial handshake.

While it's always possible to simply forward traffic, it is of course more efficient for the master accepting process to just get out of the way completely and let the application process handle all system calls and buffering itself.

On UNIX, you can use [http://www.opengroup.org/onlinepubs/000095399/functions/sendmsg.html the sendmsg] [http://www.opengroup.org/onlinepubs/007908799/xns/recvmsg.html and recvmsg] functions along with the  [http://www.opengroup.org/onlinepubs/007908799/xns/syssocket.h.html SCM_RIGHTS ancillary data type] to transmit file descriptors between processes via an `AF_UNIX` socket.

On Windows, I believe [http://msdn.microsoft.com/en-us/library/ms741565(VS.85).aspx the WSADuplicateSocket function] serves the same purpose, albeit in a more restricted form (only sockets may be transferred, not arbitrary handles).

You can already implement bits and pieces of this using Twisted, by inspecting internal state in the reactor, but we should have a high-level, portable interface for establishing the channel to send sockets to a different process.

[http://www.fluendo.com/ Fluendo] has also generously voiced some interest in contributing [https://code.fluendo.com/flumotion/trac/browser/flumotion/trunk/flumotion/twisted/fdserver.py their implementation of this idiom] to Twisted.

I have also [http://trac.calendarserver.org/changeset/5425 recently done some work] on [http://calendarserver.org/ Darwin Calendar & Contacts Server] to implement this feature.  I can use that code as a basis for the API in Twisted as well.

While I tend to prefer my own implementation for its minimalist usage of C (fluendo implements FD-sending at the C level, whereas I just have a very literal-minded `sendmsg` wrapper, with FD-sending implemented in Python) it isn't quite general enough to use as-is.  At least, the high-level bits aren't, as they rely on application-level (protocol-level) hooks to report status back to the parent process, whereas that should be the responsibility of the transport.  However, as I said, the lowest-level bits are a fairly straightforward binding to the `sendmsg` and `recvmsg` C library functions; essentially a marginally improved rewrite of [http://twistedmatrix.com/trac/browser/sandbox/exarkun/papers/2004/pycon/migration/sendmsg the sendmsg code which has been floating around in] [http://twistedmatrix.com/trac/browser/sandbox/pahan/sendmsg various people's]  [http://twistedmatrix.com/trac/browser/sandbox/therve/prefork sandboxes] for quite a while.

While I mention the Windows here to provide a reference point for the portability restrictions of a high-level API, we can definitely defer that work to a separate ticket.

Also, given that you can inherit the connection in multiple different ways - via file descriptor inheritance, via `connect()` to a numeric UNIX socket, `connect()` to a UNIX named socket, or via a process identifier on Windows, we may need to provide some platform-specific bootstrapping APIs which result in a common interface once invoked."	enhancement	new	normal		core			thomas@… marcandre.lureau@… bra@… manishtomar.public@…