[Twisted-Python] What is the minimum effort solution to make inetd-managed twisted-based application?

[exarkun at twistedmatrix.com]

On 14 Jul, 04:00 pm, twisted-web at udmvt.ru wrote:
>On Wed, Jul 14, 2010 at 02:31:46AM -0000, exarkun at twistedmatrix.com 
>wrote:
>>I'll answer whatever questions I can. :)
>
>Oh, I have some questions...
>I asked some questions in the form of suggestions, well, don't take 
>them seriously,
>I only ask, not really propose anything. However I'm ready to hear you
>about what I'm wrong with.
>
>Here, what I have discovered so far:
>There are transport factories in twisted, not defined as such, but 
>really they are,
>like tcp.Port and tcp.Connector.
>
>They interface with protocol factories. That interface is asymmetric 
>between
>server protocol factory and client protocol factory. Well, the 
>implementation
>of client's side protocol factory and server's side should differ, but 
>why should interfaces?

startedConnecting, clientConnectionFailed, and clientConnectionLost only 
exist to facilitate client reconnection.  See the end of the email for 
more about that.
>That is okay, while you only have different and asymmetric transport 
>factories
>the ones for client transports and the others for server transports.
>But is there really something in created and connected transport,
>that makes it server's or client's ? I guess nothing, except protocol 
>instance attached.

That's probably true.  And there could probably be a little less 
duplication of code between the Client and Server transports (although 
there's not a *lot* now).
>
>Q: Why should protocol factory interface "bother" about client/server 
>dichotomy?
>For example, why should I be limited to only using PBClientFactory with 
>connectXXX()
>variants of transport factories?
>Why should I do not want (under some crazy circumstances) to use 
>reverse-connects
>and run PBServerFactory with connectXXX()?
>Well, maybe not PB, but what about other protocols?
>Should it work? Well, I suppose, it would break if you try now, but is
>is supposed to be that asymmetric?

You can certainly implement protocols that don't care about which side 
of the underlying transport is the client and which side is the server. 
It probably wouldn't be much work to get PB to be such an 
implementation.  But apparently no one really cares about this, 
otherwise it might have come up before. ;)
>And after all, there is absolute symmetry in UDP-based transports.

Okay.  But SOCK_DGRAM is completely different from SOCK_STREAM, so I 
don't know if this comparison really means anything.
>Q: Consider socketpair() variant - a pair of completely symmetric 
>mutually
>connected sockets.
>Should I provide transport factory for use with server protocol 
>factories
>and another transport factory for use with client protocol factories?
>
>That idea sounds stupid to me, there can be no difference in the 
>implementation,
>except in the interfaces to protocol factories.

I'd probably start off with a low-level interface that's completely 
symmetric and doesn't involve factories.  After all, it's not like 
socketpairs can spontaneously spring into existing (like client 
connections on a server can).  On top of that, if you want, construct an 
API using factories to support multiple connections.
>
>I see there are some code, that can be moved from tcp.Port and 
>tcp.Connector
>to some common base class of an abstract transport factory. That class 
>could
>interfere with protocol factory without knowing whether it is server or 
>client.
>Protocol factory already knows about it's own asymmetry.
>
>I do identify the events, that protocol factory receives from transport 
>factory as these,
>please correct me if I'm wrong:
>
>global (factory-context) events:
>- start factory                         doStart
>- entering transport creation phase     startedConnecting 
><server: when bind&listen succeeds why does anybody
>                                                                 not 
>invoke something like startedListenig ?>
>- stop factory                          doStop

Make sure you don't get confused when you use "start factory" and "stop 
factory" as labels for doStart and doStop.  The base implementation of 
doStart calls startFactory and the base implementation of doStop calls 
stopFactory.

There's no startedListening because no one imagined there might be a 
kind of port that could only be created asynchronously.  listenTCP, 
listenSSL, listenUNIX, listenUDP, listenMulticast can all complete 
immediately, so they all return an IPort provider.  So if you wanted the 
startedListening callback, you can just call it yourself immediately 
after you call the listenXXX method.

This may be an oversight for the general case.  Vertex, for example, 
certainly wants to expose an asynchronous listen API.  Other similar 
use-cases might be a listen API that uses UPnP to request a hole in a 
firewall.

Still, listening is a one-off event.  So if you can at least return a 
Deferred from your listen API, then when the Deferred fires, you have 
your "started listening" notification.
>
>per transport connection events:
>- on transport creation                 <none exists?why?>      <after 
>connect() or listen() returns result, at this point
>                                                                 we may 
>decide to prevent further transport instantiation,
>especially if we are server-side protocol factory>

The factory can choose to do this in buildProtocol.
>- transport creation failed             clientConnectionFailed 
><server: direct analogy - accept() may return an error,
>                                                                 but 
>that event not exists for server. why?>

I don't think accept() failing is really analogous to 
clientConnectionFailed.  Nevertheless, exposing accept() failures to 
application code is definitely a good idea.  There's currently a ticket 
open for adding that feature.
>- transport created succesfully         buildProtocol           <a 
>request to build a protocol instance implies, that
>transport instance have been succesfully created. or not?>
>- build protocol                        buildProtocol

In general, no such implication exists.  It just means someone wants a 
new protocol instance to do something with.  Usually that's because 
there's a new transport to connect it to, but maybe it's just a unit 
test calling the method.
>- after transport and protocol creation clientConnectionMade    <in 
>t.s.pb.PBXXXXFactory, event sent by protocol instance,
>                                                                 name 
>not defined in any Interface, why not?>


PB wanted to put some logic on the factory when a new connection was 
first established.  I'm not sure why it does this instead of putting the 
logic in Protocol.connectionMade.
>- transport closed                      clientConnectionLost    <why 
>not define this for server too to be consistent?>

Consistency is alright, but sometimes it's not a sufficient 
justification.  Over the years, I think the *existence* of 
ClientFactory.clientConnectionLost has caused more confusion and 
questions than the *lack* of ServerFactory.clientConnectionLost.  In 
both cases, the information is available from Protocol.connectionLost.
>They are not client/servers asymmetric as I see them, but have 
>asymmetric names in twisted
>and sometimes when server variants exists, they have no fixed names 
>defined in Interface classes :(
>
>What is the difference between doStart and startedConnecting (together 
>with imaginary startedListening)?

doStart is called when a factory is hooked up to a network event source. 
That means there'll probably be just one call to doStart.  For a 
reconnecting client factory, startedConnecting may be called many times.

But really, doStart/startFactory/doStop/stopFactory are pretty obscure 
and rarely used.  I wouldn't be surprised if we could come up with a 
better way to present this information pretty easily.
>And who is responsible for sending which events? That is not defined in 
>documentation.

The reactor implementation.  There's no "how to implement a reactor" 
document because people don't do that very often.
>What do you think about these names:
>doStart()
>transportFactoryStarted()
>onBeforeTransportCreate()
>connectionFailed()
>buildProtocol()
>connectionMade()
>connectionLost()
>doStop()
>Well, I understand, you will veto them arguing by lot of code using old 
>names,
>but I am interested to know will they break the abstraction of protocol 
>factory?

What are they methods on?  What are the precise semantics of each?
>Can please anyone explain the asymmetry of the interfaces of protocol 
>factories
>and tell how can it be useful for me.

There's a lot of noise amongst the long-time Twisted developers about 
how client factories are no good and we should just dump them, leaving 
only server factories.  That doesn't exactly make things symmetric, but 
it gets rid of the asymmetric factory interfaces. :)
>Thanks for your time.
>
>Alexey.
>
>--
>D1 D3C2C0C6C5CDC8C5CC,
>A0CBC5CAD1C5C9 B8CFC0C3C8CD
>D1C8D1D2C5CCCDDBC9 C0C4CCC8CDC8D1D2D0C0D2CED0
>D6C5D5C0 CFC5D0C5C4C0D7C8 C4C0CDCDDBD5
>D2C5D5CDC8D7C5D1CACEC3CE D6C5CDD2D0C0 D2C5CBC5CACECCCCD3CDC8CAC0D6C8C9
>AEA0AE "A2CECBC3C0B2C5CBC5CACECC"
>D4C8CBC8C0CB C2 B3C4CCD3D0D2D1CACEC9 D0C5D1CFD3C1CBC8CAC5.
>
>_______________________________________________
>Twisted-Python mailing list
>Twisted-Python at twistedmatrix.com
>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python