[Twisted-Python] Setting socket options before connect

Glyph glyph at twistedmatrix.com
Wed Jul 17 11:44:05 MDT 2013


On Jul 15, 2013, at 4:44 AM, exarkun at twistedmatrix.com wrote:

> On 01:01 am, glyph at twistedmatrix.com wrote:
>> 
>> On Jul 12, 2013, at 5:09 AM, exarkun at twistedmatrix.com wrote:
>>> On 10:42 am, p.mayers at imperial.ac.uk wrote:
>>>> On 12/07/13 11:34, Itamar Turner-Trauring wrote:
>>>>> Subclass twisted.internet.tcp.Client, override createInternetSocket() so
>>>>> it calls setsockopt() on the socket after you've called base
>>>>> implementation to create it. This breaks some abstraction boundaries, so
>>>>> it isn't great, but very little code duplication is involved.
>>>> 
>>>> Ah, ok. Presumably I also need to subclass Connector and override _makeTransport to use MyClient, then call MyConnector() directly (or subclass the reactor... shudder)
>>>> 
>>>> Should there be something built in to Twisted for this? Should I open a ticket?
>>> 
>>> If you want your code to keep working, or to work with alternate reactor implementations, then you'd *really* rather use a documented, tested interface rather than the hack outlined above.
>> 
>> Does such an API exist today, or should a ticket be filed for one?
> 
> Hm, I'm not *totally* sure what you mean.

Mostly just socratic method: you said "don't do this, do something else", I am just curious what "do something else" is :-).  Given that no documented, tested interfaces actually exist, "use a documented, tested interface" is not presently an option - "create a documented, tested interface" seems to be what you (well, we, really) are suggesting, which means somebody needs to come up with a specification and file a ticket to make the change to Twisted.

> There's the approach Itamar outlined, using APIs such as `twisted.internet.tcp.{Client,Server}`.  I don't think we should codify this as the public, stable, encouraged API to use - for precisely the reasons you give below.

Yup.

> There's various other APIs that are clearly related but definitely don't currently allow you to wedge this functionality in:
> 
> 1) reactor.connectTCP - nowhere to pass extra socket options now, but we could add more arguments to it I suppose.  Doesn't sound very nice to me.

There are two general approaches I can think of here:

 1. pass a callable object to be invoked on the ... socket? transport? ... before connect().  This seems problematic because it could violate lots of assumptions Twisted makes about the socket and create arbitrary I/O problems in the main loop, which won't be reported well.
 2. pass a static description of things to do pre-connect.

To me, 1. analogizes to 'preexec_fn' in subprocess vs. 2. posix_spawn static description of process-state options, which means I like 2. a lot better.  So the question is: is there anything other than setting socket options might want to do pre-connect?  If we can conclusively say not, then I think adding an argument to pass extra socket options seems fine.  If so, then if we can enumerate the other things you might want to do, we could just add arguments for each of them.

> 2) endpoints?  Again, no current support, but it's a place you could add new parameters.  Of course, this isn't a complete solution, since endpoints mostly just use reactor methods to set things up - but if we had a nice endpoints-based API then we could have a gross lower-level API that no one actually has to use.  Still, is "tcp:host=A:port=B:sockopt=TCP_CORK|TCP_QUICKACK" the road we want to go down?

This would still require support from connectTCP anyway, no?  The sockopt:... key is just the, uh, "user interface" (for lack of a better word) for some structured functionality exposed at a lower level.  So... that suggests that it just reverts to case 1.

> 3) More transport methods

That is already supported, in a sense, you can call '.getHandle()' and set socket options on it if you want to be just a little bit platform-specific.  And we have things like setTcpNoDelay already, too, to avoid depending on python socket objects.

> - but this is an incomplete solution, as certain sockopts only make sense before a connection, so once you have a transport it's too late.

Right, it's the timing that seems to be the issue.

> Maybe someone else has some suggestions from a totally different ballpark that solve the problem more pleasantly?

Passing socket options to connectTCP is growing on me, but it sure would be nice to have something nicer.

> Anyhow, I think this certainly means a ticket should be filed for introducing some API - but it seems that a little more discussion about what the API should be will still be necessary.

Agreed.  But then, it's been a couple days and nobody else has contributed to this thread :-).

-g
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130717/80694f8c/attachment-0001.html>


More information about the Twisted-Python mailing list