[Twisted-Python] UDP servers and socket queues

Phil Mayers p.mayers at imperial.ac.uk
Mon Apr 21 14:48:09 EDT 2003


On Mon, Apr 21, 2003 at 07:29:56PM +0100, Phil Mayers wrote:
> 
> I figured something like that. Does anyone know if the various socket
> readiness abstractions (select(), poll(), kqueue, RT signals, etc.)
> given you sufficient information to know when you can do multiple reads?
> 
> The other option I guess would be to for connection-less sockets to do:
> 
> while 1:
>     try:
>         data, addr = self.socket.recvfrom(65535)
>         self.protocol.datagramReceived(data, addr)
>     except socket.error, se:
>         if se.args[0]==errno.EWOULDBLOCK:
>             break
>     except:
>         log.deferr()
> 
> ...in doRead (line 91, twisted/internet/udp.py)?
> 
> -- 
> 

On second thoughts, (having just tested it) that alone won't be
sufficient, although it does make things a lot cleaner. The problem also
is:

proto = RPCProtocol()
for host in hosts:
    ep = endpoint(proto, host)
    reactor.callLater(0, ep.go)

...the in mainLoop of SelectReactor (or rather the subclass which reads
from the network *first*):


while self.running:
    try:
        while self.running:
            # Still need to reverse the order of the read
            # versus the call of functions
            t2 = self.timeout()
            t = self.running and t2
            self.doIteration(t)
            # Advance simulation time in delayed event
            # processors.
            self.runUntilCurrent()
    except:
        log.msg("Unexpected error in main loop.")
        log.deferr()
    else:
        log.msg('Main loop terminated.')

...the runUntilCurrent will execute *all* the callLater's that are
queued, which takes a couple of seconds (say) - however, halfway though
the runUntilCurrent, a sufficient number of replies has been returned to
overflow the socket input buffer - so you still need to somehow limit
the amount of work done in runUntilCurrent.

Now, I could put "reactor.callLater(when, ep.go)" then "when += 0.1" but
as the process runs more callLaters will happen, will pile up against
each other, and eventually the same problem will likely occur.

Interesting behaviour though!

-- 

Regards,
Phil

+------------------------------------------+
| Phil Mayers                              |
| Network & Infrastructure Group           |
| Information & Communication Technologies |
| Imperial College                         |
+------------------------------------------+




More information about the Twisted-Python mailing list