[Twisted-Python] any progress for Ticket #6346, #6378?

Gelin Yan dynamicgl at gmail.com
Tue May 28 22:09:15 MDT 2013


On Wed, May 29, 2013 at 10:37 AM, Alec Matusis <matusis at yahoo.com> wrote:

> Hi Gelin,
>
> > As I know there are two epoll implementations in twisted, one that from
> twisted itself and another one comes from standard library since py 2.6.
>
> I was slightly incorrect in my email: the combination where it happens
> infrequently is actually Twisted 10.2 with python 2.7.3 (not Twisted 10.1
> as
> I wrote).
> How can I check whether twisted is using Python’s epoll or its own in this
> combination? And how can I switch between epoll implementations in my app?
>
> Alec
>
> From: Gelin Yan [mailto:dynamicgl at gmail.com]
> Sent: Tuesday, May 28, 2013 6:42 PM
> To: Alec Matusis
> Subject: Re: [Twisted-Python] any progress for Ticket #6346, #6378?
>
>
> On Wed, May 29, 2013 at 8:31 AM, Alec Matusis <matusis at yahoo.com> wrote:
> >   By reading these comments, It looks like recent version of twisted
> >may
> >suffer from this problem. So what version of twisted should  I use? Do
> >I
> >need to keep using 10.1?
> This problem is very frequent on Linux with Python 2.7 / Twisted 12
> combination (we had to quickly roll back- cannot operate with that combo).
>
> It never happened with Python 2.6 / Twisted 10 (as in years).
>
> It happens very infrequently with Python 2.7 / Twisted 10.1 (as 1x in
> weeks)
>
> Right now I have a server in this state on Python 2.7 / Twisted 10.1. It
> refuses 30% of new connections, spewing this error:
>
> 2013-05-28 17:28:25-0700 [twisted.internet.protocol.ServerFactory]
> Unhandled
> Error
>         Traceback (most recent call last):
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/python/log.py", line 69, in callWithContext
>             return context.call({ILogContext: newCtx}, func, *args, **kw)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/python/context.py", line 59, in
> callWithContext
>             return self.currentContext().callWithContext(ctx, func, *args,
> **kw)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/python/context.py", line 37, in
> callWithContext
>             return func(*args,**kw)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/epollreactor.py", line 217, in
> _doReadOrWrite
>             why = selectable.doRead()
>         --- <exception caught here> ---
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/tcp.py", line 938, in doRead
>             transport = self.transport(skt, protocol, addr, self, s,
> self.reactor)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/tcp.py", line 752, in __init__
>             self.startReading()
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/abstract.py", line 365, in
> startReading
>             self.reactor.addReader(self)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/epollreactor.py", line 100, in
> addReader
>             self._add(reader, self._reads, self._writes, self._selectables,
> _epoll.IN, _epoll.OUT)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/epollreactor.py", line 88, in _add
>             self._poller._control(cmd, fd, flags)
>           File "_epoll.pyx", line 125, in _epoll.epoll._control
>
>         exceptions.IOError: [Errno 2] No such file or directory
>
>
> If I restart the server, it will be stable for a couple of weeks, until
> some
> event triggers this problem again. We hope that this problem is fixed
> before
> we are really forced to migrate to newer Python, otherwise there's no real
> upgrade path for us using Twisted.
>
> From: twisted-python-bounces at twistedmatrix.com
> [mailto:twisted-python-bounces at twistedmatrix.com] On Behalf Of Gelin Yan
> Sent: Saturday, May 25, 2013 7:51 AM
> To: Twisted general discussion
> Subject: Re: [Twisted-Python] any progress for Ticket #6346, #6378?
>
>
>
> On Sat, May 25, 2013 at 3:22 AM, <exarkun at twistedmatrix.com> wrote:
> On 04:21 pm, dynamicgl at gmail.com wrote:
> >Hi All
>
> Hi Gelin,
>
> I don't have anything extra to add about the status of those two tickets
> (I think you meant #6373 rather than #6378, by the way) - which is
> exactly why I'm replying.
>
> Twisted development doesn't take place in secret.  The issue tracker
> should always reflect the latest state of the development process.  If
> an issue has had no comments and a branch no commits for a couple
> months, then it's almost certainly because no one has worked on the
> issue for a couple months.
>
> So you generally don't need to ask if there has been any progress,
> because progress will always be reflected in ticket history or in
> version control.
> >    I am quite concerned about these two tickets due to recently I may
> >port
> >one of my systems to Ubuntu.
> Perhaps you can help out with the tickets, then!  As hopefully you've
> noticed, one of them is currently up for review.  This means a developer
> believes a fix for the problem has been created and is just waiting for
> someone else to look it over and agree that it should be applied to
> trunk.  If this happens, then the fix will be available in the next
> release of Twisted, 13.1.0, which will probably happen in just a few
> weeks.
>
> For more details about how the review process works, please take a look
> at this wiki page:
>
>     http://twistedmatrix.com/trac/wiki/ReviewProcess
>
> Jean-Paul
> >   By reading these comments, It looks like recent version of twisted
> >may
> >suffer from this problem. So what version of twisted should  I use? Do
> >I
> >need to keep using 10.1?
> >
> >  I always use the latest version on windows.
> >
> >Regards
> >
> >gelin yan
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
> Hi exarkun
>
>     I noticed "review" tag has been removed for a while so does it imply
> this ticket is still pending?
>
>    I am not quite familiar how epoll works on linux so my help probably can
> be neglected. I event can't sure how to reproduce this bug.
>
>    By the way, Is it possible to use the "old" epoll implementation instead
> of the built in epoll from python with the latest version? It looks like
> this old one works quite well.  Thanks
>
> Regards
>
> gelin yan
>
>
> Hi Alec
>
>    Thanks for your info. As I know there are two epoll implementations in
> twisted, one that from twisted itself and another one comes from standard
> library since py 2.6. If I was correct, since 10.2, twisted has been used
> the standard epoll if applicable. I am curious which one did you use? Have
> you tried py 2.7 & epoll from twisted itself?
>
> Regards
>
> gelin yan
>
>
Hi Alec

  there is a link:
http://twistedmatrix.com/trac/browser/tags/releases/twisted-13.0.0/twisted/internet/epollreactor.py

I strip the related part and paste here:

try:
  from select import epoll
except ImportError:
  from twisted.python import _epoll

in the latest trunk  there is only  from select import epoll.

I am not sure which one you are using. epoll has been available in standard
library since Py 2.6. Twisted has been used its own epoll wrapper for years
until recently.

Regards

gelin yan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20130529/4578b081/attachment.html>


More information about the Twisted-Python mailing list