[Twisted-Python] any progress for Ticket #6346, #6378?

Gelin Yan dynamicgl at gmail.com
Wed May 29 02:51:41 MDT 2013


On Wed, May 29, 2013 at 1:40 PM, Alec Matusis <matusis at yahoo.com> wrote:

> Hi Gelin,
>
> Apparently in the twisted 10.2 / python 2.7 combination in which the bug
> occurs only rarely (every couple of weeks) I am still using twisted’s
> epoll:
>
> epollreactor.py:
>
> from twisted.python import _epoll
>
>
> From: twisted-python-bounces at twistedmatrix.com
> [mailto:twisted-python-bounces at twistedmatrix.com] On Behalf Of Gelin Yan
> Sent: Tuesday, May 28, 2013 9:09 PM
> To: Twisted general discussion
> Subject: Re: [Twisted-Python] any progress for Ticket #6346, #6378?
>
>
>
> On Wed, May 29, 2013 at 10:37 AM, Alec Matusis <matusis at yahoo.com> wrote:
> Hi Gelin,
>
> > As I know there are two epoll implementations in twisted, one that from
> twisted itself and another one comes from standard library since py 2.6.
> I was slightly incorrect in my email: the combination where it happens
> infrequently is actually Twisted 10.2 with python 2.7.3 (not Twisted 10.1
> as
> I wrote).
> How can I check whether twisted is using Python’s epoll or its own in this
> combination? And how can I switch between epoll implementations in my app?
>
> Alec
>
> From: Gelin Yan [mailto:dynamicgl at gmail.com]
> Sent: Tuesday, May 28, 2013 6:42 PM
> To: Alec Matusis
> Subject: Re: [Twisted-Python] any progress for Ticket #6346, #6378?
>
>
> On Wed, May 29, 2013 at 8:31 AM, Alec Matusis <matusis at yahoo.com> wrote:
> >   By reading these comments, It looks like recent version of twisted
> >may
> >suffer from this problem. So what version of twisted should  I use? Do
> >I
> >need to keep using 10.1?
> This problem is very frequent on Linux with Python 2.7 / Twisted 12
> combination (we had to quickly roll back- cannot operate with that combo).
>
> It never happened with Python 2.6 / Twisted 10 (as in years).
>
> It happens very infrequently with Python 2.7 / Twisted 10.1 (as 1x in
> weeks)
>
> Right now I have a server in this state on Python 2.7 / Twisted 10.1. It
> refuses 30% of new connections, spewing this error:
>
> 2013-05-28 17:28:25-0700 [twisted.internet.protocol.ServerFactory]
> Unhandled
> Error
>         Traceback (most recent call last):
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/python/log.py", line 69, in callWithContext
>             return context.call({ILogContext: newCtx}, func, *args, **kw)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/python/context.py", line 59, in
> callWithContext
>             return self.currentContext().callWithContext(ctx, func, *args,
> **kw)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/python/context.py", line 37, in
> callWithContext
>             return func(*args,**kw)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/epollreactor.py", line 217, in
> _doReadOrWrite
>             why = selectable.doRead()
>         --- <exception caught here> ---
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/tcp.py", line 938, in doRead
>             transport = self.transport(skt, protocol, addr, self, s,
> self.reactor)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/tcp.py", line 752, in __init__
>             self.startReading()
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/abstract.py", line 365, in
> startReading
>             self.reactor.addReader(self)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/epollreactor.py", line 100, in
> addReader
>             self._add(reader, self._reads, self._writes, self._selectables,
> _epoll.IN, _epoll.OUT)
>           File
>
> "/usr/local/encap/Python-2.7.3/lib/python2.7/site-packages/Twisted-10.2.0-py
> 2.7-linux-x86_64.egg/twisted/internet/epollreactor.py", line 88, in _add
>             self._poller._control(cmd, fd, flags)
>           File "_epoll.pyx", line 125, in _epoll.epoll._control
>
>         exceptions.IOError: [Errno 2] No such file or directory
>
>
> If I restart the server, it will be stable for a couple of weeks, until
> some
> event triggers this problem again. We hope that this problem is fixed
> before
> we are really forced to migrate to newer Python, otherwise there's no real
> upgrade path for us using Twisted.
>
> From: twisted-python-bounces at twistedmatrix.com
> [mailto:twisted-python-bounces at twistedmatrix.com] On Behalf Of Gelin Yan
> Sent: Saturday, May 25, 2013 7:51 AM
> To: Twisted general discussion
> Subject: Re: [Twisted-Python] any progress for Ticket #6346, #6378?
>
>
>
> On Sat, May 25, 2013 at 3:22 AM, <exarkun at twistedmatrix.com> wrote:
> On 04:21 pm, dynamicgl at gmail.com wrote:
> >Hi All
>
> Hi Gelin,
>
> I don't have anything extra to add about the status of those two tickets
> (I think you meant #6373 rather than #6378, by the way) - which is
> exactly why I'm replying.
>
> Twisted development doesn't take place in secret.  The issue tracker
> should always reflect the latest state of the development process.  If
> an issue has had no comments and a branch no commits for a couple
> months, then it's almost certainly because no one has worked on the
> issue for a couple months.
>
> So you generally don't need to ask if there has been any progress,
> because progress will always be reflected in ticket history or in
> version control.
> >    I am quite concerned about these two tickets due to recently I may
> >port
> >one of my systems to Ubuntu.
> Perhaps you can help out with the tickets, then!  As hopefully you've
> noticed, one of them is currently up for review.  This means a developer
> believes a fix for the problem has been created and is just waiting for
> someone else to look it over and agree that it should be applied to
> trunk.  If this happens, then the fix will be available in the next
> release of Twisted, 13.1.0, which will probably happen in just a few
> weeks.
>
> For more details about how the review process works, please take a look
> at this wiki page:
>
>     http://twistedmatrix.com/trac/wiki/ReviewProcess
>
> Jean-Paul
> >   By reading these comments, It looks like recent version of twisted
> >may
> >suffer from this problem. So what version of twisted should  I use? Do
> >I
> >need to keep using 10.1?
> >
> >  I always use the latest version on windows.
> >
> >Regards
> >
> >gelin yan
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
> Hi exarkun
>
>     I noticed "review" tag has been removed for a while so does it imply
> this ticket is still pending?
>
>    I am not quite familiar how epoll works on linux so my help probably can
> be neglected. I event can't sure how to reproduce this bug.
>
>    By the way, Is it possible to use the "old" epoll implementation instead
> of the built in epoll from python with the latest version? It looks like
> this old one works quite well.  Thanks
>
> Regards
>
> gelin yan
>
>
> Hi Alec
>
>    Thanks for your info. As I know there are two epoll implementations in
> twisted, one that from twisted itself and another one comes from standard
> library since py 2.6. If I was correct, since 10.2, twisted has been used
> the standard epoll if applicable. I am curious which one did you use? Have
> you tried py 2.7 & epoll from twisted itself?
>
> Regards
>
> gelin yan
>
> Hi Alec
>
>   there is a
> link:
> http://twistedmatrix.com/trac/browser/tags/releases/twisted-13.0.0/twi
> sted/internet/epollreactor.py
>
> I strip the related part and paste here:
>
> try:
>   from select import epoll
> except ImportError:
>   from twisted.python import _epoll
>
> in the latest trunk  there is only  from select import epoll.
>
> I am not sure which one you are using. epoll has been available in standard
> library since Py 2.6. Twisted has been used its own epoll wrapper for years
> until recently.
>
> Regards
>
> gelin yan
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

Hi Alec

   I suggest giving the work around provided by Itamar a try. It might
alleviate your situation.

Regards

gelin yan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20130529/572f9227/attachment.html>


More information about the Twisted-Python mailing list