[Twisted-Python] Weird epoller bug

[Robert DiFalco]

One more thing, there seems to be an error that can occur in _newclient.py
where _finishedRequest will be type None and it's errback will be attempted
to be called. Consider this in _newclient.py. If an exception occurs in in
maybeDeferred before _finishedRequest is assigned, then it will be None
when it is chained elsewhere.

def request(self, request):
    """
    Issue C{request} over C{self.transport} and return a L{Deferred} which
    will fire with a L{Response} instance or an error.

    @param request: The object defining the parameters of the request to
       issue.
    @type request: L{Request}

    @rtype: L{Deferred}
    @return: The deferred may errback with L{RequestGenerationFailed} if
        the request was not fully written to the transport due to a local
        error.  It may errback with L{RequestTransmissionFailed} if it was
        not fully written to the transport due to a network error.  It may
        errback with L{ResponseFailed} if the request was sent (not
        necessarily received) but some or all of the response was lost.  It
        may errback with L{RequestNotSent} if it is not possible to send
        any more requests using this L{HTTP11ClientProtocol}.
    """
    if self._state != 'QUIESCENT':
        return fail(RequestNotSent())

    self._state = 'TRANSMITTING'
    _requestDeferred = maybeDeferred(request.writeTo, self.transport)

    def cancelRequest(ign):
        # Explicitly cancel the request's deferred if it's still trying to
        # write when this request is cancelled.
        if self._state in (
                'TRANSMITTING', 'TRANSMITTING_AFTER_RECEIVING_RESPONSE'):
            _requestDeferred.cancel()
        else:
            self.transport.abortConnection()
            self._disconnectParser(Failure(CancelledError()))
    self._finishedRequest = Deferred(cancelRequest)




On Tue, Dec 17, 2019 at 11:19 AM Robert DiFalco <robert.difalco at gmail.com>
wrote:

> I have a theory that since this runs every 20 seconds that it is falling
> behind somehow. Ulimits is too low, twisted is maybe eating the too many
> file handles exception and then trying to close a handle that doesn't
> actually exist. Dunno.
>
> On Tue, Dec 17, 2019 at 10:18 AM Robert DiFalco <robert.difalco at gmail.com>
> wrote:
>
>> We recently switched from these versions and recently our TLS health
>> check using treq seems to be using more and more file handles and getting
>> this exception. Additionally our health check gets response time outs at a
>> much higher frequency. Any tips on how I would go about debugging this? Or
>> is this perhaps a known issue in one of the versions below and I need to
>> either bump up or down a release version? Thanks!
>>
>> twisted 16.6.0 -> 19.19.0
>> treq 15.1.0 -> 18.6.0
>>
>>   File
>> "/home/foo/foo/local/lib/python2.7/site-packages/twisted/internet/epollreactor.py",
>> line 183, in removeWriter
>>     EPOLLOUT, EPOLLIN)
>>   File
>> "/home/foo/foo/local/lib/python2.7/site-packages/twisted/internet/epollreactor.py",
>> line 160, in _remove
>>     self._poller.unregister(fd)
>> exceptions.IOError: [Errno 2] No such file or directory
>>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20191217/eacc6ce7/attachment.html>