Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#3831 defect closed fixed (fixed)

Twisted sometimes can't create the waker on Windows when AVG is installed and Active Surf Shield is enabled

Reported by: glyph Owned by:
Priority: highest Milestone:
Component: core Keywords: win32
Cc: itamar, marienz, thijs Branch: branches/windows-getsockname-3831
(diff, github, buildbot, log)
Author: pahan Launchpad Bug:

Description

Sometimes, rarely, on certain Windows installations, users see this error:

Traceback (most recent call last):

  File "whatever.py", line 1, in <module>

    from twisted.internet import reactor

  File "twisted\internet\reactor.py", line 38, in <module>

    selectreactor.install()

  File "twisted\internet\selectreactor.py", line 200, in install

    reactor = SelectReactor()

  File "twisted\internet\selectreactor.py", line 72, in __init__

    posixbase.PosixReactorBase.__init__(self)

  File "twisted\internet\posixbase.py", line 170, in __init__

    ReactorBase.__init__(self)

  File "twisted\internet\base.py", line 446, in __init__

    self._initThreads()

  File "twisted\internet\base.py", line 869, in _initThreads

    self.installWaker()

  File "twisted\internet\posixbase.py", line 206, in installWaker

    self.waker = _Waker(self)

  File "twisted\internet\posixbase.py", line 77, in __init__

    client.connect(server.getsockname())

  File "<string>", line 1, in connect

socket.error: [Errno 10049] The requested address is not valid in its context

This occurs because socket.getsockname() spuriously returns "0.0.0.0" for the host part of the address, despite the fact that it was explicitly bound to "127.0.0.1".

Possibly related tickets:

Attachments (1)

getsockfail.py (331 bytes) - added by PenguinOfDoom 5 years ago.
minimal example

Download all attachments as: .zip

Change History (19)

comment:1 Changed 5 years ago by glyph

On IRC, some users are reporting that this may have something to do with the AVG virus scanner. One user who had it installed ("Ballig"), but had the firewall disabled, was experiencing the problem. Hopefully they can fill in some more details here.

No amount of stress-testing socket.getsockname() causes the problem to happen for me on a vanilla Windows XP installation.

comment:2 follow-ups: Changed 5 years ago by glyph

  • Summary changed from Twisted sometimes can't create the waker on Windows to Twisted sometimes can't create the waker on Windows when AVG is installed and Active Surf Shield is enabled

I installed AVG Free and the problem immediately manifested.

I uninstalled and it went away.

Specifically, I disabled "Active Surf Shield" and I could make the problem start and stop on demand in a running Python process. (Open the "advanced settings" UI, click on "LinkScanner", check/uncheck "Active Surf Shield" and hit "Apply".)

Some data points:

  • the host is occasionally wrong (it's "0.0.0.0" approximately 1 out of every 3000 sockets, but the number is definitely non-deterministic) but the port number seems to remain correct.
  • the behavior is an intermittent failure of bind(), not of getsockname(); if a socket erroneously reports its address as "0.0.0.0", it will continue to do so forever.

Someone should report this problem to AVG, but given that probably a zillion people have this installed on their computers and inevitably don't know how to upgrade, I think we should work around the buggy behavior by always passing "127.0.0.1" for the host.

comment:3 Changed 5 years ago by glyph

  • Owner changed from glyph to PenguinOfDoom

comment:4 Changed 5 years ago by pahan

  • Author set to pahan
  • Branch set to branches/windows-getsockname-3831

(In [26924]) Branching to 'windows-getsockname-3831'

comment:5 follow-up: Changed 5 years ago by PenguinOfDoom

I attached a minimal example. If you run it, you might observe that the failing port is always 1028.

Port 1028 is part of some heinous undocumented ninja DCOM shit which might or might have disappeared before Windows XP. Naturally, there are exploits that use this port. AVG, in its infinite kindness, protects port 1028.

Changed 5 years ago by PenguinOfDoom

minimal example

comment:6 Changed 5 years ago by itamar

  • Cc itamar added

If we are certain 0.0.0.0 means 127.0.0.1, we should also fix this for transport.getHost().

comment:7 in reply to: ↑ 2 Changed 5 years ago by exarkun

Replying to glyph:

I installed AVG Free and the problem immediately manifested.

What test were you performing?

comment:8 in reply to: ↑ 5 Changed 5 years ago by exarkun

Replying to PenguinOfDoom:

I attached a minimal example. If you run it, you might observe that the failing port is always 1028.

Hmm. This never fails on my machine, either with AVG or without it, either with LinkScanner enabled or disabled.

comment:9 follow-up: Changed 5 years ago by exarkun

After jiggling some things around, the minimal example did start to fail, but the failing port jumps around a lot; it is not limited to 1028. Still, this seems to demonstrate the misbehavior pretty well. After reverting to a system snapshot taken prior to AVG installation, the example stopped failing.

If we are certain 0.0.0.0 means 127.0.0.1, we should also fix this for transport.getHost().

I'm not sure we're certain of that. I don't think anyone has tested other addresses.

comment:10 in reply to: ↑ 9 Changed 5 years ago by PenguinOfDoom

Replying to exarkun:

If we are certain 0.0.0.0 means 127.0.0.1, we should also fix this for transport.getHost().

I'm not sure we're certain of that. I don't think anyone has tested other addresses.

I did! No matter what address you pick, getsockname() returns 0.0.0.0 (as long as you choose or get assigned a broken port). It also returns 0.0.0.0 if you bound to 0.0.0.0 to begin with :)

comment:11 Changed 5 years ago by PenguinOfDoom

  • Keywords review added
  • Owner PenguinOfDoom deleted
  • Priority changed from normal to highest

comment:12 Changed 5 years ago by glyph

  • Keywords review removed
  • Owner set to PenguinOfDoom
  1. The test docstring really needs an explanation of why we would want to ignore the getsockname result. If I didn't know about this ticket and I saw that test, I would probably think it was added mistakenly, and "fix" it to pay attention to getsockname (unless I noticed the comment in the implementation, of course).
  2. The patching that the test does is overbroad and might have some bad consequences if any other Python program attempts to do something with sockets while this test is running. At the very least I would recommend patching posixbase rather than socket; better would be be to move the waker code to its own module and patch that; even better would be to patch only the _Win32Waker class.

comment:13 Changed 5 years ago by exarkun

Don't forget that functions can take parameters too. We don't always have to use patch.

comment:14 Changed 5 years ago by marienz

  • Cc marienz added

comment:15 in reply to: ↑ 2 Changed 5 years ago by thijs

  • Cc thijs added

Replying to glyph:

I installed AVG Free and the problem immediately manifested.

I uninstalled and it went away.

Which version of AVG Free was that? Maybe it's fixed in more recent versions?

comment:16 Changed 5 years ago by PenguinOfDoom

  • Resolution set to fixed
  • Status changed from new to closed

Seems to work fine with AVG 9.0. Hooray.

comment:17 Changed 5 years ago by Szumo

Definitely appears with AVG 8.5 though.

comment:18 Changed 4 years ago by <automation>

  • Owner PenguinOfDoom deleted
Note: See TracTickets for help on using tickets.