SO_REUSEADDR should be enabled for TCP on windows

[hraban]

hraban:
If my server app crashes, it often leaves its ports open, so that I get a 
CannotListenError 'Address already in use'. (Windows 2000)

itamarst:
SO_REUSEADDR is supposed to allow this. I think I disabled that for TCP
on windows as it caused some failing tests, but in retrospect those
tests are probably less important than the actual functionality it
provides.

[itamarst]

This causes a problem where the same application can bind twice or something
like that to the same port
(twisted.test.test_tcp.CannotBindTestCase.testCannotBind fails). If *other*
processes can't bind to the port though this is fine; we can fix by remembering
ourselves what ports we bound to and not relying on OS to do so. So we should
have a test for other processes, and if that work it can be fixed.

Also there should be a test for SO_REUSEADDR's functionality; someone should dig
up UNPv1 and find what case it is where you need it (disconnected TCP clients in
TIME_WAIT or something?)

Someone should make a policy decision about this for 2.1; we may have to revert
it and downgrade urgency.

[spiv]

Hmm.  It appears SO_REUSEADDR means something different on win32 than it does
for Linux sockets.  See:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp
http://blogs.msdn.com/wndp/archive/2005/08/03/Anthony_Jones.aspx

I don't see any way on win32 to achieve the desired behaviour of allowing
binding a listening port regardless of outstanding connections in wait states
without also allowing this overriding of existing port bindings.  Maybe I'm
misreading these docs?

[itamarst]

If we set both SO_REUSEADDR and SO_EXCLUSIVEADDRUSE this would do what we
want... except that it would also grab the socket from other processes that
haven't set SO_EXCLUSIVEADDRUSE. Windows is *awesome*. Though it does seem to
have some useful options (e.g. SO_CONDITIONAL_ACCEPT would be nice to have in
certain situations). OOh, and read up on SO_PROTECT, it's crackalicious.

Anyway, looks like this can't be fixed. I suggest reverting the patch and
closing the bug :/

[radix]

Well, I won't change the state of this bug, but I also won't consider it as
blocking the release (or is there buggy code in trunk@head?)

[itamarst]

There is buggy code in trunk. Revert my patch to tcp.py that enabled
SO_REUSEADDR on windows.

[radix]

I reverted it, and I'm downgrading this issue to bug.

[ghazel]

The comment that "If *other* processes can't bind to the port though this is 
fine" makes it sound like the assumption is that if you don't set SO_REUSEADDR 
you're in the clear. That is not the case. Other applications can set 
SO_REUSEADDR even if you have not, and smash your binding.
The only way to prevent this is by using SO_EXCLUSIVEADDRUSE, which you cannot 
use in conjunction with SO_REUSEADDR. It was incorrectly implied that this was 
possible.

So my vote is, make this an option the user can specify.

- It is useful for reclaiming ports in the TIME_WAIT state
- It does not open up the server to any vulnerabilities

The only caveat is that you could potentially be binding to a port that is in 
use by another application. Applications that want to avoid this behavior could 
not use the option, or attempt to bind without flags to see if the port is in 
use when they expect it not to be.

In addition, an option to enable SO_EXCLUSIVEADDRUSE should be considered, in 
order to prevent other applications from reusing a port whether the 
SO_REUSEADDR flag is used or not, as explained earlier.

[Jean-Paul Calderone]

A patch with unit tests would speed this along.

[bwh]

My understanding of Winsock's implementation of SO_REUSEADDR is that if you bind
a socket to a port that has another listening socket, all connection requests
will go to the earlier listener. So enabling SO_REUSEADDR can result in a silent
failure to listen. I don't know whether there's any way to test for this case.

We're working on network drivers and a WSP (Winsock Service Provider) and in the
course of testing we need to install different versions of the WSP and then
restart services. We really do not want an SSH server to fail to restart due to
SO_REUSEADDR being disabled! Currently I am patching
twisted.internet.tcp.Port.createInternetSocket at runtime to achieve the desired
behaviour.

[itamarst]

If there was some way to test whether someone was already listening on a socket
we could enable SO_REUSEADDR, yes. Modulo race conditions, I suppose, so ideally
one could test for the silent failure mode bwh mentioned after the "successful"
bind.

[ghazel]

You could get the entire TCP table and look for the port you're interested in.
http://msdn.microsoft.com/library/default.asp?url=/library/en-
us/iphlp/iphlp/gettcptable.asp

[Jean-Paul Calderone]

Why do we care about Windows, again?

[ghazel]

Well I care about Windows, but I hope this topic doesn't turn in to a debate.

[itamarst]

We will of course try to fix problems Twisted has on Windows, it's just that for
most Twisted developers it's lower on the priority list than other things. I
don't have a Windows development environment set up at all, for example.

Which is another way of saying that patches will get win32 problems solved a lot
quicker :)

[jknight]

I'd like to note that the msdn article below explicitly recommends against using 
SO_REUSEADDR, as they changed sockets to not allow _other_ users to steal your 
sockets, unless you have set SO_REUSEADDR. Of course on windows, you're probably 
screwed if you have multiple users on a box, in any case, so maybe it doesn't 
matter much.

[Trent.Nelson]

Replying to hraban:

hraban:
If my server app crashes, it often leaves its ports open, so that I get a
CannotListenError 'Address already in use'. (Windows 2000)

itamarst:
SO_REUSEADDR is supposed to allow this. I think I disabled that for TCP
on windows as it caused some failing tests, but in retrospect those
tests are probably less important than the actual functionality it
provides.

The correct way to handle this on Windows is to *always* use SO_EXCLUSIVEADDRUSE on your server sockets. If your server process dies and restarts, it will have no problem binding back to the same host/port if you use this socket option. Never ever ever ever *EVER* use SO_REUSEADDR on Windows unless you're writing a UDP multicast server, and you wish to have multiple processes listening to the same host/port, such that they all receive every packet destined for that host.

You can test the SO_EXCLUSIVEADDRUSE semantics pretty easily. Here are two ipython sessions, the first is my server, the second demonstrates SO_EXCLUSIVEADDRUSE behaviour:

In [1]: from socket import *

In [2]: s = socket()

In [3]: s.setsockopt(SOL_SOCKET, SO_EXCLUSIVEADDRUSE, 1)

In [4]: s.bind(('localhost', 5645))

In [5]: s.listen(1)

In [6]: s.accept()

In a second session: In [1]: from socket import *

In [2]: s = socket()

In [3]: s.set s.setblocking s.setsockopt s.settimeout

In [3]: s.setsockopt(SOL_SOCKET, SO_EXCLUSIVEADDRUSE, 1)

In [4]: s.bind(('localhost', 5645))

---

error Traceback (most recent call last)

C:\Documents and Settings\tnelson\<ipython console> in <module>()

C:\Documents and Settings\tnelson\<string> in bind(self, *args)

error: (10048, 'Address already in use')

That's exactly the behaviour we want -- if SO_REUSEADDR was specified instead, both python processes would wedge.

Now, if I kill the first python session and immediately do the following in the second session:

In [5]: s.bind(('localhost', 5645))

In [6]:

Wallah, it works, no 'Address in use' complaints.

[jknight]

The following test:

import socket, sys, time

def server():
    s=socket.socket()
    # s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind(('localhost', 4567))
    s.listen(1)
    c,a=s.accept()
    print "got client!", c, a
    c.send("Hello!")
    c.close()
    print "done with client!", c, a


def client():
    s=socket.socket()
    s.connect(('localhost', 4567))
    print "connected!"
    print "ha ha, just sleeping, not reading!"
    time.sleep(100)

eval(sys.argv[1]+'()')
# Run this test:
# in shell #1: python test.py server
# in shell #2: python test.py client
# in shell #1: python test.py server

has shown that not setting any flags on windows is equivalent to setting SO_REUSEADDR on linux. The second server is allowed to bind to the socket despite an outstanding FIN_WAIT2 socket from the client.

Therefore twisted's current behavior of setting SO_REUSEADDR only on non-windows platforms causes the intended behavior of allowing restarting of servers with outstanding sockets in TIME_WAIT, FIN_WAIT2, etc states.

So, resolving this bug invalid.