Opened 13 years ago
Closed 10 years ago
#1151 enhancement closed invalid (invalid)
SO_REUSEADDR should be enabled for TCP on windows
Reported by: | hraban | Owned by: | |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | core | Keywords: | win32 |
Cc: | radix, Jean-Paul Calderone, spiv, itamarst, jknight, hraban, ghazel, bwh, teratorn | Branch: | |
Author: |
Description
Change History (20)
comment:2 Changed 13 years ago by
This causes a problem where the same application can bind twice or something like that to the same port (twisted.test.test_tcp.CannotBindTestCase.testCannotBind fails). If *other* processes can't bind to the port though this is fine; we can fix by remembering ourselves what ports we bound to and not relying on OS to do so. So we should have a test for other processes, and if that work it can be fixed. Also there should be a test for SO_REUSEADDR's functionality; someone should dig up UNPv1 and find what case it is where you need it (disconnected TCP clients in TIME_WAIT or something?) Someone should make a policy decision about this for 2.1; we may have to revert it and downgrade urgency.
comment:3 Changed 13 years ago by
Hmm. It appears SO_REUSEADDR means something different on win32 than it does for Linux sockets. See: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/using_so_exclusiveaddruse.asp http://blogs.msdn.com/wndp/archive/2005/08/03/Anthony_Jones.aspx I don't see any way on win32 to achieve the desired behaviour of allowing binding a listening port regardless of outstanding connections in wait states without also allowing this overriding of existing port bindings. Maybe I'm misreading these docs?
comment:4 Changed 13 years ago by
If we set both SO_REUSEADDR and SO_EXCLUSIVEADDRUSE this would do what we want... except that it would also grab the socket from other processes that haven't set SO_EXCLUSIVEADDRUSE. Windows is *awesome*. Though it does seem to have some useful options (e.g. SO_CONDITIONAL_ACCEPT would be nice to have in certain situations). OOh, and read up on SO_PROTECT, it's crackalicious. Anyway, looks like this can't be fixed. I suggest reverting the patch and closing the bug :/
comment:5 Changed 13 years ago by
Well, I won't change the state of this bug, but I also won't consider it as blocking the release (or is there buggy code in trunk@head?)
comment:6 Changed 13 years ago by
There is buggy code in trunk. Revert my patch to tcp.py that enabled SO_REUSEADDR on windows.
comment:8 Changed 12 years ago by
The comment that "If *other* processes can't bind to the port though this is fine" makes it sound like the assumption is that if you don't set SO_REUSEADDR you're in the clear. That is not the case. Other applications can set SO_REUSEADDR even if you have not, and smash your binding. The only way to prevent this is by using SO_EXCLUSIVEADDRUSE, which you cannot use in conjunction with SO_REUSEADDR. It was incorrectly implied that this was possible. So my vote is, make this an option the user can specify. - It is useful for reclaiming ports in the TIME_WAIT state - It does not open up the server to any vulnerabilities The only caveat is that you could potentially be binding to a port that is in use by another application. Applications that want to avoid this behavior could not use the option, or attempt to bind without flags to see if the port is in use when they expect it not to be. In addition, an option to enable SO_EXCLUSIVEADDRUSE should be considered, in order to prevent other applications from reusing a port whether the SO_REUSEADDR flag is used or not, as explained earlier.
comment:10 Changed 12 years ago by
My understanding of Winsock's implementation of SO_REUSEADDR is that if you bind a socket to a port that has another listening socket, all connection requests will go to the earlier listener. So enabling SO_REUSEADDR can result in a silent failure to listen. I don't know whether there's any way to test for this case. We're working on network drivers and a WSP (Winsock Service Provider) and in the course of testing we need to install different versions of the WSP and then restart services. We really do not want an SSH server to fail to restart due to SO_REUSEADDR being disabled! Currently I am patching twisted.internet.tcp.Port.createInternetSocket at runtime to achieve the desired behaviour.
comment:11 Changed 12 years ago by
If there was some way to test whether someone was already listening on a socket we could enable SO_REUSEADDR, yes. Modulo race conditions, I suppose, so ideally one could test for the silent failure mode bwh mentioned after the "successful" bind.
comment:12 Changed 12 years ago by
You could get the entire TCP table and look for the port you're interested in. http://msdn.microsoft.com/library/default.asp?url=/library/en- us/iphlp/iphlp/gettcptable.asp
comment:14 Changed 12 years ago by
Well I care about Windows, but I hope this topic doesn't turn in to a debate.
comment:15 Changed 12 years ago by
We will of course try to fix problems Twisted has on Windows, it's just that for most Twisted developers it's lower on the priority list than other things. I don't have a Windows development environment set up at all, for example. Which is another way of saying that patches will get win32 problems solved a lot quicker :)
comment:16 Changed 12 years ago by
I'd like to note that the msdn article below explicitly recommends against using SO_REUSEADDR, as they changed sockets to not allow _other_ users to steal your sockets, unless you have set SO_REUSEADDR. Of course on windows, you're probably screwed if you have multiple users on a box, in any case, so maybe it doesn't matter much.
comment:17 Changed 11 years ago by
Cc: | teratorn added |
---|
comment:18 Changed 10 years ago by
Replying to hraban:
hraban: If my server app crashes, it often leaves its ports open, so that I get a CannotListenError 'Address already in use'. (Windows 2000) itamarst: SO_REUSEADDR is supposed to allow this. I think I disabled that for TCP on windows as it caused some failing tests, but in retrospect those tests are probably less important than the actual functionality it provides.
The correct way to handle this on Windows is to *always* use SO_EXCLUSIVEADDRUSE on your server sockets. If your server process dies and restarts, it will have no problem binding back to the same host/port if you use this socket option. Never ever ever ever *EVER* use SO_REUSEADDR on Windows unless you're writing a UDP multicast server, and you wish to have multiple processes listening to the same host/port, such that they all receive every packet destined for that host.
You can test the SO_EXCLUSIVEADDRUSE semantics pretty easily. Here are two ipython sessions, the first is my server, the second demonstrates SO_EXCLUSIVEADDRUSE behaviour:
In [1]: from socket import *
In [2]: s = socket()
In [3]: s.setsockopt(SOL_SOCKET, SO_EXCLUSIVEADDRUSE, 1)
In [4]: s.bind(('localhost', 5645))
In [5]: s.listen(1)
In [6]: s.accept()
In a second session: In [1]: from socket import *
In [2]: s = socket()
In [3]: s.set s.setblocking s.setsockopt s.settimeout
In [3]: s.setsockopt(SOL_SOCKET, SO_EXCLUSIVEADDRUSE, 1)
In [4]: s.bind(('localhost', 5645))
error Traceback (most recent call last)
C:\Documents and Settings\tnelson\<ipython console> in <module>()
C:\Documents and Settings\tnelson\<string> in bind(self, *args)
error: (10048, 'Address already in use')
That's exactly the behaviour we want -- if SO_REUSEADDR was specified instead, both python processes would wedge.
Now, if I kill the first python session and immediately do the following in the second session:
In [5]: s.bind(('localhost', 5645))
In [6]:
Wallah, it works, no 'Address in use' complaints.
comment:19 Changed 10 years ago by
Resolution: | → invalid |
---|---|
Status: | new → closed |
The following test:
import socket, sys, time def server(): s=socket.socket() # s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) s.bind(('localhost', 4567)) s.listen(1) c,a=s.accept() print "got client!", c, a c.send("Hello!") c.close() print "done with client!", c, a def client(): s=socket.socket() s.connect(('localhost', 4567)) print "connected!" print "ha ha, just sleeping, not reading!" time.sleep(100) eval(sys.argv[1]+'()') # Run this test: # in shell #1: python test.py server # in shell #2: python test.py client # in shell #1: python test.py server
has shown that not setting any flags on windows is equivalent to setting SO_REUSEADDR on linux. The second server is allowed to bind to the socket despite an outstanding FIN_WAIT2 socket from the client.
Therefore twisted's current behavior of setting SO_REUSEADDR only on non-windows platforms causes the intended behavior of allowing restarting of servers with outstanding sockets in TIME_WAIT, FIN_WAIT2, etc states.
So, resolving this bug invalid.
comment:20 Changed 7 years ago by
Owner: | itamarst deleted |
---|