[Twisted-Python] Checking whether a server started successfully

[Christian Simms]

On 8/21/07, Maarten ter Huurne <maarten at treewalker.org> wrote:
>
> Hi,
>
> I am starting a server process using "twistd -oy" (don't save state;
> application is described by Python source). If all is well, the server is
> started in the background. However, if all is not well, I'd like to detect
> that startup failed.
>
> The particular scenario that I'm using is testing a server using a test
> framework that does HTTP requests. Before the tests are run, the server is
> started and it tries to bind to a fixed port. If there is still an old
> instance of the server running, the bind will fail and any requests made
> will
> be handled by the old instance instead. That will lead to invalid test
> results.
>
> I worked around it currently by waiting for 3 seconds (probably sufficient
> time to try the binding) and then checking if the process of which the ID
> stored in "twisted.pid" is still running. However, there is no guarantee
> that
> 3 seconds is always enough time for the server to have bound the port. So
> I'm
> wondering if there is a reliable and elegant way of solving this.
>
> One approach would be to check which process has bound a particular port.
> I'm
> running this server on Linux and the "netstat" command line tool has
> access
> to the right information, but not in an easily parseable format. I guess
> the
> same info can be found in /proc somehow, but are those files guaranteed to
> stay compatible with newer kernels?
>
> Another approach would be to check in advance if something is listening on
> that particular port, but between the time this check is done and the time
> the server tries to bind, some other process might have grabbed the port.
>
> In any case, the port already being bound is just one of the reasons why
> the
> startup might fail; I'd prefer to have a way to detect startup failures
> for
> other reasons as well.
>
> One question is when exactly "startup" is finished. For my purpose, the
> moment
> the reactor is running would be the moment I consider startup to be
> successful. Maybe I can write a file somewhere to explicitly signal that,
> but
> that's a bit messy (where to put it? is the server process allowed to
> write
> there? who cleans up the file?).
>
> Maybe it would be useful to have twistd (optionally) wait until the
> reactor is
> running before returning. I don't want the test script to rely on a
> patched
> version of Twisted, so this would only be an option if it would make sense
> to
> integrate such a feature into Twisted itself.
>
> Have you encountered this problem as well? And if so, how did you deal
> with
> it?
>
> Bye,
>                 Maarten
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python



Yes, I've encountered this problem before.  And yes, I initially did the
same thing as you, by waiting a hardcoded time.  But eventually I got tired
of this race condition, and that it doesn't detect when the server failed to
start.  What I'm doing right now for my non-unit functional testing is
running a bash script which:

- make sure twistd isn't running using ps:  ps aux | grep twistd | grep -v
grep
- start up the app in non-daemon mode, send its output to the console and to
a file:
     twistd -noy my-app.tac | tee server.log &
- poll the server.log by grep'ing for a string I print from my app when the
app is completely started
- launch the tests against the running server (I think I'm using mechanize
right now for web client library)

This works for me in my development environment, on Linux.

Cheers,
Christian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20070822/5b70fff9/attachment.html>