[Twisted-web] spawning a xvfb and firefox with a timeout

[Stephan]

Thanks for the prompt reply response inline. I'm still having issues
even though I've changed some code based on your advice.

On Thu, Aug 18, 2011 at 9:51 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote:
>
> On Aug 19, 2011, at 12:42 AM, Stephan wrote:
>
>> I'm attempting to spawn xvfb with firefox using twisted's spawnProcess.
>>
>> I'm having two issues:
>>
>> 1) when I call a normal process (not xvfb) it seems to work, but when I call xvfb the process goes defunct
>
> Defunct processes have exited, but not been reaped.  In older versions of Twisted, this might happen if some other library were competing to register the SIGCHLD signal.  What version of Twisted are you using?

I'm using the latest version (I installed it today). 11.0.0 on an ubuntu machine

>
>> 2) perhaps because it goes defunct but the timeout I set in place in the processprotocol does
>> not trigger.
>
> It should never go defunct, whether or not your timeout triggers.

I agree, I can't figure out why it's going defunct.

>
>> ==============
>> here is the code for spawning a process
>> ==============
>> 108         command = ["/usr/bin/xvfb-run", "--auto-servernum",
>> "/usr/bin/firefox", "-P", profile_id]
>> 109         subprocess = reactor.spawnProcess(TimedProcessProtocol(20),
>> 110                                             command[0], command)
>> 111         logging.debug(type(subprocess))
>> 112         logging.debug("spawned a process: pid: %d", subprocess.pid)
>> 113
>> 114         # this file will be created when firefox starts
>> 115         while (not os.path.exists(self.profile_dir + "/importantfile")):
>> 116             logging.debug("in while loop waiting for
>> firesharkReady file to exist")
>> 117             pass
>
> You should really not busy-loop like this waiting for the subprocess.  Have a repeating callLater (or LoopingCall) that does this check cooperatively with the reactor, so if something goes wrong (for example: firefox fails to launch), you'll be able to react to it effectively rather than just hanging your Twisted process forever.  If firefox isn't actually starting correctly, or that file doesn't actually get created for some reason, this could cause the defunct process.


I agree with your point, I've changed it a bit (see the code at the
bottom) I'm not sure if this is correct or not.

>
>> 118
>> 119         logging.debug("file exist")
>>
>> ==============
>> here is the process class
>> ==============
>>
>> 27 class TimedProcessProtocol(protocol.ProcessProtocol):
>> 28
>> 29     def __init__(self, timeout):
>> 30         self.timeout = timeout
>> 31
>> 32     def connectionMade(self):
>> 33         logging.debug("connection made timeout = %d", self.timeout)
>> 34         @defer.inlineCallbacks
>> 35         def killIfAlive():
>> 36             logging.debug("timeout reached - killing process")
>> 37             try:
>> 38                 yield self.transport.signalProcess('KILL')
>> 39             except error.ProcessExitedAlready:
>> 40                 logging.debug("process already exited")
>> 41                 pass
>> 42
>> 43         d = reactor.callLater(self.timeout, killIfAlive)
>> 44
>> 45     def outReceived(self, data):
>> 46         logging.debug("output: %s", data)
>> 47
>> 48     def errReceived(self, data):
>> 49         logging.debug("errReceived %s", data)
>>
>> _______________________________________________
>> Twisted-web mailing list
>> Twisted-web at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
>
>
> _______________________________________________
> Twisted-web mailing list
> Twisted-web at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
>

============
new code (still does not work)
=============
108         profile_id = self.profile['name']
109         command = ["/usr/bin/xvfb-run", "--auto-servernum",
"/usr/bin/firefox", "-P", profile_id]
110         subprocess = reactor.spawnProcess(TimedProcessProtocol(20),
111                                             command[0], command)
112         logging.debug(type(subprocess))
113         logging.debug("spawned a process: pid: %d", subprocess.pid)
114
115         lc = LoopingCall(self.check_if_file)
116         self.check = lc
117         lc.start(0.5)
118
119     def check_if_file(self):
120         if os.path.exists(self.fireshark_profile_dir + "/firesharkReady"):
121             logging.debug("firesharkReady file to exist")
122             self.check.stop()

============
new code (log)
=============

478C0-CA07-11E0-B595-E88E810DE385
2011-08-18 22:06:58,038 DEBUG:FirefoxProcess run called
2011-08-18 22:06:58,038 DEBUG:intialize firefox files
2011-08-18 22:06:58,040 DEBUG:connection made timeout = 20
2011-08-18 22:06:58,041 DEBUG:<class 'twisted.internet.process.Process'>
2011-08-18 22:06:58,041 DEBUG:spawned a process: pid: 30562
2011-08-18 22:07:01,374 DEBUG:inConnectionLost
2011-08-18 22:07:01,374 DEBUG:errConnectionLost
2011-08-18 22:07:01,375 DEBUG:process exited, status 1


=======================

this time the xvfb process did not go defunct but it did exit, which
shouldn't really happen, firefox should just stay open really.