<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><div><blockquote type="cite" class="clean_bq" style="color: rgb(0, 0, 0); font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">spawnProcess is somewhat low-level. It would be very nice indeed if Twisted had some sort of Service object you could easily use for spawning worker processes, along the lines of Ampoule, that would take care of all the platform-specific nonsense of setting up communication with subprocesses, spawning them, monitoring them, figuring out the right Python to spawn, the right environment variables to get sys.path correct, and other things, such as...</div></blockquote></div><p><br></p><p>Ampoule is very close. Any chance of polishing and absorbing it into Twisted? Parallel computing is the weakest point of entire Python ecosystem, and Ampoule is so very close to the perfection, unless, like me, you end up getting stuck on a corner case. </p><p><br></p><p>For this, I am almost entirely sure that the imports are failing, most probably because it’s trying to import the system python, not the one PyInstaller packages up. I have no idea how to fix it though. I’m feeling rather guilty about it, Aether v1.1 is a vastly improved version, but I can’t package it, so the release is off for a while. For personal reasons (Moved to SF, started a new full–time job) I am unlikely to find enough time to debug such a complex issue in the near future, while the code is fading from my memory and rotting in drive. Such a shame.</p><p> </p><p>Best,</p><p>Burak</p><p><br></p><p><br></p></div> <div style="color:black"><br>From: <span style="color:black">Glyph Lefkowitz</span> <a href="mailto:glyph@twistedmatrix.com">glyph@twistedmatrix.com</a><br>Reply: <span style="color:black">Twisted general discussion</span> <a href="mailto:twisted-python@twistedmatrix.com">twisted-python@twistedmatrix.com</a><br>Date: <span style="color:black">March 10, 2014 at 11:39:36 AM</span><br>To: <span style="color:black">Twisted general discussion</span> <a href="mailto:twisted-python@twistedmatrix.com">twisted-python@twistedmatrix.com</a><br>Subject: <span style="color:black"> Re: [Twisted-Python] Freezing Ampoule or the Invasion of Infinite Zombie Processes <br></span></div><br> <blockquote type="cite" class="clean_bq"><span><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div>
<title></title>
<br>
<div>
<div>On Mar 9, 2014, at 12:31 AM, Burak Nehbit <<a href="mailto:burak@nehbit.net">burak@nehbit.net</a>> wrote:</div>
<blockquote type="cite">
<div style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div id="bloop_customfont" style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;">
<div>
<p>I agree. Though it’s also very useful as an drop–in
implementation of a Twisted spawnProcess for one single other
process.</p>
</div>
</div>
</div>
</blockquote>
<div>spawnProcess is somewhat low-level. It would be very
nice indeed if Twisted had some sort of Service object you could
easily use for spawning worker processes, along the lines of
Ampoule, that would take care of all the platform-specific nonsense
of setting up communication with subprocesses, spawning them,
monitoring them, figuring out the right Python to spawn, the right
environment variables to get sys.path correct, and other things,
such as...</div>
<br>
<blockquote type="cite">
<div style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div id="bloop_customfont" style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;">
<div>
<div>
<div>
<blockquote type="cite" class="clean_bq" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">
It sounds like the basic process support included in
Twisted, <br>
reactor.spawnProcess, could address your
requirements. </blockquote>
</div>
<p>I have just found your answer here: </p>
<p><a href="https://stackoverflow.com/questions/6678763/simple-example-with-spawnprocess">
https://stackoverflow.com/questions/6678763/simple-example-with-spawnprocess</a></p>
<p>which seems useful. If nothing else works, I will try to reduce
complexity by converting this to bare spawnProcess. </p>
<p>Any ideas on what might be going wrong in my current stack?
After all, processes being repeatedly created is just a symptom of
something going wrong in the process very early, so ampoule tries
to spawn another one. I am mainly looking to find a way to see the
exception and where it happens. Twisted’s logging module seem to
kick in too late. </p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
Twisted's logging typically gets initialized after all modules are
imported. A tool like Ampoule really ought to set up logging
as early as possible in its child, and handle the case where it's
not getting set up on the parent side.
<div><br></div>
<div>The important thing to do is to make sure you have an
errReceived callback that actually prints or logs the stderr it
receives; there's likely a traceback in there which you may have
missed if the process is just crashing.<br>
<div>
<div>
<div>
<div><br></div>
<div>-glyph</div>
</div>
</div>
</div>
</div>
<hr> <div>- smime.p7s, 6 KB</div> _______________________________________________
<br>Twisted-Python mailing list
<br>Twisted-Python@twistedmatrix.com
<br>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
<br></div></div></span></blockquote></body></html>