Opened 7 years ago

Closed 7 years ago

#4827 defect closed duplicate (duplicate)

Twisted on FreeBSD has issues with child processes

Reported by: lewq Owned by:
Priority: normal Milestone:
Component: core Keywords: buildbot, process
Cc: Branch:
Author:

Description

This also seems to cause the builder to hang after it appears to have finished...

FAILED (skips=579, expectedFailures=18, failures=4, errors=39, successes=6161)

command timed out: 1200 seconds without output

Even after this point, buildbot claims that the test is still running.

I think it's waiting for some child processes to die, because there seems to be a bunch of stray processes...

hybrid@fdf07a6bdb1745c5:~/buildbot/freebsd-8.1-amd64/Twisted/_trial_temp$ sudo ps ax |grep build78131  ??  I      0:52.00 /usr/local/bin/python2.6 /usr/local/bin/buildbot start .
79420  ??  I      0:00.00 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/internet/test/process_helper.py
79422  ??  I      0:00.00 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/internet/test/process_helper.py
79656  ??  I      0:00.00 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/internet/test/process_helper.py
79659  ??  I      0:00.00 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/internet/test/process_helper.py
80089  ??  I      0:00.06 /usr/local/bin/python -u /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/test/process_fds.py
80697  ??  I      0:00.34 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/test/process_twisted.py
80999  ??  I      0:00.36 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/test/process_twisted.py
81350  ??  I      0:00.40 /usr/local/bin/python -u /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/test/process_twisted.py
81644  ??  I      0:01.11 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/test/process_echoer.py
81948  ??  I      0:00.05 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/test/process_echoer.py
82586  ??  I      0:00.37 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/twisted/test/stdio_test_halfclose.py tw
82956  ??  I      0:00.06 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/_trial_temp/twisted.web.test.test_cgi/C
82995  ??  I      0:00.05 /usr/local/bin/python /usr/home/hybrid/buildbot/freebsd-8.1-amd64/Twisted/_trial_temp/twisted.web2.test.test_cgi/

I'm pretty sure this is related to tests such as twisted.internet.test.test_process.test_childConnectionLost (which shows a continuous spew of ReactorBuilder.tearDown reaping some processes ...).

In fact many tests which spawn a subprocess seems to suffer from this problem where the subprocess doesn't die when it should.

<exarkun> lewq: I think there's some assumption that's being made about what will cause the _child_ processes to exit
<exarkun> lewq: So it's not that the reaping code is broken, but the processes are living too long.
<lewq> oh, hmm.. living too long after being sent a signal?
<exarkun> lewq: no, I think they're expected to exit for some other reason
<exarkun> lewq: eg the process test_childConnectionLost runs reads lines from stdin until it gets an empty one, then it exits
<exarkun> lewq: So maybe buffering is messing with that?  Or something is preventing the Python process from actually exiting after it runs out of Python code to run?

Simply running the simple test code in test_childConnectionLost (which loops until it reads a blank line) behaves as you would expect.

Any thoughts?

Change History (2)

comment:1 Changed 7 years ago by <automation>

comment:2 Changed 7 years ago by lewq

Resolution: duplicate
Status: newclosed

Duplicate of #4881, closing.

Note: See TracTickets for help on using tickets.