[Twisted-Python] Re: plus mode was Re: how winnt fileops work and what to do about it

Brian Warner warner at lothar.com
Sat Dec 31 17:16:39 EST 2005


glyph at divmod.com writes:

> It seems like we can work around this more easily than that, considering
> that flush and seek are available from Twisted; the file object causing
> problems in the tests is being returned from the open() method of a
> FilePath object, if I understand it correctly. FilePath could include the
> workaround far in advance of Python deciding to.

I'm pretty sure that the real problem we're trying to solve here is caused by
a stuck process keeping a .pyd file open. Indeed, if you look at the
buildslave's logs, you'll see the exception is as follows:

 exceptions.OSError: [Errno 13] Permission denied: 'c:\\buildslave\\win32-win32er\\W32-full2.4-win32er\\Twisted\\twisted\\protocols\\_c_urlarg.pyd'

So changing the way Twisted or its unit tests open a file is just not going
to help. What matters is the way python (or.. pyrex?) opens a file.

(for context: the buildbot is currently configured to do SVN checkout/updates
into one directory, then copy the tree into a second directory, then run
tests on that second directory. This mode='copy' approach uses 'svn update'
to minimizes network bandwidth, but at the expense of doubling the disk usage
with the extra copy. At the beginning of each build, the buildslave deletes
the second directory with a function named rmdirRecursive() that bear
provided, which does a chmod() of any mis-permissioned files before deleting
them. It was an os.remove() inside this rmdirRecursive which raised the
exception).


I've run into a similar problem in the past, under Solaris, using NFS, where
a test case spawned off a daemon process which then didn't die when it was
supposed to, somehow held on to a file (I think solaris won't let you delete
a file that is being used as the backing store for an executable), and that
prevented the unlink() from succeeding.

In that environment, I just renamed the top-level directory to something
unique, spawned off an 'rm -rf' into the background to delete the old
directory if it was possible, then continued on with the next build. If the
code had to try too hard to come up with a unique name, it would flag a
warning that there might be a stuck process somewhere.

Perhaps we could use something similar here?

Of course, the real fix would be to find a way to let the testing code kill
off any stuck processes, but that'll probably be very windows-specific.


cheers,
 -Brian




More information about the Twisted-Python mailing list