[Twisted-Python] Re: Twisted 1.0.4 - Heavy search for `plugins.tml'
pinard at iro.umontreal.ca
Sun Apr 27 20:28:48 EDT 2003
> > Is Twisted modifying `sys.path' to include the first part of argv?
> > Or is it standard Python behaviour?
> Yes, Python always adds the location of the main script to sys.path [...]
Thanks for enlightening me.
For the case of the heaviness for `plugins.tml', at least on Unix, there
might be something we could do that would be reasonable on average,
but I'm not fully sure, I'll merely let judge those who are grown up! :-)
Usually, `/usr/bin/' and such do not have subdirectories, so if a
directory contains 1000 entries, one may immediately check if one of
these is `plugins.tml' by a direct try. Then, if the number of links of
that directory -- itself -- is exactly 2, we can conclude that it has no
sub-directories, and that there is nothing more to check for this one.
We could then get more speed even for directories other than `/usr/bin/'.
This trick is heavily used in GNU find as a way to avoid `stat'ing all
entries in a directory to find sub-directories, when it can be proven
in advance there is no such sub-directories. When the trick was added,
the speed-up that resulted was judged spectacular.
But I do not remember if Linux existed at the time. As Linux seems to
beat many other systems at properly caching disk accesses, the benefits
of the trick might be more hidden. A year ago, maybe, I tried the same
trick on Linux in hope of increasing the performance of `os.path.walk',
and the speed-up was not so significant. This is part of my hesitation.
Moreover, maybe the above trick is meaningless on Windows or on MacOS.
I would be surprised if there are Unix filesystems where the link count of
2 for directories (`/' excepted) is not dependable; this might be another
thing to check. With some luck, GNU find sources would tell us what the
problems may be. The maintainers surely met them all! :-)
François Pinard http://www.iro.umontreal.ca/~pinard
More information about the Twisted-Python