[Twisted-Python] Log Rotation Blocks w/ Large Number of Log Files
Jason J. W. Williams
jasonjwwilliams at gmail.com
Tue Mar 16 19:57:08 EDT 2010
True. Technically, you could also just kick it out to syslog and let
logrotate.d do the rotation for you.
The code I put up had a major "concurrency" error in that multiple
rotation threads could be launched and conflict with each other (try
to remove the same file).
Updated the Gist to deal with this: http://gist.github.com/334545
On Tue, Mar 16, 2010 at 5:01 PM, Maarten ter Huurne
<maarten at treewalker.org> wrote:
> On Tuesday 16 March 2010, Jason J. W. Williams wrote:
>> Twistd blocks for a long period of time when the number of log files
>> built up is large, thereby causing connections to freeze and
>> (depending on the length of the rotation) time out.
>> Given I've only used Twisted in the past and not contributed to it, I
>> was curious if someone could take a look at my changes to
>> twisted.python.logfile.LogFile's rotate() method and see if there's
>> any reason the changes would be a bad idea.
>> What I've attempted to do is defer the rotation of the old log files
>> to a thread, and only rotate the current log file in the foreground.
> An alternative approach could be to name the log files differently.
> For example, use the timestamp of the rotation as a filename suffix:
> This is not the traditional way logging is done, but it would get good
> performance with simple code. (It would still need resolution of duplicate
> file names, in case the log rotates very quickly or the clock is moved
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
More information about the Twisted-Python