[Twisted-Python] daemon thread support

Andrew Bennetts andrew-twisted at puzzling.org
Tue Dec 2 23:26:09 MST 2003


On Tue, Dec 02, 2003 at 10:58:05PM -0500, Bob Ippolito wrote:
> On Dec 2, 2003, at 10:10 PM, Andrew Bennetts wrote:
> >On Tue, Dec 02, 2003 at 09:46:20PM -0500, Bob Ippolito wrote:
> >>
> >>What about list as the function, and itertools.imap as the argument to
> >>that function?
> >
> >Or maybe sets.Set?  So long as process always returns the same value
> >(probably None), this won't eat memory the way list would...
> 
> sets.Set eats more memory than list does (it's a python class that uses 
> a dict for storage).. if you don't want to return anything useful 
> there's always reduce(lambda a,b:None, iterable)

Uh, not if the iterable returns the same element repeatedly.  Compare the
memory use of:

    sets.Set(itertools.repeat(None))

vs.

    list(itertools.repeat(None))

Your idea with reduce is better -- except that it uses a lambda.  Somehow
that feels like cheating ;)

So, to sum up:

    reactor.callInThread(sets.Set, itertools.imap(process, iter(queue.get, None)))

Or:
    threading.Thread(
        target=sets.Set,
        args=(itertools.imap(process, iter(queue.get, None)),)
    )

Is our generic one-line worker thread?  What about catching possible
exceptions from process?  ;)

-Andrew.





More information about the Twisted-Python mailing list