[Twisted-Python] twisted thumbnail server
p.mayers at imperial.ac.uk
Fri Nov 2 12:13:37 EDT 2012
On 02/11/12 15:42, Paul Wiseman wrote:
> I hope this will be an easy question for some of you guys :)
> I'm trying to set up a simple server which will accept requests over GET
> to create a thumbnail for an image, and server it back as the response.
> The images are stored in two S3 buckets, the originals are in one bucket
> (store), and the generated thumbnails are stored in another (thumb) as a
> cache so that the work doesn't need to be repeated.
> Currently I'm checking if the thumbnail already exists in the thumb
> bucket. I'm redirecting the request if it is or if not I'm downloading
> the image from store, generating the thumb using PIL, uploading the
> thumbnail to the thumb bucket and then redirecting the request.
This isn't a criticism, but I trust you are aware of the implications
and problems of doing work in threads?
FWIW we usually use a child process pool for intensive tasks; this has
the advantage you can sensibly kill a long-lived child (just kill the
process) and you side-step the lack of concurrency in the python
[In this case, I'd just start up a bunch of python interpreters using a
ProcessProtocol and use a simple request/response command protocol on
stdin/stdout - the child interpreters can be non-Twisted processes able
to block on PIL operations]
If you really do want threads, is there any reason to not use the
Twisted threadpool stuff?
It's often a personal/style choice, but I don't use StringIO for large
volumes of data personally (not Twisted-specific).
I'm sure someone will mention tests ;o)
More information about the Twisted-Python