[Twisted-Python] twisted application server
glyph at divmod.com
glyph at divmod.com
Wed Feb 27 19:20:29 EST 2008
On 11:37 pm, clemesha at gmail.com wrote:
>Would you say that this method of using a separate process instead of
>threads to do work also is possibly good for database operations?
Sure.
>I'm always reading 'beware of threads' ;) with respect to Twisted,
>so using a very simple control protocol to manage a separate
>dedicate database process might be better than
>relying on the threading that 'adbapi' uses?
It would definitely work. The main reason adbapi itself isn't done this
way is that setting up a process pool is unfortunately more work than
spawning a thread - you have to decide on a proper set of environment
variables, locate the python interpreter, locate the script that you're
going to run, make sure that script is installed by setup.py, decide on
a control protocol, wait for the subprocess to start up before sending
it messages, shut down the process appropriately, catch process
termination and restart (or not), etc etc etc. Twisted should really
have a nice API that does all that stuff for you, though, and it's a
shame that it doesn't.
The other reason is that adbapi is old. If we were going to implement
something like ADBAPI today, we'd probably write a process pool first,
but adbapi was written as a quick hack to get some database integration
a long time ago.
The only caveat is that additional Python interpreters take up more RAM
than additional threads. If the database processes are doing any heavy
CPU lifting though, this cost could well be worth it.
More information about the Twisted-Python
mailing list