[Twisted-Python] Twisted.Web memory usage, and other things

[Glyph Lefkowitz]

On Sunday 29 April 2001 02:55, you wrote:
> I'm planning on setting up a web server and was wondering how much
> memory TwistedPython uses up compared to Apache, especially since my
> RAM is limited.

RAM usage is hard to measure on UNIX, but if you can run Python on your 
limited-RAM machine (e.g. it's not an embedded system with sub-megabyte hard 
allocation requirements), you ought to be able to bring up a basic Twisted 
Web server; it doesn't have many requirements in excess of just a standard 
python interpreter.  As far as how it measures up to Apache; it depends on 
what you configure each of them to do :).  I would guess that in the average 
case, twisted web would be lighter, memory-wise, than apache, but that apache 
can be configured to take up much less RAM if you know what you're doing.

Here is some sample output of PS, indicating what a bunch of average users do 
with twisted web:

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
jh       17110  0.0  1.8  6312 2372 ?        S    Mar10   3:38 python2.0 
shae     24700  0.0  1.4  3708 1796 ?        S    Mar15   0:02 python 
glyph    22519  0.0  2.1  5380 2804 ?        S    Mar27   0:44 python 
carmstro 19275  0.0  1.7  4568 2280 ?        S    Apr03   0:22 python 
matsaleh  8402  0.0  1.6  3804 2096 ?        S    Apr06   0:01 python
washort  27718  0.1  1.4  5052 1816 ?        S    Apr12  28:52 python
det       3675  0.0  1.9  4160 2464 ?        S    Apr20   0:01 python 
www-data 30020  0.0  3.7  5568 4760 ?        S    Apr26   0:39 python
rikyu    12063  0.0  2.8  4480 3676 ?        S    Apr27   0:01 python

By contrast, here's my single-user testing machine with a single instance of 
Apache running (debian's default config)

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
root       435  0.0  0.9  2840 1164 ?        S    Apr04   0:00 apache
www-data 14321  0.0  0.8  2900 1044 ?        S    07:35   0:00 apache
www-data 14322  0.0  0.8  2888 1076 ?        S    07:35   0:00 apache
www-data 14323  0.0  0.8  2888 1060 ?        S    07:35   0:00 apache
www-data 14324  0.0  0.8  2900 1064 ?        S    07:35   0:00 apache
www-data 14325  0.0  0.8  2900 1060 ?        S    07:35   0:00 apache
www-data 14733  0.0  0.5  2852  764 ?        S    10:17   0:00 apache
www-data 14738  0.0  0.6  2852  776 ?        S    10:17   0:00 apache
www-data 14739  0.0  0.9  2852 1212 ?        S    10:17   0:00 apache
www-data 14740  0.0  0.9  2852 1212 ?        S    10:17   0:00 apache
www-data 14741  0.0  0.9  2852 1212 ?        S    10:17   0:00 apache

(Note, also, that "root" is not one of the users in that first list! ^_^)

> I would also like to know is TwistedPython supports virtual hosts at this
> time,

While at the time that you sent the message that wasn't true, named virtual 
host is such a trivial feature that I added it: it took 7 minutes to add and 
5 minutes to test.  Also, the implementation is 29 lines, which includes 11 
lines of API documentation and 4 blank lines.  Other features are just as 
easy to add :)

> and whether there's any way to have a persistant interpreter like mod_perl
> and mod_php.

Twisted Python *is* a persistent interpreter.  There are a number of 
different ways to run code in it, depending on your preferred modality of 
thinking about web servers.  However, it only supports Python (obviously).

It wouldn't be too hard to implement a filtered process which talked to a 
persistent interpreter of some other language, but I don't see the point, 
since python is more than good enough for me :)

> Oh, and while I'm asking questions, how does one create a multi-user
> interactive fiction? Strikes me as an odd concept as I've never seen one,
> but an interesting possibility.

Good question!  I've been hacking away trying to solve it for years, and I'm 
still not sure it can be done :).  When we get something good done, I'll tell 
you... :)

> Python is in some ways similar to TADS
> (text adventure development system)'s scripting language, but TADS is
> geared towards a single user. Wouldn't a multiplayer IF game be like a
> singleplayer IF game except there's only one set of objects for the
> players to fight over?

Yes, and that's a tough nut to crack.  The idea being that you create LOTS of 
objects to fight over.  Then you make it hard to fight...

> By the way, there's a MUD base derived from LPC
> called MudOS that tried to accomplish some of the same goals as
> TwistedPython, like network services including HTTP, FTP, Intermud Chat,
> etc, and providing sort of an internal shell with programs.

I'm familiar with MudOS.  It's certainly older and more established than TPy, 
but I think you'll find the integration of services in TPy somewhat more ... 
streamlined.

> And another quick question: What's the GTK stuff for?

I am a very lazy person and I am highly resistant to change.  So I have these 
crazy ideas, like writing servers should be about the same as writing clients 
on the network level.  The GTK stuff allows me to do that, by integrating the 
network layer API (twisted net) with a GUI API (gtk).  This means, for 
example, you can run a webserver inside a PyGTK application with almost no 
extra work; the glue code will be on the order of 3 lines.  Not only that, 
but it'll be a relatively high-performance webserver, that you could handle a 
decent amount of traffic on, and one that won't hang your UI while doing 
stuff, rather than the standard "joke" blocking webserver included with 
Python.  If it still works, the map "editor" should demonstrate this :)

> Anyways:
>   Thank you in advance for any help you can provide,

And you're welcome.  Happy twisting.


-- 
                      ______      __   __  _____  _     _
                     |  ____ |      \_/   |_____] |_____|
                     |_____| |_____  |    |       |     |
                     @ t w i s t e d m a t r i x  . c o m
                     http://twistedmatrix.com/users/glyph