[Twisted-Python] Re: Newbie - guidance required please

Nicola Larosa nico at tekNico.net
Mon Nov 7 15:01:10 EST 2005

> I am developing a multi-user accounting/business system. I use a database
> for the back end - PostgreSQL/psycopg on Linux, MS SQL Server/pywin32.odbc
> on MSW.

FWIW, PostgreSQL 8.0 runs on MS Windows too.

> The solution that I have in mind is to move all of my authentication and
> business logic to a server program, which will be the only program that
> makes a connection to the database. Each client will connect to my server
> program, and I will have to devise some message formats for the passing of
> information between the client and the server.

You may also convert your program to a web interface, and use a standard
web browser as client, connected to the server via the HTTP[S] protocol.

> Should I worry about the
> number of active objects, or is it safe to assume that virtual memory will
> handle it, on the assumption that most of them will be idle most of the
> time?

Virtual memory surely helps. A server will comfortably handle thousands of
objects: if and when you'll have problems, only then you'll explore ways to
minimize their number and size, no use worrying now.

> As I understand it, I will use Twisted in the following roles -
> 1. in my server program, as an interface to DB-API for all database access.
> 2. in my server program, to handle requests from and pass information to
> clients.
> 3. in my client program, to connect with and interact with my server
> program.

Unless you go the web interface route, of course, which by the way has the
added benefit of sidestepping client maintenance issues.

> 1. Once a client has established a connection with the server, I would
> prefer to keep the connection open until they log out, which could be
> several hours later. Is this an acceptable way of doing it, or should I keep
> the number of open connections down to a minimum, closing and reopening as
> required?

You have a thousand sockets available per process, use them judiciously. :-)

> 2. For authentication, the user enters a userid and password. The userid is
> looked up in a database table. One of the columns contains the hashed (sha)
> password. If the userid does not exist or the hashed password does not
> match, I do not allow the user to proceed.
> I looked at cred.py, but it seems more complicated to use this than stick to
> my own method. Is this a valid observation?

You have to grasp a few concepts, and then you have a general, reusable
piece fo infrastructure.

This is often true of Twisted: you first have to make an effort
understanding stuff, but then, after it "clicks", you feel turbocharged. :-)

> A related question. With my present system, the authentication happens on
> the client, so I do not have to send the password over the network. With the
> new method, I will have to send it to the server. Should I be looking at
> using SSL for protection against sniffing, or is there an easier way?

No, there isn't. You need to encrypt communication, if you want to protect
it. Of course, security involves more than just protecting communication.

> 3. Whether or not I use cred, it looks as if I should be using Perspective
> Broker (PB). I would normally have been thinking in terms of defining
> message formats for exchanging data between server and client, so the
> concept of defining objects that make calls on each other feels a bit
> strange. If I maintain the full state of the connection on the server side,
> I do not want to duplicate this on the client side, so I assume I must
> define some form of stripped down object to represent the client state, and
> use this to exchange information. Are there any simple examples I can look
> at to explain this concept in more detail?

You make a remote call, pass the data you want to transmit as parameters,
get data back as returned values (through Deferreds, of course). What could
be simpler than that? :-)

> I feel I am at a crossroads. Either I can cobble something together that
> works, but does not fit in with the Twisted philosophy, and will therefore
> limit me as my application grows. Or I can get my mind more in tune with
> Twisted, and set up structures that will work with me as things evolve.

We've all been there. Twisted seems daunting at first, and it's not clear
whether it will be worth the effort.

The winning point, for me, more than the impressive amount of built-in
network infrastructure, was its asynchronous, event-driven concurrency
model. Once you read this:

Generalization of Deferred Execution in Python

you stop and think: "How could have I been relying on *threads* all this

(The signature is quite on-topic here. :-) )

> If you have read this far, thanks for your perseverance. Any pointers or
> suggestions will be much appreciated.

After you've finished the above paper, study this:

Asynchronous Programming with Twisted

and the linked docs at the end. You're not going anywhere with Twisted
until Deferreds are firmly under your belt.

Other important things are interfaces and adapters:

Components: Interfaces and Adapters

More details here:


Of course, Python and Twisted are just layers in the stack. Above them
there are Nevow, and Axiom, and Mantissa, and... :-)

Good luck in your endeavors, you're fiddling with some powerful tools here!

Nicola Larosa - nico at tekNico.net

No inventions have really significantly eased the cognitive difficulty
of writing scalable concurrent applications and it is unlikely that any
will in the near term. [...] Most of all, threads do not help, in fact,
they make the problem worse in many cases. -- G. Lefkowitz, August 2005

More information about the Twisted-Python mailing list