[Twisted-Python] Twisted enterprise deployment models

[Glyph]

On Oct 20, 2011, at 6:50 AM, Brad Milne wrote:

> We have a core message-handling platform which encompasses an ever-increasing array of modules and services run inside one Twisted process. This is good as it means the reactor is in control of most of the stuff relevant to it. However, as the number of deployed services within it grows, it becomes increasingly burdensome to have all the modules in a single deployed codebase running in a single launched process.
> Java has JEE; erlang has hot code loading; x has y; what does Twisted have? pb?

Erlang can do this because it has a strict separation of state and behavior (in the vernacular, "it is not object-oriented") and J2EE can do this because it requires you to externalize all your state - usually to a relational database.  Part of the appeal of Twisted is that you can freely use simple, in-memory mutable data structures to store your mutable state.

Introducing these kinds of limits into the Twisted ecosystem would not be impossible, but it is very challenging, because you'd have to come up with a model that applied equally well to all of the supported protocols and all the ways you might want to store state within those protocols.  

> To date we have begun introducing RabbitMQ (AMQP), with separate Twisted processes running as adapters at either end of queues. On the surface this fits nicely as it provides distributed architecture, resilience through isolation, and decoupled upgrade paths. On the flipside though, it means more technology and more processes running outside of a singleton Twisted reactor (read: OS process management, which Twisted's designed to be better than).

I'm not sure what you mean by "OS process management" versus "Twisted process management".  It's true that you get some benefits from using spawnProcess to manage your subprocesses, since you can write code to automatically start and stop them in response to certain stimuli, but this relies completely on however it is your OS starts and stops processes.  Why not just have a Twisted monitor process that starts and monitors the health of all of your queue producer/consumer processes?  Then you should get the best of both worlds.

> So my questions:
> 1) I'm right, right? Twisted doesn't have mechanisms or a library for hot swapping code modules or similar solutions for low impact upgrades and high availability? (I originally read about Twisted *plugins* hoping for this, but was disappointed to find how basic they actually were)

Reza already replied with a pretty good list of possible options for this, but: no, there's no single, one-size-fits-all solution for this.

And there could be!  As I said above, it's tricky, but if you could think of a good design for such a system, I'd absolutely LOVE for it to be in Twisted.  Please feel free to contribute something :).

> 2) Is RabbitMQ with txAMQP-empowered Twisted instances on the ends of the queues a sensible, compatible, best-practice approach as agreed by the community?

No.

I don't mean that it doesn't work, I just don't like the term "best-practice".

Maybe your implementation is a good idea, maybe not.  But if I said it's a "best practice", that would relieve you of the need to think critically about whether or not this approach was satisfying your requirements.  There are some cases where this is a good idea, some not.  There are some people in the community who have had good experiences with this approach; I personally don't use it.

You'll have to be the judge of whether it's working for you or not.