[Twisted-Python] Using Twisted in a network project

Brian Warner warner at lothar.com
Thu Dec 4 16:43:57 EST 2003


"Hegedus, Matthew S" <matthew.s.hegedus at lmco.com> writes:

> I would like to use Twisted if I could to write the code running on these
> Custom Routers to achieve what I've described. 

Sounds perfectly reasonable. That's the same approach used by standard IPv4
multicast routers. Most of the multicast routing protocols are concerned with
optimizing a tree of possible sources and members: your design simplifies
that to a single source and a strict two-level hierarchy (level 1 are the
distant gateway router, level 2 is the set of remote endpoints), which should
remove much of the complexity from those protocols.

You'll need to implement a protocol from the end point to their local
"router" (a host that can multicast to the end point and which is running
your router program) which says "I want to join group X". That router then
needs a protocol to the data source to say "somebody behind me wants to join
group X, please sign me up".

It will probably be easier to have the original data source do both the local
multicast and the series of remote-router UDP unicasts. (I'm assuming you're
writing the code on that end too). You can have a local-router too, but then
you have to coordinate an additional host.

Think of things in terms of trees of subscriptions. You probably have
reasonable multicast support within your local subnet (most OSes ship with
multicast-capable IP stacks), so use it (i.e. don't broadcast your local data
packets, because then you'll waste the time and interrupts of all the local
non-participating nodes). The data source should pick a multicast group
address, everyone local should join that group (the IP stack and the ethernet
hardware will take care of the details), the local router can join it too.
You'll need some kind of service advertisement so the remote routers and
remote nodes can know which groups they can join.

> Question #3: If this idea is reasonable in theory, should I be worried about
> Twisted/Python network performance?

Depends upon how much data you're sending and what kind of connection you
have to the outside world. Unless you've got a really nice pipe, it is likely
that Twisted running on reasonable hardware will be able to saturate it. The
local traffic is all multicast (one packet on the wire each), the remote
traffic to N remote routers is unicast (N packets on the wire each), so the
number of packets that your userspace code has to deal with is N+1 each, and
your upstream bandwidth has to handle N of those. Once you've got more than
two or three remote routers, N/(N+1) goes to 1 and Twisted doesn't have to
run any faster than it takes to saturate your upsteam pipe.

If you live on an OC-48 or something then you can set up multiple local
routers and have them decide amongst themselves which one should be
responsible for sending to each remote router.

sounds like fun,
 -Brian




More information about the Twisted-Python mailing list