[Twisted-web] [Athena] Is ReliableMessageDelivery really necessary?

[Jean-Paul Calderone]

On Wed, 1 Jul 2009 11:15:35 +0100, Paul Thomas <spongelavapaul at googlemail.com> wrote:
>Hi,
>
>I've hit a problem as my app has got bigger (about 30-40 widgets now,  all 
>chattering roughly once every 2 seconds) where the reliable  message 
>delivery mechanism is spiralling out of control. It seems that  the constant 
>back and forth means that large 'baskets' of messages are  resent. The more 
>this happens, the busier everything gets until the  browser becomes 
>unresponsive.

If you can produce a minimal example which demonstrates this behavior, it
would probably be very helpful in improving the situation.

>
>There's a fix for it: [Divmod-dev] athena duplicate messages issue but  I'm 
>slightly concerned about the potential for lost messages - and  also 
>confused about how this could happen. Given that HTTP is a  reliable 
>connection-oriented transport, where is the gap that messages  can fall 
>through?

Actually, HTTP is not a reliable transport.  The most obvious shortcoming
it has is that there is no way for a server to know if a client received
a response or not, but there are others.  So ReliableMessageDelivery is
necessary.

>
>I think I can cope with lost messages in most cases, so would it be  useful 
>to add a kind of 'sendRemote' that was like 'callRemote' but  didn't care 
>about a response? Or maybe this already exists and I've  missed it?

This is an interesting idea.  I haven't considered having such a feature in
Athena before.  It may be worth exploring.  The first problem that comes to
mind is that if any part of a page uses callRemote, sendRemote's advantages
are largely lost.  This would be because the messages generated by callRemote
will still need to be sent, so whatever retransmission logic is present in
ReliableMessageDelivery will still be invoked.

>Paul.
>
>P.S. this app is likely to get more noisy - is it likely that I'll  have to 
>abandon Athena for Orbited or similar? I mean, are there  architectural 
>differences that will prevent Athena scaling?
>

I certainly hope that Athena can handle whatever load you intended to put on
it, or that we can work together to fix whatever problems it has which would
prevent that. :)

Jean-Paul