[Twisted-Python] Looking for help dealing with ClientService reconnections

[Daniel Sutcliffe]

Thanks Glyph, I think you have given me a push back in the 'right'
direction - more thoughts and commentary embedded below if you, or
anyone else, has the time.

On May 6, 2016, at 10:19 AM, Daniel Sutcliffe <dansut at gmail.com> wrote:
[...]
>> The new ClientService class seems like it will fit my needs very
>> closely but I am struggling with how to handle the reconnections... I
>> have been using the whenConnected() method to grab the Protocol for
>> the initial connection and then use a method of this to poll the
>> connected slave. When the connection is lost I get an errback from
>> this method's deferred which I use as a signal to abandon the Protocol
>> and call whenConnected() again... at this point I have an issue though
>> as the returned deferred immediately gives me a callback with the same
>> Protocol which has just lost its connection, and thus loop...

On Mon, May 9, 2016 at 6:19 PM, Glyph <glyph at twistedmatrix.com> wrote:
> If you want a hook each time a new protocol is created, you're probably
> better off writing a wrapper protocol factory, and passing that to your
> ClientService, then doing any set-up work you want to do in your
> buildProtocol implementation, which delegates to the real, pymodbus
> implementation.

Understood, if this is the way the framework is intended to be used I
realize doing anything else is going to be fighting against the flow.

However, just to probe the situation I found myself in further, for
the sake probing broken code to see how it might be fixed:

Given the ClientService.whenConnected() method is intended to provide
access to my connected Protocol through the deferred it returns, is it
not a little unfriendly that this Protocol may turn out to be
disconnected? OK occasionally due to timing but for this to be a
possible condition which can loop with the same disconnected Protocol
returned until the ClientService has its _currentConnection set to
None, suggests to me that I can't safely use my Protocol from
whenConnected() for much other than as a notification the first
connection has occurred... but how do I avoid this?

I have looked at the source and it seems to me the fact that the
connection has been lost should bubble up to the ClientService through
a t.a.i._DisconnectFactory and t.a.i._ReconnectingProtocolProxy once
my Protocol's connectionLost() is called. My issue seems to be that I
errback on a Protocol method's deferred returned to code at or above
the ClientService level which gives up on that Protocol and calls
whenConnected() to get the next one, only the Protocol's
connectionLost() has yet to be called and then doesn't have chance to
because my code is looping around calling whenConnected() and getting
the same Protocol back. I hope that makes sense :-/

My Q on this is if I should be internally calling my Protocol's
connectionLost() so it can bubble up to the ClientService before I
errback on the Protocol method - whose responsibility is it to call
this?

>> Before I got on this mailing list I posted this Q to stackoverflow
>> with some example code:
>>    http://stackoverflow.com/q/37061807/3448214
>> but no solution or much attention there yet.
>>
>> As I say there, I realize I have probably just made a bad pattern
>> choice for how to use this API, but I have not been able to work out a
>> better choice which seems clean and fits my needs/understanding well.
>> I have tried deriving my own Protocol/Factory and handling the polling
>> there but this seems to get really messy once I start to add code to
>> get the collected data to a destination at that level, involving
>> giving the Protocol too much knowledge of how the data is to be
>> handled.
>
> I am curious as to why you say that this is "messy".

Honestly, this was just a gut feeling at the time, probably more
sourced in my implementation from lack of experience in using Twisted;
after reading around the subject, looking at many more examples, and
your advice, I think I am convinced I need to back to looking at my
own Protocol derived from the pymodbus one with a Factory that
contains the persistent config and access to an interface to pump the
polled data upstream.

>> Any advice, good patterns, or pointers to other projects which do
>> something similar is appreciated,
>
> I spent a while thinking about your question, and I'm sorry that I can't give
> a more thorough answer, but I think you need to be a bit more specific
> about what it is you don't like about your potential solution.  It seems to me
> that having a delegating Factory, especially if all you need to do is set up
> some state on each Protocol that gets produced, should be sufficient...

Looking at the code again I think it just seemed to make sense to me
at the time to have something that is (or has) a ClientService be the
object I am calling a DataSource have more control over when it polls
that data and what it does with it - and not to have to make the
Protocol and its Factory aware of this at all. Does that make it any
clearer? I am happy to push ahead with building this into my
Factory/Protocol if that is more normal usage for Twisted as I am sure
there will be benefits of encapsulating it here beyond the other side
I was seeing during that moment of confusion.

Thanks for the advice, I think it was enough to nudge me in a
direction that will work better to get me going, and if with more
experience other usage makes more sense I can always refactor,
refactor, refactor :)
Cheers
/dan
-- 
Daniel Sutcliffe <dansut at gmail.com>