[Twisted-Python] Looking for help dealing with ClientService reconnections

[Glyph]

> On May 6, 2016, at 10:19 AM, Daniel Sutcliffe <dansut at gmail.com> wrote:
> 
> Hello all, my first post here - only been using Twisted for about a
> month and am also a relative newcomer to Python but have been coding
> professionally for 20+ years. I was attracted to Twisted and Python
> for a particular project purely because after research it seemed to be
> the best tool for the job, and have actually been enjoying both Python
> and Twisted much more than I ever thought I would.
> 
> The project I am coding towards is creating a sensor data collection
> gateway. First iteration needs are simply pulling data from ModBus TCP
> slave PLCs and writing it to a MySQL database, but goals beyond that
> are making the source of the data and its destination(s) very
> flexible(pluggable). Therefore I am trying to create a good clean
> architecture from the outset so as I iterate forwards I don't finish
> up having to take too many steps backwards before heading forwards.
> 
> I am using pymodbus to pull the data which works well for my devices,
> has a twisted async API, and have created more than a few prototypes
> that demonstrate all works as I expect. Where I am a bit stalled is
> getting to grips with a good architecture that fulfills my needs - my
> intention is that the application that meets my first goal will be a
> twistd plugin.
> 
> The new ClientService class seems like it will fit my needs very
> closely but I am struggling with how to handle the reconnections... I
> have been using the whenConnected() method to grab the Protocol for
> the initial connection and then use a method of this to poll the
> connected slave. When the connection is lost I get an errback from
> this method's deferred which I use as a signal to abandon the Protocol
> and call whenConnected() again... at this point I have an issue though
> as the returned deferred immediately gives me a callback with the same
> Protocol which has just lost its connection, and thus loop...

If you want a hook each time a new protocol is created, you're probably better off writing a wrapper protocol factory, and passing that to your ClientService, then doing any set-up work you want to do in your buildProtocol implementation, which delegates to the real, pymodbus implementation.

> Before I got on this mailing list I posted this Q to stackoverflow
> with some example code:
>    http://stackoverflow.com/q/37061807/3448214
> but no solution or much attention there yet.
> 
> As I say there, I realize I have probably just made a bad pattern
> choice for how to use this API, but I have not been able to work out a
> better choice which seems clean and fits my needs/understanding well.
> I have tried deriving my own Protocol/Factory and handling the polling
> there but this seems to get really messy once I start to add code to
> get the collected data to a destination at that level, involving
> giving the Protocol too much knowledge of how the data is to be
> handled.

I am curious as to why you say that this is "messy".

> Any advice, good patterns, or pointers to other projects which do
> something similar is appreciated,

I spent a while thinking about your question, and I'm sorry that I can't give a more thorough answer, but I think you need to be a bit more specific about what it is you don't like about your potential solution.  It seems to me that having a delegating Factory, especially if all you need to do is set up some state on each Protocol that gets produced, should be sufficient...

-glyph