[Twisted-Python] Retry: Conch Issues

[Adi Roiban]

Hi

On Fri, 25 Sep 2020 at 00:15, Robert DiFalco <robert.difalco at gmail.com>
wrote:

> I am having troubles with Conch. I've been able to reproduce them in a
> unit test and all the code is here along with requirements.txt:
>    https://github.com/radifalco/conch_issues
>
> I'm trying to reuse my connection, but if it is idle too long the server
> shuts it down. What I don't understand is why the next call to
> `FileTransferClient` does not call back or errback? It will just hang
> forever even though the Factory and Transport know the connection was
> closed (as evidenced by the log messages).
>
> I don't ever want to hang, if I use FileTransferClient I would like it
> always call a callback or errback. I feel like Conch is missing a deferred
> somewhere but damned if I can find it.
>
> Thanks!
>
> Have you checked
https://twistedmatrix.com/documents/current/core/howto/clients.html#reconnection
?

I see that `connector.connect()` is commented in your code.

The theory is that once a client connection is lost, you will need to call
again the whole chain starting with reactor.connectTCP.

---------

I know the pain here with SFTP as there is a lot of wrapping: TCP client,
SSH high level protocol, SSH session, SSH channel, SSH subsystem.

One option is to use MySSHClientFactory as a proxy for any SFTP operation.
When the SSH channel is open and you have received the response from the
subsystem request, pass the SFTP client all the way down to the factory.
The factory will know when the connection is lost and then trigger a new
connection... which will update the SFTP client instance on the factory.

That is, instead of getting the SFTP client, the SFTP client will be pushed
to the factory.

I think that the main issue with your code is that it tries to work
directly with the SFTP client, instead of using the factory as a proxy for
SFTP operations.

A protocol represents a single connection that is already made and it can't
reconnect itself.
The factory is responsible for handling new connections, or triggering new
connections.

So, your high-level API is using a very low-level object.
The SFTP client represents an SSH connection that was already made,
authenticated and the SFTP subsystem requested.

For my production code I am doing something like this:
MySSHClientFactory is instantiated with the reactor and the
SFTPClientOptions.
Then, when I want to list a directory I call
sftp_factory.listDir('some/path')
The factory can then check if it needs to get an SFTP client or reuse an
existing one.
A SFTP client is obtained by triggering a new connectTCP to the reactor
with `self` as the factory.

Does it make sense?

-- 
Adi Roiban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20200925/e5dbe4ce/attachment.htm>