[Twisted-Python] Twisted and Thread, thread not running when twisted app is running as service

[Sereysethy TOUCH]

Hi,

After I tested it on Linux, everything works fine. So I think the problem
is related to MacOS restriction. I already reported the problem to the
maintainers, because the library is supposed to work on both Linux and
MacOS. But this new thread is killing the twisted main reactor.

By the way I have another question regarding ssh client. I developed an ssh
client, after authentication is made, I dont want to open a channel
directly in serviceStarted in connection, I want to keep the connection so
that I can use to open channel for doing exec later, my question is where
should I keep that connection? Currently in my transport I add a defer in
which it will be later called to create another object that I control, the
defer is called back in serviceStarted, which I pass the connection object.
I am not sure what is a correct way to do it. For now I feel like it is a
cycling reference: transport -> connection -> transport -> my object to
control connection.

Thanks,
TS

On Fri, May 22, 2020 at 6:16 PM Barry Scott <barry.scott at forcepoint.com>
wrote:

> On Thursday, 21 May 2020 21:45:44 BST Sereysethy TOUCH wrote:
> > I am on MacOS, it is a development phase, but deployment will be on
> Linux.
> >
> > It is hard to run dtrace/dtruss on MacOS due to the "system integrity
> > protection" on MacOS. I got some output but very limited information.
> >
> > So if there is a workaround you might suggest?
>
> Why bother debugging it on macOS it never has to work there?
>
> If you are targetting linux then develop and test in linux I'd recommend.
> I use VMware fusion on the mac and run VMs for the target
> environements I need. Works very well.
>
> Barry
>
> >
> > TS
> >
> > On Thu, May 21, 2020 at 10:20 PM Jean-Paul Calderone <
> >
> > exarkun at twistedmatrix.com> wrote:
> > > On Thu, May 21, 2020 at 3:56 PM Sereysethy TOUCH <
> > >
> > > touch.sereysethy at gmail.com> wrote:
> > >> Hi Jean-Paul,
> > >>
> > >> I found the error after working on a short, self contained, correcte
> > >> example. It is not about twisted and normal thread in general, but
> > >> something else.
> > >>
> > >> After more tests, I come to realise that whenever I instantiate this
> > >> class PolicyClient from this library
> > >>
> https://github.com/ray-project/ray/blob/master/rllib/env/policy_client.py
> > >> in my protocol or just anywhere, twisted app either freezes or
> > >> terminates, and this *only* happens when the app runs as a daemon,
> > >> created using a service (.tac file).
> > >>
> > >> I hope you can take a look at the class PolicyClient, why it causes
> > >> problem to reactor main loop. Please point me to where the idea might
> > >> come
> > >> from.
> > >>
> > >> I can give you the example but not sure if it is enough, as the client
> > >> needs to connect to a server.
> > >
> > > It looks like PolicyClient does a blocking HTTP request in __init__.
> > > Since PolicyClient is instantiated in the reactor thread, this will
> block
> > > the reactor.  Perhaps this call is hanging for some reason when the
> > > process
> > > has daemonized?  This would be unusual but not unheard of.  For
> example,
> > > there are macOS environments where a process does not have access to
> the
> > > network if it is not associated with a windowing session.
> > >
> > > What platform do you observe the problem on, and what more can you
> learn
> > > about what the process does on its way to hanging (eg can you run
> strace
> > > on
> > > it)?
> > >
> > > Jean-Paul
> > >
> > >> Thank you,
> > >> TS
> > >>
> > >>
> > >> On Thu, May 21, 2020 at 5:23 PM Jean-Paul Calderone <
> > >>
> > >> exarkun at twistedmatrix.com> wrote:
> > >>> On Thu, May 21, 2020 at 10:58 AM Sereysethy TOUCH <
> > >>>
> > >>> touch.sereysethy at gmail.com> wrote:
> > >>>> Hello,
> > >>>>
> > >>>> I am developing a twisted app, and it runs as a service using
> twistd -y
> > >>>> to start the app.
> > >>>> I am having a problem of running a thread. I know it is not
> recommended
> > >>>> to use thread, but the library that I use, the object created is
> > >>>> running in
> > >>>> a thread.
> > >>>>
> > >>>> Here is the problem:
> > >>>>
> > >>>> 1) if I start reactor by running reactor.run() directly, thread is
> > >>>> running fine
> > >>>> 2) if I run it as a service using twisted, the thread is not
> running,
> > >>>> it runs but it seems to be blocked, because I tried to write
> something
> > >>>> to
> > >>>> file using time.sleep(), but file is empty.
> > >>>>
> > >>>> Is there something that I miss? How can I debug this?
> > >>>
> > >>> A good next step would be to create at Short, Self Contained, Correct
> > >>> (Compilable), Example <http://sscce.org/> and share it.
> > >>>
> > >>> Jean-Paul
> > >>>
> > >>>> Thank you,
> > >>>>
> > >>>> TS
> > >>>> _______________________________________________
> > >>>> Twisted-Python mailing list
> > >>>> Twisted-Python at twistedmatrix.com
> > >>>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> > >>>
> > >>> _______________________________________________
> > >>> Twisted-Python mailing list
> > >>> Twisted-Python at twistedmatrix.com
> > >>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> > >>
> > >> _______________________________________________
> > >> Twisted-Python mailing list
> > >> Twisted-Python at twistedmatrix.com
> > >> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> > >
> > > _______________________________________________
> > > Twisted-Python mailing list
> > > Twisted-Python at twistedmatrix.com
> > > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20200522/93cbb55a/attachment.htm>