[Twisted-Python] Decorator @inlinecallbacks prohibiting memory profiling

[Vikas]

Ok this experiment works fine because it doesn't have any decorator & 
generator.
Here is the actual method (with @inlineCallback and generator) where I 
see method(my_daemon_fun) being profiled with @profile decorator is not 
being profiled.
I am doubting that *generator* or *inlineCallback* decorator is 
prohibiting the correct code from getting profiled.

To profile the code all I need to do is:
  STEP#1 ->  decorate the _monitor_daemons() with "profile" decorator #1
  STEP#2 ->  start memory profiling  
  STEP#3 ->*  *end  memory profiling

Something like this:

@profile                                               => STEP#1
@inlineCallbacks
def my_daemon_fun(self):
     """
     Monitor my daemons
     """
     if self.running == True:
         try:
hp = hpy() => STEP#2
before = hp.heap()
             yield self._lock.acquire()
             .....
             if ((x_running == False) or (b_running == False)):
                 log.error('Daemons are not running, restarting...')
                 yield self._restart_daemons()
         except:
             log.exception('Exception restarting daemons')
         finally:
             yield self._lock.release()
             reactor.callLater(MY_DAEMON_MONITOR_INTERVAL, 
self.my_daemon_fun)
after = hp.heap() => STEP#3
leftover = after - before


Now we run the script to call method: my_daemon_fun(). After running the 
script though we see memory profiling happening but the function being 
profiled is *not* the one we decorated with @profile:

Line #    Mem usage    Increment      Line Contents
1176     42.3 MiB      0.0 MiB       def unwindGenerator(*args, **kwargs):
1177     42.3 MiB      0.0 MiB           try:
1178     42.3 MiB      0.0 MiB               gen = f(*args, **kwargs)
1183     42.3 MiB      0.0 MiB           if not isinstance(gen, 
types.GeneratorType):
1187     42.3 MiB      0.0 MiB           return _inlineCallbacks(None, 
gen, Deferred())


I am investigating why the actual function(i.e. my_daemon_fun()) was not 
profiled? Meanwhile if you can help us here that would be great.

Regards
Vikas

On 21/10/15 7:07 pm, Vikas wrote:
> Hi,
>
> I am trying to instrument method: run() for memory profiling(I am 
> using memory_profiler & guppy tools for this). All I need to do is 
> decorate the run() with "profile" decorator and mention *start* and 
> *end* of memory profiling.
>
> @profile
> def run():
>     d = {}
>     l = []
>     hp = hpy()
>     before = hp.heap()            => *start* memory profiling from here
>
>     d["k1"] = 'val1'
>     d["k2"] = 10
>     count = 0
>     while (count < 9):
>         l.append(count)
>         print 'The count is:', count
>         count = count + 1
>     print "Good bye!"
>     after = hp.heap()         => *end* memory profiling here
>     leftover = after - before
>
>     print leftover
>     del hp
>
>
> So far works fine. But moment I introduce @inlineCallbacks before 
> @profile(see below), I don't see memory getting profiled. Am I doing 
> something wrong here?
> @inlinecallbacks
> @profile
> def run():
>     ....
>
> Regards
> Vikas
>
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20151022/e9dbbcf6/attachment-0002.html>