[Twisted-Python] finding out what's blocking the reactor

[Werner Thie]

Aloha

the code below has proven to be a life saver together with manhole. It
might be a bit outdated but I usually create a manhole access allowing
me to inspect the server from within then using dumpObjects to look at
the sea of objects in the running process.

Even only watching the inner workings of a server is quite fun and
illuminating.

Werner

exc =[
"function",
"type",
"list",
"dict",
"tuple",
"wrapper_descriptor",
"module",
"method_descriptor",
"member_descriptor",
"instancemethod",
"builtin_function_or_method",
"frame",
"classmethod",
"classmethod_descriptor",
"_Environ",
"MemoryError",
"_Printer",
"_Helper",
"getset_descriptor",
"weakref"
]
inc =[
]
prev ={}
#delta - print delta between current and previous run
#limit - only print object count bigger than limit
#inc - include named objects in the dump
#exc - exclude named obects in the dump
defdumpObjects(delta=True, limit=0, include=inc, exclude=[]):
globalprev
ifinclude !=[] andexclude !=[]:
print'cannot use include and exclude at the same time'
return
print'working with:'
print' delta: ', delta
print' limit: ', limit
print' include: ', include
print' exclude: ', exclude
objects ={}
gc.collect()
oo =gc.get_objects()
foro inoo:
ifgetattr(o, "__class__", None):
name =o.__class__.__name__
if((exclude ==[] andinclude ==[]) or\
(exclude !=[] andname notinexclude) or\
(include !=[] andname ininclude)):
objects[name] =objects.get(name, 0) +1
## if more:
## print o
pk =prev.keys()
pk.sort()
names =objects.keys()
names.sort()
forname innames:
iflimit ==0orobjects[name] >limit:
ifnotprev.has_key(name):
prev[name] =objects[name]
dt =objects[name] -prev[name]
ifdelta ordt !=0:
print'%0.6d-- %0.6d-- '%(dt, objects[name]), name
prev[name] =objects[name]
defgetObjects(oname):
"""
gets an object list with all the named objects out of the sea of
gc'ed objects
"""
olist =[]
objects ={}
gc.collect()
oo =gc.get_objects()
foro inoo:
ifgetattr(o, "__class__", None):
name =o.__class__.__name__
if(name ==oname):
olist.append(o)
returnolist
On 5/15/19 02:19, Maarten ter Huurne wrote:
> On Wednesday, 15 May 2019 13:11:29 CEST Scott, Barry wrote:
>
>> In my experience gc is low level noise only.
>> Our work load is very heavy and its never been an issue.
> It depends entirely on how many objects you have. With a few GBs worth 
> of relatively small objects, a full garbage collection cycle can take 
> over a minute. But in normal circumstances, the overhead indeed isn't 
> relevant.
>
> Bye,
> 		Maarten
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20190515/d3e9841f/attachment-0002.html>