deluge fails with twistedCore 12.1.0

[rm]

Good day.

It was reported there:  http://lists.freebsd.org/pipermail/freebsd-python/2012-June/004275.html

That issuing:

from twisted.internet import gtk2reactor
reactor = gtk2reactor.install()

fails with this message:

/usr/local/lib/python2.7/site-packages/gtk-2.0/gtk/__init__.py:40:
RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  from gtk import _gtk
ImportError: No module named gi

This breaks deluge for example. It looks related to changeset from: #5494 And this fix seems breaks the things with pygtk2. Would you please check? Thanks.

[rm]

Actual code to reproduce the issue:

> from twisted.internet import gtk2reactor 
> reactor = gtk2reactor.install()

[itamar]

My guess is you're automatically accessing pygtk via the gi compat layer; perhaps this is a FreeBSD packaging thing? Or maybe the gi compat layer kicks in automatically if gi is installed but not pygtk. The code snippet above works in Ubuntu 12.04, for me at least.

1. What version pygtk, python gi bindings, glib, gtk, etc. are you running?
2. If you import gtk, does sys.modules["gi.pygtkcompat"] exist?

If the answer to the second question is yes, this could probably be fixed by extending #5676 slightly, in which case I'll probably close this as a duplicate and move the discussion there.

[rm]

Sorry, I'm not sure about "accessing pygtk via the gi compat layer". I only guess that "gi" is for "gobject introspection". Can't say about packaging - I looked at pygtk port and didn't find anything different from upstream (we only have numpy disabled by default and --enable-thread is passing by default). Just can say that until 12.1.0 I never see such an error.

1. Here is a list of relevant packages you requested. Please note that I use more fresh versions of some of them, that we have in ports tree. And I also have both gtk2 and gtk3 installed (using gnome2 though), dunno if this is important. In brackets you may see the versions that is now in official ports tree.

gtk-2.24.10 (2.24.6) gtk-3.0.12 gobject-introspection-0.10.8 py-gobject-2.28.6 glib-2.30.3 (2.28.8)

2. No, it is not:

>>> import gtk
>>> import sys
>>> sys.modules["gi.pygtkcompat"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'gi.pygtkcompat'

I'm open to any tests. May be you have some script that's able to collect all the info that helps.

[itamar]

Hm. And just to be clear, if you just start up python, and then run only those two lines importing and installing gtk2reactor, you get an exception?

[rm]

If I import gtk first, I can't reproduce anymore.

>>> import gtk, sys
>>> from twisted.internet import gtk2reactor
>>> reactor = gtk2reactor.install()
>>> sys.modules["gi.pygtkcompat"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'gi.pygtkcompat'
>>>

[rm]

But if I didn't imported gtk previously I got:

>>> from twisted.internet import gtk2reactor
>>> reactor = gtk2reactor.install()
ImportError: No module named gi
/usr/local/lib/python2.7/site-packages/gtk-2.0/gtk/__init__.py:40: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  from gtk import _gtk
ImportError: No module named gi

[itamar]

I verified that pygtkcompat did not exist in the packages installed on the system exhibiting the problem, so that's not it.

[itamar]

And, reading through pygtk and pygobject source code has gotten me nowhere.

[itamar]

I tried this on the FreeBSD build slave, thanks to the very helpful Tobias Oberstein. I was able to reproduce the problem... but I also note that it doesn't *break* anything, the program continues to run after printing the ImportError. The original report says it broke things, though.

I also note that the behavior in comment 6 above is not what I'm seeing; if I import gtk then you basically can't use gtk2reactor because gi is imported and it complains. I can solve this second problem, at least, and given the sufficient similarity in symptoms it'll probably fix the original reporter's problems.

[itamarst]

(In [35162]) Branching to 'gtk2-freebsd-5737'

[rm]

itamar, did you installed twistedCore from FreeBSD port while testing it on FreeBSD? If yes, please note, that the port is patched against this:

In twisted/internet/gtk2reactor.py there is such change:

-    preventImports=["gi"])
+    preventImports=[""])

[itamar]

I was testing against a checkout of twisted from SVN.

[glyph]

Replying to rm:

itamar, did you installed twistedCore from FreeBSD port while testing it on FreeBSD? If yes, please note, that the port is patched against this:

Hi rm,

I should point out that if the bug is in the FreeBSD packaging of Twisted, you should be reporting bugs against FreeBSD, not Twisted. We support the code that we release, not whatever random mangling of that code your operating system has chosen to provide.

Is there a bug report for what makes this patch necessary? If not, then this patch should be removed.

Thanks for reporting this difference though,

-glyph

[itamar]

Glyph, in case it wasn't clear, the patch against FreeBSD is a fix for the problem; this isn't a packaging bug, this is a bug in Twisted.

[rm]

Hi, this patch was added in first place because after updating twistedCore port to 12.1.0, deluge stops to working. So the original problem is reported there, in #5737, and the local patch was added as workaround to this issue (until it will be solved upstream) and to let the FreeBSD users to use deluge. Also, as I wrote before, there is no any other specifics - just regular setuptools install and that's all.

[glyph]

Replying to rm:

Hi, this patch was added in first place because after updating twistedCore port to 12.1.0, deluge stops to working. So the original problem is reported there, in #5737, and the local patch was added as workaround to this issue (until it will be solved upstream) and to let the FreeBSD users to use deluge. Also, as I wrote before, there is no any other specifics - just regular setuptools install and that's all.

Ah. I see. I interpreted your comment exactly backwards then - I apologize! Thanks for properly reporting the bug upstream :).

[itamar]

 http://buildbot.twistedmatrix.com/boxes-supported?branch=/branches/gtk2-freebsd-5737 failures will be fixed by #5676, so maybe that should be reviewed and merged first.

[itamar]

(As alternative to merging #5676 branch first, I could fix #5790 in this branch, which is a smaller change.)

[itamar]

Since FreeBSD has a workaround already, I can't actually reproduce the exact behavior and we've had no reports on other platforms, I'm going to take this off list for 12.2.

[itamarst]

(In [35286]) Branching to 'gtk2-freebsd-5737-2'

[itamar]

Merged forward, buildbot looks happier now:  http://buildbot.twistedmatrix.com/boxes-supported?branch=/branches/gtk2-freebsd-5737-2

[therve]

Looks good to me, please merge.

[itamarst]

(In [35851]) Merge gtk2-freebsd-5737-2: Make gtk2 reactor work on FreeBSD again.

Author: itamar Review: therve Fixes: #5737

Preventing the import of gi in the gtk2/glib2 reactor was, apparently, a bad idea; it is in any case unnecessary.

[exarkun]

It's really nice to talk about actual versions of things rather than certain versions. The former is helpful for future maintainers, the latter often sends them on wild goose chases.