Support pygobject 3.4 / gtk 3.6 in gireactor

[itamar]

The combination of pygobject 3.4 and gtk 3.6 (possibly only one is necessary - they come together on Ubuntu 12.10) make the gi.repository.Gtk module unimportable if X11 is unavailable. That is, if there's no DISPLAY. This breaks our code, which assumes that if Gi is available then Gtk is available too.

This is a requirement for #6055 since pygobject only supports Python 3.3 as of pygobject 3.4.

[itamarst]

(In [36330]) Branching to 'pygobject34-6170'

[itamar]

Here's what the problem looks like:

$ DISPLAY="" PYTHONPATH=`pwd` trial -r gi twisted.internet.test.test_gireactor
twisted.internet
  test
    test_gireactor ...                                                  [ERROR]

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/itamarst/Devel/Twisted/trunk/twisted/trial/runner.py", line 650, in loadByNames
    things.append(self.findByName(name))
  File "/home/itamarst/Devel/Twisted/trunk/twisted/trial/runner.py", line 460, in findByName
    return reflect.namedAny(name)
  File "/home/itamarst/Devel/Twisted/trunk/twisted/python/_reflectpy3.py", line 208, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/home/itamarst/Devel/Twisted/trunk/twisted/python/_reflectpy3.py", line 144, in _importAndCheckStack
    return __import__(importName)
  File "/home/itamarst/Devel/Twisted/trunk/twisted/internet/test/test_gireactor.py", line 11, in <module>
    from gi.repository import Gtk, Gio
  File "/usr/lib/python2.7/dist-packages/gi/importer.py", line 76, in load_module
    dynamic_module._load()
  File "/usr/lib/python2.7/dist-packages/gi/module.py", line 244, in _load
    overrides_modules = __import__('gi.overrides', fromlist=[self._namespace])
  File "/usr/lib/python2.7/dist-packages/gi/overrides/Gtk.py", line 1624, in <module>
    raise RuntimeError("Gtk couldn't be initialized")
exceptions.RuntimeError: Gtk couldn't be initialized

twisted.internet.test.test_gireactor
-------------------------------------------------------------------------------
Ran 1 tests in 0.004s

FAILED (errors=1)

[itamar]

 http://buildbot.twistedmatrix.com/boxes-supported?branch=/branches/pygobject34-6170 is running.

The stupid way that pygobject determines it won't create Gtk for you is to call Gtk.init_check; if it returns False then raise RuntimeError. This means if you import gi.repository.Gtk a second time, you get a module back... except it's a half-broken module. Merely checking for RuntimeError is therefore insufficient; you get crap like this where it crashes half-way, or you get an exception when you try to _exit_ the gtk3 mainloop because apparently starting the mainloop works, etc.:

$ trial -r gi twisted.internet.test.test_udp twisted.internet.test.test_gireactor
twisted.internet.test.test_gireactor
  GApplicationRegistration
    test_cantRegisterAfterRun ...                                          [OK]
    test_cantRegisterTwice ...                                             [OK]
    test_gApplicationActivate ...                                          [OK]
    test_gtkApplicationActivate ... 
(trial:12455): Gtk-WARNING **: cannot open display: 
$

The only way I could find to check if Gtk was imported brokenly was re-doing the init_check.

[exarkun]

Supposing one didn't have Ubuntu 12.10 installed, and observing that none of the GIReactor tests are currently running on the Python 3.3 builder, how might one verify that this actually works?

[itamar]

Ask Barry to install pygobject for Python 2 on the 12.10 buildslave he provided (if it isn't already), and configure buildbot to use it for python 2 trial runs as well, running under gi reactor.

[exarkun]

Since this is more complex than I can deal with, and because there is no BuildBot slave set up with an environment which can exercise this code, and because the whole universe of gtk-related functionality is so poorly tested already, and because this issue isn't really a blocker to porting gireactor, I'm kicking this out of the minimal milestone and not reviewing it.

[itamar]

An alternative, possibly better implementation would make gtk3reactor.py raise ImportError if it was on a POSIX environment, not Mac OS X, and the DISPLAY env variable was not set.

[itamarst]

(In [36499]) A simpler, more robust solution. Refs #6170

[itamarst]

(In [36500]) OS X doesn't use X11. Refs #6170

[itamar]

I've switched to simply requiring DISPLAY env variable, as described above.

 http://buildbot.twistedmatrix.com/boxes-supported?branch=/branches/pygobject34-6170

[exarkun]

Thanks. I'm re-adding this to the minimal milestone, since the conclusion we came to is that this is the easiest way to be able to run the gireactor tests on our Python 3.3 builder.

1. Please refactor the redundant skip logic/message so there's less repetition
2. Somehow you upset pydoctor -  http://buildbot.twistedmatrix.com/builders/documentation/builds/2745/steps/api-documentation
3. The comment in twisted/internet/gtk3reactor.py feels a bit misleading. It explains a subtlety that the code is going to ignore, but without trying to explain what the point of the code is.
4. There's a big pile of pyflakes warnings in twisted/internet/test/test_gireactor.py, which it would be nice to eliminate. Moving the global reactor import into test_noCompatibilityLayer would probably do that.

Please merge after addressing these.

[itamarst]

(In [36513]) Address review comments 1, 3 and 4. Refs #6170

[itamarst]

(In [36515]) Merge pygobject-6170: Passing tests with newer versions of pygobject.

Author: itamar Review: exarkun Fixes: #6170

On newer versions of pygobject (e.g. v3.4, included in Ubuntu 12.10) Gtk is unimportable if X11 isn't available. This fixes the tests to deal with this brokenness.