get twisted to run on jython

[glyph]

There are some other tickets which imply this as a goal, but it hasn't been made explicit that this is desired.

This ticket should be closed when we have a jython buildbot that is green.

Related tickets

• #3410
• #3099
• #3411
• #3718
• #3725
• #4002

[glyph]

Some nasty hacks just so we can see trial start up and report bugs in Jython

[glyph]

Accompanying the nasty hacks mentioned above, this is a signal module that provides almost enough stuff to get started

[glyph]

Signal module derived from one pasted at  http://pylonshq.com/pasties/646 - copying it here because pastebins typically aren't permanent.

[thijs]

The Jython buildbot is back up:  http://buildbot.twistedmatrix.com/builders/ubuntu64-jython2.5-select/builds/19/steps/trial/logs/stdio

[vschiavoni]

what is the exact steps to locally reproduce the error that is now stopping the buildbot ( http://buildbot.twistedmatrix.com/builders/ubuntu64-jython2.5-select/builds/99) ? That is, which version to checkout ? which build tool can I execute ?

[vschiavoni]

Currently, the build process fails due to a missing dependency toward jruby:

jython -v  setup.py install
import: 'exceptions' as org.python.core.exceptions in builtin modules
import: import site # precompiled from /Users/veleno/jython2.5b1/Lib/site$py.class
import: 'sys' as sys in builtin modules
import: import os # precompiled from /Users/veleno/jython2.5b1/Lib/os$py.class
import: 'errno' as org.python.modules.errno in builtin modules
import: 'jarray' as org.python.modules.jarray in builtin modules
import: 'java' as java package
import: 'System' as java class
import: 'time' as org.python.modules.time.Time in builtin modules
import: import stat # precompiled from /Users/veleno/jython2.5b1/Lib/stat$py.class
import: 'File' as java class
import: 'org' as java package
import: 'FileDescriptors' as java class
import: 'FileIO' as java class
import: 'IOBase' as java class
import: 'Py' as java class
import: 'Errno' as java class
import: 'JavaPOSIX' as java class
import: 'POSIXHandler' as java class
import: 'POSIXFactory' as java class
import: import posixpath # precompiled from /Users/veleno/jython2.5b1/Lib/posixpath$py.class
import: 'IOException' as java class
import: 'SecureRandom' as java class
import: 'FileDescriptor' as java class
import: import twisted # precompiled from ./twisted/__init__$py.class
import: import twisted.python # precompiled from ./twisted/python/__init__$py.class
import: import twisted.python.compat # precompiled from ./twisted/python/compat$py.class
import: import string # precompiled from /Users/veleno/jython2.5b1/Lib/string$py.class
import: import re # precompiled from /Users/veleno/jython2.5b1/Lib/re$py.class
import: import sre_compile # precompiled from /Users/veleno/jython2.5b1/Lib/sre_compile$py.class
import: '_sre' as org.python.modules._sre in builtin modules
import: import sre_constants # precompiled from /Users/veleno/jython2.5b1/Lib/sre_constants$py.class
import: import sre_parse # precompiled from /Users/veleno/jython2.5b1/Lib/sre_parse$py.class
import: import copy_reg # precompiled from /Users/veleno/jython2.5b1/Lib/copy_reg$py.class
import: import types # precompiled from /Users/veleno/jython2.5b1/Lib/types$py.class
import: import socket # precompiled from /Users/veleno/jython2.5b1/Lib/socket$py.class
import: 'struct' as org.python.modules.struct in builtin modules
import: import threading # precompiled from /Users/veleno/jython2.5b1/Lib/threading$py.class
import: 'InterruptedException' as java class
import: 'Collections' as java class
import: 'WeakHashMap' as java class
import: 'Semaphore' as java class
import: 'CyclicBarrier' as java class
import: 'ReentrantLock' as java class
import: 'jython' as java class
import: 'thread' as org.python.modules.thread.thread in builtin modules
import: 'Thread' as java class
import: import weakref # precompiled from /Users/veleno/jython2.5b1/Lib/weakref$py.class
import: import UserDict # precompiled from /Users/veleno/jython2.5b1/Lib/UserDict$py.class
import: '_weakref' as org.python.modules._weakref.WeakrefModule in builtin modules
import: import traceback # precompiled from /Users/veleno/jython2.5b1/Lib/traceback$py.class
import: import linecache # precompiled from /Users/veleno/jython2.5b1/Lib/linecache$py.class
import: import atexit # precompiled from /Users/veleno/jython2.5b1/Lib/atexit$py.class
import: 'BufferedInputStream' as java class
import: 'BufferedOutputStream' as java class
import: 'InterruptedIOException' as java class
import: 'String' as java class
import: 'Exception' as java class
import: 'DatagramPacket' as java class
import: 'InetAddress' as java class
import: 'InetSocketAddress' as java class
import: 'Socket' as java class
import: 'BindException' as java class
import: 'ConnectException' as java class
import: 'NoRouteToHostException' as java class
import: 'PortUnreachableException' as java class
import: 'ProtocolException' as java class
import: 'SocketException' as java class
import: 'SocketTimeoutException' as java class
import: 'UnknownHostException' as java class
import: 'ByteBuffer' as java class
import: 'DatagramChannel' as java class
import: 'ServerSocketChannel' as java class
import: 'SocketChannel' as java class
import: 'AlreadyConnectedException' as java class
import: 'AsynchronousCloseException' as java class
import: 'CancelledKeyException' as java class
import: 'ClosedByInterruptException' as java class
import: 'ClosedChannelException' as java class
import: 'ClosedSelectorException' as java class
import: 'ConnectionPendingException' as java class
import: 'IllegalBlockingModeException' as java class
import: 'IllegalSelectorException' as java class
import: 'NoConnectionPendingException' as java class
import: 'NonReadableChannelException' as java class
import: 'NonWritableChannelException' as java class
import: 'NotYetBoundException' as java class
import: 'NotYetConnectedException' as java class
import: 'UnresolvedAddressException' as java class
import: 'UnsupportedAddressTypeException' as java class
import: 'javax' as java package
import: 'SSLSocketFactory' as java class
import: 'DatagramSocketIO' as java class
import: 'ServerSocketIO' as java class
import: 'SocketIO' as java class
import: 'operator' as org.python.modules.operator in builtin modules
import: import twisted._version # precompiled from ./twisted/_version$py.class
import: import twisted.python.versions # precompiled from ./twisted/python/versions$py.class
import: import twisted.copyright # precompiled from ./twisted/copyright$py.class
import: import twisted.python.dist # precompiled from ./twisted/python/dist$py.class
import: import distutils # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/__init__$py.class
import: import distutils.command # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/command/__init__$py.class
import: import distutils.command.build_scripts # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/command/build_scripts$py.class
import: import distutils.sysconfig # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/sysconfig$py.class
import: import distutils.errors # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/errors$py.class
import: import distutils.core # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/core$py.class
import: import distutils.debug # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/debug$py.class
import: import distutils.util # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/util$py.class
import: import distutils.dep_util # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/dep_util$py.class
import: import distutils.spawn # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/spawn$py.class
import: import distutils.log # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/log$py.class
import: import distutils.dist # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/dist$py.class
import: import copy # precompiled from /Users/veleno/jython2.5b1/Lib/copy$py.class
import: 'PyStringMap' as java class
import: import warnings # precompiled from /Users/veleno/jython2.5b1/Lib/warnings$py.class
import: import distutils.fancy_getopt # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/fancy_getopt$py.class
import: import getopt # precompiled from /Users/veleno/jython2.5b1/Lib/getopt$py.class
import: import distutils.cmd # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/cmd$py.class
import: import distutils.dir_util # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/dir_util$py.class
import: import distutils.file_util # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/file_util$py.class
import: import distutils.archive_util # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/archive_util$py.class
import: import distutils.extension # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/extension$py.class
import: import distutils.command.install_data # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/command/install_data$py.class
import: import distutils.command.build_ext # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/command/build_ext$py.class
import: import distutils.command.build_py # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/command/build_py$py.class
import: import glob # precompiled from /Users/veleno/jython2.5b1/Lib/glob$py.class
import: import fnmatch # precompiled from /Users/veleno/jython2.5b1/Lib/fnmatch$py.class
import: import ConfigParser # precompiled from /Users/veleno/jython2.5b1/Lib/ConfigParser$py.class
import: import distutils.command.install # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/command/install$py.class
running install
import: import distutils.command.build # precompiled from /Users/veleno/jython2.5b1/Lib/distutils/command/build$py.class
running build
running build_py
copying twisted/_version.py -> build/lib/twisted
import: 'Integer' as java class
import: 'Long' as java class
Traceback (most recent call last):
  File "setup.py", line 97, in <module>
    main(sys.argv[1:])
  File "setup.py", line 97, in <module>
    main(sys.argv[1:])
  File "setup.py", line 92, in main
    setup(**setup_args)
  File "./twisted/python/dist.py", line 47, in setup
    return core.setup(**get_setup_args(**kw))
  File "/Users/veleno/jython2.5b1/Lib/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/Users/veleno/jython2.5b1/Lib/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/Users/veleno/jython2.5b1/Lib/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/Users/veleno/jython2.5b1/Lib/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Users/veleno/jython2.5b1/Lib/distutils/command/install.py", line 513, in run
    self.run_command('build')
  File "/Users/veleno/jython2.5b1/Lib/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/Users/veleno/jython2.5b1/Lib/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Users/veleno/jython2.5b1/Lib/distutils/command/build.py", line 112, in run
    self.run_command(cmd_name)
  File "/Users/veleno/jython2.5b1/Lib/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/Users/veleno/jython2.5b1/Lib/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/Users/veleno/jython2.5b1/Lib/distutils/command/build_py.py", line 97, in run
    self.build_packages()
  File "/Users/veleno/jython2.5b1/Lib/distutils/command/build_py.py", line 415, in build_packages
    self.build_module(module, module_file, package)
  File "/Users/veleno/jython2.5b1/Lib/distutils/command/build_py.py", line 378, in build_module
    return self.copy_file(module_file, outfile, preserve_mode=0)
  File "/Users/veleno/jython2.5b1/Lib/distutils/cmd.py", line 371, in copy_file
    return file_util.copy_file(
  File "/Users/veleno/jython2.5b1/Lib/distutils/file_util.py", line 172, in copy_file
    os.utime(dst, (st[ST_ATIME], st[ST_MTIME]))
  File "/Users/veleno/jython2.5b1/Lib/os.py", line 557, in utime
    atimeval = _to_timeval(times[0])
  File "/Users/veleno/jython2.5b1/Lib/os.py", line 574, in _to_timeval
    from org.jruby.ext.posix.util import Platform
ImportError: No module named jruby

[thijs]

Replying to glyph:

Signal module derived from one pasted at  http://pylonshq.com/pasties/646 - copying it here because pastebins typically aren't permanent.

They added support for signals based on that module:  http://bugs.jython.org/issue1074

The builder now fails with:  http://buildbot.twistedmatrix.com/builders/ubuntu64-jython2.5-select/builds/102/steps/trial/logs/stdio

so we're one step closer :)

[thijs]

Replying to vschiavoni:

Currently, the build process fails due to a missing dependency toward jruby:

I tried installing twisted 8.2 on the latest jython trunk (r5970) and it works great (no jruby errors here) but getting some UnicodeDecodeError:

byte-compiling build/bdist.java1.5.0_06/egg/twisted/words/test/test_xpath.py to test_xpath$py.class
byte-compiling build/bdist.java1.5.0_06/egg/twisted/words/test/test_domish.py to test_domish$py.class
byte-compiling build/bdist.java1.5.0_06/egg/twisted/words/test/test_jabberxmppstringprep.py to test_jabberxmppstringprep$py.class
Sorry: UnicodeDecodeError: ('unicodeescape', "u'\\udf42'", 2, 9, 'illegal Unicode character')
byte-compiling build/bdist.java1.5.0_06/egg/twisted/words/test/test_irc.py to test_irc$py.class
byte-compiling build/bdist.java1.5.0_06/egg/twisted/words/test/__init__.py to __init__$py.class

[thijs]

Replying to thijs:

The builder now fails with:  http://buildbot.twistedmatrix.com/builders/ubuntu64-jython2.5-select/builds/102/steps/trial/logs/stdio

Opened a Jython  ticket for this issue.

[glyph]

(In [26586]) Branching to 'jython-3413'

[glyph]

I'm not intending to merge this branch; every fix here should be explained, tested, and merged separately. However, it's easier to smash the fixes together to push Jython as far along as possible to enumerate the other issues. Also, the Jython folks can run tests against this branch and their fix issues in advance of our patches.

[thijs]

Added some more related tickets to this ticket description.

[thijs]

adding more related tickets to the ticket description

[thijs]

Our Jython buildslave is now failing with the following  error:

Traceback (most recent call last):
  File "./bin/trial", line 22, in <module>
    run()
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-ubuntu/Twisted/twisted/scripts/trial.py", line 363, in run
    test_result = trialRunner.run(suite)
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-ubuntu/Twisted/twisted/trial/runner.py", line 833, in run
    return self._runWithoutDecoration(test)
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-ubuntu/Twisted/twisted/trial/runner.py", line 859, in _runWithoutDecoration
    oldDir = self._setUpTestdir()
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-ubuntu/Twisted/twisted/trial/runner.py", line 749, in _setUpTestdir
    if self._testDirLock.lock():
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-ubuntu/Twisted/twisted/python/lockfile.py", line 125, in lock
    symlink(str(os.getpid()), self.name)
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-ubuntu/Twisted/twisted/python/lockfile.py", line 125, in lock
    symlink(str(os.getpid()), self.name)
AttributeError: 'module' object has no attribute 'getpid'

Seems like a Jython issue to me so opened a  ticket for them.

[thijs]

(In [27705]) Branching to 'jython-3413-2'

[thijs]

(In [27706]) Merge forward. refs #3413

[thijs]

 Issue with ctypes/Jython is logged at  http://bugs.jython.org/issue1554

[thijs]

fyi, created  http://wiki.python.org/jython/TwistedOnJython linking to this ticket.

[therve]

Thijs: is there a plan to get the buildbot slave back? Progress will stale otherwise.