Opened 23 months ago

Last modified 3 months ago

#5965 enhancement new

Port twisted.trial.runner to Python 3

Reported by: exarkun Owned by: cpdean
Priority: normal Milestone: Python-3.x
Component: trial Keywords: patch python3
Cc: jml, kmike, jon@… Branch:
Author: realcr Launchpad Bug:

Description

twisted.trial.runner is a big part of the guts of the trial command line tool. In order to be able to run the trial command line tool on Python 3, we need to port twisted.trial.runner to Python 3.

Attachments (3)

python3_porting1.patch (23.3 KB) - added by realcr 7 months ago.
Python3 porting fixes
twisted_383e25.patch (23.3 KB) - added by realcr 7 months ago.
Some porting to Python3. This time the right order :)
runner_port3.patch (1.8 KB) - added by realcr 6 months ago.
minor changes to runner.py for python3 porting.

Download all attachments as: .zip

Change History (15)

comment:1 Changed 23 months ago by DefaultCC Plugin

  • Cc jml added

comment:2 Changed 22 months ago by thijs

  • Milestone changed from Python-3.x to Python 3.3 Minimal

comment:3 Changed 21 months ago by itamar

  • Milestone changed from Python 3.3 Minimal to Python-3.x

Not going to happen in minimal port.

Changed 7 months ago by realcr

Python3 porting fixes

comment:4 Changed 7 months ago by realcr

  • Author set to realcr
  • Keywords review patch added

I Ported some parts of files to be Python3 compatible.
Mostly did simple things:

  • Used the 'as' keyword in the except statements.
  • Dealt with lazy evaluation of map.
  • Dealt with sys.maxint.
  • Some names of packages that have changed, like StringIO are dealt with.
  • Sorting using Key Function instead of Compare function.

The trial tests for python2 passed. Tests for python3 didn't pass for me in the beginning, and they still don't pass in the same way.

It's my first time, please tell me if something is need to be done better or in another way.
Regards,
real.

Changed 7 months ago by realcr

Some porting to Python3. This time the right order :)

comment:5 Changed 6 months ago by itamar

  • Keywords review removed

Thanks for working on this! Please read https://twistedmatrix.com/trac/wiki/Plan/Python3#Methodology as well as the rest of the page and resubmit (presumably as multiple tickets).

comment:6 Changed 6 months ago by realcr

  • Keywords python3 review added

Some minor changes to trial.runner to work on python3.3 too:

  • implementer instead of implements.
  • open instead of file.
  • FilePath should only be constructed with bytes, so added encode('utf-8') to self.workingDirectory.

Note that the trial.runner is not fully ported yet.

Included patch file runner_port3.patch

Changed 6 months ago by realcr

minor changes to runner.py for python3 porting.

comment:7 Changed 5 months ago by kmike

  • Cc kmike added

Hey Twisted developers,

we need twisted.trial.runner to start porting Scrapy, so I tried to understand better what should be done for this task. I used 'modulegraph' package to get a list of modules that need to be ported before twisted.trial.runner. Here are the results (modules that are marked as ported are excluded):

twisted.internet

  • twisted.internet._baseprocess (imported by ported twisted.internet.posixbase)
  • twisted.internet._dumbwin32proc (imported by ported twisted.internet.posixbase)
  • twisted.internet._oldtls (imported by ported twisted.internet.tcp)
  • twisted.internet._pollingfile (imported by ported twisted.internet.posixbase)
  • twisted.internet._ssl (imported by ported twisted.internet.tcp)
  • twisted.internet._sslverify (imported by ported twisted.internet.ssl)
  • twisted.internet.fdesc (imported by ported twisted.internet.base)
  • twisted.internet.process (imported by ported twisted.internet.posixbase)
  • twisted.internet.unix (imported by ported twisted.internet.posixbase)
  • twisted.internet.win32eventreactor (imported by ported twisted.internet.selectreactor)

twisted.internet looks in a good shape. All modules that are not explicitly ported are dependencies of modules that are explicitly ported.

twisted.persisted

  • twisted.persisted
  • twisted.persisted.styles

It seems some other twisted.persisted modules should be also ported.

twisted.python

  • twisted.python._initgroups (imported by ported twisted.python.util)
  • twisted.python._inotify (imported by ported twisted.python.runtime)
  • twisted.python.modules
  • twisted.python.reflect
  • twisted.python.win32 (imported from several ported modules)
  • twisted.python.zippath
  • twisted.python.zipstream (not needed, see http://twistedmatrix.com/trac/ticket/4035)

twisted.python.zippath and twisted.python.modules must be ported.

It looks like all twisted.python.reflect usages in trial can be replaced with twisted.python._reflectpy3.

twisted.trial

  • twisted.trial._asyncrunner (used by ported twisted.trial.unittest)
  • twisted.trial.reporter

Both must be ported. For twisted.trial.reporter see http://twistedmatrix.com/trac/ticket/5964

Conclusion

twisted.trial.reporter, twisted.python.zippath and modules from twisted.persisted don't depend on other unported modules and can be ported first.

After that twisted.python.modules (which depends on twisted.python.zippath) and
twisted.trial._asyncrunner (which depends on twisted.trial.reporter) can be ported.

After that it should become possible to port twisted.trial.runner. Porting of twisted.python.reflect is not necessary for this sub-task.

I haven't looked into code in much details, so I may have missed something. But hope this helps.

comment:8 Changed 5 months ago by exarkun

Hi kmike,

Your comment is incredibly helpful. Thank you very much!

comment:9 Changed 5 months ago by multani

  • Cc jon@… added

This still doesn't meet the criteria from https://twistedmatrix.com/trac/wiki/Plan/Python3#Methodology
I'm removing the review keyword in the meanwhile.

comment:10 Changed 5 months ago by multani

  • Keywords review removed

comment:11 Changed 3 months ago by cpdean

I'm going to start looking at this to port twisted.trial.test.test_reporter #5964

comment:12 Changed 3 months ago by cpdean

  • Owner set to cpdean
Note: See TracTickets for help on using tickets.