[Twisted-Python] porting twisted.spread to Python3

[Wolfgang Rohdewald]

I now have a local git branch with about 70 commits, always 
rebased onto current trunk.

Those test suites pass with Python2 and Python3:

test_banana, test_jelly, test_pb, test_pbfailure, test_persisted

I also have a real app using PB (kajongg, a KDE game). Running
the server and the client with any combination of Python 2 and
Python 3 works but more testing is needed there. Especially
transferring failures with non-ascii test, so I need to inject
random errors.

spread.banana is now fully covered by tests (was at 85% before).
There is still a lot of code in twisted.spread which has no
tests yet.
I do not think I want to add missing tests for everything,
only for those regions where I do changes or where it is not
too difficult. And of course if an application triggers a bug
yet uncovered by tests.

Can you point me to other PB applications I could use for testing?
Preferrably ones that are already mostly compatible with PY3
themselves, but that is not a _must_, I would try porting them
unless it turns out to be too much work. And preferrably ones
developed by people with a non-ascii native language. I do not
yet trust Failures and logging with non-ascii.


Below is the current list of commits. Everything you see
there is still much in flux. This includes shuffling code
between commits, combining and splitting commits, changing order.
I want to open one ticket per commit. Simple things hopefully 
not needing much discussion first.

Estimating one week per review/commit to trunk, this process 
will take more than one year. Often I can only add a new ticket
after the previous one has been integrated because they 
depend on each other.

So if it really seems to take that long, at some point I might
just decide to put what I have to github and not care anymore.
Which of course means I might have to distribute a patched
twisted version with my application. I really want to leave
Python2 behind, and only twisted is in the way. Latest versions
of KDE do not have Python2 bindings anymore (KDE Frameworks 5).

This log of course includes things being in tickets up for review.

git log --pretty='format:%s' --reverse trunk..pb3
spread.banana: selecting an unsupported dialect now raises an Exception
spread.banana now raises BananaError when receiving pb messages without pb being the selected dialect
spread.banana: specific error message about unicode objects
spread.banana: test coverage goes up from 85% to 100%
test.test_banana: eliminate duplicate code
twisted.python.reflect.safe_str on Python 3 converts utf-8 encoded bytes to clean str instead of "b'a'"
test.test_{failure,banana,persisted,reflect}: consistently name all test methods test_*
test.test_jelly: add tests for yet untested code
spread.jelly: support the sets module only for Python 2
twisted.spread: Assume that Python always has unicode support enabled
Ticket 7653. spread.jelly: deprecate unjellying instances
spread.jelly can now transfer objects with new style classes
cred.credentials: implements->implementer
cred.checkers: implements->implementer
test.test_pb: implements->implementer
persisted.styles: add new tests
persisted.styles.unpickleMethod: remove dead code
spread.flavors: fix import paths, making them work with Python 3 too
spread: Python 3: add from __future__
do not use argument tuple unpacking anymore, not available in Python 3
spread.publish and test.test_pb: replace file by open
test.test_pb: use a constant for "BRAINS!"
python.compat: get_im_func, get_im_self, get_im_class and use them in spread
persisted.styles: simplify unpickleMethod
persisted.styles: Port to Python 3
python.compat: for PY3, define long=int and xrange=range
spread.banana: replace StringIO.StringIO by io.BytesIO
python.compat.networkChar: New, and use it in spread.banana
python.compat.networkString: improve Exception message
spread.banana: change literals from "" to b"" where appropriate
test.test_banana: fix a wrong assertion message
test.test_banana: use itertools.chain
test.test_banana: Python 3 does not know type long
spread: Python 3 does not have sys.maxint
spread.banana and test: Complete port to Python 3: handling of data
spread.jelly: Unjellying instance is not supported with Python 3
spread.jelly: unify usage of the types module
spread.jelly: consistently use _atom identifiers
spread.jelly: complete port to Python 3
spread.pb, test.test_pbfailure: new syntax for except
spread.pb: do not use argument tuple unpacking anymore, not available in Python 3
test.test_jelly now passes all tests with Python 3
cred.credentials.updateHash accepts unicode
add ported modules to python/dist3.py
spread.flavors: consistently use _atom identifiers
spread.flavors: Python 3: eliminate __cmp__. Only support __eq__ and __ne__
spread: Use networkString/nativeString where appropriate
spread.flavors: switch atoms from str to b""
test.test_pb: improve test_concurrentLogin: use a non-ascii password
spread.pb: eliminate __cmp__, using FancyEqMixin
test.test_pb: add comment to test_tooManyRefs: too slow
spread.pb: introduce *_atom constants
spread.pb: Python 3: use list() around some generators
spread.pb.Broker: improve Exception message
spread.pb and test: Python 3: xrange = range
spread.pb and test: Python 3: use byte literals
spread.pb: failure2Copyable: use _newInstance from jelly
spread.pb: replace types.ListType with list
spread.pb and test: use networkChar where appropriate
test.test_pb: change from StringIO to io.BytesIO
spread.pb: Python 3: make *_atom bytes
python.logger.formatWithCall: support non-ascii strings. Why exactly does kajongg need this?
python.compat: networkIdentifier, nativeIdentifier. Not yet used. SQUASH
spread.pb: CopiedFailure now handles non-ascii strings. TEST MISSING
spread.flavors: RemoteCopy supports non-ascii identifiers. TEST MISSING. Why does kajongg need this?
python.reflect: _safeFormat must handle unicode
python.util: new debug function stack(). REMOVE AGAIN
python.compat: nativeIdentifier accepts non-unicode. NEEDED WHY? TEST CASE MISSING
test.test_{banana,jelly,pb,pbfailure,persisted}: from __future__
test.test_pbfailure: make literal strings bytes
test.test_pbfailure: replace StringIO with io.BytesIO
spread.flavors: translate keys of received dict into nativeString. INCOMPLETE


-- 

Wolfgang