[Twisted-Python] Python 3: bytes vs. str in twisted.python.filepath

Harry Bock bock.harryw at gmail.com
Sat Jul 13 20:00:09 MDT 2013


Hi all,

My name is Harry Bock.  I'm interested in helping out porting Twisted to
Python 3, and I've popped in IRC a few times to introduce myself and ask a
few questions. A few developers agreed that working on trial dependencies
would be a big help.

In doing some porting work on trial, I stumbled upon a previous porting
effort (possibly by Itamar?) for twisted.python.filepath and related
modules.  It seemed like the porting effort included forcing all pathname
inputs to be byte strings instead of native strings. After some
investigation, I believe this is the wrong approach, but I wanted to start
a discussion here first.  Some thoughts:

(a) As of Python 3.3, use of the ANSI API in Windows is deprecated[1], so
many functions in os and os.path raise DeprecationWarning when given byte
strings as input.  Although win32 is not an initial target of the porting
effort, we'll have to support it eventually and the API should be supported
before then.

(b) Misunderstandings at the application level about the underlying
filesystem's path encoding is not the problem of the Twisted API.  Correct
me if I'm wrong, but that's the responsibility of the system administrator
or individual user (at least on UNIX) to set the LANG environment variable,
or for the application to call setlocale(3) to explicitly override it.

(c) If we do not allow unicode strings, we will be forcing the application
developer to decide how to encode paths when using the FilePath API.  Per
(b) above, the user will have to call sys.getfilesystemencoding()[2] to
divine what encoding to use before using the API at all, which to me is
terribly annoying and would just add str.encode calls everywhere.

Thus, my vote is that on Python 2.x, Twisted should accept either the
native str or unicode types for path names, and on Python 3.x, only accept
the str type to prevent deprecation issues with system calls.  I have a
patch set that will make this happen including unittest modifications; if
there's a consensus I'm happy to open a ticket and submit the patches.

Thanks!

[1] http://bugs.python.org/issue13374
[2] http://docs.python.org/3/library/sys.html#sys.getfilesystemencoding
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130713/e4a9a989/attachment.html>


More information about the Twisted-Python mailing list