[Twisted-Python] utf-8 support for ftp

Paul Wiseman poalman at gmail.com
Thu Nov 22 12:15:01 EST 2012

On 22 November 2012 11:35, Adi Roiban <adi at roiban.ro> wrote:

> On 21 November 2012 19:45, Paul Wiseman <poalman at gmail.com> wrote:
> > On 21 November 2012 17:35, Paul Wiseman <poalman at gmail.com> wrote:
> >>
> >> I know this has been asked before, I've found it in several trackers.
> >>
> >> http://twistedmatrix.com/trac/ticket/5411
> >> http://twistedmatrix.com/trac/ticket/4515
> >> http://twistedmatrix.com/trac/ticket/5100
> >>
> >> Some of these were created a while ago, some with patches. I wondered
> what
> >> the current status was with regards to utf-8 in ftp? Is it currently
> >> possible?
> >>
> >> I'm trying to list a folder called 'Paul’s Mac Pro', this gets returned
> as
> >> 'Paulâ s Mac Pro'. The client tries to list that folder, and the server
> >> returns 'Paulâ  s Mac Pro: No such file or directory.' I think the
> client
> >> is interpreting the bytes that make up the unicode char as separate
> ascii
> >> characters. (maybe a problem with the client? FileZilla
> >>
> >> This problem may be made worse by my implementation of IFTPShell, I can
> >> try to make an example if this isn't the expected result.
> >>
> >> I'm returning all names and paths back encoded in utf-8, but maybe the
> >> problem is the client isn't expecting it because there's no FEAT
> command (is
> >> the patch in 4515 ok to add?)
> >>
> >> I just want to get an idea of how I can best go about getting this to
> >> work. Will I need to build in support, or is it available in a newer
> >> version, or by applying some patches?
> >>
> >> Thanks very much!!
> >>
> >> Paul
> The patch from #4515 does little to provide a functional UTF-8 FTP
> implementation.
> It is there only to help advertise future implementation of UTF-8 support.
> Thanks for reminding about #5411 . I have submitted a patch for review
> together with required tests.
> I have a few more FTP ticket in work/waiting for review. As soon as
> they are done, I will look at UTF-8 support.
Awesome- sounds good! I'll keep an eye out for it! :)

> ----
> Patch #4515 is already in trunk, and together with patch for #5411 it
> should be a good start for UTF-8 support.
> Ticket #5100 is mainly about the default IFTPShell implementation, but
> if you have your own IFTPShell implementation just make sure it
> accepts input as str encoded utf-8 and that it output str encoded
> utf-8

My IFPShell does this, I have crudely added ftp_FEAT and ftp_OPT to a
subclass of FTP which returns UTF8 in the list of features and responds
with CMD_OK for "UTF8 ON" OPT. Which will hopefully get me by until there
is support.

I'd like to help add support, but I'm not sure how you'd do it. The FTP
protocol needs to know what to return if a client asks UTF8 ON in OPT, but
it's up to the shell which needs to make sure it complies with this option
and return things in the right encoding? unless everything is returned in
unicode and the FTP protocol encodes it if utf8 is on, or throws an error
if anything is outside an ascii range.

but there are certain parts of it sound quite tricky to get right:

   "Prior to transmitting response code 200 in response to the OPTS UTF-8

   command, the Server-FTP must not transmit UTF-8 encoded pathnames and
   should not accept them on commands: the Server-FTP should transmit
   either response code 501 or 553 in reply to any command which
   includes a pathname outside the range of 7-bit ASCII; and the Server-
   FTP should transmit response code 550 in reply to any command to
   which the server would otherwise have sent a UTF-8 encoded pathname."


> Cheers,
> --
> Adi Roiban
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20121122/1a9798b2/attachment.htm 

More information about the Twisted-Python mailing list