Opened 6 years ago
Closed 6 years ago
#77 closed defect (fixed)
`UnicodeDecodeError` when server sends non-ASCII error messages
Reported by: | schwa | Owned by: | schwa |
---|---|---|---|
Priority: | major | Milestone: | 3.1 |
Component: | Library | Version: | 3.0 |
Keywords: | makedirs, UnicodeDecodeError, error handling | Cc: |
Description
reported by Roger Demetrescu:
"""
I've found an issue with ftputil 3.0 when trying to use host.makedirs() and part of the path already exists AND the FTP server gives error messages with accented characters.
I created 2 virtualenvs:
- python 2.7 (also happens with python 2.6)
- ftputil 2.8 and ftputil 3.0
FTP Server messages are in brazilian portuguese language.
When I use:
host.makedirs('/aaa/bbb/ccc')
and /aaa doesn't exist, all directories are created successfully.
BUT, when "/aaa" already exists, that's what happens:
ftputil 2.8
>>> host.makedirs('/aaa/bbb/ccc') *cmd* 'CWD /' *put* 'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'CWD /' *put* 'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'MKD aaa' *put* 'MKD aaa\r\n' *get* '550 aaa: N\xe3o \xe9 poss\xedvel criar um arquivo j\xe1 existente.\r\n' *resp* '550 aaa: N\xe3o \xe9 poss\xedvel criar um arquivo j\xe1 existente.' *cmd* 'CWD /' *put* 'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'CWD /' *put* 'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'CWD /aaa' *put* 'CWD /aaa\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'MKD bbb' *put* 'MKD bbb\r\n' *get* '257 "bbb" directory created.\r\n' *resp* '257 "bbb" directory created.' *cmd* 'CWD /' *put* 'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'CWD /' *put* 'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'CWD /aaa/bbb' *put* 'CWD /aaa/bbb\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* 'MKD ccc' *put* 'MKD ccc\r\n' *get* '257 "ccc" directory created.\r\n' *resp* '257 "ccc" directory created.' *cmd* 'CWD /' *put* 'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.'
ftputil 3.0
>>> host.makedirs('/aaa/bbb/ccc') *cmd* u'CWD /' *put* u'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* u'CWD /' *put* u'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' *cmd* u'MKD aaa' *put* u'MKD aaa\r\n' *get* '550 aaa: N\xe3o \xe9 poss\xedvel criar um arquivo j\xe1 existente.\r\n' *resp* '550 aaa: N\xe3o \xe9 poss\xedvel criar um arquivo j\xe1 existente.' *cmd* u'CWD /' *put* u'CWD /\r\n' *get* '250 CWD command successful.\r\n' *resp* '250 CWD command successful.' Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/roger/.virtualenvs/version30/local/lib/python2.7/site-packages/ftputil/host.py", line 628, in makedirs self.mkdir(next_directory) File "/home/roger/.virtualenvs/version30/local/lib/python2.7/site-packages/ftputil/host.py", line 608, in mkdir self._robust_ftp_command(command, path) File "/home/roger/.virtualenvs/version30/local/lib/python2.7/site-packages/ftputil/host.py", line 574, in _robust_ftp_command return command(self, tail) File "/home/roger/.virtualenvs/version30/local/lib/python2.7/site-packages/ftputil/host.py", line 607, in command self._session.mkd(path) File "/home/roger/.virtualenvs/version30/local/lib/python2.7/site-packages/ftputil/error.py", line 128, in __exit__ if exc_value.args and exc_value.args[0].startswith("502"): UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 10: ordinal not in range(128)
The error message in pt_br is "Não é possível criar um arquivo já existente."
"""
Change History (3)
comment:1 Changed 6 years ago by
comment:2 Changed 6 years ago by
Here's an untested workaround until a fix is available. Use this code in one of your modules to monkey-patch error.ftplib_error_to_ftp_os_error.__exit__
.
import ftputil.error _original_exit = ftputil.error.ftplib_error_to_ftp_os_error.__exit__ # Bound method, don't use `self` as first argument. def _new_exit(exc_type, exc_value, traceback): # `exc_value.args` is a tuple and thus can't be # modified in-place below. args = [] for arg in exc_value.args: if isinstance(arg, str): # Since latin1 is an 8-bit encoding, the # `decode` call should never cause an exception. arg = arg.decode("latin1") args.append(arg) exc_value.args = tuple(args) return _original_exit(exc_type, exc_value, traceback) ftputil.error.ftplib_error_to_ftp_os_error.__exit__ = _new_exit
If you use the workaround and it fails, please tell me the error you get.
It seems the actual error message is a byte string and ftputil checks if the string starts with the string "502".
In ftputil 2.8, "502" is a byte string, so everything works.
On the other hand, in ftputil 3.0, "502" is a unicode string, since I have
from __future__ import unicode_literals
at the top of the module. For thestartswith
check, Python implicitly tries to convert the server message, a byte string, to a unicode string and fails because of the non-ASCII characters.The responsible code is
in
error.py
. Note thestartswith
call about in the middle of the__exit__
method.