~sschwarzer/ftputil#103: Error `open`ing new file-like objects when a connection is timed-out but not yet closed.

Here's the sample stack trace:

    File "build/bdist.linux-x86_64/egg/ftputil/host.py", line 195, in open
        
      File "build/bdist.linux-x86_64/egg/ftputil/host.py", line 166, in _available_child
        
      File "/usr/lib/python2.7/ftplib.py", line 583, in pwd
        resp = self.sendcmd('PWD')
      File "/usr/lib/python2.7/ftplib.py", line 249, in sendcmd
        return self.getresp()
      File "/usr/lib/python2.7/ftplib.py", line 215, in getresp
        resp = self.getmultiline()
      File "/usr/lib/python2.7/ftplib.py", line 201, in getmultiline
        line = self.getline()
      File "/usr/lib/python2.7/ftplib.py", line 191, in getline
        if not line: raise EOFError
    exceptions.EOFError:

The reason might be that the server has already seen the connection as a timed-out one, but not yet closed it.

Related lines: line #164 to #172 of ftputil/host.py

Status: REPORTED
Submitter: ftputiluser (unverified)
Assigned to: No-one
Submitted: 8 years ago
Updated: 8 years ago
Labels: enhancement library

schwa (unverified) 8 years ago · edit

Thanks for entering this ticket. :-)

Replying to ftputiluser:

The reason might be that the server has already seen the connection as a timed-out one, but not yet closed it.

Assuming this is the reason for the stack trace:

I'd expect that the stack trace would be difficult to reproduce. How often have you seen it?

Probably the problem could happen in other situations when using ftputil. On the other hand, I assume the EOFError would happen most of the time in the lines you have in the stack trace.

After all, if client code closes a file-like object, the client code assumes that it's done with the file and wouldn't expect that any resources could still be in an "open" state. So a timeout when getting a file-like object is probably more surprising than a timeout during "normal operation" without using file-like objects. That could be a reason to fix the problem here, even if not catching EOFErrors in lots of other parts of the ftputil code.

I just did a web search and found this interesting discussion on StackOverflow. Together with the information in CPython's ticket 148036 that EOFError is contained in ftplib.all_errors it appears that ftputil is already implicitly catching EOFErrors in its context managers ftplib_error_to_ftp_os_error and ftplib_error_to_ftp_io_error in ftputil.error.

As it happens, _available_child uses a direct host._session.pwd() call (where _session usually is an ftplib.FTP instance) instead of the context managers.

I guess I'll use the ftplib_error_to_ftp_os_error context manager in _available_child and consider the child session unusable if the context manager gives me an FTPOSError. TemporaryError and PermanentError inherit from FTPOSError, so this would also implicitly catch the ftplib.error_temp I'm catching now in _available_child.