Add 'rest' argument to file()

[ftputiluser]

The method file() of FTPHost lacks extra numeric argument "rest" (or "offset") which is given to _FTPFile method _open() in line 228:

host._file._open(effective_file, mode)

The method _open() uses this argument for transfercmd() method in lines 85-86:

self._conn = ftp_error._try_with_ioerror(
               self._session.transfercmd, command)

This will give functionality of byte offset supported by transfercmd (​http://docs.python.org/library/ftplib.html#ftplib.FTP.transfercmd). It does not fully cover ticket #14, however it will save lots of time of developers who simply need byte offset when reading files.

[ftputiluser]

This is critical for me too.

Regards, Łukasz (lm at zork.pl)

[schwa]

The FTP REST command was originally described in ​RFC 959, section 3.5. The RFC describes REST only in the context of the "block" and "compressed" modes, which aren't used by Python's ftplib. RFC 959 describes REST markers which don't have to consist of digits.

On the other hand, ​RFC 3659, ​chapter 5 describes an extension for the use of REST in "stream" mode. For binary transfer, the REST parameter is the 0-based byte count from where reading from or writing to a remote file should start.

For example, if a remote file contains the bytes

abcdefghijkl

(represented as ASCII letters here), a REST 3, followed by a RETR command, will read the string "defghijkl". A REST 3, followed by a STOR command, will overwrite the bytes starting at "d".

If the data written to the server has fewer bytes than the remaining bytes on the remote side, the "extra" remote bytes (here "jkl") will get lost. For example, writing "123" to the above remote file "abcdefghijkl" will result in the new remote content "abc123". I don't know if the shortening of the remote file is officially specified anywhere, but it's the behavior I see with my FTP server (Pure FTPd).

[schwa]

As suggested in the ticket description, I'm going to add a rest argument to FTPHost.open (formerly file). For now, this argument will only be supported for binary transfer modes since applying the concept to text files may give confusing results.

Since ftputil is about simulating Python's API for local file systems, I had considered implementing the "offset" functionality with a seek method on FTPFile objects (the type of object returned by FTPHost.open). However, this is far more complicated to implement:

• seek would have to reopen the file with a REST argument and an identical path.
• Having a seek without tell (local files have a tell method) would be awkward.
• Implementing tell would require keeping track of a counter when reading from or writing to a remote file.

All in all, implementing the "rest" functionality via a file seek method may be an interesting experiment, but offers little gain compared to the implementation effort.

[schwa]

I implemented passing rest from FTPHost.open to FTP.transfercmd in [3cb69eca3e41] and [66ef713f4fe8], [525a118b3085]. The documentation update is in [60662e7b943d].

The documentation contains the following caveat:

If you pass rest values which point after the file, the behavior is undefined and may even differ from one FTP server to another. Therefore, use the rest argument only for error recovery in case of interrupted transfers. You need to keep track of the transferred data so that you can provide a valid rest argument for a resumed transfer.