~sschwarzer/ftputil#110: 
Directory listings are empty if a server doesn't support the`-a` option

This issue comes up from time to time (for example ​http://lists.sschwarzer.net/pipermail/ftputil/2017q2/thread.html).

Since ftputil 2.8 the library by default tries to list hidden files (i. e. files starting with a dot) by using the -a option. (Everything in this ticket also applies to hidden directories.)

There are three cases:

  • The server supports the -a option and is configured to list hidden files. This is what we ideally want.
  • The server supports the -a option but is configured to not list hidden files. In this case, the directory listing misses hidden files, which is what the server intended.
  • The server does not support the -a option. In this case, the server understands -a as a file name.

This third case usually means that the returned listing won't have any entries. After all, ftputil told the server (from its perspective) to list the file -a and the file isn't there. This behavior is most obvious in FTPHost.listdir, but probably effects other functionality as well.

Status
RESOLVED FIXED
Submitter
schwa (unverified)
Assigned to
No-one
Submitted
6 years ago
Updated
6 years ago
Labels
bug library

schwa (unverified) 6 years ago · edit

You can switch off the automatic addition of the -a option by using

ftp_host = ftputil.FTPHost(server, user, password)
ftp_host.use_list_a_option = False

Although this kind of solves the problem, it's not what a user thinks of in the first place. With FTPHost.listdir, the returned list will be empty and from there it's not intuitive where to look for a solution. A user can read the documentation where the problem and workaround are described, but this probably won't happen smoothly because the user doesn't expect the problem to be related to hidden files.

schwa (unverified) 6 years ago · edit

The most obvious remedy to this problem is not to use -a by default. This will have the consequence that even if the server supports the -a option and is configured to list hidden files when using the option, these files won't be listed. But in this case it's relatively clear that the problem is related to hidden files and I could describe in the documentation how to use the -a option.

A problem with this remedy, and the reason why I didn't implement it, is that it breaks backward compatibility. When some code used to use ftputil on a server supporting the -a option and allows to list hidden files, these files will be unexpectedly absent if ftputil changes the default to not include the -a option for listings.

schwa (unverified) 6 years ago · edit

Mattias Bokesand ​came up with another suggestion to use a similar approach as for the "user-supported" timezone detection. By using some user-visible method, ftputil could try to write a "hidden" file to the current directory, check if it gets listed with -a and if yes, use the -a option from then on.

Obviously, this requires that the user has write permissions for the current directory.

If it's added, such a detection should not happen by default because I think it's bad practice to write something on the host without the user knowing about it. Moreover, some servers are configured to offer only write-only access to some directories and if ftputil is logged into such a directory, it won't be able to delete the file.

schwa (unverified) 6 years ago · edit

Added deprecation warning in d816c740a1064e6e47265839134c0f68f975bac6.

schwa (unverified) 5 years ago · edit

Replying to schwa:

The most obvious remedy to this problem is not to use -a by default. This will have the consequence that even if the server supports the -a option and is configured to list hidden files when using the option, these files won't be listed. But in this case it's relatively clear that the problem is related to hidden files and I could describe in the documentation how to use the -a option.

A problem with this remedy, and the reason why I didn't implement it, is that it breaks backward compatibility. When some code used to use ftputil on a server supporting the -a option and allows to list hidden files, these files will be unexpectedly absent if ftputil changes the default to not include the -a option for listings.

ftputil 4.x will set use_list_a_option to False by default.

schwa (unverified) 5 years ago · edit

This is changed in master (changeset [846b2f5740a673e84af71bb8eef28253c49079d7](https://git.sr.ht/~sschwarzer/ftputil/commit/846b2f5740a673e84af71bb8eef28253c49079d7 "Set use_list_a_option to False by default See ticket #110 for the ...")).

~sschwarzer referenced this from #157 5 months ago

Register here or Log in to comment, or comment via email.