Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win 32 Type "help", "copyright", "credits" or "license" for more information.

import ftplib ftp = ftplib.FTP(..) ftp.cwd(..)

'250 CWD command successful'

ftp.retrlines('LIST')

drw-rw-rw- 1 cidsauto cidsauto 1 Sep 24 17:51 Agents drw-rw-rw- 1 cidsauto cidsauto 1 Sep 24 17:51 CentralServer? drw-rw-rw- 1 cidsauto cidsauto 1 Sep 24 17:51 Packages drw-rw-rw- 1 cidsauto cidsauto 1 Sep 24 17:51 Working -rw-rw-rw- 1 cidsauto cidsauto 3099 Aug 1 19:39 LUXStyleSheet.xsl '226 Transfer complete.'

ftp.retrlines('LIST -a')

'226 Transfer complete.'

Thanks for the report!

What you describe is a general limitation of the auto-probing. When I implemented it, I wasn't able to come up with a reliable way so that it would work under all circumstances, for all servers (see last comment in ticket #23). On the other hand, I still added support for -a because not supporting it had other implications (see ticket #63). I assumed that most likely a server would either support the -a option, ignore it or cause an error condition if used.

Which FTP server and version do you use? Is there any documentation on how the server interprets the LIST -a option or even if there's a supposed way to handle any LIST options?

I can think of two approaches now:

• Remove auto-probing and let the user switch on using -a explicitly if he/she wants to get hidden files or directories.

• Auto-probe by default, but give the user a way not to use it, for example with a keyword argument in the FTPHost constructor.

I tend to use the second approach because then ftputil will behave "as expected" for most users. On the other hand, if I knew that there were many servers behaving as yours, I would prefer the first approach. But maybe there are other approaches?

(I edited the description field to use code markup, so __init__ doesn't appear as underlined init. Unfortunately, I can't edit your comment as easily.)

Replying to schwa:

I can think of two approaches now:

• Remove auto-probing and let the user switch on using -a explicitly if he/she wants to get hidden files or directories.

• Auto-probe by default, but give the user a way not to use it, for example with a keyword argument in the FTPHost constructor.

Given the unrealiability of auto-probing, I'm aware that from a software design point of view the first approach is clearly preferable. But the second approach might still be better from the point of view of overall usefulness for all users. In my experience, many users don't look at the documentation if something doesn't behave as it "obviously" should and maybe won't use the library at all.

This is a Perforce FTP server. It is managed by our IT so unfortunately I do not have intimate knowledge of the actual host.

I am not sure about the prevalence of this problem. I will say that if a new user falls victim to this, it is very likely that they won't use the library either.

Replying to ftputiluser:

This is a Perforce FTP server. It is managed by our IT so unfortunately I do not have intimate knowledge of the actual host.

Thanks nonetheless. :-)

I am not sure about the prevalence of this problem. I will say that if a new user falls victim to this, it is very likely that they won't use the library either.

That's true. On the other hand, I assume it's very unlikely that LIST -a leads to empty directory listings. So my assumption is that of the two cases

• a user doesn't see hidden files and abandons the library vs.
• a user gets an empty directory listing because of use of LIST -a and abandons the library,

the first one is more likely.

Besides, after I wrote my reply I thought more about the problem. Maybe it's a reasonable compromise to add a check in the auto-probing, so that if there are fewer entries in the directory listing if LIST -a is used, I'll drop the -a option.

In the "extreme" case that the directory is empty even if using LIST alone, I could attempt to write a "hidden" file like ._ftputil_hidden_ and if the writing succeeds but the file still doesn't show up, don't use the -a option. If the user doesn't have write permission, I would rather stick with using the -a option because of what I said above. (Note that this default then only applies if the directory seemed empty even without -a.)

In addition to that I'm still not against a way to override the auto-probing result. :-)

What do you think?

Well, I think those are interesting ideas but I would not try to fix something like this by adding more code. I was able to figure out this problem because the code was well written and straightforward. Adding more layers on top layers to hide a basic limitation is not the path I would take.

Replying to ftputiluser:

Well, I think those are interesting ideas but I would not try to fix something like this by adding more code.

Thanks for your feedback.

I was able to figure out this problem because the code was well written and straightforward.

Thanks again! :-)

Adding more layers on top layers to hide a basic limitation is not the path I would take.

I think for a library like ftputil to be useful you have to jump through hoops (at least a few). Imagine a web server that serves only clients strictly adhering to the applicable RFCs. I don't fulfill each and every wish though (see #26, #58, #60). ;-)

Regarding the LIST -a heuristics, it came to my mind that in addition to being more code, the LIST calls upon connection burden everyone making a connection with ftputil, even if LIST -a would work without problems or not having any effect. Clearly, for only a few entries in the login directory it won't matter, but for larger directories it might.

So I guess I'll remove the test I now have, activate use of -a by default, offer a boolean attribute to deactivate the use of -a and document this flag.

Replying to schwa:

So I guess I'll remove the test I now have, activate use of -a by default, offer a boolean attribute to deactivate the use of -a and document this flag.

I did this in changesets 62e7f2ec5e66e58c0fa83404db459eab01c7eb37 for the code and 69aa319877e35fffbe8b9d4f3d5c8a9e7e29294f for the documentation.