Custom Query (115 matches)

Filters
 
Or
 
  
 
Columns

Show under each result:


Results (19 - 21 of 115)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Ticket Resolution Summary Owner Reporter
#108 duplicate Broken symlinks generate needless lstat calls schwa ftputiluser
Description

Hi,

I'm trying to walk() a directory on an FTP site that contains a large amount of broken symlinks to files in a different directory. The site in question is:

ftp://anonymous:anonymous@ftp.ebi.ac.uk/pub/databases/embl/new/

When walk('/pub/databases/embl/new/') runs, it sees a symlink to e.g. '../tsa/tsa_gfcx01_inv.dat.gz', and tries to follow it (even though followlinks is False). This starts an lstat of '/pub/databases/embl/tsa/' and the result of the directory listing is cached. If finds the file is missing and discards it.

However, when walk() hits the next broken symlink, it finds there's a cache miss for the file and does a new lstat of '/pub/databases/embl/tsa/' even though we just did it some milliseconds ago. This is then repeated for the hundreds of broken symlinks.

I have found two possible solutions:

1) Use the cache to detect that the directory has already been visited, and interpret a cache miss on the file as a broken link.

2) Change the semantics of walk(dir, followlinks=False) to not just discard directory symlinks, but also file symlinks. This is a small change in 'host.py':

--- host.py.orig	2017-01-27 16:20:04.633080980 +0100
+++ host.py	2017-01-27 16:20:22.027566063 +0100
@@ -905,7 +905,10 @@
             return
         dirs, nondirs = [], []
         for name in names:
-            if self.path.isdir(self.path.join(top, name)):
+            path = self.path.join(top, name)
+            if self.path.islink(path) and not followlinks:
+                continue
+            if self.path.isdir(path):
                 dirs.append(name)
             else:
                 nondirs.append(name)
#107 fixed untrapped exception schwa ftputiluser
Description
     for root,dirs,files in ftp_host.walk(ftp_host.curdir, topdown=True, onerror=None, followlinks=False):
  File "/usr/local/lib/python3.4/dist-packages/ftputil/host.py", line 917, in walk
    for item in self.walk(path, topdown, onerror, followlinks):
  File "/usr/local/lib/python3.4/dist-packages/ftputil/host.py", line 917, in walk
    for item in self.walk(path, topdown, onerror, followlinks):
  File "/usr/local/lib/python3.4/dist-packages/ftputil/host.py", line 917, in walk
    for item in self.walk(path, topdown, onerror, followlinks):
  File "/usr/local/lib/python3.4/dist-packages/ftputil/host.py", line 908, in walk
    if self.path.isdir(self.path.join(top, name)):
  File "/usr/local/lib/python3.4/dist-packages/ftputil/path.py", line 159, in isdir
    path, _exception_for_missing_path=False)
  File "/usr/local/lib/python3.4/dist-packages/ftputil/host.py", line 889, in stat
    return self._stat._stat(path, _exception_for_missing_path)
  File "/usr/local/lib/python3.4/dist-packages/ftputil/stat.py", line 747, in _stat
    _exception_for_missing_path)
  File "/usr/local/lib/python3.4/dist-packages/ftputil/stat.py", line 701, in __call_with_parser_retry
    result = method(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/ftputil/stat.py", line 685, in _real_stat
    format(original_path))
ftputil.error.PermanentError: recursive link structure detected for remote path './old-gnu/Manuals/automake-1.3/automake-1.4'
Debugging info: ftputil 3.3.1, Python 3.4.3 (linux)
Uncaught exception. Entering post mortem debugging

wrapped by
try:
            result = method(*args, **kwargs)
            # If a `listdir` call didn't find anything, we can't
            # say anything about the usefulness of the parser.
            if (method is not self._real_listdir) and result:
                self._allow_parser_switching = False
            return result
        except ftputil.error.ParserError:
            if self._allow_parser_switching:
                self._allow_parser_switching = False
                self._parser = MSParser()
                return method(*args, **kwargs)
            else:
                raisetry:
            result = method(*args, **kwargs)
            # If a `listdir` call didn't find anything, we can't
            # say anything about the usefulness of the parser.
            if (method is not self._real_listdir) and result:
                self._allow_parser_switching = False
            return result
        except ftputil.error.ParserError:
            if self._allow_parser_switching:
                self._allow_parser_switching = False
                self._parser = MSParser()
                return method(*args, **kwargs)
            else:
                raise

so no handler for Permanent error

#106 invalid anonymous user ? schwa ftputiluser
Description

The signature of FTPHost function requires a user and a password.

It would be good if the default was to user an anonymous user (like in basic ftp clients).

I also could not find an example about how to login with anonymous user in the documentation

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Note: See TracQuery for help on using queries.