source: doc/ftputil.txt @ 1508:452a03a5b1ff

Last change on this file since 1508:452a03a5b1ff was 1508:452a03a5b1ff, checked in by Stefan Schwarzer <sschwarzer@…>, 7 years ago
Updated documentation for `FTPHost.walk` (`followlinks` parameter).
File size: 52.5 KB
Line 
1``ftputil`` -- a high-level FTP client library
2==============================================
3
4:Version:   3.0
5:Date:      2013-11-17
6:Summary:   high-level FTP client library for Python
7:Keywords:  FTP, ``ftplib`` substitute, virtual filesystem, pure Python
8:Author:    Stefan Schwarzer <sschwarzer@sschwarzer.net>
9
10.. contents::
11
12
13Introduction
14------------
15
16The ``ftputil`` module is a high-level interface to the ftplib_
17module. The `FTPHost objects`_ generated from it allow many operations
18similar to those of os_, `os.path`_ and `shutil`_.
19
20.. _ftplib: http://www.python.org/doc/current/lib/module-ftplib.html
21.. _os: http://www.python.org/doc/current/lib/module-os.html
22.. _`os.stat`: http://www.python.org/doc/current/lib/module-os.html#os.stat
23.. _`os.path`: http://www.python.org/doc/current/lib/module-os.path.html
24.. _`shutil`: http://www.python.org/doc/current/lib/module-shutil.html
25
26Examples::
27
28    import ftputil
29
30    # Download some files from the login directory.
31    with ftputil.FTPHost("ftp.domain.com", "user", "password") as host:
32        names = host.listdir(host.curdir)
33        for name in names:
34            if host.path.isfile(name):
35                host.download(name, name)  # remote, local
36        # Make a new directory and copy a remote file into it.
37        host.mkdir("newdir")
38        with host.open("index.html", "rb") as source:
39            with host.open("newdir/index.html", "wb") as target:
40                host.copyfileobj(source, target)  # similar to shutil.copyfileobj
41
42Also, there are `FTPHost.lstat`_ and `FTPHost.stat`_ to request size and
43modification time of a file. The latter can also follow links, similar
44to `os.stat`_. `FTPHost.walk`_ and `FTPHost.path.walk`_ work, too.
45
46
47``ftputil`` features
48--------------------
49
50* Method names are familiar from Python's ``os``, ``os.path`` and
51  ``shutil`` modules
52
53* Remote file system navigation (``getcwd``, ``chdir``)
54
55* Upload and download files (``upload``, ``upload_if_newer``,
56  ``download``, ``download_if_newer``)
57
58* Time zone synchronization between client and server (needed
59  for ``upload_if_newer`` and ``download_if_newer``)
60
61* Create and remove directories (``mkdir``, ``makedirs``, ``rmdir``,
62  ``rmtree``) and remove files (``remove``)
63
64* Get information about directories, files and links (``listdir``,
65  ``stat``, ``lstat``, ``exists``, ``isdir``, ``isfile``, ``islink``,
66  ``abspath``, ``split``, ``join``, ``dirname``, ``basename`` etc.)
67
68* Iterate over remote file systems (``walk``)
69
70* Local caching of results from ``lstat`` and ``stat`` calls to reduce
71  network access (also applies to ``exists``, ``getmtime`` etc.).
72
73* Read files from and write files to remote hosts via
74  file-like objects (``FTPHost.open``; the generated file-like objects
75  have many common methods like ``read``, ``readline``, ``readlines``,
76  ``write``, ``writelines``, ``close`` and can do automatic line
77  ending conversions on the fly, i. e. text/binary mode).
78
79
80Exception hierarchy
81-------------------
82
83The exceptions are in the namespace of the ``ftputil.error`` module, e. g.
84``ftputil.error.TemporaryError``.
85
86The exception classes are organized as follows::
87
88    FTPError
89        FTPOSError(FTPError, OSError)
90            PermanentError(FTPOSError)
91                CommandNotImplementedError(PermanentError)
92            TemporaryError(FTPOSError)
93        FTPIOError(FTPError)
94        InternalError(FTPError)
95            InaccessibleLoginDirError(InternalError)
96            ParserError(InternalError)
97            RootDirError(InternalError)
98            TimeShiftError(InternalError)
99
100and are described here:
101
102- ``FTPError``
103
104  is the root of the exception hierarchy of the module.
105
106- ``FTPOSError``
107
108  is derived from ``OSError``. This is for similarity between the
109  os module and ``FTPHost`` objects. Compare
110
111  ::
112
113    try:
114        os.chdir("nonexisting_directory")
115    except OSError:
116        ...
117
118  with
119
120  ::
121
122    host = ftputil.FTPHost("host", "user", "password")
123    try:
124        host.chdir("nonexisting_directory")
125    except OSError:
126        ...
127
128  Imagine a function
129
130  ::
131
132    def func(path, file):
133        ...
134
135  which works on the local file system and catches ``OSErrors``. If you
136  change the parameter list to
137
138  ::
139
140    def func(path, file, os=os):
141        ...
142
143  where ``os`` denotes the ``os`` module, you can call the function also as
144
145  ::
146
147    host = ftputil.FTPHost("host", "user", "password")
148    func(path, file, os=host)
149
150  to use the same code for both a local and remote file system.
151  Another similarity between ``OSError`` and ``FTPOSError`` is that
152  the latter holds the FTP server return code in the ``errno``
153  attribute of the exception object and the error text in
154  ``strerror``.
155
156- ``PermanentError``
157
158  is raised for 5xx return codes from the FTP server. This
159  corresponds to ``ftplib.error_perm`` (though ``PermanentError`` and
160  ``ftplib.error_perm`` are *not* identical).
161
162- ``CommandNotImplementedError``
163
164  indicates that an underlying command the code tries to use is not
165  implemented. For an example, see the description of the
166  `FTPHost.chmod`_ method.
167
168- ``TemporaryError``
169
170  is raised for FTP return codes from the 4xx category. This
171  corresponds to ``ftplib.error_temp`` (though ``TemporaryError`` and
172  ``ftplib.error_temp`` are *not* identical).
173
174- ``FTPIOError``
175
176  denotes an I/O error on the remote host. This appears
177  mainly with file-like objects that are retrieved by calling
178  ``FTPHost.open``. Compare
179
180  ::
181
182    >>> try:
183    ...     f = open("not_there")
184    ... except IOError as obj:
185    ...     print obj.errno
186    ...     print obj.strerror
187    ...
188    2
189    No such file or directory
190
191  with
192
193  ::
194
195    >>> host = ftputil.FTPHost("host", "user", "password")
196    >>> try:
197    ...     f = host.open("not_there")
198    ... except IOError as obj:
199    ...     print obj.errno
200    ...     print obj.strerror
201    ...
202    550
203    550 not_there: No such file or directory.
204
205  As you can see, both code snippets are similar. However, the error
206  codes aren't the same.
207
208- ``InternalError``
209
210  subsumes exception classes for signaling errors due to limitations
211  of the FTP protocol or the concrete implementation of ``ftputil``.
212
213- ``InaccessibleLoginDirError``
214
215  This exception is raised if the directory in which "you" are placed
216  upon login is not accessible, i. e. a ``chdir`` call with the
217  directory as argument would fail.
218
219- ``ParserError``
220
221  is used for errors during the parsing of directory
222  listings from the server. This exception is used by the ``FTPHost``
223  methods ``stat``, ``lstat``, and ``listdir``.
224
225- ``RootDirError``
226
227  Because of the implementation of the ``lstat`` method it is not
228  possible to do a ``stat`` call on the root directory ``/``.
229  If you know *any* way to do it, please let me know. :-)
230
231  This problem does *not* affect stat calls on items *in* the root
232  directory.
233
234- ``TimeShiftError``
235
236  is used to denote errors which relate to setting the `time shift`_,
237  *for example* trying to set a value which is no multiple of a full
238  hour.
239
240
241Directory and file names
242------------------------
243
244Methods that take names of directories and files can take either byte
245strings (``str`` on Python 2, ``bytes`` on Python 3) or unicode
246strings (``unicode`` on Python 2, ``str`` on Python 3).
247
248Byte strings will be sent to the FTP server as-is. Unicode strings
249will be encoded with the encoding returned from
250``locale.getpreferredencoding``. This is the same semantics as for
251locally used names in Python 3.
252
253Methods that take and return a directory or file name will return the
254same string type as they're given. For example, if the argument to
255``FTPHost.path.abspath`` is a byte string, you'll get a byte string
256back. This behavior is the same as for the local file system API in
257Python 2 and 3.
258
259
260``FTPHost`` objects
261-------------------
262
263.. _`FTPHost construction`:
264
265Construction
266~~~~~~~~~~~~
267
268Basics
269``````
270
271``FTPHost`` instances can be generated with the following call::
272
273    host = ftputil.FTPHost(host, user, password, account,
274                           session_factory=ftplib.FTP)
275
276The first four parameters are strings with the same meaning as for the
277FTP class in the ``ftplib`` module.
278
279Session factories
280`````````````````
281
282The keyword argument ``session_factory`` may be used to generate FTP
283connections with other factories than the default ``ftplib.FTP``. For
284example, the standard library of Python 2.7 contains a class
285``ftplib.FTP_TLS`` which extends ``ftplib.FTP`` to use an encrypted
286connection.
287
288In fact, all positional and keyword arguments other than
289``session_factory`` are passed to the factory to generate a new
290background session. This happens for every remote file that is opened;
291see below.
292
293This functionality of the constructor also allows to wrap
294``ftplib.FTP`` objects to do something that wouldn't be possible with
295the ``ftplib.FTP`` constructor alone.
296
297As an example, assume you want to connect to another than the default
298port, but ``ftplib.FTP`` only offers this by means of its ``connect``
299method, not via its constructor. The solution is to use a wrapper
300class::
301
302    import ftplib
303    import ftputil
304
305    EXAMPLE_PORT = 50001
306
307    class MySession(ftplib.FTP):
308
309        def __init__(self, host, userid, password, port):
310            """Act like ftplib.FTP's constructor but connect to another port."""
311            ftplib.FTP.__init__(self)
312            self.connect(host, port)
313            self.login(userid, password)
314
315    # Try _not_ to use an instance `MySession()` as factory, -
316    # use the class itself.
317    host = ftputil.FTPHost(host, userid, password,
318                           port=EXAMPLE_PORT, session_factory=MySession)
319    # Use `host` as usual.
320
321On login, the format of the directory listings (needed for stat'ing
322files and directories) should be determined automatically. If not,
323please `file a bug report`_.
324
325.. _`file a bug report`: http://ftputil.sschwarzer.net/issuetrackernotes
326
327Support for the ``with`` statement
328``````````````````````````````````
329
330`FTPHost` objects can be used in the ``with`` statement::
331
332    import ftputil
333
334    with ftputil.FTPHost(host, user, password) as host:
335        print host.listdir(host.curdir)
336
337After the ``with`` block, the ``FTPHost`` instance and the
338associated FTP sessions will be closed automatically.
339
340If something goes wrong during the ``FTPHost`` construction or in the
341body of the ``with`` statement, the instance is closed as well.
342Exceptions will be propagated (as with ``try ... finally``).
343
344Hidden files and directories
345~~~~~~~~~~~~~~~~~~~~~~~~~~~~
346
347Whether ftputil sees "hidden" files and directories (usually files or
348directories whose names start with a dot) depends on the FTP server
349configuration. By default, ftputil uses the ``-a`` option in the FTP
350``LIST`` command to find hidden files. However, the server may ignore
351this.
352
353If using the ``-a`` option leads to problems, for example if an
354FTP server causes an exception, you may switch off the use of the
355option::
356
357    host = ftputil.FTPHost(host, user, password, account,
358                           session_factory=ftplib.FTP)
359    host.use_list_a_option = False
360
361``FTPHost`` attributes and methods
362~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
363
364Attributes
365``````````
366
367- ``curdir``, ``pardir``, ``sep``
368
369  are strings which denote the current and the parent directory on the
370  remote server. ``sep`` holds the path separator. Though `RFC 959`_
371  (File Transfer Protocol) notes that these values may depend on the
372  FTP server implementation, the Unix variants seem to work well in
373  practice, even for non-Unix servers.
374
375  Nevertheless, it's recommended that you don't hardcode these values
376  for remote paths, but use `FTPHost.path`_ as you would use
377  ``os.path`` to write platform-independent Python code for local
378  filesystems. Keep in mind that most, *but not all*, arguments of
379  ``FTPHost`` methods refer to remote directories or files. For
380  example, in `FTPHost.upload`_, the first argument is a local
381  path and the second a remote path. Both of these should use their
382  respective path separators.
383
384.. _`FTPHost.upload`: `Uploading and downloading files`_
385
386Remote file system navigation
387`````````````````````````````
388
389- ``getcwd()``
390
391  returns the absolute current directory on the remote host. This
392  method works like ``os.getcwd``.
393
394- ``chdir(directory)``
395
396  sets the current directory on the FTP server. This resembles
397  ``os.chdir``, as you may have expected.
398
399.. _`callback function`:
400
401Uploading and downloading files
402```````````````````````````````
403
404- ``upload(source, target, callback=None)``
405
406  copies a local source file (given by a filename, i. e. a string)
407  to the remote host under the name target. Both ``source`` and
408  ``target`` may be absolute paths or relative to their corresponding
409  current directory (on the local or the remote host, respectively).
410
411  The file content is always transferred in binary mode.
412
413  The callback, if given, will be invoked for each transferred chunk
414  of data::
415
416    callback(chunk)
417
418  where ``chunk`` is a bytestring. An example usage of a callback
419  method is to display a progress indicator.
420
421- ``download(source, target, callback=None)``
422
423  performs a download from the remote source file to a local target
424  file. Both ``source`` and ``target`` are strings. See the
425  description of ``upload`` for more details.
426
427.. _`upload_if_newer`:
428
429- ``upload_if_newer(source, target, callback=None)``
430
431  is similar to the ``upload`` method. The only difference is that the
432  upload is only invoked if the time of the last modification for the
433  source file is more recent than that of the target file or the
434  target doesn't exist at all. The check for the last modification
435  time considers the precision of the timestamps and transfers a file
436  "if in doubt". Consequently the code
437
438  ::
439
440    host.upload_if_newer("source_file", "target_file")
441    time.sleep(10)
442    host.upload_if_newer("source_file", "target_file")
443
444  might upload the file again if the timestamp of the target file is
445  precise up to a minute, which is typically the case because the
446  remote datetime is determined by parsing a directory listing from
447  the server. To avoid unnecessary transfers, wait at least a minute
448  between calls of ``upload_if_newer`` for the same file. If it still
449  seems that a file is uploaded unnecessarily (or not when it should),
450  read the subsection on `time shift`_ settings.
451
452  If an upload actually happened, the return value of
453  ``upload_if_newer`` is a ``True``, else ``False``.
454
455  Note that the method only checks the existence and/or the
456  modification time of the source and target file; it doesn't
457  compare any other file properties, say, the file size.
458
459  This also means that if a transfer is interrupted, the remote file
460  will have a newer modification time than the local file, and thus
461  the transfer won't be repeated if ``upload_if_newer`` is used a
462  second time. There are at least two possibilities after a failed
463  upload:
464
465  - use ``upload`` instead of ``upload_if_newer``, or
466
467  - remove the incomplete target file with ``FTPHost.remove``, then
468    use ``upload`` or ``upload_if_newer`` to transfer it again.
469
470.. _`download_if_newer`:
471
472- ``download_if_newer(source, target, callback=None)``
473
474  corresponds to ``upload_if_newer`` but performs a download from the
475  server to the local host. Read the descriptions of download and
476  ``upload_if_newer`` for more information. If a download actually
477  happened, the return value is ``True``, else ``False``.
478
479.. _`time shift`:
480.. _`time zone correction`:
481
482Time zone correction
483````````````````````
484
485If the client where ``ftputil`` runs and the server have a different
486understanding of their local times, this has to be taken into account
487for ``upload_if_newer`` and ``download_if_newer`` to work correctly.
488
489Note that even if the client and the server are in the same time zone
490(or even on the same computer), the time shift value (see below) may
491be different from zero. For example, my computer is set to use local
492time whereas the server running on the very same host insists on using
493UTC time.
494
495.. _`set_time_shift`:
496
497- ``set_time_shift(time_shift)``
498
499  sets the so-called time shift value, measured in seconds. The time
500  shift is the difference between the local time of the server and the
501  local time of the client at a given moment, i. e. by definition
502
503  ::
504
505    time_shift = server_time - client_time
506
507  Setting this value is important for `upload_if_newer`_ and
508  `download_if_newer`_ to work correctly even if the time zone of the
509  FTP server differs from that of the client. Note that the time shift
510  value *can be negative*.
511
512  If the time shift value is invalid, e. g. no multiple of a full hour
513  or its absolute value larger than 24 hours, a ``TimeShiftError`` is
514  raised.
515
516  See also `synchronize_times`_ for a way to set the time shift with a
517  simple method call.
518
519- ``time_shift()``
520
521  returns the currently-set time shift value. See ``set_time_shift``
522  above for its definition.
523
524.. _`synchronize_times`:
525
526- ``synchronize_times()``
527
528  synchronizes the local times of the server and the client, so that
529  `upload_if_newer`_ and `download_if_newer`_ work as expected, even
530  if the client and the server use different time zones. For this
531  to work, *all* of the following conditions must be true:
532
533  - The connection between server and client is established.
534
535  - The client has write access to the directory that is current when
536    ``synchronize_times`` is called.
537
538  If you can't fulfill these conditions, you can nevertheless set the
539  time shift value explicitly with `set_time_shift`_. Trying to call
540  ``synchronize_times`` if the above conditions aren't met results in
541  a ``TimeShiftError`` exception.
542
543Creating and removing directories
544`````````````````````````````````
545
546- ``mkdir(path, [mode])``
547
548  makes the given directory on the remote host. This does *not*
549  construct "intermediate" directories that don't already exist. The
550  ``mode`` parameter is ignored; this is for compatibility with
551  ``os.mkdir`` if an ``FTPHost`` object is passed into a function
552  instead of the ``os`` module. See the explanation in the subsection
553  `Exception hierarchy`_.
554
555- ``makedirs(path, [mode])``
556
557  works similar to ``mkdir`` (see above), but also makes intermediate
558  directories like ``os.makedirs``. The ``mode`` parameter is only
559  there for compatibility with ``os.makedirs`` and is ignored.
560
561- ``rmdir(path)``
562
563  removes the given remote directory. If it's not empty, raise
564  a ``PermanentError``.
565
566- ``rmtree(path, ignore_errors=False, onerror=None)``
567
568  removes the given remote, possibly non-empty, directory tree.
569  The interface of this method is rather complex, in favor of
570  compatibility with ``shutil.rmtree``.
571
572  If ``ignore_errors`` is set to a true value, errors are ignored.
573  If ``ignore_errors`` is a false value *and* ``onerror`` isn't
574  set, all exceptions occurring during the tree iteration and
575  processing are raised. These exceptions are all of type
576  ``PermanentError``.
577
578  To distinguish between different kinds of errors, pass in a callable
579  for ``onerror``. This callable must accept three arguments:
580  ``func``, ``path`` and ``exc_info``. ``func`` is a bound method
581  object, *for example* ``your_host_object.listdir``. ``path`` is the
582  path that was the recent argument of the respective method
583  (``listdir``, ``remove``, ``rmdir``). ``exc_info`` is the exception
584  info as it is gotten from ``sys.exc_info``.
585
586  The code of ``rmtree`` is taken from Python's ``shutil`` module
587  and adapted for ``ftputil``.
588
589Removing files and links
590````````````````````````
591
592- ``remove(path)``
593
594  removes a file or link on the remote host, similar to ``os.remove``.
595
596- ``unlink(path)``
597
598  is an alias for ``remove``.
599
600Retrieving information about directories, files and links
601`````````````````````````````````````````````````````````
602
603- ``listdir(path)``
604
605  returns a list containing the names of the files and directories
606  in the given path, similar to `os.listdir`_. The special names
607  ``.`` and ``..`` are not in the list.
608
609The methods ``lstat`` and ``stat`` (and some others) rely on the
610directory listing format used by the FTP server. When connecting to a
611host, ``FTPHost``'s constructor tries to guess the right format, which
612succeeds in most cases. However, if you get strange results or
613``ParserError`` exceptions by a mere ``lstat`` call, please `file a
614bug report`_.
615
616If ``lstat`` or ``stat`` give wrong modification dates or times, look
617at the methods that deal with time zone differences (`time zone
618correction`_).
619
620.. _`FTPHost.lstat`:
621
622- ``lstat(path)``
623
624  returns an object similar to that from `os.lstat`_. This is a
625  "tuple" with additional attributes; see the documentation of the
626  ``os`` module for details.
627
628  The result is derived by parsing the output of a ``LIST`` command on
629  the server. Therefore, the result from ``FTPHost.lstat`` can not
630  contain more information than the received text. In particular:
631
632  - User and group ids can only be determined as strings, not as
633    numbers, and that only if the server supplies them. This is
634    usually the case with Unix servers but maybe not for other FTP
635    server programs.
636
637  - Values for the time of the last modification may be rough,
638    depending on the information from the server. For timestamps
639    older than a year, this usually means that the precision of the
640    modification timestamp value is not better than days. For newer
641    files, the information may be accurate to a minute.
642
643  - Links can only be recognized on servers that provide this
644    information in the ``LIST`` output.
645
646  - Stat attributes that can't be determined at all are set to
647        ``None``. For example, a line of a directory listing may not
648        contain the date/time of a directory's last modification.
649
650  - There's a special problem with stat'ing the root directory.
651    (Stat'ing things *in* the root directory is fine though.) In
652    this case, a ``RootDirError`` is raised. This has to do with the
653    algorithm used by ``(l)stat``, and I know of no approach which
654    mends this problem.
655
656  Currently, ``ftputil`` recognizes the common Unix-style and
657  Microsoft/DOS-style directory formats. If you need to parse output
658  from another server type, please write to the `ftputil mailing
659  list`_. You may consider `writing your own parser`_.
660
661.. _`os.listdir`: http://www.python.org/doc/current/lib/module-os.html#os.listdir
662.. _`os.lstat`: http://www.python.org/doc/current/lib/module-os.html#os.lstat
663.. _`ftputil mailing list`: http://ftputil.sschwarzer.net/mailinglist
664.. _`writing your own parser`: `Writing directory parsers`_
665
666.. _`FTPHost.stat`:
667
668- ``stat(path)``
669
670  returns ``stat`` information also for files which are pointed to by a
671  link. This method follows multiple links until a regular file or
672  directory is found. If an infinite link chain is encountered or the
673  target of the last link in the chain doesn't exist, a
674  ``PermanentError`` is raised.
675
676  The limitations of the ``lstat`` method also apply to ``stat``.
677
678.. _`FTPHost.path`:
679
680``FTPHost`` objects contain an attribute named ``path``, similar to
681`os.path`_. The following methods can be applied to the remote host
682with the same semantics as for ``os.path``:
683
684::
685
686    abspath(path)
687    basename(path)
688    commonprefix(path_list)
689    dirname(path)
690    exists(path)
691    getmtime(path)
692    getsize(path)
693    isabs(path)
694    isdir(path)
695    isfile(path)
696    islink(path)
697    join(path1, path2, ...)
698    normcase(path)
699    normpath(path)
700    split(path)
701    splitdrive(path)
702    splitext(path)
703    walk(path, func, arg)
704
705Like Python's counterparts under `os.path`_, ``ftputil``'s ``is...``
706methods return ``False`` if they can't find the path given by their
707argument.
708
709Local caching of file system information
710````````````````````````````````````````
711
712Many of the above methods need access to the remote file system to
713obtain data on directories and files. To get the most recent data,
714*each* call to ``lstat``, ``stat``, ``exists``, ``getmtime`` etc.
715would require to fetch a directory listing from the server, which can
716make the program *very* slow. This effect is more pronounced for
717operations which mostly scan the file system rather than transferring
718file data.
719
720For this reason, ``ftputil`` by default saves the results from
721directory listings locally and reuses those results. This reduces
722network accesses and so speeds up the software a lot. However, since
723data is more rarely fetched from the server, the risk of obsolete data
724also increases. This will be discussed below.
725
726Caching can be controlled -- if necessary at all -- via the
727``stat_cache`` object in an ``FTPHost``'s namespace. For example,
728after calling
729
730::
731
732    host = ftputil.FTPHost(host, user, password)
733
734the cache can be accessed as ``host.stat_cache``.
735
736While ``ftputil`` usually manages the cache quite well, there are two
737possible reasons for modifying cache parameters.
738
739The first is when the number of possible entries is too low. You may
740notice that when you are processing very large directories and the
741program becomes much slower than before. It's common for code to read
742a directory with ``listdir`` and then process the found directories
743and files. This can also happen implicitly by a call to
744``FTPHost.walk``. Since version 2.6 ``ftputil`` automatically
745increases the cache size if directories with more entries than the
746current maximum cache size are to be scanned. Most of the time, this
747works fine.
748
749However, if you need access to stat data for several directories at
750the same time, you may need to increase the cache explicitly. This is
751done by the ``resize`` method::
752
753    host.stat_cache.resize(20000)
754
755where the argument is the maximum number of ``lstat`` results to store
756(the default is 5000, in versions before 2.6 it was 1000). Note that
757each path on the server, e. g. "/home/schwa/some_dir", corresponds to
758a single cache entry. Methods like ``exists`` or ``getmtime`` all
759derive their results from a previously fetched ``lstat`` result.
760
761The value 5000 above means that the cache will hold *at most* 5000
762entries (unless increased automatically by an explicit or implicit
763``listdir`` call, see above). If more are about to be stored, the
764entries which haven't been used for the longest time will be deleted
765to make place for newer entries.
766
767The second possible reason to change the cache parameters is to avoid
768stale cache data. Caching is so effective because it reduces network
769accesses. This can also be a disadvantage if the file system data on
770the remote server changes after a stat result has been retrieved; the
771client, when looking at the cached stat data, will use obsolete
772information.
773
774There are two potential ways to get such out-of-date stat data. The
775first happens when an ``FTPHost`` instance modifies a file path for
776which it has a cache entry, e. g. by calling ``remove`` or ``rmdir``.
777Such changes are handled transparently; the path will be deleted from
778the cache. A different matter are changes unknown to the ``FTPHost``
779object which inspects its cache. Obviously, for example, these are
780changes by programs running on the remote host. On the other hand,
781cache inconsistencies can also occur if two ``FTPHost`` objects change
782a file system simultaneously::
783
784    with (
785      ftputil.FTPHost(server, user1, password1) as host1,
786      ftputil.FTPHost(server, user1, password1) as host2
787    ):
788        stat_result1 = host1.stat("some_file")
789        stat_result2 = host2.stat("some_file")
790        host2.remove("some_file")
791        # `host1` will still see the obsolete cache entry!
792        print host1.stat("some_file")
793        # Will raise an exception since an `FTPHost` object
794        # knows of its own changes.
795        print host2.stat("some_file")
796
797At first sight, it may appear to be a good idea to have a shared cache
798among several ``FTPHost`` objects. After some thinking, this turns out
799to be very error-prone. For example, it won't help with different
800processes using ``ftputil``. So, if you have to deal with concurrent
801write/read accesses to a server, you have to handle them explicitly.
802
803The most useful tool for this is the ``invalidate`` method. In the
804example above, it could be used like this::
805
806    with (
807      ftputil.FTPHost(server, user1, password1) as host1,
808      ftputil.FTPHost(server, user1, password1) as host2
809    ):
810        stat_result1 = host1.stat("some_file")
811        stat_result2 = host2.stat("some_file")
812        host2.remove("some_file")
813        # Invalidate using an absolute path.
814        absolute_path = host1.path.abspath(
815                          host1.path.join(host1.getcwd(), "some_file"))
816        host1.stat_cache.invalidate(absolute_path)
817        # Will now raise an exception as it should.
818        print host1.stat("some_file")
819        # Would raise an exception since an `FTPHost` object
820        # knows of its own changes, even without `invalidate`.
821        print host2.stat("some_file")
822
823The method ``invalidate`` can be used on any *absolute* path, be it a
824directory, a file or a link.
825
826By default, the cache entries (if not replaced by newer ones) are
827stored for an infinite time. That is, if you start your Python process
828using ``ftputil`` and let it run for three days a stat call may still
829access cache data that old. To avoid this, you can set the ``max_age``
830attribute::
831
832    host = ftputil.FTPHost(server, user, password)
833    host.stat_cache.max_age = 60 * 60  # = 3600 seconds
834
835This sets the maximum age of entries in the cache to an hour. This
836means any entry older won't be retrieved from the cache but its data
837instead fetched again from the remote host and then again stored for
838up to an hour. To reset `max_age` to the default of unlimited age,
839i. e. cache entries never expire, use ``None`` as value.
840
841If you are certain that the cache will be in the way, you can disable
842and later re-enable it completely with ``disable`` and ``enable``::
843
844    host = ftputil.FTPHost(server, user, password)
845    host.stat_cache.disable()
846    ...
847    host.stat_cache.enable()
848
849During that time, the cache won't be used; all data will be fetched
850from the network. After enabling the cache, its entries will be the
851same as when the cache was disabled, that is, entries won't get
852updated with newer data during this period. Note that even when the
853cache is disabled, the file system data in the code can become
854inconsistent::
855
856    host = ftputil.FTPHost(server, user, password)
857    host.stat_cache.disable()
858    if host.path.exists("some_file"):
859        mtime = host.path.getmtime("some_file")
860
861In that case, the file ``some_file`` may have been removed by another
862process between the calls to ``exists`` and ``getmtime``!
863
864Iteration over directories
865``````````````````````````
866
867.. _`FTPHost.walk`:
868
869- ``walk(top, topdown=True, onerror=None, followlinks=False)``
870
871  iterates over a directory tree, similar to `os.walk`_. Actually,
872  ``FTPHost.walk`` uses the code from Python with just the necessary
873  modifications, so see the linked documentation.
874
875.. _`os.walk`: https://docs.python.org/2/library/os.html#os.walk
876
877.. _`FTPHost.path.walk`:
878
879- ``path.walk(path, func, arg)``
880
881  Similar to ``os.path.walk``, the ``walk`` method in
882  `FTPHost.path`_ can be used, though ``FTPHost.walk`` is probably
883  easier to use.
884
885Other methods
886`````````````
887
888- ``close()``
889
890  closes the connection to the remote host. After this, no more
891  interaction with the FTP server is possible with this ``FTPHost``
892  object.
893
894- ``rename(source, target)``
895
896  renames the source file (or directory) on the FTP server.
897
898.. _`FTPHost.chmod`:
899
900- ``chmod(path, mode)``
901
902  sets the access mode (permission flags) for the given path. The mode
903  is an integer as returned for the mode by the ``stat`` and ``lstat``
904  methods. Be careful: Usually, mode values are written as octal
905  numbers, for example 0755 to make a directory readable and writable
906  for the owner, but not writable for the group and others. If you
907  want to use such octal values, rely on Python's support for them::
908
909    host.chmod("some_directory", 0o755)
910
911  Not all FTP servers support the ``chmod`` command. In case of
912  an exception, how do you know if the path doesn't exist or if
913  the command itself is invalid? If the FTP server complies with
914  `RFC 959`_, it should return a status code 502 if the ``SITE CHMOD``
915  command isn't allowed. ``ftputil`` maps this special error
916  response to a ``CommandNotImplementedError`` which is derived from
917  ``PermanentError``.
918
919  So you need to code like this::
920
921    host = ftputil.FTPHost(server, user, password)
922    try:
923        host.chmod("some_file", 0o644)
924    except ftputil.error.CommandNotImplementedError:
925        # `chmod` not supported
926        ...
927    except ftputil.error.PermanentError:
928        # Possibly a non-existent file
929        ...
930
931  Because the ``CommandNotImplementedError`` is more specific, you
932  have to test for it first.
933
934.. _`RFC 959`: `RFC 959 - File Transfer Protocol (FTP)`_
935
936- ``copyfileobj(source, target, length=64*1024)``
937
938  copies the contents from the file-like object ``source`` to the
939  file-like object ``target``. The only difference to
940  ``shutil.copyfileobj`` is the default buffer size. Note that
941  arbitrary file-like objects can be used as arguments (e. g. local
942  files, remote FTP files).
943
944  However, the interfaces of ``source`` and ``target`` have to match;
945  the string type read from ``source`` must be an accepted string type
946  when written to ``target``. For example, if you open ``source`` in
947  Python 3 as a local text file and ``target`` as a remote file object
948  in binary mode, the transfer will fail since ``source.read`` gives
949  unicode strings whereas ``target.write`` only accepts byte strings.
950
951  See `File-like objects`_ for the construction and use of remote
952  file-like objects.
953
954.. _`set_parser`:
955
956- ``set_parser(parser)``
957
958  sets a custom parser for FTP directories. Note that you have to pass
959  in a parser *instance*, not the class.
960
961  An `extra section`_ shows how to write own parsers if the default
962  parsers in ``ftputil`` don't work for you.
963
964.. _`extra section`: `Writing directory parsers`_
965
966.. _`keep_alive`:
967
968- ``keep_alive()``
969
970  attempts to keep the connection to the remote server active in order
971  to prevent timeouts from happening. This method is primarily
972  intended to keep the underlying FTP connection of an ``FTPHost``
973  object alive while a file is uploaded or downloaded. This will
974  require either an extra thread while the upload or download is in
975  progress or calling ``keep_alive`` from a `callback function`_.
976
977  The ``keep_alive`` method won't help if the connection has already
978  timed out. In this case, a ``ftputil.error.TemporaryError`` is raised.
979
980  If you want to use this method, keep in mind that FTP servers define
981  a timeout for a reason. A timeout prevents running out of server
982  connections because of clients that never disconnect on their own.
983
984  Note that the ``keep_alive`` method does *not* affect the "hidden"
985  FTP connections established by ``FTPHost.open`` (see below). You
986  *can't* use ``keep_alive`` to avoid a timeout in a stalling transfer
987  like this::
988
989      with ftputil.FTPHost(server, userid, password) as host:
990          fobj = host.open("some_remote_file", "rb")
991          data = fobj.read(100)
992          # _Futile_ attempt to avoid file connection timeout.
993          for i in xrange(15):
994              time.sleep(60)
995              host.keep_alive()
996          # Will raise an `ftputil.error.TemporaryError`.
997          data += fobj.read()
998
999
1000File-like objects
1001-----------------
1002
1003Construction
1004~~~~~~~~~~~~
1005
1006Basics
1007``````
1008
1009``FTPFile`` objects are returned by a call to ``FTPHost.open``;
1010never use the constructor directly.
1011
1012The API of remote file-like objects are is modeled after the API of
1013the io_ module in Python 3, which has also been backported to Python
10142.6 and 2.7.
1015
1016.. _io: http://docs.python.org/library/io.html
1017
1018- ``FTPHost.open(path, mode="r", buffering=None, encoding=None,
1019  errors=None, newline=None)``
1020
1021  returns a file-like object that refers to the path on the remote
1022  host. This path may be absolute or relative to the current directory
1023  on the remote host (this directory can be determined with the
1024  ``getcwd`` method). As with local file objects, the default mode is
1025  "r", i. e. reading text files. Valid modes are "r", "rb", "w", and
1026  "wb".
1027
1028  If a file is opened in binary mode, you *must not* specify an
1029  encoding. On the other hand, if you open a file in text mode, an
1030  encoding is used. By default, this is the return value of
1031  ``locale.getpreferredencoding``, but you can (and probably should)
1032  specify a distinct encoding.
1033
1034  If you open a file in binary mode, the read and write operations use
1035  byte strings (``str`` in Python 2, ``bytes`` in Python 3). That is,
1036  read operations return byte strings and write operations only accept
1037  byte strings.
1038
1039  Similarly, text files always work with unicode strings (``unicode``
1040  in Python 2, ``str`` in Python 3). Here, read operations return
1041  unicode strings and write operations only accept unicode strings.
1042
1043  The arguments ``errors`` and ``newline`` have the same semantics as
1044  in `io.open`_. The argument ``buffering`` currently is ignored.
1045  It's only there for compatibility with the ``io.open`` interface.
1046
1047.. _`io.open`: http://docs.python.org/library/io.html#io.open
1048
1049Note that the semantics of "text mode" has changed fundamentally from
1050ftputil 2.8 and earlier. Previously, "text mode" implied converting
1051newline characters to ``\r\n`` when writing remote files and
1052converting newlines to ``\n`` when reading remote files. This is
1053in line with the "text mode" notion of FTP command line clients.
1054Now, "text mode" follows the semantics in Python's ``io`` module.
1055
1056Support for the ``with`` statement
1057``````````````````````````````````
1058
1059``FTPFile`` objects can be used with the ``with`` statement::
1060
1061    import ftputil
1062
1063    # Get an ``FTPHost`` object.
1064    ...
1065
1066    with host.open("new_file", "w") as fobj:
1067        fobj.write("This is some text.")
1068
1069At the end of the ``with`` block, the file will be closed
1070automatically.
1071
1072If something goes wrong during the construction of the file or in the
1073body of the ``with`` statement, the file will be closed as well.
1074Exceptions will be propagated as with ``try ... finally``.
1075
1076Attributes and methods
1077~~~~~~~~~~~~~~~~~~~~~~
1078
1079The methods
1080
1081::
1082
1083    close()
1084    read([count])
1085    readline([count])
1086    readlines()
1087    write(data)
1088    writelines(string_sequence)
1089
1090and the attribute ``closed`` have the same semantics as for file
1091objects of a local disk file system. The iterator protocol is
1092supported as well, i. e. you can use a loop to read a file line by
1093line::
1094
1095    with ftputil.FTPHost(...) as host:
1096        with host.open("some_file") as input_file:
1097            for line in input_file:
1098                # Do something with the line, e. g.
1099                print line.strip().replace("ftplib", "ftputil")
1100
1101For more on file objects, see the section `File objects`_ in the
1102Python Library Reference.
1103
1104.. _`file objects`: http://www.python.org/doc/current/lib/bltin-file-objects.html
1105
1106
1107Writing directory parsers
1108-------------------------
1109
1110``ftputil`` recognizes the two most widely-used FTP directory formats,
1111Unix and MS style, and adjusts itself automatically. Almost every FTP
1112server uses one of these formats.
1113
1114However, if your server uses a format which is different from the two
1115provided by ``ftputil``, you can plug in a custom parser with a single
1116method call and have ``ftputil`` use this parser.
1117
1118For this, you need to write a parser class by inheriting from the
1119class ``Parser`` in the ``ftputil.stat`` module. Here's an example::
1120
1121    import ftputil.error
1122    import ftputil.stat
1123
1124    class XyzParser(ftputil.stat.Parser):
1125        """
1126        Parse the default format of the FTP server of the XYZ
1127        corporation.
1128        """
1129
1130        def parse_line(self, line, time_shift=0.0):
1131            """
1132            Parse a `line` from the directory listing and return a
1133            corresponding `StatResult` object. If the line can't
1134            be parsed, raise `ftputil.error.ParserError`.
1135
1136            The `time_shift` argument can be used to fine-tune the
1137            parsing of dates and times. See the class
1138            `ftputil.stat.UnixParser` for an example.
1139            """
1140            # Split the `line` argument and examine it further; if
1141            # something goes wrong, raise an `ftputil.error.ParserError`.
1142            ...
1143            # Make a `StatResult` object from the parts above.
1144            stat_result = ftputil.stat.StatResult(...)
1145            # `_st_name`, `_st_target` and `_st_mtime_precision` are optional.
1146            stat_result._st_name = ...
1147            stat_result._st_target = ...
1148            stat_result._st_mtime_precision = ...
1149            return stat_result
1150
1151        # Define `ignores_line` only if the default in the base class
1152        # doesn't do enough!
1153        def ignores_line(self, line):
1154            """
1155            Return a true value if the line should be ignored. For
1156            example, the implementation in the base class handles
1157            lines like "total 17". On the other hand, if the line
1158            should be used for stat'ing, return a false value.
1159            """
1160            is_total_line = super(XyzParser, self).ignores_line(line)
1161            my_test = ...
1162            return is_total_line or my_test
1163
1164A ``StatResult`` object is similar to the value returned by
1165`os.stat`_ and is usually built with statements like
1166
1167::
1168
1169    stat_result = StatResult(
1170                    (st_mode, st_ino, st_dev, st_nlink, st_uid,
1171                     st_gid, st_size, st_atime, st_mtime, st_ctime))
1172    stat_result._st_name = ...
1173    stat_result._st_target = ...
1174    stat_result._st_mtime_precision = ...
1175
1176with the arguments of the ``StatResult`` constructor described in
1177the following table.
1178
1179===== =================== ============ =================== =======================
1180Index Attribute           os.stat type ``StatResult`` type Notes
1181===== =================== ============ =================== =======================
11820     st_mode             int          int
11831     st_ino              long         long
11842     st_dev              long         long
11853     st_nlink            int          int
11864     st_uid              int          str                 usually only available as string
11875     st_gid              int          str                 usually only available as string
11886     st_size             long         long
11897     st_atime            int/float    float
11908     st_mtime            int/float    float
11919     st_ctime            int/float    float
1192\-    _st_name            \-           str                 file name without directory part
1193\-    _st_target          \-           str                 link target (may be absolute or relative)
1194\-    _st_mtime_precision \-           int                 ``st_mtime`` precision in seconds
1195===== =================== ============ =================== =======================
1196
1197If you can't extract all the desirable data from a line (for
1198example, the MS format doesn't contain any information about the
1199owner of a file), set the corresponding values in the ``StatResult``
1200instance to ``None``.
1201
1202Parser classes can use several helper methods which are defined in
1203the class ``Parser``:
1204
1205- ``parse_unix_mode`` parses strings like "drwxr-xr-x" and returns
1206  an appropriate ``st_mode`` integer value.
1207
1208- ``parse_unix_time`` returns a float number usable for the
1209  ``st_...time`` values by parsing arguments like "Nov"/"23"/"02:33" or
1210  "May"/"26"/"2005". Note that the method expects the timestamp string
1211  already split at whitespace.
1212
1213- ``parse_ms_time`` parses arguments like "10-23-01"/"03:25PM" and
1214  returns a float number like from ``time.mktime``. Note that the
1215  method expects the timestamp string already split at whitespace.
1216
1217Additionally, there's an attribute ``_month_numbers`` which maps
1218lowercase three-letter month abbreviations to integers.
1219
1220For more details, see the two "standard" parsers ``UnixParser`` and
1221``MSParser`` in the module ``ftputil/stat.py``.
1222
1223To actually *use* the parser, call the method `set_parser`_ of the
1224``FTPHost`` instance.
1225
1226If you can't write a parser or don't want to, please ask on the
1227`ftputil mailing list`_. Possibly someone has already written a parser
1228for your server or can help with it.
1229
1230
1231FAQ / Tips and tricks
1232---------------------
1233
1234Where can I get the latest version?
1235~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1236
1237See the `download page`_. Announcements will be sent to the `mailing
1238list`_. Announcements on major updates will also be posted to the
1239newsgroup `comp.lang.python.announce`_ .
1240
1241.. _`download page`: http://ftputil.sschwarzer.net/download
1242.. _`mailing list`: http://ftputil.sschwarzer.net/mailinglist
1243.. _`comp.lang.python.announce`: news:comp.lang.python.announce
1244
1245Is there a mailing list on ``ftputil``?
1246~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1247
1248Yes, please visit http://ftputil.sschwarzer.net/mailinglist to
1249subscribe or read the archives.
1250
1251Though you can *technically* post without subscribing first I can't
1252recommend it: The mails from non-subscribers have to be approved by
1253me and because the arriving mails contain *lots* of spam, I rarely go
1254through these mails.
1255
1256I found a bug! What now?
1257~~~~~~~~~~~~~~~~~~~~~~~~
1258
1259Before reporting a bug, make sure that you already read this manual
1260and tried the `latest version`_ of ``ftputil``. There the bug might
1261have already been fixed.
1262
1263.. _`latest version`: http://ftputil.sschwarzer.net/download
1264
1265Please see http://ftputil.sschwarzer.net/issuetrackernotes for
1266guidelines on entering a bug in ``ftputil``'s ticket system. If you
1267are unsure if the behaviour you found is a bug or not, you should write
1268to the `ftputil mailing list`_. In *either* case you *must not*
1269include confidential information (user id, password, file names, etc.)
1270in the problem report! Be careful!
1271
1272Does ``ftputil`` support SSL/TLS?
1273~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1274
1275``ftputil`` has no *built-in* SSL/TLS support.
1276
1277On the other hand, there are two ways to get TLS support with
1278ftputil:
1279
1280- In Python 2.7 and Python 3.2 and up, the ``ftplib`` library has a
1281  class ``FTP_TLS`` that you can use for the ``session_factory``
1282  keyword argument in the ``FTPHost`` constructor. You can't use the
1283  class directly though if you need additional setup code in
1284  comparison to ``ftplib.FTP``, for example calling ``prot_p``, to
1285  secure the data connection. In that case, create your own session
1286  factory by inheriting from ``ftplib.FTP_TLS``::
1287
1288    import ftplib
1289
1290    import ftputil
1291
1292
1293    class FTPTLSSession(ftplib.FTP_TLS):
1294
1295        def __init__(self, host, user, password):
1296            ftplib.FTP_TLS.__init__(self)
1297            self.connect(host, port)
1298            self.login(user, password)
1299            # Set up encrypted data connection.
1300            self.prot_p()
1301
1302    # Note the `session_factory` parameter. Pass the class, not
1303    # an instance.
1304    host = ftputil.FTPHost(host, user, password,
1305                           session_factory=FTPTLSSession)
1306    # Use `host` as usual.
1307
1308- If you need to work with Python 2.6, you can use the
1309  ``ftpslib.FTP_TLS`` class from the M2Crypto_ project. Again, you
1310  can't use the class directly but need a recipe similar to that
1311  above.
1312
1313  Unfortunately, ``M2Crypto.ftpslib.FTP_TLS`` (at least in version
1314  0.22.3) doesn't work correctly if you pass unicode strings to its
1315  methods. Since ``ftputil`` does exactly that at some point (even if
1316  you used byte strings in ``ftputil`` calls) you need a workaround in
1317  the session factory class::
1318
1319    import M2Crypto
1320
1321    import ftputil
1322    import ftputil.tool
1323
1324
1325    class M2CryptoSession(M2Crypto.ftpslib.FTP_TLS):
1326
1327        def __init__(self, host, user, password):
1328            M2Crypto.ftpslib.FTP_TLS.__init__(self)
1329            # Change the port number if needed.
1330            self.connect(host, 21)
1331            self.auth_tls()
1332            self.login(user, password)
1333            self.prot_p()
1334            self._fix_socket()
1335
1336        def _fix_socket(self):
1337            """
1338            Change the socket object so that arguments to `sendall`
1339            are converted to byte strings before being used.
1340            """
1341            original_sendall = self.sock.sendall
1342            # Bound method, therefore no `self` argument.
1343            def sendall(data):
1344                data = ftputil.tool.as_bytes(data)
1345                return original_sendall(data)
1346            self.sock.sendall = sendall
1347
1348    # Note the `session_factory` parameter. Pass the class, not
1349    # an instance.
1350    host = ftputil.FTPHost(host, user, password,
1351                           session_factory=M2CryptoSession)
1352    # Use `host` as usual.
1353
1354.. _M2Crypto: https://github.com/martinpaljak/M2Crypto
1355
1356How do I connect to a non-default port?
1357~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1358
1359By default, an instantiated ``FTPHost`` object connects on the usual
1360FTP ports. If you have to use a different port, refer to the
1361section `FTPHost construction`_.
1362
1363You can use the same approach to connect in active or passive mode.
1364
1365How do I set active or passive mode?
1366~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1367
1368See the previous question and answer.
1369
1370How can I debug an FTP connection problem?
1371~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1372
1373Similarly to the tip above, write a session factory which calls
1374`ftplib.FTP.set_debuglevel`_ before the actual login.
1375
1376.. _`ftplib.FTP.set_debuglevel`: http://docs.python.org/library/ftplib.html#ftplib.FTP.set_debuglevel
1377
1378If you want to change the debug level only temporarily after the
1379connection is established, you can reach the `session object`_ as the
1380``_session`` attribute of the ``FTPHost`` instance. Note that this
1381interface should *only* be used for debugging. Calling arbitrary
1382``ftplib.FTP`` methods on the session object may *cause* bugs!
1383
1384.. _`session object`: #session-factories
1385
1386Conditional upload/download to/from a server in a different time zone
1387~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1388
1389You may find that ``ftputil`` uploads or downloads files
1390unnecessarily, or not when it should. This can happen when the FTP
1391server is in a different time zone than the client on which
1392``ftputil`` runs. Please see the section on `time zone correction`_.
1393It may even be sufficient to call `synchronize_times`_.
1394
1395When I use ``ftputil``, all I get is a ``ParserError`` exception
1396~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1397
1398The FTP server you connect to may use a directory format that
1399``ftputil`` doesn't understand. You can either write and
1400`plug in an own parser`_ or ask on the `mailing list`_ for
1401help.
1402
1403.. _`plug in an own parser`: `Writing directory parsers`_
1404
1405``isdir``, ``isfile`` or ``islink`` incorrectly return ``False``
1406~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1407
1408Like Python's counterparts under `os.path`_, ``ftputil``'s methods
1409return ``False`` if they can't find the given path.
1410
1411Probably you used ``listdir`` on a directory and called ``is...()`` on
1412the returned names. But if the argument for ``listdir`` wasn't the
1413current directory, the paths won't be found and so all ``is...()``
1414variants will return ``False``.
1415
1416I don't find an answer to my problem in this document
1417~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1418
1419Please send an email with your problem report or question to the
1420`ftputil mailing list`_, and we'll see what we can do for you. :-)
1421
1422
1423Bugs and limitations
1424--------------------
1425
1426- ``ftputil`` needs at least Python 2.6 to work.
1427
1428- Whether ``ftputil`` "sees" "hidden" directory and file names (i. e.
1429  names starting with a dot) depends on the configuration of the FTP
1430  server. See `Hidden files and directories`_ for details.
1431
1432- Due to the implementation of ``lstat`` it can not return a sensible
1433  value for the root directory ``/`` though stat'ing entries *in* the
1434  root directory isn't a problem. If you know an implementation that
1435  can do this, please let me know. The root directory is handled
1436  appropriately in ``FTPHost.path.exists/isfile/isdir/islink``, though.
1437
1438- In multithreaded programs, you can have each thread use one or more
1439  ``FTPHost`` instances as long as no instance is shared with other
1440  threads.
1441
1442- Currently, it is not possible to continue an interrupted upload or
1443  download. Contact me if this causes problems for you.
1444
1445- There's exactly one cache for ``lstat`` results for each ``FTPHost``
1446  object, i. e. there's no sharing of cache results determined by
1447  several ``FTPHost`` objects. See `Local caching of file system
1448  information`_ for the reasons.
1449
1450
1451Files
1452-----
1453
1454If not overwritten via installation options, the ``ftputil`` files
1455reside in the ``ftputil`` package. There's also documentation in
1456`reStructuredText`_ and in HTML format. The locations of these
1457files after installation is system-dependent.
1458
1459.. _`reStructuredText`: http://docutils.sourceforge.net/rst.html
1460
1461The files ``test_*.py`` and ``mock_ftplib.py`` are for unit-testing.
1462If you only *use* ``ftputil``, i. e. *don't* modify it, you can
1463delete these files.
1464
1465
1466References
1467----------
1468
1469- Mackinnon T, Freeman S, Craig P. 2000. `Endo-Testing:
1470  Unit Testing with Mock Objects`_.
1471
1472- Postel J, Reynolds J. 1985. `RFC 959 - File Transfer Protocol (FTP)`_.
1473
1474- Van Rossum G et al. 2013. `Python Library Reference`_.
1475
1476.. _`Endo-Testing: Unit Testing with Mock Objects`:
1477   http://www.connextra.com/aboutUs/mockobjects.pdf
1478.. _`RFC 959 - File Transfer Protocol (FTP)`: http://www.ietf.org/rfc/rfc959.txt
1479.. _`Python Library Reference`: http://www.python.org/doc/current/lib/lib.html
1480
1481
1482Authors
1483-------
1484
1485``ftputil`` is written by Stefan Schwarzer
1486<sschwarzer@sschwarzer.net>, in part based on suggestions
1487from users.
1488
1489The ``lrucache`` module is written by Evan Prodromou
1490<evan@prodromou.name>.
1491
1492Feedback is appreciated. :-)
Note: See TracBrowser for help on using the repository browser.