Changes between Version 3 and Version 4 of PreReleaseDocumentation

May 31, 2020, 11:08:37 PM (14 months ago)



  • PreReleaseDocumentation

    v3 v4  
     3``ftputil`` -- a high-level FTP client library
     6:Version:   4.0.0-beta
     7:Date:      2020-05-31
     8:Summary:   high-level FTP client library for Python
     9:Keywords:  FTP, ``ftplib`` substitute, virtual filesystem, pure Python
     10:Author:    Stefan Schwarzer <>
     12.. contents::
     18The ``ftputil`` module is a high-level interface to the ftplib_
     19module. The `FTPHost objects`_ generated from it allow many operations
     20similar to those of os_, `os.path`_ and `shutil`_.
     22.. _ftplib:
     23.. _os:
     24.. _`os.stat`:
     25.. _`os.path`:
     26.. _`shutil`:
     30    import ftputil
     32    # Download some files from the login directory.
     33    with ftputil.FTPHost("", "user", "password") as ftp_host:
     34        names = ftp_host.listdir(ftp_host.curdir)
     35        for name in names:
     36            if ftp_host.path.isfile(name):
     37      , name)  # remote, local
     38        # Make a new directory and copy a remote file into it.
     39        ftp_host.mkdir("newdir")
     40        with"index.html", "rb") as source:
     41            with"newdir/index.html", "wb") as target:
     42                ftp_host.copyfileobj(source, target)  # similar to shutil.copyfileobj
     44Also, there are `FTPHost.lstat`_ and `FTPHost.stat`_ to request size and
     45modification time of a file. The latter can also follow links, similar
     46to `os.stat`_. `FTPHost.walk`_ and `FTPHost.path.walk`_ work, too.
     49``ftputil`` features
     52* Method names are familiar from Python's ``os``, ``os.path`` and
     53  ``shutil`` modules. For example, use ``os.path.join`` to join
     54  paths for a local file system and ``ftp_host.path.join`` to join
     55  paths for a remote FTP file system.
     57* Remote file system navigation (``getcwd``, ``chdir``)
     59* Upload and download files (``upload``, ``upload_if_newer``,
     60  ``download``, ``download_if_newer``)
     62* Time zone synchronization between client and server (needed
     63  for ``upload_if_newer`` and ``download_if_newer``)
     65* Create and remove directories (``mkdir``, ``makedirs``, ``rmdir``,
     66  ``rmtree``) and remove files (``remove``)
     68* Get information about directories, files and links (``listdir``,
     69  ``stat``, ``lstat``, ``exists``, ``isdir``, ``isfile``, ``islink``,
     70  ``abspath``, ``dirname``, ``basename`` etc.)
     72* Iterate over remote file systems (``walk``)
     74* Local caching of results from ``lstat`` and ``stat`` calls to reduce
     75  network access (also applies to ``exists``, ``getmtime`` etc.).
     77* Read files from and write files to remote hosts via
     78  file-like objects (````; the generated file-like objects
     79  have the familiar methods like ``read``, ``readline``, ``readlines``,
     80  ``write``, ``writelines`` and ``close``. You can also iterate over
     81  these files line by line in a ``for`` loop.
     84Exception hierarchy
     87The exceptions are in the namespace of the ``ftputil.error`` module, e. g.
     90The exception classes are organized as follows::
     92    FTPError
     93        FTPOSError(FTPError, OSError)
     94            PermanentError(FTPOSError)
     95                CommandNotImplementedError(PermanentError)
     96            TemporaryError(FTPOSError)
     97        FTPIOError(FTPError)
     98        InternalError(FTPError)
     99            InaccessibleLoginDirError(InternalError)
     100            ParserError(InternalError)
     101            RootDirError(InternalError)
     102            TimeShiftError(InternalError)
     104and are described here:
     106- ``FTPError``
     108  is the root of the exception hierarchy of the module.
     110- ``FTPOSError``
     112  is derived from ``OSError``. This is for similarity between the
     113  os module and ``FTPHost`` objects. Compare
     115  ::
     117    try:
     118        os.chdir("nonexisting_directory")
     119    except OSError:
     120        ...
     122  with
     124  ::
     126    host = ftputil.FTPHost("host", "user", "password")
     127    try:
     128        host.chdir("nonexisting_directory")
     129    except OSError:
     130        ...
     132  Imagine a function
     134  ::
     136    def func(path, file):
     137        ...
     139  which works on the local file system and catches ``OSErrors``. If you
     140  change the parameter list to
     142  ::
     144    def func(path, file, os=os):
     145        ...
     147  where ``os`` denotes the ``os`` module, you can call the function also as
     149  ::
     151    host = ftputil.FTPHost("host", "user", "password")
     152    func(path, file, os=host)
     154  to use the same code for both a local and remote file system.
     155  Another similarity between ``OSError`` and ``FTPOSError`` is that
     156  the latter holds the FTP server return code in the ``errno``
     157  attribute of the exception object and the error text in
     158  ``strerror``.
     160- ``PermanentError``
     162  is raised for 5xx return codes from the FTP server. This
     163  corresponds to ``ftplib.error_perm`` (though ``PermanentError`` and
     164  ``ftplib.error_perm`` are *not* identical).
     166- ``CommandNotImplementedError``
     168  indicates that an underlying command the code tries to use is not
     169  implemented. For an example, see the description of the
     170  `FTPHost.chmod`_ method.
     172- ``TemporaryError``
     174  is raised for FTP return codes from the 4xx category. This
     175  corresponds to ``ftplib.error_temp`` (though ``TemporaryError`` and
     176  ``ftplib.error_temp`` are *not* identical).
     178- ``FTPIOError``
     180  denotes an I/O error on the remote host. This appears
     181  mainly with file-like objects that are retrieved by calling
     182  ````. Compare
     184  ::
     186    >>> try:
     187    ...     f = open("not_there")
     188    ... except IOError as obj:
     189    ...     print(obj.errno)
     190    ...     print(obj.strerror)
     191    ...
     192    2
     193    No such file or directory
     195  with
     197  ::
     199    >>> ftp_host = ftputil.FTPHost("host", "user", "password")
     200    >>> try:
     201    ...     f ="not_there")
     202    ... except IOError as obj:
     203    ...     print(obj.errno)
     204    ...     print(obj.strerror)
     205    ...
     206    550
     207    550 not_there: No such file or directory.
     209  As you can see, both code snippets are similar. However, the error
     210  codes aren't the same.
     212- ``InternalError``
     214  subsumes exception classes for signaling errors due to limitations
     215  of the FTP protocol or the concrete implementation of ``ftputil``.
     217- ``InaccessibleLoginDirError``
     219  This exception is raised if the directory in which "you" are placed
     220  upon login is not accessible, i. e. a ``chdir`` call with the
     221  directory as argument would fail.
     223- ``ParserError``
     225  is used for errors during the parsing of directory
     226  listings from the server. This exception is used by the ``FTPHost``
     227  methods ``stat``, ``lstat``, and ``listdir``.
     229- ``RootDirError``
     231  Because of the implementation of the ``lstat`` method it is not
     232  possible to do a ``stat`` call on the root directory ``/``.
     233  If you know *any* way to do it, please let me know. :-)
     235  This problem does *not* affect stat calls on items *in* the root
     236  directory.
     238- ``TimeShiftError``
     240  is used to denote errors which relate to setting the `time shift`_.
     243Directory and file names
     246.. note::
     248   Keep in mind that this section only applies to directory and file
     249   *names*, not file *contents*. Encoding and decoding for file
     250   contents is handled by the ``encoding`` argument for
     251   ``_.
     253First off: If your directory and file names (both as arguments and on
     254the server) contain only ISO 8859-1 (latin-1) characters, you can use
     255such names in the form of ``bytes`` or ``str`` objects. However, you
     256can't mix different string types (``bytes`` and ``str``) in one call
     257(for example in ``FTPHost.path.join``).
     259If you have directory or file names with characters that aren't in
     260latin-1, it's recommended to use ``bytes`` objects. In that case,
     261returned paths will be ``bytes`` objects, too.
     263Read on for details.
     265.. note::
     267   The approach described below may look awkward and in a way it is.
     268   The intention of ``ftputil`` is to behave like the local file
     269   system APIs of Python 3 as far as it makes sense. Moreover, the
     270   taken approach makes sure that directory and file names that were
     271   used with Python 3's native ``ftplib`` module will be compatible
     272   with ``ftputil`` and vice versa. Otherwise you may be able to use a
     273   file name with ``ftputil``, but get an exception when trying to
     274   read the same file with Python 3's ``ftplib`` module.
     276Methods that take paths of directories and/or files can take either
     277``bytes`` or ``str`` objects, or `PathLike`_ objects that can be
     278converted to ``bytes`` or ``str``.
     280.. _PathLike:
     282If a method gets a string argument (or a string argument wrapped in a
     283PathLike_ object) and returns one or more strings, these strings will
     284have the same string type (``bytes`` or ``str``) as the argument(s).
     285Mixing different string types in one call (for example in
     286``FTPHost.path.join``) isn't allowed and will cause a ``TypeError``.
     287These rules are the same as for local file system operations in Python 3.
     289``bytes`` objects for directory and file names will be sent to the
     290server as-is. On the other hand, ``str`` objects will be encoded to
     291``bytes`` objects, assuming latin-1 encoding. This implies that such
     292``str`` objects must only contain code points 0-255 for the latin-1
     293character set. Using any other characters will result in a
     294``UnicodeEncodeError`` exception.
     296If you have directory or file names as ``str`` objects with
     297non-latin-1 characters, encode the strings to ``bytes`` yourself,
     298using the encoding you know the server uses for its file system.
     299Decode received paths with the same encoding. Encapsulate these
     300conversions as far as you can. Otherwise, you'd have to adapt
     301potentially a lot of code if the server encoding changes.
     303If you *don't* know the encoding on the server side, it's probably the
     304best to only use ``bytes`` for directory and file names. That said, as
     305soon as you *show* the names to a user, you -- or the library you use
     306for displaying the names -- has to guess an encoding.
     308If you can decide about paths yourself, it's generally safest to use
     309only ASCII characters in FTP paths.
     312``FTPHost`` objects
     315.. _`FTPHost construction`:
     323``FTPHost`` instances can be created with the following call::
     325    ftp_host = ftputil.FTPHost(server, user, password, account,
     326                               session_factory=ftplib.FTP)
     328The first four parameters are strings with the same meaning as for the
     329FTP class in the ``ftplib`` module. Usually the ``account`` and
     330``session_factory`` arguments aren't needed though.
     332``FTPHost`` objects can also be used in a ``with`` statement::
     334    import ftputil
     336    with ftputil.FTPHost(server, user, password) as ftp_host:
     337        print(ftp_host.listdir(ftp_host.curdir))
     339After the ``with`` block, the ``FTPHost`` instance and the
     340associated FTP sessions will be closed automatically.
     342If something goes wrong during the ``FTPHost`` construction or in the
     343body of the ``with`` statement, the instance is closed as well.
     344Exceptions will be propagated (as with ``try ... finally``).
     346Session factories
     349The keyword argument ``session_factory`` may be used to generate FTP
     350connections with other factories than the default ``ftplib.FTP``. For
     351example, the standard library of Python 3 contains a class
     352``ftplib.FTP_TLS`` which extends ``ftplib.FTP`` to use an encrypted
     355In fact, all positional and keyword arguments other than
     356``session_factory`` are passed to the factory to generate a new
     357background session. This also happens for every remote file that is
     358opened; see below.
     360This functionality of the constructor also allows to wrap
     361``ftplib.FTP`` objects to do something that wouldn't be possible with
     362the ``ftplib.FTP`` constructor alone.
     364As an example, assume you want to connect to another than the default
     365port, but ``ftplib.FTP`` only offers this by means of its ``connect``
     366method, not via its constructor. One solution is to use a custom
     367class as a session factory::
     369    import ftplib
     370    import ftputil
     372    EXAMPLE_PORT = 50001
     374    class MySession(ftplib.FTP):
     376        def __init__(self, host, userid, password, port):
     377            """Act like ftplib.FTP's constructor but connect to another port."""
     378            ftplib.FTP.__init__(self)
     379            self.connect(host, port)
     380            self.login(userid, password)
     382    # Try _not_ to use an _instance_ `MySession()` as factory, -
     383    # use the class itself.
     384    with ftputil.FTPHost(host, userid, password, port=EXAMPLE_PORT,
     385                         session_factory=MySession) as ftp_host:
     386        # Use `ftp_host` as usual.
     387        ...
     389On login, the format of the directory listings (needed for stat'ing
     390files and directories) should be determined automatically. If not,
     391please `enter a ticket`_.
     393.. _`enter a ticket`:
     395For the most common uses you don't need to create your own session
     396factory class though. The ``ftputil.session`` module has a function
     397``session_factory`` that can create session factories for a variety
     398of parameters::
     400    session_factory(base_class=ftplib.FTP,
     401                    port=21,
     402                    use_passive_mode=None,
     403                    encrypt_data_channel=True,
     404                    debug_level=None)
     408- ``base_class`` is a base class to inherit a new session factory
     409  class from. By default, this is ``ftplib.FTP`` from the Python
     410  standard library.
     412- ``port`` is the command channel port. The default is 21, used in most
     413  FTP server configurations.
     415- ``use_passive_mode`` is either a boolean that determines whether
     416  passive mode should be used or ``None``. ``None`` means to let the
     417  base class choose active or passive mode.
     419- ``encrypt_data_channel`` defines whether to encrypt the data channel
     420  for secure connections. This is only supported for the base classes
     421  ``ftplib.FTP_TLS`` and ``M2Crypto.ftpslib.FTP_TLS``, otherwise the
     422  parameter is ignored.
     424- ``debug_level`` sets the debug level for FTP session instances. The
     425  semantics is defined by the base class. For example, a debug level
     426  of 2 causes the most verbose output for Python's ``ftplib.FTP``
     427  class.
     429All of these parameters can be combined. For example, you could use
     433    import ftplib
     435    import ftputil
     436    import ftputil.session
     439    my_session_factory = ftputil.session.session_factory(
     440                           base_class=ftpslib.FTP_TLS,
     441                           port=31,
     442                           encrypt_data_channel=True,
     443                           debug_level=2)
     445    with ftputil.FTPHost(server, user, password,
     446                         session_factory=my_session_factory) as ftp_host:
     447        ...
     449to create and use a session factory derived from ``ftplib.FTP_TLS``
     450that connects on command channel 31, will encrypt the data channel and
     451print output for debug level 2.
     453Note: Generally, you can achieve everything you can do with
     454``ftputil.session.session_factory`` with an explicit session factory
     455as described at the start of this section. However, the class
     456``M2Crypto.ftpslib.FTP_TLS`` has a limitation so that you can't use
     457it with ftputil out of the box. The function ``session_factory``
     458contains a workaround for this limitation. For details refer to `this
     461.. _`this ticket`:
     463Hidden files and directories
     466Whether ftputil sees "hidden" files and directories (usually files or
     467directories whose names start with a dot) depends on the FTP server
     468configuration. By default, ftputil does *not* use the ``-a`` option in
     469the FTP ``LIST`` command to find hidden files.
     471To tell the server to list hidden directories and files, set
     472``FTPHost.use_list_a_option`` to ``True``::
     474    ftp_host = ftputil.FTPHost(server, user, password, account,
     475                               session_factory=ftplib.FTP)
     476    ftp_host.use_list_a_option = True
     480- If the server doesn't understand the ``-a`` option at all, the
     481  server may interpret ``-a`` as the name of a file or directory,
     482  which can result in odd behavior. Therefore, use ``-a`` only if
     483  you're sure the server you're talking to supports it. Another
     484  approach is to have test code for ``-a`` support and fall back to
     485  not using the option.
     487- Even if the server knows about the ``-a`` option, the server may
     488  be configured to ignore it.
     490``FTPHost`` attributes and methods
     496- ``curdir``, ``pardir``, ``sep``
     498  are strings which denote the current and the parent directory on the
     499  remote server. ``sep`` holds the path separator. Though `RFC 959`_
     500  (File Transfer Protocol) notes that these values may depend on the
     501  FTP server implementation, the Unix variants seem to work well in
     502  practice, even for non-Unix servers.
     504  Nevertheless, it's recommended that you don't hardcode these values
     505  for remote paths, but use `FTPHost.path`_ as you would use
     506  ``os.path`` to write platform-independent Python code for local
     507  filesystems. Keep in mind that most, *but not all*, arguments of
     508  ``FTPHost`` methods refer to remote directories or files. For
     509  example, in `FTPHost.upload`_, the first argument is a local
     510  path and the second a remote path. Both of these should use their
     511  respective path separators.
     513.. _`FTPHost.upload`: `Uploading and downloading files`_
     515Remote file system navigation
     518- ``getcwd()``
     520  returns the absolute current directory on the remote host. This
     521  method works like ``os.getcwd``.
     523- ``chdir(directory)``
     525  sets the current directory on the FTP server. This resembles
     526  ``os.chdir``, as you may have expected.
     528.. _`callback function`:
     530Uploading and downloading files
     533- ``upload(source, target, callback=None)``
     535  copies a local source file (given by a filename, i. e. a string)
     536  to the remote host under the name target. Both ``source`` and
     537  ``target`` may be absolute paths or relative to their corresponding
     538  current directory (on the local or the remote host, respectively).
     540  The file content is always transferred in binary mode.
     542  The callback, if given, will be invoked for each transferred chunk
     543  of data::
     545    callback(chunk)
     547  where ``chunk`` is a bytestring. An example usage of a callback
     548  method is to display a progress indicator.
     550- ``download(source, target, callback=None)``
     552  performs a download from the remote source file to a local target
     553  file. Both ``source`` and ``target`` are strings. See the
     554  description of ``upload`` for more details.
     556.. _`upload_if_newer`:
     558- ``upload_if_newer(source, target, callback=None)``
     560  is similar to the ``upload`` method. The only difference is that the
     561  upload is only invoked if the time of the last modification for the
     562  source file is more recent than that of the target file or the
     563  target doesn't exist at all. The check for the last modification
     564  time considers the precision of the timestamps and transfers a file
     565  "if in doubt". Consequently the code
     567  ::
     569    ftp_host.upload_if_newer("source_file", "target_file")
     570    time.sleep(10)
     571    ftp_host.upload_if_newer("source_file", "target_file")
     573  might upload the file again if the timestamp of the target file is
     574  precise up to a minute, which is typically the case because the
     575  remote datetime is determined by parsing a directory listing from
     576  the server. To avoid unnecessary transfers, wait at least a minute
     577  between calls of ``upload_if_newer`` for the same file. If it still
     578  seems that a file is uploaded unnecessarily (or not when it should),
     579  read the subsection on `time shift`_ settings.
     581  If an upload actually happened, the return value of
     582  ``upload_if_newer`` is a ``True``, else ``False``.
     584  Note that the method only checks the existence and/or the
     585  modification time of the source and target file; it doesn't
     586  compare any other file properties, say, the file size.
     588  This also means that if a transfer is interrupted, the remote file
     589  will have a newer modification time than the local file, and thus
     590  the transfer won't be repeated if ``upload_if_newer`` is used a
     591  second time. There are at least two possibilities after a failed
     592  upload:
     594  - use ``upload`` instead of ``upload_if_newer``, or
     596  - remove the incomplete target file with ``FTPHost.remove``, then
     597    use ``upload`` or ``upload_if_newer`` to transfer it again.
     599.. _`download_if_newer`:
     601- ``download_if_newer(source, target, callback=None)``
     603  corresponds to ``upload_if_newer`` but performs a download from the
     604  server to the local host. Read the descriptions of download and
     605  ``upload_if_newer`` for more information. If a download actually
     606  happened, the return value is ``True``, else ``False``.
     608.. _`time shift`:
     609.. _`time zone correction`:
     611Time zone correction
     614For ``upload_if_newer`` and ``download_if_newer`` to work correctly,
     615the time zone of the server must be taken into account. By default,
     616ftputil assumes that the timestamps in server listings are in UTC_.
     618.. _UTC:
     620.. _`set_time_shift`:
     622- ``set_time_shift(time_shift)``
     624  sets the so-called time shift value, measured in seconds. The time
     625  shift here is defined as the difference between the time used in
     626  server listings and UTC.
     628  ::
     630    time_shift = server_time - utc_time
     632  For example, a server in Berlin/Germany set to the local time
     633  (currently UTC+03:00), would require a time shift value of 3 *
     634  3600.0 = 10800.0 seconds to be handled correctly by ftputil's
     635  ``upload_if_newer`` and ``download_if_newer``, as well as the
     636  ``stat`` and ``lstat`` calls.
     638  Note that servers don't necessarily send their file system listings
     639  in their local time zone. Some use UTC, which actually makes sense
     640  because UTC doesn't lead to an ambiguity when there's a switch back
     641  from the daylight saving time to the "normal" time of the server
     642  location.
     644  If the time shift value is invalid, for example its absolute value
     645  is larger than 24 hours, a ``TimeShiftError`` is raised.
     647  .. note::
     649     Versions of ftputil before 4.0.0 used a different definition of
     650     "time shift", server_time - local_client_time.
     652     This had the advantage that the default of 0.0 would be correct
     653     *if* the server was set to the same time zone as the client
     654     where ftputil runs. On the other hand, this approach meant that
     655     the time shift depended on *two* time zones, not only the one
     656     used on the server side. This could be confusing if server
     657     and client *didn't* use the same time zone.
     659  See also `synchronize_times`_ for a way to set the time shift with a
     660  simple method call. If you can't use ``synchronize_times`` *and* the
     661  server uses the same time zone as the client, you can set the time
     662  shift value with
     664  ::
     666    set_time_shift(
     667      round( ( - datetime.datetime.utcnow()).seconds, -2 )
     668    )
     670- ``time_shift()``
     672  returns the currently-set time shift value. See ``set_time_shift``
     673  above for its definition.
     675.. _`synchronize_times`:
     677- ``synchronize_times()``
     679  synchronizes the local times of the server and the client, so that
     680  `upload_if_newer`_ and `download_if_newer`_ work as expected, even
     681  if the client and the server use different time zones. For this
     682  to work, *all* of the following conditions must be true:
     684  - The connection between server and client is established.
     686  - The client has write access to the directory that is current when
     687    ``synchronize_times`` is called.
     689  If you can't fulfill these conditions, you can nevertheless set the
     690  time shift value explicitly with `set_time_shift`_. Trying to call
     691  ``synchronize_times`` if the above conditions aren't met results in
     692  a ``TimeShiftError`` exception.
     694Creating and removing directories
     697- ``mkdir(path, [mode])``
     699  makes the given directory on the remote host. This does *not*
     700  construct "intermediate" directories that don't already exist. The
     701  ``mode`` parameter is ignored; this is for compatibility with
     702  ``os.mkdir`` if an ``FTPHost`` object is passed into a function
     703  instead of the ``os`` module. See the explanation in the subsection
     704  `Exception hierarchy`_.
     706- ``makedirs(path, [mode])``
     708  works similar to ``mkdir`` (see above), but also makes intermediate
     709  directories like ``os.makedirs``. The ``mode`` parameter is only
     710  there for compatibility with ``os.makedirs`` and is ignored.
     712- ``rmdir(path)``
     714  removes the given remote directory. If it's not empty, raise
     715  a ``PermanentError``.
     717- ``rmtree(path, ignore_errors=False, onerror=None)``
     719  removes the given remote, possibly non-empty, directory tree.
     720  The interface of this method is rather complex, in favor of
     721  compatibility with ``shutil.rmtree``.
     723  If ``ignore_errors`` is set to a true value, errors are ignored.
     724  If ``ignore_errors`` is a false value *and* ``onerror`` isn't
     725  set, all exceptions occurring during the tree iteration and
     726  processing are raised. These exceptions are all of type
     727  ``PermanentError``.
     729  To distinguish between different kinds of errors, pass in a callable
     730  for ``onerror``. This callable must accept three arguments:
     731  ``func``, ``path`` and ``exc_info``. ``func`` is a bound method
     732  object, *for example* ``your_host_object.listdir``. ``path`` is the
     733  path that was the recent argument of the respective method
     734  (``listdir``, ``remove``, ``rmdir``). ``exc_info`` is the exception
     735  info as it is gotten from ``sys.exc_info``.
     737  The code of ``rmtree`` is taken from Python's ``shutil`` module
     738  and adapted for ``ftputil``.
     740Removing files and links
     743- ``remove(path)``
     745  removes a file or link on the remote host, similar to ``os.remove``.
     747- ``unlink(path)``
     749  is an alias for ``remove``.
     751Retrieving information about directories, files and links
     754- ``listdir(path)``
     756  returns a list containing the names of the files and directories
     757  in the given path, similar to `os.listdir`_. The special names
     758  ``.`` and ``..`` are not in the list.
     760The methods ``lstat`` and ``stat`` (and some others) rely on the
     761directory listing format used by the FTP server. When connecting to a
     762host, ``FTPHost``'s constructor tries to guess the right format, which
     763succeeds in most cases. However, if you get strange results or
     764``ParserError`` exceptions by a mere ``lstat`` call, please `enter a
     767If ``lstat`` or ``stat`` give wrong modification dates or times, look
     768at the methods that deal with time zone differences (`time zone
     771.. _`FTPHost.lstat`:
     773- ``lstat(path)``
     775  returns an object similar to that from `os.lstat`_. This is a kind
     776  of tuple with additional attributes; see the documentation of the
     777  ``os`` module for details.
     779  The result is derived by parsing the output of a ``LIST`` command on
     780  the server. Therefore, the result from ``FTPHost.lstat`` can not
     781  contain more information than the received text. In particular:
     783  - User and group ids can only be determined as strings, not as
     784    numbers, and that only if the server supplies them. This is
     785    usually the case with Unix servers but maybe not for other FTP
     786    servers.
     788  - Values for the time of the last modification may be rough,
     789    depending on the information from the server. For timestamps
     790    older than a year, this usually means that the precision of the
     791    modification timestamp value is not better than a day. For newer
     792    files, the information may be accurate to a minute.
     794    If the time of the last modification is before the epoch (usually
     795    1970-01-01 UTC), set the time of the last modification to 0.0.
     797  - Links can only be recognized on servers that provide this
     798    information in the ``LIST`` output.
     800  - Stat attributes that can't be determined at all are set to
     801        ``None``. For example, a line of a directory listing may not
     802        contain the date/time of a directory's last modification.
     804  - There's a special problem with stat'ing the root directory.
     805    (Stat'ing things *in* the root directory is fine though.) In
     806    this case, a ``RootDirError`` is raised. This has to do with the
     807    algorithm used by ``(l)stat``, and I know of no approach which
     808    mends this problem.
     810  Currently, ``ftputil`` recognizes the common Unix-style and
     811  Microsoft/DOS-style directory formats. If you need to parse output
     812  from another server type, please write to the `ftputil mailing
     813  list`_. You may consider `writing your own parser`_.
     815.. _`os.listdir`:
     816.. _`os.lstat`:
     817.. _`ftputil mailing list`:
     818.. _`writing your own parser`: `Writing directory parsers`_
     820.. _`FTPHost.stat`:
     822- ``stat(path)``
     824  returns ``stat`` information also for files which are pointed to by a
     825  link. This method follows multiple links until a regular file or
     826  directory is found. If an infinite link chain is encountered or the
     827  target of the last link in the chain doesn't exist, a
     828  ``PermanentError`` is raised.
     830  The limitations of the ``lstat`` method also apply to ``stat``.
     832.. _`FTPHost.path`:
     834``FTPHost`` objects contain an attribute named ``path``, similar to
     835`os.path`_. The following methods can be applied to the remote host
     836with the same semantics as for ``os.path``:
     840    abspath(path)
     841    basename(path)
     842    commonprefix(path_list)
     843    dirname(path)
     844    exists(path)
     845    getmtime(path)
     846    getsize(path)
     847    isabs(path)
     848    isdir(path)
     849    isfile(path)
     850    islink(path)
     851    join(path1, path2, ...)
     852    normcase(path)
     853    normpath(path)
     854    split(path)
     855    splitdrive(path)
     856    splitext(path)
     857    walk(path, func, arg)
     859Like Python's counterparts under `os.path`_, ``ftputil``'s ``is...``
     860methods return ``False`` if they can't find the path given by their
     863Local caching of file system information
     866Many of the above methods need access to the remote file system to
     867obtain data on directories and files. To get the most recent data,
     868*each* call to ``lstat``, ``stat``, ``exists``, ``getmtime`` etc.
     869would require to fetch a directory listing from the server, which can
     870make the program *very* slow. This effect is more pronounced for
     871operations which mostly scan the file system rather than transferring
     872file data.
     874For this reason, ``ftputil`` by default saves the results from
     875directory listings locally and reuses those results. This reduces
     876network accesses and so speeds up the software a lot. However, since
     877data is more rarely fetched from the server, the risk of obsolete data
     878also increases. This will be discussed below.
     880Caching can be controlled -- if necessary at all -- via the
     881``stat_cache`` object in an ``FTPHost``'s namespace. For example,
     882after calling
     886    ftp_host = ftputil.FTPHost(host, user, password)
     888the cache can be accessed as ``ftp_host.stat_cache``.
     890While ``ftputil`` usually manages the cache quite well, there are two
     891possible reasons for modifying cache parameters.
     893The first is when the number of possible entries is too low. You may
     894notice that when you are processing very large directories and the
     895program becomes much slower than before. It's common for code to read
     896a directory with ``listdir`` and then process the found directories
     897and files. This can also happen implicitly by a call to
     898``FTPHost.walk``. Since version 2.6 ``ftputil`` automatically
     899increases the cache size if directories with more entries than the
     900current maximum cache size are to be scanned. Most of the time, this
     901works fine.
     903However, if you need access to stat data for several directories at
     904the same time, you may need to increase the cache explicitly. This is
     905done by the ``resize`` method::
     907    ftp_host.stat_cache.resize(20000)
     909where the argument is the maximum number of ``lstat`` results to store
     910(the default is 5000, in versions before 2.6 it was 1000). Note that
     911each path on the server, e. g. "/home/schwa/some_dir", corresponds to
     912a single cache entry. Methods like ``exists`` or ``getmtime`` all
     913derive their results from a previously fetched ``lstat`` result.
     915The value 5000 above means that the cache will hold *at most* 5000
     916entries (unless increased automatically by an explicit or implicit
     917``listdir`` call, see above). If more are about to be stored, the
     918entries which haven't been used for the longest time will be deleted
     919to make place for newer entries.
     921The second possible reason to change the cache parameters is to avoid
     922stale cache data. Caching is so effective because it reduces network
     923accesses. This can also be a disadvantage if the file system data on
     924the remote server changes after a stat result has been retrieved; the
     925client, when looking at the cached stat data, will use obsolete
     928There are two potential ways to get such out-of-date stat data. The
     929first happens when an ``FTPHost`` instance modifies a file path for
     930which it has a cache entry, e. g. by calling ``remove`` or ``rmdir``.
     931Such changes are handled transparently; the path will be deleted from
     932the cache. A different matter are changes unknown to the ``FTPHost``
     933object which inspects its cache. Obviously, for example, these are
     934changes by programs running on the remote host. On the other hand,
     935cache inconsistencies can also occur if two ``FTPHost`` objects change
     936a file system simultaneously::
     938    with (
     939      ftputil.FTPHost(server, user1, password1) as ftp_host1,
     940      ftputil.FTPHost(server, user1, password1) as ftp_host2
     941    ):
     942        stat_result1 = ftp_host1.stat("some_file")
     943        stat_result2 = ftp_host2.stat("some_file")
     944        ftp_host2.remove("some_file")
     945        # `ftp_host1` will still see the obsolete cache entry!
     946        print(ftp_host1.stat("some_file"))
     947        # Will raise an exception since an `FTPHost` object
     948        # knows of its own changes.
     949        print(ftp_host2.stat("some_file"))
     951At first sight, it may appear to be a good idea to have a shared cache
     952among several ``FTPHost`` objects. After some thinking, this turns out
     953to be very error-prone. For example, it won't help with different
     954processes using ``ftputil``. So, if you have to deal with concurrent
     955write/read accesses to a server, you have to handle them explicitly.
     957The most useful tool for this is the ``invalidate`` method. In the
     958example above, it could be used like this::
     960    with (
     961      ftputil.FTPHost(server, user1, password1) as ftp_host1,
     962      ftputil.FTPHost(server, user1, password1) as ftp_host2
     963    ):
     964        stat_result1 = ftp_host1.stat("some_file")
     965        stat_result2 = ftp_host2.stat("some_file")
     966        ftp_host2.remove("some_file")
     967        # Invalidate using an absolute path.
     968        absolute_path = ftp_host1.path.abspath(
     969                          ftp_host1.path.join(ftp_host1.getcwd(), "some_file"))
     970        ftp_host1.stat_cache.invalidate(absolute_path)
     971        # Will now raise an exception as it should.
     972        print(ftp_host1.stat("some_file"))
     973        # Would raise an exception since an `FTPHost` object
     974        # knows of its own changes, even without `invalidate`.
     975        print(ftp_host2.stat("some_file"))
     977The method ``invalidate`` can be used on any *absolute* path, be it a
     978directory, a file or a link.
     980By default, the cache entries (if not replaced by newer ones) are
     981stored for an infinite time. That is, if you start your Python process
     982using ``ftputil`` and let it run for three days a stat call may still
     983access cache data that old. To avoid this, you can set the ``max_age``
     986    with ftputil.FTPHost(server, user, password) as ftp_host:
     987        ftp_host.stat_cache.max_age = 60 * 60  # = 3600 seconds
     989This sets the maximum age of entries in the cache to an hour. This
     990means any entry older won't be retrieved from the cache but its data
     991instead fetched again from the remote host and then again stored for
     992up to an hour. To reset `max_age` to the default of unlimited age,
     993i. e. cache entries never expire, use ``None`` as value.
     995If you are certain that the cache will be in the way, you can disable
     996and later re-enable it completely with ``disable`` and ``enable``::
     998    with ftputil.FTPHost(server, user, password) as ftp_host:
     999        ftp_host.stat_cache.disable()
     1000        ...
     1001        ftp_host.stat_cache.enable()
     1003During that time, the cache won't be used; all data will be fetched
     1004from the network. After enabling the cache again, its entries will be
     1005the same as when the cache was disabled, that is, entries won't get
     1006updated with newer data during this period. Note that even when the
     1007cache is disabled, the file system data in the code can become
     1010    with ftputil.FTPHost(server, user, password) as ftp_host:
     1011        ftp_host.stat_cache.disable()
     1012        if ftp_host.path.exists("some_file"):
     1013            mtime = ftp_host.path.getmtime("some_file")
     1015In that case, the file ``some_file`` may have been removed by another
     1016process between the calls to ``exists`` and ``getmtime``!
     1018Iteration over directories
     1021.. _`FTPHost.walk`:
     1023- ``walk(top, topdown=True, onerror=None, followlinks=False)``
     1025  iterates over a directory tree, similar to `os.walk`_. Actually,
     1026  ``FTPHost.walk`` uses the code from Python with just the necessary
     1027  modifications, so see the linked documentation.
     1029.. _`os.walk`:
     1031.. _`FTPHost.path.walk`:
     1033- ``path.walk(path, func, arg)``
     1035  Similar to ``os.path.walk``, the ``walk`` method in
     1036  `FTPHost.path`_ can be used, though ``FTPHost.walk`` is probably
     1037  easier to use.
     1039Other methods
     1042- ``close()``
     1044  closes the connection to the remote host. After this, no more
     1045  interaction with the FTP server is possible with this ``FTPHost``
     1046  object. Usually you don't need to close an ``FTPHost`` instance
     1047  with ``close`` if you set up the instance in a ``with`` statement.
     1049- ``rename(source, target)``
     1051  renames the source file (or directory) on the FTP server.
     1053.. _`FTPHost.chmod`:
     1055- ``chmod(path, mode)``
     1057  sets the access mode (permission flags) for the given path. The mode
     1058  is an integer as returned for the mode by the ``stat`` and ``lstat``
     1059  methods. Be careful: Usually, mode values are written as octal
     1060  numbers, for example 0755 to make a directory readable and writable
     1061  for the owner, but not writable for the group and others. If you
     1062  want to use such octal values, rely on Python's support for them::
     1064    ftp_host.chmod("some_directory", 0o755)
     1066  Not all FTP servers support the ``chmod`` command. In case of
     1067  an exception, how do you know if the path doesn't exist or if
     1068  the command itself is invalid? If the FTP server complies with
     1069  `RFC 959`_, it should return a status code 502 if the ``SITE CHMOD``
     1070  command isn't allowed. ``ftputil`` maps this special error
     1071  response to a ``CommandNotImplementedError`` which is derived from
     1072  ``PermanentError``.
     1074  So you need to code like this::
     1076    with ftputil.FTPHost(server, user, password) as ftp_host:
     1077        try:
     1078            ftp_host.chmod("some_file", 0o644)
     1079        except ftputil.error.CommandNotImplementedError:
     1080            # `chmod` not supported
     1081            ...
     1082        except ftputil.error.PermanentError:
     1083            # Possibly a non-existent file
     1084            ...
     1086  Because the ``CommandNotImplementedError`` is more specific, you
     1087  have to test for it first.
     1089.. _`RFC 959`: `RFC 959 - File Transfer Protocol (FTP)`_
     1091- ``copyfileobj(source, target, length=64*1024)``
     1093  copies the contents from the file-like object ``source`` to the
     1094  file-like object ``target``. The only difference to
     1095  ``shutil.copyfileobj`` is the default buffer size. Note that
     1096  arbitrary file-like objects can be used as arguments (e. g. local
     1097  files, remote FTP files).
     1099  However, the interfaces of ``source`` and ``target`` have to match;
     1100  the string type read from ``source`` must be an accepted string type
     1101  when written to ``target``. For example, if you open ``source`` in
     1102  Python 3 as a local text file and ``target`` as a remote file object
     1103  in binary mode, the transfer will fail since ```` gives
     1104  unicode strings (``str``) whereas ``target.write`` only accepts byte
     1105  strings (``bytes``).
     1107  See `File-like objects`_ for the construction and use of remote
     1108  file-like objects.
     1110.. _`set_parser`:
     1112- ``set_parser(parser)``
     1114  sets a custom parser for FTP directories. Note that you have to pass
     1115  in a parser *instance*, not the class.
     1117  An `extra section`_ shows how to write own parsers if the default
     1118  parsers in ``ftputil`` don't work for you.
     1120.. _`extra section`: `Writing directory parsers`_
     1122.. _`keep_alive`:
     1124- ``keep_alive()``
     1126  attempts to keep the connection to the remote server active in order
     1127  to prevent timeouts from happening. This method is primarily
     1128  intended to keep the underlying FTP connection of an ``FTPHost``
     1129  object alive while a file is uploaded or downloaded. This will
     1130  require either an extra thread while the upload or download is in
     1131  progress or calling ``keep_alive`` from a `callback function`_.
     1133  The ``keep_alive`` method won't help if the connection has already
     1134  timed out. In this case, a ``ftputil.error.TemporaryError`` is raised.
     1136  If you want to use this method, keep in mind that FTP servers define
     1137  a timeout for a reason. A timeout prevents running out of server
     1138  connections because of clients that never disconnect on their own.
     1140  Note that the ``keep_alive`` method does *not* affect the "hidden"
     1141  FTP child connections established by ```` (see section
     1142  `FTPHost instances vs. FTP connections`_ for details). You *can't*
     1143  use ``keep_alive`` to avoid a timeout in a stalling transfer like
     1144  this::
     1146      with ftputil.FTPHost(server, userid, password) as ftp_host:
     1147          with"some_remote_file", "rb") as fobj:
     1148              data =
     1149              # _Futile_ attempt to avoid file connection timeout.
     1150              for i in range(15):
     1151                  time.sleep(60)
     1152                  ftp_host.keep_alive()
     1153              # Will raise an `ftputil.error.TemporaryError`.
     1154              data +=
     1157.. _``:
     1159File-like objects
     1168``FTPFile`` objects are returned by a call to ````;
     1169never use the ``FTPFile`` constructor directly.
     1171The APIs for remote file-like objects is modeled after the APIs of
     1172the built-in ``open`` function and its return value.
     1174- ``, mode="r", buffering=None, encoding=None,
     1175  errors=None, newline=None, rest=None)``
     1177  returns a file-like object that refers to the path on the remote
     1178  host. This path may be absolute or relative to the current directory
     1179  on the remote host (this directory can be determined with the
     1180  ``getcwd`` method). As with local file objects, the default mode is
     1181  "r", i. e. reading text files. Valid modes are "r", "rb", "w", and
     1182  "wb".
     1184  If a file is opened in binary mode, you *must not* specify an
     1185  encoding. On the other hand, if you open a file in text mode, an
     1186  encoding is used. By default, this is the return value of
     1187  ``locale.getpreferredencoding``, but you can (and probably should)
     1188  specify a distinct encoding.
     1190  If you open a file in binary mode, the read and write operations use
     1191  ``bytes`` objects. That is, read operations return ``bytes`` and
     1192  write operations only accept ``bytes``.
     1194  Similarly, text files always work with strings (``str``). Here, read
     1195  operations return string and write operations only accept strings.
     1197  The arguments ``buffering``, ``errors`` and ``newline`` have the
     1198  same semantics as in open_.
     1200  If the file is opened in binary mode, you may pass 0 or a positive
     1201  integer for the ``rest`` argument. The argument is passed to the
     1202  underlying FTP session instance (for example an instance of
     1203  ``ftplib.FTP``) to start reading or writing at the given byte
     1204  offset. For example, if a remote file contains the letters
     1205  "abcdef" in ASCII encoding, ``rest=3`` will start reading at "d".
     1207  .. warning::
     1209     If you pass ``rest`` values which point *after* the file, the
     1210     behavior is undefined and may even differ from one FTP server to
     1211     another. Therefore, use the ``rest`` argument only for error
     1212     recovery in case of interrupted transfers. You need to keep track
     1213     of the transferred data so that you can provide a valid ``rest``
     1214     argument for a resumed transfer.
     1216.. _`open`:
     1218```` can also be used in a ``with`` statement::
     1220    import ftputil
     1222    with ftputil.FTPHost(...) as ftp_host:
     1223        ...
     1224        with"new_file", "w", encoding="utf8") as fobj:
     1225            fobj.write("This is some text.")
     1227At the end of the ``with`` block, the remote file will be closed
     1230If something goes wrong during the construction of the file or in the
     1231body of the ``with`` statement, the file will be closed as well.
     1232Exceptions will be propagated as with ``try ... finally``.
     1234Attributes and methods
     1237The methods
     1241    close()
     1242    read([count])
     1243    readline([count])
     1244    readlines()
     1245    write(data)
     1246    writelines(string_sequence)
     1248and the attribute ``closed`` have the same semantics as for file
     1249objects of a local disk file system. The iterator protocol is
     1250supported as well, i. e. you can use a loop to read a file line by
     1253    with ftputil.FTPHost(server, user, password) as ftp_host:
     1254        with"some_file") as input_file:
     1255            for line in input_file:
     1256                # Do something with the line, e. g.
     1257                print(line.strip().replace("ftplib", "ftputil"))
     1259For more on file objects, see the section `File objects`_ in the
     1260Python Library Reference.
     1262.. _`file objects`:
     1265.. _`child_connections`:
     1267``FTPHost`` instances vs. FTP connections
     1270This section explains why keeping an ``FTPHost`` instance "alive"
     1271without timing out sometimes isn't trivial. If you always finish your
     1272FTP operations in time, you don't need to read this section.
     1274The file transfer protocol is a stateful protocol. That means an FTP
     1275connection always is in a certain state. Each of these states can only
     1276change to certain other states under certain conditions triggered by
     1277the client or the server.
     1279One of the consequences is that a single FTP connection can't be used
     1280at the same time, say, to transfer data on the FTP data channel and to
     1281create a directory on the remote host.
     1283For example, consider this::
     1285    >>> import ftplib
     1286    >>> ftp = ftplib.FTP(server, user, password)
     1287    >>> ftp.pwd()
     1288    '/'
     1289    >>> # Start transfer. `CONTENTS` is a text file on the server.
     1290    >>> socket = ftp.transfercmd("RETR CONTENTS")
     1291    >>> socket
     1292    <socket._socketobject object at 0x7f801a6386e0>
     1293    >>> ftp.pwd()
     1294    Traceback (most recent call last):
     1295      File "<stdin>", line 1, in <module>
     1296      File "/usr/lib64/python2.7/", line 578, in pwd
     1297        return parse257(resp)
     1298      File "/usr/lib64/python2.7/", line 842, in parse257
     1299        raise error_reply, resp
     1300    ftplib.error_reply: 226-File successfully transferred
     1301    226 0.000 seconds (measured here), 5.60 Mbytes per second
     1302    >>>
     1304Note that ``ftp`` is a single FTP connection, represented by an
     1305``ftplib.FTP`` instance, not an ``ftputil.FTPHost`` instance.
     1307On the other hand, consider this::
     1309    >>> import ftputil
     1310    >>> ftp_host = ftputil.FTPHost(server, user, password)
     1311    >>> ftp_host.getcwd()
     1312    >>> fobj ="CONTENTS")
     1313    >>> fobj
     1314    <ftputil.file.FTPFile object at 0x7f8019d3aa50>
     1315    >>> ftp_host.getcwd()
     1316    u'/'
     1317    >>> fobj.readline()
     1318    u'Contents of FTP test directory\n'
     1319    >>> fobj.close()
     1320    >>>
     1322To be able to start a file transfer (i. e. open a remote file for
     1323reading or writing) and still be able to use other FTP commands,
     1324ftputil uses a trick. For every remote file, ftputil creates a new FTP
     1325connection, called a child connection in the ftputil source code.
     1326(Actually, FTP connections belonging to closed remote files are
     1327re-used if they haven't timed out yet.)
     1329In most cases this approach isn't noticeable by code using ftputil.
     1330However, the nice abstraction of dealing with a single FTP connection
     1331falls apart if one of the child connections times out. For example, if
     1332you open a remote file and work only with the initial "main"
     1333connection to navigate the file system, the FTP connection for the
     1334remote file may eventually time out.
     1336While it's often relatively easy to prevent the "main" connection from
     1337timing out it's unfortunately practically impossible to do this for a
     1338remote file connection (apart from transferring some data, of course).
     1339For this reason, `FTPHost.keep_alive`_ affects only the main
     1340connection. Child connections may still time out if they're idle for
     1341too long.
     1343.. _`FTPHost.keep_alive`: `keep_alive`_
     1345Some more details:
     1347- A kind of "straightforward" way of keeping the main connection alive
     1348  would be to call ``ftp_host.getcwd()``. However, this doesn't work
     1349  because ftputil caches the current directory and returns it without
     1350  actually contacting the server. That's the main reason why there's
     1351  a ``keep_alive`` method since it calls ``pwd`` on the FTP connection
     1352  (i. e. the session object), which isn't a public attribute.
     1354- Some servers define not only an idle timeout but also a transfer
     1355  timeout. This means the connection times out unless there's some
     1356  transfer on the data channel for this connection. So ftputil's
     1357  ``keep_alive`` doesn't prevent this timeout, but an
     1358  ``ftp_host.listdir(ftp_host.curdir)`` call should do it. However,
     1359  this transfers the data for the whole directory listing which might
     1360  take some time if the directory has many entries.
     1362Bottom line: If you can, you should organize your FTP actions so that
     1363you finish everything before a timeout happens.
     1366Writing directory parsers
     1369``ftputil`` recognizes the two most widely-used FTP directory formats,
     1370Unix and MS style, and adjusts itself automatically. Almost every FTP
     1371server uses one of these formats.
     1373However, if your server uses a format which is different from the two
     1374provided by ``ftputil``, you can plug in a custom parser with a single
     1375method call and have ``ftputil`` use this parser.
     1377For this, you need to write a parser class by inheriting from the
     1378class ``Parser`` in the ``ftputil.stat`` module. Here's an example::
     1380    import ftputil.error
     1381    import ftputil.stat
     1383    class XyzParser(ftputil.stat.Parser):
     1384        """
     1385        Parse the default format of the FTP server of the XYZ
     1386        corporation.
     1387        """
     1389        def parse_line(self, line, time_shift=0.0):
     1390            """
     1391            Parse a `line` from the directory listing and return a
     1392            corresponding `StatResult` object. If the line can't
     1393            be parsed, raise `ftputil.error.ParserError`.
     1395            The `time_shift` argument can be used to fine-tune the
     1396            parsing of dates and times. See the class
     1397            `ftputil.stat.UnixParser` for an example.
     1398            """
     1399            # Split the `line` argument and examine it further; if
     1400            # something goes wrong, raise an `ftputil.error.ParserError`.
     1401            ...
     1402            # Make a `StatResult` object from the parts above.
     1403            stat_result = ftputil.stat.StatResult(...)
     1404            # `_st_name`, `_st_target` and `_st_mtime_precision` are optional.
     1405            stat_result._st_name = ...
     1406            stat_result._st_target = ...
     1407            stat_result._st_mtime_precision = ...
     1408            return stat_result
     1410        # Define `ignores_line` only if the default in the base class
     1411        # doesn't do enough!
     1412        def ignores_line(self, line):
     1413            """
     1414            Return a true value if the line should be ignored. For
     1415            example, the implementation in the base class handles
     1416            lines like "total 17". On the other hand, if the line
     1417            should be used for stat'ing, return a false value.
     1418            """
     1419            is_total_line = super().ignores_line(line)
     1420            my_test = ...
     1421            return is_total_line or my_test
     1423A ``StatResult`` object is similar to the value returned by
     1424`os.stat`_ and is usually built with statements like
     1428    stat_result = StatResult(
     1429                    (st_mode, st_ino, st_dev, st_nlink, st_uid,
     1430                     st_gid, st_size, st_atime, st_mtime, st_ctime))
     1431    stat_result._st_name = ...
     1432    stat_result._st_target = ...
     1433    stat_result._st_mtime_precision = ...
     1435with the arguments of the ``StatResult`` constructor described in
     1436the following table.
     1438===== =================== ============ =================== =======================
     1439Index Attribute           os.stat type ``StatResult`` type Notes
     1440===== =================== ============ =================== =======================
     14410     st_mode             int          int
     14421     st_ino              long         long
     14432     st_dev              long         long
     14443     st_nlink            int          int
     14454     st_uid              int          str                 usually only available as string
     14465     st_gid              int          str                 usually only available as string
     14476     st_size             long         long
     14487     st_atime            int/float    float
     14498     st_mtime            int/float    float
     14509     st_ctime            int/float    float
     1451\-    _st_name            \-           str                 file name without directory part
     1452\-    _st_target          \-           str                 link target (may be absolute or relative)
     1453\-    _st_mtime_precision \-           int                 ``st_mtime`` precision in seconds
     1454===== =================== ============ =================== =======================
     1456If you can't extract all the desirable data from a line (for
     1457example, the MS format doesn't contain any information about the
     1458owner of a file), set the corresponding values in the ``StatResult``
     1459instance to ``None``.
     1461Parser classes can use several helper methods which are defined in
     1462the class ``Parser``:
     1464- ``parse_unix_mode`` parses strings like "drwxr-xr-x" and returns
     1465  an appropriate ``st_mode`` integer value.
     1467- ``parse_unix_time`` returns a float number usable for the
     1468  ``st_...time`` values by parsing arguments like "Nov"/"23"/"02:33" or
     1469  "May"/"26"/"2005". Note that the method expects the timestamp string
     1470  already split at whitespace.
     1472- ``parse_ms_time`` parses arguments like "10-23-01"/"03:25PM" and
     1473  returns a float number like from ``time.mktime``. Note that the
     1474  method expects the timestamp string already split at whitespace.
     1476Additionally, there's an attribute ``_month_numbers`` which maps
     1477lowercase three-letter month abbreviations to integers.
     1479For more details, see the two "standard" parsers ``UnixParser`` and
     1480``MSParser`` in the module ``ftputil/``.
     1482To actually *use* the parser, call the method `set_parser`_ of the
     1483``FTPHost`` instance.
     1485If you can't write a parser or don't want to, please ask on the
     1486`ftputil mailing list`_. Possibly someone has already written a parser
     1487for your server or can help with it.
     1490FAQ / Tips and tricks
     1493Where can I get the latest version?
     1496See the `download page`_. Announcements will be sent to the `mailing
     1497list`_. Announcements on major updates will also be posted to the
     1498`Python announcements list`_.
     1500.. _`download page`:
     1501.. _`mailing list`:
     1502.. _`Python announcements list`:
     1504Is there a mailing list on ``ftputil``?
     1507Yes, please visit to
     1508subscribe or read the archives.
     1510Though you can *technically* post without subscribing first I can't
     1511recommend it: The mails from non-subscribers have to be approved by
     1512me and because the arriving mails contain *lots* of spam, I rarely go
     1513through these mails.
     1515I found a bug! What now?
     1518Before reporting a bug, make sure that you already read this manual
     1519and tried the `latest version`_ of ``ftputil``. There the bug might
     1520have already been fixed.
     1522.. _`latest version`:
     1524Please see for
     1525guidelines on entering a bug in ``ftputil``'s ticket system. If you
     1526are unsure if the behaviour you found is a bug or not, you should write
     1527to the `ftputil mailing list`_. *Never* include confidential information
     1528(user id, password, file names, etc.) in the problem report! Be
     1531Does ``ftputil`` support TLS?
     1534``ftputil`` has no *built-in* TLS support.
     1536On the other hand, there are two ways to get TLS support with
     1539- The ``ftplib`` library has a class ``FTP_TLS`` that you can use for
     1540  the ``session_factory`` keyword argument in the ``FTPHost``
     1541  constructor. You can't use the class directly though *if* you need
     1542  additional setup code in comparison to ``ftplib.FTP``, for example
     1543  calling ``prot_p``, to secure the data connection. On the other
     1544  hand, `ftputil.session.session_factory`_ can be used to create a
     1545  custom session factory.
     1547- If you have other requirements that ``session_factory`` can't
     1548  fulfill, you may create your own session factory by inheriting from
     1549  ``ftplib.FTP_TLS``::
     1551    import ftplib
     1553    import ftputil
     1556    class FTPTLSSession(ftplib.FTP_TLS):
     1558        def __init__(self, host, user, password):
     1559            ftplib.FTP_TLS.__init__(self)
     1560            self.connect(host, port)
     1561            self.login(user, password)
     1562            # Set up encrypted data connection.
     1563            self.prot_p()
     1564            ...
     1566    # Note the `session_factory` parameter. Pass the class, not
     1567    # an instance.
     1568    with ftputil.FTPHost(server, user, password,
     1569                         session_factory=FTPTLSSession) as ftp_host:
     1570        # Use `ftp_host` as usual.
     1571        ...
     1573.. _`ftputil.session.session_factory`: `Session factories`_
     1576How do I connect to a non-default port?
     1579By default, an instantiated ``FTPHost`` object connects on the usual
     1580FTP port. If you have to use a different port, refer to the section
     1581`Session factories`_.
     1583How do I set active or passive mode?
     1586Please see the section `Session factories`_.
     1588How can I debug an FTP connection problem?
     1591You can do this with a session factory. See `Session factories`_.
     1593If you want to change the debug level only temporarily after the
     1594connection is established, you can reach the `session object`_ as the
     1595``_session`` attribute of the ``FTPHost`` instance and call
     1596``_session.set_debuglevel``. Note that the ``_session`` attribute
     1597should *only* be accessed for debugging. Calling arbitrary
     1598``ftplib.FTP`` methods on the session object may *cause* bugs!
     1600.. _`session object`: `Session factories`_
     1602Conditional upload/download to/from a server in a different time zone
     1605You may find that ``ftputil`` uploads or downloads files
     1606unnecessarily, or not when it should. Please see the section on `time
     1607zone correction`_. It may even be sufficient to call
     1610When I use ``ftputil``, all I get is a ``ParserError`` exception
     1613The FTP server you connect to may use a directory format that
     1614``ftputil`` doesn't understand. You can either write and
     1615`plug in your own parser`_ or ask on the `mailing list`_ for
     1618.. _`plug in your own parser`: `Writing directory parsers`_
     1620``isdir``, ``isfile`` or ``islink`` incorrectly return ``False``
     1623Like Python's counterparts under `os.path`_, ``ftputil``'s methods
     1624return ``False`` if they can't find the given path.
     1626Probably you used ``listdir`` on a directory and called ``is...()`` on
     1627the returned names. But if the argument for ``listdir`` wasn't the
     1628current directory, the paths won't be found and so all ``is...()``
     1629variants will return ``False``.
     1631I don't find an answer to my problem in this document
     1634Please send an email with your problem report or question to the
     1635`ftputil mailing list`_, and we'll see what we can do for you. :-)
     1638Bugs and limitations
     1641- ``ftputil`` needs at least Python 3.5 to work.
     1643- Whether ``ftputil`` "sees" "hidden" directory and file names (i. e.
     1644  names starting with a dot) depends on the configuration of the FTP
     1645  server. See `Hidden files and directories`_ for details.
     1647- Due to the implementation of ``lstat`` it can not return a sensible
     1648  value for the root directory ``/`` though stat'ing entries *in* the
     1649  root directory isn't a problem. If you know an implementation that
     1650  can do this, please let me know. The root directory is handled
     1651  appropriately in ``FTPHost.path.exists/isfile/isdir/islink``, though.
     1653- In multithreaded programs, you can have each thread use one or more
     1654  ``FTPHost`` instances as long as no instance is shared with other
     1655  threads.
     1657- Currently, it is not possible to continue an interrupted upload or
     1658  download. Contact me if this causes problems for you.
     1660- There's exactly one cache for ``lstat`` results for each ``FTPHost``
     1661  object, i. e. there's no sharing of cache results determined by
     1662  several ``FTPHost`` objects. See `Local caching of file system
     1663  information`_ for the reasons.
     1669If not overwritten via installation options, the ``ftputil`` files
     1670reside in the ``ftputil`` package. There's also documentation in
     1671`reStructuredText`_ and in HTML format. The locations of these
     1672files after installation is system-dependent.
     1674.. _`reStructuredText`:
     1676The files ``test_*.py`` and ```` are for
     1677unit-testing. If you only *use* ``ftputil``, i. e. *don't* modify it,
     1678you can delete these files.
     1684- Postel J, Reynolds J. 1985. `RFC 959 - File Transfer Protocol (FTP)`_.
     1686- Python Software Foundation. 2020. `The Python Standard Library`_.
     1688.. _`RFC 959 - File Transfer Protocol (FTP)`:
     1689.. _`The Python Standard Library`:
     1695``ftputil`` is written by Stefan Schwarzer
     1696<> and contributors (see
     1699The original ``lrucache`` module was written by Evan Prodromou
     1702Feedback is appreciated. :-)