source: doc/ftputil.txt @ 1524:75f09a84fa26

Last change on this file since 1524:75f09a84fa26 was 1524:75f09a84fa26, checked in by Stefan Schwarzer <sschwarzer@…>, 5 years ago
Updated FAQ to refer to new `session.session_factory` function.
File size: 55.3 KB
Line 
1``ftputil`` -- a high-level FTP client library
2==============================================
3
4:Version:   3.0
5:Date:      2013-11-17
6:Summary:   high-level FTP client library for Python
7:Keywords:  FTP, ``ftplib`` substitute, virtual filesystem, pure Python
8:Author:    Stefan Schwarzer <sschwarzer@sschwarzer.net>
9
10.. contents::
11
12
13Introduction
14------------
15
16The ``ftputil`` module is a high-level interface to the ftplib_
17module. The `FTPHost objects`_ generated from it allow many operations
18similar to those of os_, `os.path`_ and `shutil`_.
19
20.. _ftplib: https://docs.python.org/library/ftplib.html
21.. _os: https://docs.python.org/library/os.html
22.. _`os.stat`: https://docs.python.org/library/os.html#os.stat
23.. _`os.path`: https://docs.python.org/library/os.path.html
24.. _`shutil`: https://docs.python.org/library/shutil.html
25
26Examples::
27
28    import ftputil
29
30    # Download some files from the login directory.
31    with ftputil.FTPHost("ftp.domain.com", "user", "password") as host:
32        names = host.listdir(host.curdir)
33        for name in names:
34            if host.path.isfile(name):
35                host.download(name, name)  # remote, local
36        # Make a new directory and copy a remote file into it.
37        host.mkdir("newdir")
38        with host.open("index.html", "rb") as source:
39            with host.open("newdir/index.html", "wb") as target:
40                host.copyfileobj(source, target)  # similar to shutil.copyfileobj
41
42Also, there are `FTPHost.lstat`_ and `FTPHost.stat`_ to request size and
43modification time of a file. The latter can also follow links, similar
44to `os.stat`_. `FTPHost.walk`_ and `FTPHost.path.walk`_ work, too.
45
46
47``ftputil`` features
48--------------------
49
50* Method names are familiar from Python's ``os``, ``os.path`` and
51  ``shutil`` modules
52
53* Remote file system navigation (``getcwd``, ``chdir``)
54
55* Upload and download files (``upload``, ``upload_if_newer``,
56  ``download``, ``download_if_newer``)
57
58* Time zone synchronization between client and server (needed
59  for ``upload_if_newer`` and ``download_if_newer``)
60
61* Create and remove directories (``mkdir``, ``makedirs``, ``rmdir``,
62  ``rmtree``) and remove files (``remove``)
63
64* Get information about directories, files and links (``listdir``,
65  ``stat``, ``lstat``, ``exists``, ``isdir``, ``isfile``, ``islink``,
66  ``abspath``, ``split``, ``join``, ``dirname``, ``basename`` etc.)
67
68* Iterate over remote file systems (``walk``)
69
70* Local caching of results from ``lstat`` and ``stat`` calls to reduce
71  network access (also applies to ``exists``, ``getmtime`` etc.).
72
73* Read files from and write files to remote hosts via
74  file-like objects (``FTPHost.open``; the generated file-like objects
75  have many common methods like ``read``, ``readline``, ``readlines``,
76  ``write``, ``writelines``, ``close`` and can do automatic line
77  ending conversions on the fly, i. e. text/binary mode).
78
79
80Exception hierarchy
81-------------------
82
83The exceptions are in the namespace of the ``ftputil.error`` module, e. g.
84``ftputil.error.TemporaryError``.
85
86The exception classes are organized as follows::
87
88    FTPError
89        FTPOSError(FTPError, OSError)
90            PermanentError(FTPOSError)
91                CommandNotImplementedError(PermanentError)
92            TemporaryError(FTPOSError)
93        FTPIOError(FTPError)
94        InternalError(FTPError)
95            InaccessibleLoginDirError(InternalError)
96            ParserError(InternalError)
97            RootDirError(InternalError)
98            TimeShiftError(InternalError)
99
100and are described here:
101
102- ``FTPError``
103
104  is the root of the exception hierarchy of the module.
105
106- ``FTPOSError``
107
108  is derived from ``OSError``. This is for similarity between the
109  os module and ``FTPHost`` objects. Compare
110
111  ::
112
113    try:
114        os.chdir("nonexisting_directory")
115    except OSError:
116        ...
117
118  with
119
120  ::
121
122    host = ftputil.FTPHost("host", "user", "password")
123    try:
124        host.chdir("nonexisting_directory")
125    except OSError:
126        ...
127
128  Imagine a function
129
130  ::
131
132    def func(path, file):
133        ...
134
135  which works on the local file system and catches ``OSErrors``. If you
136  change the parameter list to
137
138  ::
139
140    def func(path, file, os=os):
141        ...
142
143  where ``os`` denotes the ``os`` module, you can call the function also as
144
145  ::
146
147    host = ftputil.FTPHost("host", "user", "password")
148    func(path, file, os=host)
149
150  to use the same code for both a local and remote file system.
151  Another similarity between ``OSError`` and ``FTPOSError`` is that
152  the latter holds the FTP server return code in the ``errno``
153  attribute of the exception object and the error text in
154  ``strerror``.
155
156- ``PermanentError``
157
158  is raised for 5xx return codes from the FTP server. This
159  corresponds to ``ftplib.error_perm`` (though ``PermanentError`` and
160  ``ftplib.error_perm`` are *not* identical).
161
162- ``CommandNotImplementedError``
163
164  indicates that an underlying command the code tries to use is not
165  implemented. For an example, see the description of the
166  `FTPHost.chmod`_ method.
167
168- ``TemporaryError``
169
170  is raised for FTP return codes from the 4xx category. This
171  corresponds to ``ftplib.error_temp`` (though ``TemporaryError`` and
172  ``ftplib.error_temp`` are *not* identical).
173
174- ``FTPIOError``
175
176  denotes an I/O error on the remote host. This appears
177  mainly with file-like objects that are retrieved by calling
178  ``FTPHost.open``. Compare
179
180  ::
181
182    >>> try:
183    ...     f = open("not_there")
184    ... except IOError as obj:
185    ...     print obj.errno
186    ...     print obj.strerror
187    ...
188    2
189    No such file or directory
190
191  with
192
193  ::
194
195    >>> host = ftputil.FTPHost("host", "user", "password")
196    >>> try:
197    ...     f = host.open("not_there")
198    ... except IOError as obj:
199    ...     print obj.errno
200    ...     print obj.strerror
201    ...
202    550
203    550 not_there: No such file or directory.
204
205  As you can see, both code snippets are similar. However, the error
206  codes aren't the same.
207
208- ``InternalError``
209
210  subsumes exception classes for signaling errors due to limitations
211  of the FTP protocol or the concrete implementation of ``ftputil``.
212
213- ``InaccessibleLoginDirError``
214
215  This exception is raised if the directory in which "you" are placed
216  upon login is not accessible, i. e. a ``chdir`` call with the
217  directory as argument would fail.
218
219- ``ParserError``
220
221  is used for errors during the parsing of directory
222  listings from the server. This exception is used by the ``FTPHost``
223  methods ``stat``, ``lstat``, and ``listdir``.
224
225- ``RootDirError``
226
227  Because of the implementation of the ``lstat`` method it is not
228  possible to do a ``stat`` call on the root directory ``/``.
229  If you know *any* way to do it, please let me know. :-)
230
231  This problem does *not* affect stat calls on items *in* the root
232  directory.
233
234- ``TimeShiftError``
235
236  is used to denote errors which relate to setting the `time shift`_,
237  *for example* trying to set a value which is no multiple of a full
238  hour.
239
240
241Directory and file names
242------------------------
243
244Methods that take names of directories and files can take either byte
245strings (``str`` on Python 2, ``bytes`` on Python 3) or unicode
246strings (``unicode`` on Python 2, ``str`` on Python 3).
247
248Byte strings will be sent to the FTP server as-is. Unicode strings
249will be encoded with the encoding returned from
250``locale.getpreferredencoding``. This is the same semantics as for
251locally used names in Python 3.
252
253Methods that take and return a directory or file name will return the
254same string type as they're given. For example, if the argument to
255``FTPHost.path.abspath`` is a byte string, you'll get a byte string
256back. This behavior is the same as for the local file system API in
257Python 2 and 3.
258
259
260``FTPHost`` objects
261-------------------
262
263.. _`FTPHost construction`:
264
265Construction
266~~~~~~~~~~~~
267
268Basics
269``````
270
271``FTPHost`` instances can be generated with the following call::
272
273    host = ftputil.FTPHost(host, user, password, account,
274                           session_factory=ftplib.FTP)
275
276The first four parameters are strings with the same meaning as for the
277FTP class in the ``ftplib`` module.
278
279``FTPHost`` objects can also be used in a ``with`` statement::
280
281    import ftputil
282
283    with ftputil.FTPHost(host, user, password) as ftp_host:
284        print ftp_host.listdir(ftp_host.curdir)
285
286After the ``with`` block, the ``FTPHost`` instance and the
287associated FTP sessions will be closed automatically.
288
289If something goes wrong during the ``FTPHost`` construction or in the
290body of the ``with`` statement, the instance is closed as well.
291Exceptions will be propagated (as with ``try ... finally``).
292
293Session factories
294`````````````````
295
296The keyword argument ``session_factory`` may be used to generate FTP
297connections with other factories than the default ``ftplib.FTP``. For
298example, the standard library of Python 2.7 contains a class
299``ftplib.FTP_TLS`` which extends ``ftplib.FTP`` to use an encrypted
300connection.
301
302In fact, all positional and keyword arguments other than
303``session_factory`` are passed to the factory to generate a new
304background session. This also happens for every remote file that is
305opened; see below.
306
307This functionality of the constructor also allows to wrap
308``ftplib.FTP`` objects to do something that wouldn't be possible with
309the ``ftplib.FTP`` constructor alone.
310
311As an example, assume you want to connect to another than the default
312port, but ``ftplib.FTP`` only offers this by means of its ``connect``
313method, not via its constructor. One solution is to use a custom
314class as a session factory::
315
316    import ftplib
317    import ftputil
318
319    EXAMPLE_PORT = 50001
320
321    class MySession(ftplib.FTP):
322
323        def __init__(self, host, userid, password, port):
324            """Act like ftplib.FTP's constructor but connect to another port."""
325            ftplib.FTP.__init__(self)
326            self.connect(host, port)
327            self.login(userid, password)
328
329    # Try _not_ to use an _instance_ `MySession()` as factory, -
330    # use the class itself.
331    with ftputil.FTPHost(host, userid, password, port=EXAMPLE_PORT,
332                         session_factory=MySession) as ftp_host:
333        # Use `ftp_host` as usual.
334        ...
335
336On login, the format of the directory listings (needed for stat'ing
337files and directories) should be determined automatically. If not,
338please `file a bug report`_.
339
340.. _`file a bug report`: http://ftputil.sschwarzer.net/issuetrackernotes
341
342For the most common uses you don't need to create your own session
343factory class though. The ``ftputil.session`` module has a function
344``session_factory`` that can create session factories for a variety
345of parameters::
346
347    session_factory(base_class=ftplib.FTP,
348                    port=21,
349                    use_passive_mode=None,
350                    encrypt_data_channel=True,
351                    debug_level=None)
352
353with
354
355- ``base_class`` is a base class to inherit a new session factory
356  class from. By default, this is ``ftplib.FTP`` from the Python
357  standard library.
358
359- ``port`` is the command channel port. The default is 21, used in most
360  FTP server configurations.
361
362- ``use_passive_mode`` is either a boolean that determines whether
363  passive mode should be used or ``None``. ``None`` means to let the
364  base class choose active or passive mode.
365
366- ``encrypt_data_channel`` defines whether to encrypt the data channel
367  for secure connections. This is only supported for the base classes
368  ``ftplib.FTP_TLS`` and ``M2Crypto.ftpslib.FTP_TLS``, otherwise the
369  the parameter is ignored.
370
371- ``debug_level`` sets the debug level for FTP session instances. The
372  semantics is defined by the base class. For example, a debug level
373  of 2 causes the most verbose output for Python's ``ftplib.FTP``
374  class.
375
376All of these parameters can be combined. For example, you could use
377
378::
379
380    import ftplib
381
382    import ftputil
383    import ftputil.session
384
385
386    my_session_factory = ftputil.session.session_factory(
387                           base_class=ftpslib.FTP_TLS,
388                           port=31,
389                           encrypt_data_channel=True,
390                           debug_level=2)
391
392    with ftputil.FTPHost(host, user, password,
393                         session_factory=my_session_factory) as ftp_host:
394        ...
395
396to create and use a session factory derived from ``ftplib.FTP_TLS``
397that connects on command channel 31, will encrypt the data channel and
398print output for debug level 2.
399
400Note: Generally, you can achieve everything you can do with
401``ftputil.session.session_factory`` with an explicit session factory
402as described at the start of this section. However, the class
403``M2Crypto.ftpslib.FTP_TLS`` has a limitation so that you can't use
404it with ftputil out of the box. The function ``session_factory``
405contains a workaround for this limitation. For details refer to `this
406bug report`_.
407
408.. _`this bug report`: http://ftputil.sschwarzer.net/trac/ticket/78
409
410Hidden files and directories
411~~~~~~~~~~~~~~~~~~~~~~~~~~~~
412
413Whether ftputil sees "hidden" files and directories (usually files or
414directories whose names start with a dot) depends on the FTP server
415configuration. By default, ftputil uses the ``-a`` option in the FTP
416``LIST`` command to find hidden files. However, the server may ignore
417this.
418
419If using the ``-a`` option leads to problems, for example if an
420FTP server causes an exception, you may switch off the use of the
421option::
422
423    host = ftputil.FTPHost(host, user, password, account,
424                           session_factory=ftplib.FTP)
425    host.use_list_a_option = False
426
427``FTPHost`` attributes and methods
428~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
429
430Attributes
431``````````
432
433- ``curdir``, ``pardir``, ``sep``
434
435  are strings which denote the current and the parent directory on the
436  remote server. ``sep`` holds the path separator. Though `RFC 959`_
437  (File Transfer Protocol) notes that these values may depend on the
438  FTP server implementation, the Unix variants seem to work well in
439  practice, even for non-Unix servers.
440
441  Nevertheless, it's recommended that you don't hardcode these values
442  for remote paths, but use `FTPHost.path`_ as you would use
443  ``os.path`` to write platform-independent Python code for local
444  filesystems. Keep in mind that most, *but not all*, arguments of
445  ``FTPHost`` methods refer to remote directories or files. For
446  example, in `FTPHost.upload`_, the first argument is a local
447  path and the second a remote path. Both of these should use their
448  respective path separators.
449
450.. _`FTPHost.upload`: `Uploading and downloading files`_
451
452Remote file system navigation
453`````````````````````````````
454
455- ``getcwd()``
456
457  returns the absolute current directory on the remote host. This
458  method works like ``os.getcwd``.
459
460- ``chdir(directory)``
461
462  sets the current directory on the FTP server. This resembles
463  ``os.chdir``, as you may have expected.
464
465.. _`callback function`:
466
467Uploading and downloading files
468```````````````````````````````
469
470- ``upload(source, target, callback=None)``
471
472  copies a local source file (given by a filename, i. e. a string)
473  to the remote host under the name target. Both ``source`` and
474  ``target`` may be absolute paths or relative to their corresponding
475  current directory (on the local or the remote host, respectively).
476
477  The file content is always transferred in binary mode.
478
479  The callback, if given, will be invoked for each transferred chunk
480  of data::
481
482    callback(chunk)
483
484  where ``chunk`` is a bytestring. An example usage of a callback
485  method is to display a progress indicator.
486
487- ``download(source, target, callback=None)``
488
489  performs a download from the remote source file to a local target
490  file. Both ``source`` and ``target`` are strings. See the
491  description of ``upload`` for more details.
492
493.. _`upload_if_newer`:
494
495- ``upload_if_newer(source, target, callback=None)``
496
497  is similar to the ``upload`` method. The only difference is that the
498  upload is only invoked if the time of the last modification for the
499  source file is more recent than that of the target file or the
500  target doesn't exist at all. The check for the last modification
501  time considers the precision of the timestamps and transfers a file
502  "if in doubt". Consequently the code
503
504  ::
505
506    host.upload_if_newer("source_file", "target_file")
507    time.sleep(10)
508    host.upload_if_newer("source_file", "target_file")
509
510  might upload the file again if the timestamp of the target file is
511  precise up to a minute, which is typically the case because the
512  remote datetime is determined by parsing a directory listing from
513  the server. To avoid unnecessary transfers, wait at least a minute
514  between calls of ``upload_if_newer`` for the same file. If it still
515  seems that a file is uploaded unnecessarily (or not when it should),
516  read the subsection on `time shift`_ settings.
517
518  If an upload actually happened, the return value of
519  ``upload_if_newer`` is a ``True``, else ``False``.
520
521  Note that the method only checks the existence and/or the
522  modification time of the source and target file; it doesn't
523  compare any other file properties, say, the file size.
524
525  This also means that if a transfer is interrupted, the remote file
526  will have a newer modification time than the local file, and thus
527  the transfer won't be repeated if ``upload_if_newer`` is used a
528  second time. There are at least two possibilities after a failed
529  upload:
530
531  - use ``upload`` instead of ``upload_if_newer``, or
532
533  - remove the incomplete target file with ``FTPHost.remove``, then
534    use ``upload`` or ``upload_if_newer`` to transfer it again.
535
536.. _`download_if_newer`:
537
538- ``download_if_newer(source, target, callback=None)``
539
540  corresponds to ``upload_if_newer`` but performs a download from the
541  server to the local host. Read the descriptions of download and
542  ``upload_if_newer`` for more information. If a download actually
543  happened, the return value is ``True``, else ``False``.
544
545.. _`time shift`:
546.. _`time zone correction`:
547
548Time zone correction
549````````````````````
550
551If the client where ``ftputil`` runs and the server have a different
552understanding of their local times, this has to be taken into account
553for ``upload_if_newer`` and ``download_if_newer`` to work correctly.
554
555Note that even if the client and the server are in the same time zone
556(or even on the same computer), the time shift value (see below) may
557be different from zero. For example, my computer is set to use local
558time whereas the server running on the very same host insists on using
559UTC time.
560
561.. _`set_time_shift`:
562
563- ``set_time_shift(time_shift)``
564
565  sets the so-called time shift value, measured in seconds. The time
566  shift is the difference between the local time of the server and the
567  local time of the client at a given moment, i. e. by definition
568
569  ::
570
571    time_shift = server_time - client_time
572
573  Setting this value is important for `upload_if_newer`_ and
574  `download_if_newer`_ to work correctly even if the time zone of the
575  FTP server differs from that of the client. Note that the time shift
576  value *can be negative*.
577
578  If the time shift value is invalid, e. g. no multiple of a full hour
579  or its absolute value larger than 24 hours, a ``TimeShiftError`` is
580  raised.
581
582  See also `synchronize_times`_ for a way to set the time shift with a
583  simple method call.
584
585- ``time_shift()``
586
587  returns the currently-set time shift value. See ``set_time_shift``
588  above for its definition.
589
590.. _`synchronize_times`:
591
592- ``synchronize_times()``
593
594  synchronizes the local times of the server and the client, so that
595  `upload_if_newer`_ and `download_if_newer`_ work as expected, even
596  if the client and the server use different time zones. For this
597  to work, *all* of the following conditions must be true:
598
599  - The connection between server and client is established.
600
601  - The client has write access to the directory that is current when
602    ``synchronize_times`` is called.
603
604  If you can't fulfill these conditions, you can nevertheless set the
605  time shift value explicitly with `set_time_shift`_. Trying to call
606  ``synchronize_times`` if the above conditions aren't met results in
607  a ``TimeShiftError`` exception.
608
609Creating and removing directories
610`````````````````````````````````
611
612- ``mkdir(path, [mode])``
613
614  makes the given directory on the remote host. This does *not*
615  construct "intermediate" directories that don't already exist. The
616  ``mode`` parameter is ignored; this is for compatibility with
617  ``os.mkdir`` if an ``FTPHost`` object is passed into a function
618  instead of the ``os`` module. See the explanation in the subsection
619  `Exception hierarchy`_.
620
621- ``makedirs(path, [mode])``
622
623  works similar to ``mkdir`` (see above), but also makes intermediate
624  directories like ``os.makedirs``. The ``mode`` parameter is only
625  there for compatibility with ``os.makedirs`` and is ignored.
626
627- ``rmdir(path)``
628
629  removes the given remote directory. If it's not empty, raise
630  a ``PermanentError``.
631
632- ``rmtree(path, ignore_errors=False, onerror=None)``
633
634  removes the given remote, possibly non-empty, directory tree.
635  The interface of this method is rather complex, in favor of
636  compatibility with ``shutil.rmtree``.
637
638  If ``ignore_errors`` is set to a true value, errors are ignored.
639  If ``ignore_errors`` is a false value *and* ``onerror`` isn't
640  set, all exceptions occurring during the tree iteration and
641  processing are raised. These exceptions are all of type
642  ``PermanentError``.
643
644  To distinguish between different kinds of errors, pass in a callable
645  for ``onerror``. This callable must accept three arguments:
646  ``func``, ``path`` and ``exc_info``. ``func`` is a bound method
647  object, *for example* ``your_host_object.listdir``. ``path`` is the
648  path that was the recent argument of the respective method
649  (``listdir``, ``remove``, ``rmdir``). ``exc_info`` is the exception
650  info as it is gotten from ``sys.exc_info``.
651
652  The code of ``rmtree`` is taken from Python's ``shutil`` module
653  and adapted for ``ftputil``.
654
655Removing files and links
656````````````````````````
657
658- ``remove(path)``
659
660  removes a file or link on the remote host, similar to ``os.remove``.
661
662- ``unlink(path)``
663
664  is an alias for ``remove``.
665
666Retrieving information about directories, files and links
667`````````````````````````````````````````````````````````
668
669- ``listdir(path)``
670
671  returns a list containing the names of the files and directories
672  in the given path, similar to `os.listdir`_. The special names
673  ``.`` and ``..`` are not in the list.
674
675The methods ``lstat`` and ``stat`` (and some others) rely on the
676directory listing format used by the FTP server. When connecting to a
677host, ``FTPHost``'s constructor tries to guess the right format, which
678succeeds in most cases. However, if you get strange results or
679``ParserError`` exceptions by a mere ``lstat`` call, please `file a
680bug report`_.
681
682If ``lstat`` or ``stat`` give wrong modification dates or times, look
683at the methods that deal with time zone differences (`time zone
684correction`_).
685
686.. _`FTPHost.lstat`:
687
688- ``lstat(path)``
689
690  returns an object similar to that from `os.lstat`_. This is a
691  "tuple" with additional attributes; see the documentation of the
692  ``os`` module for details.
693
694  The result is derived by parsing the output of a ``LIST`` command on
695  the server. Therefore, the result from ``FTPHost.lstat`` can not
696  contain more information than the received text. In particular:
697
698  - User and group ids can only be determined as strings, not as
699    numbers, and that only if the server supplies them. This is
700    usually the case with Unix servers but maybe not for other FTP
701    server programs.
702
703  - Values for the time of the last modification may be rough,
704    depending on the information from the server. For timestamps
705    older than a year, this usually means that the precision of the
706    modification timestamp value is not better than days. For newer
707    files, the information may be accurate to a minute.
708
709  - Links can only be recognized on servers that provide this
710    information in the ``LIST`` output.
711
712  - Stat attributes that can't be determined at all are set to
713        ``None``. For example, a line of a directory listing may not
714        contain the date/time of a directory's last modification.
715
716  - There's a special problem with stat'ing the root directory.
717    (Stat'ing things *in* the root directory is fine though.) In
718    this case, a ``RootDirError`` is raised. This has to do with the
719    algorithm used by ``(l)stat``, and I know of no approach which
720    mends this problem.
721
722  Currently, ``ftputil`` recognizes the common Unix-style and
723  Microsoft/DOS-style directory formats. If you need to parse output
724  from another server type, please write to the `ftputil mailing
725  list`_. You may consider `writing your own parser`_.
726
727.. _`os.listdir`: https://docs.python.org/library/os.html#os.listdir
728.. _`os.lstat`: https://docs.python.org/library/os.html#os.lstat
729.. _`ftputil mailing list`: http://ftputil.sschwarzer.net/mailinglist
730.. _`writing your own parser`: `Writing directory parsers`_
731
732.. _`FTPHost.stat`:
733
734- ``stat(path)``
735
736  returns ``stat`` information also for files which are pointed to by a
737  link. This method follows multiple links until a regular file or
738  directory is found. If an infinite link chain is encountered or the
739  target of the last link in the chain doesn't exist, a
740  ``PermanentError`` is raised.
741
742  The limitations of the ``lstat`` method also apply to ``stat``.
743
744.. _`FTPHost.path`:
745
746``FTPHost`` objects contain an attribute named ``path``, similar to
747`os.path`_. The following methods can be applied to the remote host
748with the same semantics as for ``os.path``:
749
750::
751
752    abspath(path)
753    basename(path)
754    commonprefix(path_list)
755    dirname(path)
756    exists(path)
757    getmtime(path)
758    getsize(path)
759    isabs(path)
760    isdir(path)
761    isfile(path)
762    islink(path)
763    join(path1, path2, ...)
764    normcase(path)
765    normpath(path)
766    split(path)
767    splitdrive(path)
768    splitext(path)
769    walk(path, func, arg)
770
771Like Python's counterparts under `os.path`_, ``ftputil``'s ``is...``
772methods return ``False`` if they can't find the path given by their
773argument.
774
775Local caching of file system information
776````````````````````````````````````````
777
778Many of the above methods need access to the remote file system to
779obtain data on directories and files. To get the most recent data,
780*each* call to ``lstat``, ``stat``, ``exists``, ``getmtime`` etc.
781would require to fetch a directory listing from the server, which can
782make the program *very* slow. This effect is more pronounced for
783operations which mostly scan the file system rather than transferring
784file data.
785
786For this reason, ``ftputil`` by default saves the results from
787directory listings locally and reuses those results. This reduces
788network accesses and so speeds up the software a lot. However, since
789data is more rarely fetched from the server, the risk of obsolete data
790also increases. This will be discussed below.
791
792Caching can be controlled -- if necessary at all -- via the
793``stat_cache`` object in an ``FTPHost``'s namespace. For example,
794after calling
795
796::
797
798    host = ftputil.FTPHost(host, user, password)
799
800the cache can be accessed as ``host.stat_cache``.
801
802While ``ftputil`` usually manages the cache quite well, there are two
803possible reasons for modifying cache parameters.
804
805The first is when the number of possible entries is too low. You may
806notice that when you are processing very large directories and the
807program becomes much slower than before. It's common for code to read
808a directory with ``listdir`` and then process the found directories
809and files. This can also happen implicitly by a call to
810``FTPHost.walk``. Since version 2.6 ``ftputil`` automatically
811increases the cache size if directories with more entries than the
812current maximum cache size are to be scanned. Most of the time, this
813works fine.
814
815However, if you need access to stat data for several directories at
816the same time, you may need to increase the cache explicitly. This is
817done by the ``resize`` method::
818
819    host.stat_cache.resize(20000)
820
821where the argument is the maximum number of ``lstat`` results to store
822(the default is 5000, in versions before 2.6 it was 1000). Note that
823each path on the server, e. g. "/home/schwa/some_dir", corresponds to
824a single cache entry. Methods like ``exists`` or ``getmtime`` all
825derive their results from a previously fetched ``lstat`` result.
826
827The value 5000 above means that the cache will hold *at most* 5000
828entries (unless increased automatically by an explicit or implicit
829``listdir`` call, see above). If more are about to be stored, the
830entries which haven't been used for the longest time will be deleted
831to make place for newer entries.
832
833The second possible reason to change the cache parameters is to avoid
834stale cache data. Caching is so effective because it reduces network
835accesses. This can also be a disadvantage if the file system data on
836the remote server changes after a stat result has been retrieved; the
837client, when looking at the cached stat data, will use obsolete
838information.
839
840There are two potential ways to get such out-of-date stat data. The
841first happens when an ``FTPHost`` instance modifies a file path for
842which it has a cache entry, e. g. by calling ``remove`` or ``rmdir``.
843Such changes are handled transparently; the path will be deleted from
844the cache. A different matter are changes unknown to the ``FTPHost``
845object which inspects its cache. Obviously, for example, these are
846changes by programs running on the remote host. On the other hand,
847cache inconsistencies can also occur if two ``FTPHost`` objects change
848a file system simultaneously::
849
850    with (
851      ftputil.FTPHost(server, user1, password1) as host1,
852      ftputil.FTPHost(server, user1, password1) as host2
853    ):
854        stat_result1 = host1.stat("some_file")
855        stat_result2 = host2.stat("some_file")
856        host2.remove("some_file")
857        # `host1` will still see the obsolete cache entry!
858        print host1.stat("some_file")
859        # Will raise an exception since an `FTPHost` object
860        # knows of its own changes.
861        print host2.stat("some_file")
862
863At first sight, it may appear to be a good idea to have a shared cache
864among several ``FTPHost`` objects. After some thinking, this turns out
865to be very error-prone. For example, it won't help with different
866processes using ``ftputil``. So, if you have to deal with concurrent
867write/read accesses to a server, you have to handle them explicitly.
868
869The most useful tool for this is the ``invalidate`` method. In the
870example above, it could be used like this::
871
872    with (
873      ftputil.FTPHost(server, user1, password1) as host1,
874      ftputil.FTPHost(server, user1, password1) as host2
875    ):
876        stat_result1 = host1.stat("some_file")
877        stat_result2 = host2.stat("some_file")
878        host2.remove("some_file")
879        # Invalidate using an absolute path.
880        absolute_path = host1.path.abspath(
881                          host1.path.join(host1.getcwd(), "some_file"))
882        host1.stat_cache.invalidate(absolute_path)
883        # Will now raise an exception as it should.
884        print host1.stat("some_file")
885        # Would raise an exception since an `FTPHost` object
886        # knows of its own changes, even without `invalidate`.
887        print host2.stat("some_file")
888
889The method ``invalidate`` can be used on any *absolute* path, be it a
890directory, a file or a link.
891
892By default, the cache entries (if not replaced by newer ones) are
893stored for an infinite time. That is, if you start your Python process
894using ``ftputil`` and let it run for three days a stat call may still
895access cache data that old. To avoid this, you can set the ``max_age``
896attribute::
897
898    host = ftputil.FTPHost(server, user, password)
899    host.stat_cache.max_age = 60 * 60  # = 3600 seconds
900
901This sets the maximum age of entries in the cache to an hour. This
902means any entry older won't be retrieved from the cache but its data
903instead fetched again from the remote host and then again stored for
904up to an hour. To reset `max_age` to the default of unlimited age,
905i. e. cache entries never expire, use ``None`` as value.
906
907If you are certain that the cache will be in the way, you can disable
908and later re-enable it completely with ``disable`` and ``enable``::
909
910    host = ftputil.FTPHost(server, user, password)
911    host.stat_cache.disable()
912    ...
913    host.stat_cache.enable()
914
915During that time, the cache won't be used; all data will be fetched
916from the network. After enabling the cache, its entries will be the
917same as when the cache was disabled, that is, entries won't get
918updated with newer data during this period. Note that even when the
919cache is disabled, the file system data in the code can become
920inconsistent::
921
922    host = ftputil.FTPHost(server, user, password)
923    host.stat_cache.disable()
924    if host.path.exists("some_file"):
925        mtime = host.path.getmtime("some_file")
926
927In that case, the file ``some_file`` may have been removed by another
928process between the calls to ``exists`` and ``getmtime``!
929
930Iteration over directories
931``````````````````````````
932
933.. _`FTPHost.walk`:
934
935- ``walk(top, topdown=True, onerror=None, followlinks=False)``
936
937  iterates over a directory tree, similar to `os.walk`_. Actually,
938  ``FTPHost.walk`` uses the code from Python with just the necessary
939  modifications, so see the linked documentation.
940
941.. _`os.walk`: https://docs.python.org/2/library/os.html#os.walk
942
943.. _`FTPHost.path.walk`:
944
945- ``path.walk(path, func, arg)``
946
947  Similar to ``os.path.walk``, the ``walk`` method in
948  `FTPHost.path`_ can be used, though ``FTPHost.walk`` is probably
949  easier to use.
950
951Other methods
952`````````````
953
954- ``close()``
955
956  closes the connection to the remote host. After this, no more
957  interaction with the FTP server is possible with this ``FTPHost``
958  object. Usually you don't need to close an ``FTPHost`` instance
959  with ``close`` if you set up the instance in a ``with`` statement.
960
961- ``rename(source, target)``
962
963  renames the source file (or directory) on the FTP server.
964
965.. _`FTPHost.chmod`:
966
967- ``chmod(path, mode)``
968
969  sets the access mode (permission flags) for the given path. The mode
970  is an integer as returned for the mode by the ``stat`` and ``lstat``
971  methods. Be careful: Usually, mode values are written as octal
972  numbers, for example 0755 to make a directory readable and writable
973  for the owner, but not writable for the group and others. If you
974  want to use such octal values, rely on Python's support for them::
975
976    host.chmod("some_directory", 0o755)
977
978  Not all FTP servers support the ``chmod`` command. In case of
979  an exception, how do you know if the path doesn't exist or if
980  the command itself is invalid? If the FTP server complies with
981  `RFC 959`_, it should return a status code 502 if the ``SITE CHMOD``
982  command isn't allowed. ``ftputil`` maps this special error
983  response to a ``CommandNotImplementedError`` which is derived from
984  ``PermanentError``.
985
986  So you need to code like this::
987
988    host = ftputil.FTPHost(server, user, password)
989    try:
990        host.chmod("some_file", 0o644)
991    except ftputil.error.CommandNotImplementedError:
992        # `chmod` not supported
993        ...
994    except ftputil.error.PermanentError:
995        # Possibly a non-existent file
996        ...
997
998  Because the ``CommandNotImplementedError`` is more specific, you
999  have to test for it first.
1000
1001.. _`RFC 959`: `RFC 959 - File Transfer Protocol (FTP)`_
1002
1003- ``copyfileobj(source, target, length=64*1024)``
1004
1005  copies the contents from the file-like object ``source`` to the
1006  file-like object ``target``. The only difference to
1007  ``shutil.copyfileobj`` is the default buffer size. Note that
1008  arbitrary file-like objects can be used as arguments (e. g. local
1009  files, remote FTP files).
1010
1011  However, the interfaces of ``source`` and ``target`` have to match;
1012  the string type read from ``source`` must be an accepted string type
1013  when written to ``target``. For example, if you open ``source`` in
1014  Python 3 as a local text file and ``target`` as a remote file object
1015  in binary mode, the transfer will fail since ``source.read`` gives
1016  unicode strings whereas ``target.write`` only accepts byte strings.
1017
1018  See `File-like objects`_ for the construction and use of remote
1019  file-like objects.
1020
1021.. _`set_parser`:
1022
1023- ``set_parser(parser)``
1024
1025  sets a custom parser for FTP directories. Note that you have to pass
1026  in a parser *instance*, not the class.
1027
1028  An `extra section`_ shows how to write own parsers if the default
1029  parsers in ``ftputil`` don't work for you.
1030
1031.. _`extra section`: `Writing directory parsers`_
1032
1033.. _`keep_alive`:
1034
1035- ``keep_alive()``
1036
1037  attempts to keep the connection to the remote server active in order
1038  to prevent timeouts from happening. This method is primarily
1039  intended to keep the underlying FTP connection of an ``FTPHost``
1040  object alive while a file is uploaded or downloaded. This will
1041  require either an extra thread while the upload or download is in
1042  progress or calling ``keep_alive`` from a `callback function`_.
1043
1044  The ``keep_alive`` method won't help if the connection has already
1045  timed out. In this case, a ``ftputil.error.TemporaryError`` is raised.
1046
1047  If you want to use this method, keep in mind that FTP servers define
1048  a timeout for a reason. A timeout prevents running out of server
1049  connections because of clients that never disconnect on their own.
1050
1051  Note that the ``keep_alive`` method does *not* affect the "hidden"
1052  FTP connections established by ``FTPHost.open`` (see below). You
1053  *can't* use ``keep_alive`` to avoid a timeout in a stalling transfer
1054  like this::
1055
1056      with ftputil.FTPHost(server, userid, password) as host:
1057          fobj = host.open("some_remote_file", "rb")
1058          data = fobj.read(100)
1059          # _Futile_ attempt to avoid file connection timeout.
1060          for i in xrange(15):
1061              time.sleep(60)
1062              host.keep_alive()
1063          # Will raise an `ftputil.error.TemporaryError`.
1064          data += fobj.read()
1065
1066
1067File-like objects
1068-----------------
1069
1070Construction
1071~~~~~~~~~~~~
1072
1073Basics
1074``````
1075
1076``FTPFile`` objects are returned by a call to ``FTPHost.open``;
1077never use the ``FTPFile`` constructor directly.
1078
1079The API of remote file-like objects are is modeled after the API of
1080the io_ module in Python 3, which has also been backported to Python
10812.6 and 2.7.
1082
1083.. _io: http://docs.python.org/library/io.html
1084
1085- ``FTPHost.open(path, mode="r", buffering=None, encoding=None,
1086  errors=None, newline=None)``
1087
1088  returns a file-like object that refers to the path on the remote
1089  host. This path may be absolute or relative to the current directory
1090  on the remote host (this directory can be determined with the
1091  ``getcwd`` method). As with local file objects, the default mode is
1092  "r", i. e. reading text files. Valid modes are "r", "rb", "w", and
1093  "wb".
1094
1095  If a file is opened in binary mode, you *must not* specify an
1096  encoding. On the other hand, if you open a file in text mode, an
1097  encoding is used. By default, this is the return value of
1098  ``locale.getpreferredencoding``, but you can (and probably should)
1099  specify a distinct encoding.
1100
1101  If you open a file in binary mode, the read and write operations use
1102  byte strings (``str`` in Python 2, ``bytes`` in Python 3). That is,
1103  read operations return byte strings and write operations only accept
1104  byte strings.
1105
1106  Similarly, text files always work with unicode strings (``unicode``
1107  in Python 2, ``str`` in Python 3). Here, read operations return
1108  unicode strings and write operations only accept unicode strings.
1109
1110  The arguments ``errors`` and ``newline`` have the same semantics as
1111  in `io.open`_. The argument ``buffering`` currently is ignored.
1112  It's only there for compatibility with the ``io.open`` interface.
1113
1114.. _`io.open`: http://docs.python.org/library/io.html#io.open
1115
1116Note that the semantics of "text mode" has changed fundamentally from
1117ftputil 2.8 and earlier. Previously, "text mode" implied converting
1118newline characters to ``\r\n`` when writing remote files and
1119converting newlines to ``\n`` when reading remote files. This is
1120in line with the "text mode" notion of FTP command line clients.
1121Now, "text mode" follows the semantics in Python's ``io`` module.
1122
1123``FTPHost.open`` can also be used in a ``with`` statement::
1124
1125    import ftputil
1126
1127    with ftputil.FTPHost(...) as ftp_host:
1128        ...
1129        with ftp_host.open("new_file", "w", encoding="utf8") as fobj:
1130            fobj.write("This is some text.")
1131
1132At the end of the ``with`` block, the remote file will be closed
1133automatically.
1134
1135If something goes wrong during the construction of the file or in the
1136body of the ``with`` statement, the file will be closed as well.
1137Exceptions will be propagated as with ``try ... finally``.
1138
1139Attributes and methods
1140~~~~~~~~~~~~~~~~~~~~~~
1141
1142The methods
1143
1144::
1145
1146    close()
1147    read([count])
1148    readline([count])
1149    readlines()
1150    write(data)
1151    writelines(string_sequence)
1152
1153and the attribute ``closed`` have the same semantics as for file
1154objects of a local disk file system. The iterator protocol is
1155supported as well, i. e. you can use a loop to read a file line by
1156line::
1157
1158    with ftputil.FTPHost(...) as host:
1159        with host.open("some_file") as input_file:
1160            for line in input_file:
1161                # Do something with the line, e. g.
1162                print line.strip().replace("ftplib", "ftputil")
1163
1164For more on file objects, see the section `File objects`_ in the
1165Python Library Reference.
1166
1167.. _`file objects`: https://docs.python.org/2.7/library/stdtypes.html#file-objects
1168
1169
1170Writing directory parsers
1171-------------------------
1172
1173``ftputil`` recognizes the two most widely-used FTP directory formats,
1174Unix and MS style, and adjusts itself automatically. Almost every FTP
1175server uses one of these formats.
1176
1177However, if your server uses a format which is different from the two
1178provided by ``ftputil``, you can plug in a custom parser with a single
1179method call and have ``ftputil`` use this parser.
1180
1181For this, you need to write a parser class by inheriting from the
1182class ``Parser`` in the ``ftputil.stat`` module. Here's an example::
1183
1184    import ftputil.error
1185    import ftputil.stat
1186
1187    class XyzParser(ftputil.stat.Parser):
1188        """
1189        Parse the default format of the FTP server of the XYZ
1190        corporation.
1191        """
1192
1193        def parse_line(self, line, time_shift=0.0):
1194            """
1195            Parse a `line` from the directory listing and return a
1196            corresponding `StatResult` object. If the line can't
1197            be parsed, raise `ftputil.error.ParserError`.
1198
1199            The `time_shift` argument can be used to fine-tune the
1200            parsing of dates and times. See the class
1201            `ftputil.stat.UnixParser` for an example.
1202            """
1203            # Split the `line` argument and examine it further; if
1204            # something goes wrong, raise an `ftputil.error.ParserError`.
1205            ...
1206            # Make a `StatResult` object from the parts above.
1207            stat_result = ftputil.stat.StatResult(...)
1208            # `_st_name`, `_st_target` and `_st_mtime_precision` are optional.
1209            stat_result._st_name = ...
1210            stat_result._st_target = ...
1211            stat_result._st_mtime_precision = ...
1212            return stat_result
1213
1214        # Define `ignores_line` only if the default in the base class
1215        # doesn't do enough!
1216        def ignores_line(self, line):
1217            """
1218            Return a true value if the line should be ignored. For
1219            example, the implementation in the base class handles
1220            lines like "total 17". On the other hand, if the line
1221            should be used for stat'ing, return a false value.
1222            """
1223            is_total_line = super(XyzParser, self).ignores_line(line)
1224            my_test = ...
1225            return is_total_line or my_test
1226
1227A ``StatResult`` object is similar to the value returned by
1228`os.stat`_ and is usually built with statements like
1229
1230::
1231
1232    stat_result = StatResult(
1233                    (st_mode, st_ino, st_dev, st_nlink, st_uid,
1234                     st_gid, st_size, st_atime, st_mtime, st_ctime))
1235    stat_result._st_name = ...
1236    stat_result._st_target = ...
1237    stat_result._st_mtime_precision = ...
1238
1239with the arguments of the ``StatResult`` constructor described in
1240the following table.
1241
1242===== =================== ============ =================== =======================
1243Index Attribute           os.stat type ``StatResult`` type Notes
1244===== =================== ============ =================== =======================
12450     st_mode             int          int
12461     st_ino              long         long
12472     st_dev              long         long
12483     st_nlink            int          int
12494     st_uid              int          str                 usually only available as string
12505     st_gid              int          str                 usually only available as string
12516     st_size             long         long
12527     st_atime            int/float    float
12538     st_mtime            int/float    float
12549     st_ctime            int/float    float
1255\-    _st_name            \-           str                 file name without directory part
1256\-    _st_target          \-           str                 link target (may be absolute or relative)
1257\-    _st_mtime_precision \-           int                 ``st_mtime`` precision in seconds
1258===== =================== ============ =================== =======================
1259
1260If you can't extract all the desirable data from a line (for
1261example, the MS format doesn't contain any information about the
1262owner of a file), set the corresponding values in the ``StatResult``
1263instance to ``None``.
1264
1265Parser classes can use several helper methods which are defined in
1266the class ``Parser``:
1267
1268- ``parse_unix_mode`` parses strings like "drwxr-xr-x" and returns
1269  an appropriate ``st_mode`` integer value.
1270
1271- ``parse_unix_time`` returns a float number usable for the
1272  ``st_...time`` values by parsing arguments like "Nov"/"23"/"02:33" or
1273  "May"/"26"/"2005". Note that the method expects the timestamp string
1274  already split at whitespace.
1275
1276- ``parse_ms_time`` parses arguments like "10-23-01"/"03:25PM" and
1277  returns a float number like from ``time.mktime``. Note that the
1278  method expects the timestamp string already split at whitespace.
1279
1280Additionally, there's an attribute ``_month_numbers`` which maps
1281lowercase three-letter month abbreviations to integers.
1282
1283For more details, see the two "standard" parsers ``UnixParser`` and
1284``MSParser`` in the module ``ftputil/stat.py``.
1285
1286To actually *use* the parser, call the method `set_parser`_ of the
1287``FTPHost`` instance.
1288
1289If you can't write a parser or don't want to, please ask on the
1290`ftputil mailing list`_. Possibly someone has already written a parser
1291for your server or can help with it.
1292
1293
1294FAQ / Tips and tricks
1295---------------------
1296
1297Where can I get the latest version?
1298~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1299
1300See the `download page`_. Announcements will be sent to the `mailing
1301list`_. Announcements on major updates will also be posted to the
1302newsgroup `comp.lang.python.announce`_ .
1303
1304.. _`download page`: http://ftputil.sschwarzer.net/download
1305.. _`mailing list`: http://ftputil.sschwarzer.net/mailinglist
1306.. _`comp.lang.python.announce`: news:comp.lang.python.announce
1307
1308Is there a mailing list on ``ftputil``?
1309~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1310
1311Yes, please visit http://ftputil.sschwarzer.net/mailinglist to
1312subscribe or read the archives.
1313
1314Though you can *technically* post without subscribing first I can't
1315recommend it: The mails from non-subscribers have to be approved by
1316me and because the arriving mails contain *lots* of spam, I rarely go
1317through these mails.
1318
1319I found a bug! What now?
1320~~~~~~~~~~~~~~~~~~~~~~~~
1321
1322Before reporting a bug, make sure that you already read this manual
1323and tried the `latest version`_ of ``ftputil``. There the bug might
1324have already been fixed.
1325
1326.. _`latest version`: http://ftputil.sschwarzer.net/download
1327
1328Please see http://ftputil.sschwarzer.net/issuetrackernotes for
1329guidelines on entering a bug in ``ftputil``'s ticket system. If you
1330are unsure if the behaviour you found is a bug or not, you should write
1331to the `ftputil mailing list`_. In *either* case you *must not*
1332include confidential information (user id, password, file names, etc.)
1333in the problem report! Be careful!
1334
1335Does ``ftputil`` support SSL/TLS?
1336~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1337
1338``ftputil`` has no *built-in* SSL/TLS support.
1339
1340On the other hand, there are two ways to get TLS support with
1341ftputil:
1342
1343- In Python 2.7 and Python 3.2 and up, the ``ftplib`` library has a
1344  class ``FTP_TLS`` that you can use for the ``session_factory``
1345  keyword argument in the ``FTPHost`` constructor. You can't use the
1346  class directly though if you need additional setup code in
1347  comparison to ``ftplib.FTP``, for example calling ``prot_p``, to
1348  secure the data connection. On the other hand,
1349  `ftputil.session.session_factory`_ can be used to create a custom
1350  session factory.
1351 
1352  If you have other requirements that ``session_factory`` can't
1353  fulfill, you may create your own session factory by inheriting from
1354  ``ftplib.FTP_TLS``::
1355
1356    import ftplib
1357
1358    import ftputil
1359
1360
1361    class FTPTLSSession(ftplib.FTP_TLS):
1362
1363        def __init__(self, host, user, password):
1364            ftplib.FTP_TLS.__init__(self)
1365            self.connect(host, port)
1366            self.login(user, password)
1367            # Set up encrypted data connection.
1368            self.prot_p()
1369            ...
1370
1371    # Note the `session_factory` parameter. Pass the class, not
1372    # an instance.
1373    with ftputil.FTPHost(host, user, password,
1374                         session_factory=FTPTLSSession) as ftp_host:
1375        # Use `ftp_host` as usual.
1376
1377.. _`ftputil.session.session_factory`: `Session factories`_
1378
1379- If you need to work with Python 2.6, you can use the
1380  ``ftpslib.FTP_TLS`` class from the M2Crypto_ project. Again, you
1381  can't use the class directly but need to use
1382  ``ftputil.session.session_factory`` or a recipe similar to that
1383  above.
1384
1385  Unfortunately, ``M2Crypto.ftpslib.FTP_TLS`` (at least in version
1386  0.22.3) doesn't work correctly if you pass unicode strings to its
1387  methods. Since ``ftputil`` does exactly that at some point (even if
1388  you used byte strings in ``ftputil`` calls) you need a workaround in
1389  the session factory class::
1390
1391    import M2Crypto
1392
1393    import ftputil
1394    import ftputil.tool
1395
1396
1397    class M2CryptoSession(M2Crypto.ftpslib.FTP_TLS):
1398
1399        def __init__(self, host, user, password):
1400            M2Crypto.ftpslib.FTP_TLS.__init__(self)
1401            # Change the port number if needed.
1402            self.connect(host, 21)
1403            self.auth_tls()
1404            self.login(user, password)
1405            self.prot_p()
1406            self._fix_socket()
1407            ...
1408
1409        def _fix_socket(self):
1410            """
1411            Change the socket object so that arguments to `sendall`
1412            are converted to byte strings before being used.
1413            """
1414            original_sendall = self.sock.sendall
1415            # Bound method, therefore no `self` argument.
1416            def sendall(data):
1417                data = ftputil.tool.as_bytes(data)
1418                return original_sendall(data)
1419            self.sock.sendall = sendall
1420
1421    # Note the `session_factory` parameter. Pass the class, not
1422    # an instance.
1423    with ftputil.FTPHost(host, user, password,
1424                         session_factory=M2CryptoSession) as ftp_host:
1425        # Use `ftp_host` as usual.
1426
1427  That said, ``session_factory`` has this workaround built in, so
1428  normally you don't need to define the session factory yourself!
1429
1430.. _M2Crypto: https://github.com/martinpaljak/M2Crypto
1431
1432How do I connect to a non-default port?
1433~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1434
1435By default, an instantiated ``FTPHost`` object connects on the usual
1436FTP port. If you have to use a different port, refer to the section
1437`Session factories`_.
1438
1439How do I set active or passive mode?
1440~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1441
1442Please see the section `Session factories`_.
1443
1444How can I debug an FTP connection problem?
1445~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1446
1447You can do this with a session factory. See `Session factories`_.
1448
1449If you want to change the debug level only temporarily after the
1450connection is established, you can reach the `session object`_ as the
1451``_session`` attribute of the ``FTPHost`` instance and call
1452``_session.set_debuglevel``. Note that the ``_session`` attribute
1453should *only* be accessed for debugging. Calling arbitrary
1454``ftplib.FTP`` methods on the session object may *cause* bugs!
1455
1456.. _`session object`: `Session factories`_
1457
1458Conditional upload/download to/from a server in a different time zone
1459~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1460
1461You may find that ``ftputil`` uploads or downloads files
1462unnecessarily, or not when it should. This can happen when the FTP
1463server is in a different time zone than the client on which
1464``ftputil`` runs. Please see the section on `time zone correction`_.
1465It may even be sufficient to call `synchronize_times`_.
1466
1467When I use ``ftputil``, all I get is a ``ParserError`` exception
1468~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1469
1470The FTP server you connect to may use a directory format that
1471``ftputil`` doesn't understand. You can either write and
1472`plug in an own parser`_ or ask on the `mailing list`_ for
1473help.
1474
1475.. _`plug in an own parser`: `Writing directory parsers`_
1476
1477``isdir``, ``isfile`` or ``islink`` incorrectly return ``False``
1478~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1479
1480Like Python's counterparts under `os.path`_, ``ftputil``'s methods
1481return ``False`` if they can't find the given path.
1482
1483Probably you used ``listdir`` on a directory and called ``is...()`` on
1484the returned names. But if the argument for ``listdir`` wasn't the
1485current directory, the paths won't be found and so all ``is...()``
1486variants will return ``False``.
1487
1488I don't find an answer to my problem in this document
1489~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1490
1491Please send an email with your problem report or question to the
1492`ftputil mailing list`_, and we'll see what we can do for you. :-)
1493
1494
1495Bugs and limitations
1496--------------------
1497
1498- ``ftputil`` needs at least Python 2.6 to work.
1499
1500- Whether ``ftputil`` "sees" "hidden" directory and file names (i. e.
1501  names starting with a dot) depends on the configuration of the FTP
1502  server. See `Hidden files and directories`_ for details.
1503
1504- Due to the implementation of ``lstat`` it can not return a sensible
1505  value for the root directory ``/`` though stat'ing entries *in* the
1506  root directory isn't a problem. If you know an implementation that
1507  can do this, please let me know. The root directory is handled
1508  appropriately in ``FTPHost.path.exists/isfile/isdir/islink``, though.
1509
1510- In multithreaded programs, you can have each thread use one or more
1511  ``FTPHost`` instances as long as no instance is shared with other
1512  threads.
1513
1514- Currently, it is not possible to continue an interrupted upload or
1515  download. Contact me if this causes problems for you.
1516
1517- There's exactly one cache for ``lstat`` results for each ``FTPHost``
1518  object, i. e. there's no sharing of cache results determined by
1519  several ``FTPHost`` objects. See `Local caching of file system
1520  information`_ for the reasons.
1521
1522
1523Files
1524-----
1525
1526If not overwritten via installation options, the ``ftputil`` files
1527reside in the ``ftputil`` package. There's also documentation in
1528`reStructuredText`_ and in HTML format. The locations of these
1529files after installation is system-dependent.
1530
1531.. _`reStructuredText`: http://docutils.sourceforge.net/rst.html
1532
1533The files ``test_*.py`` and ``mock_ftplib.py`` are for unit-testing.
1534If you only *use* ``ftputil``, i. e. *don't* modify it, you can
1535delete these files.
1536
1537
1538References
1539----------
1540
1541- Mackinnon T, Freeman S, Craig P. 2000. `Endo-Testing:
1542  Unit Testing with Mock Objects`_.
1543
1544- Postel J, Reynolds J. 1985. `RFC 959 - File Transfer Protocol (FTP)`_.
1545
1546- Van Rossum G et al. 2013. `Python Library Reference`_.
1547
1548.. _`Endo-Testing: Unit Testing with Mock Objects`:
1549   http://www.connextra.com/aboutUs/mockobjects.pdf
1550.. _`RFC 959 - File Transfer Protocol (FTP)`: http://www.ietf.org/rfc/rfc959.txt
1551.. _`Python Library Reference`: https://docs.python.org/library/index.html
1552
1553
1554Authors
1555-------
1556
1557``ftputil`` is written by Stefan Schwarzer
1558<sschwarzer@sschwarzer.net>, in part based on suggestions
1559from users.
1560
1561The original ``lrucache`` module was written by Evan Prodromou
1562<evan@prodromou.name>.
1563
1564Feedback is appreciated. :-)
Note: See TracBrowser for help on using the repository browser.