Changeset 1671:2a3350ab9374

Timestamp:

Oct 8, 2017, 8:00:38 PM (17 months ago)

Author:

Stefan Schwarzer <sschwarzer@…>

Branch:

default

Message:

Handle socket read timeout in `_available_child`

When `FTPFile.close` causes a socket read timeout, the _next_ read on
the socket will raise an `OSError`. Hence, the `OSError` can happen in
the `host._session.pwd` call in `FTPHost._available_child`. Catch this
`OSError`.

Although the fix in `ftputil.host.FTPHost._available_child` will make
the child session handling work, socket read errors could result in
child sessions being returned in a non-reproducible way. To make the
identity checks in the tests reproducible, use the small `CONTENTS`
file instead of the larger `debian-keyring.tar.gz` file. Investigation
of ticket #112 showed that socket read errors would rather occur with
larger files.

This change in the tests is slightly hackish, but should be ok, unless
it'll cause other problems later on.

ticket: 112

Files:

• ftputil/host.py (modified) (1 diff)
• test/test_real_ftp.py (modified) (4 diffs)

ftputil/host.py

@@ -175 +175 @@
                 # when `_available_child` is called the next time.
                 except ftplib.error_reply:
+                    continue
+                # Under high load, there may be a socket read timeout
+                # during the last FTP file `close` (see ticket #112).
+                except OSError:
                     continue
                 else:

test/test_real_ftp.py

@@ -675 +675 @@
 
     def test_only_closed_children(self):
-        REMOTE_FILE_NAME = "debian-keyring.tar.gz"
+        REMOTE_FILE_NAME = "CONTENTS"
         host = self.host
         with host.open(REMOTE_FILE_NAME, "rb") as file_obj1:
@@ -688 +688 @@
 
     def test_no_timed_out_children(self):
-        REMOTE_FILE_NAME = "debian-keyring.tar.gz"
+        REMOTE_FILE_NAME = "CONTENTS"
         host = self.host
         # Implicitly create child host object.
@@ -706 +706 @@
 
     def test_no_delayed_226_children(self):
-        REMOTE_FILE_NAME = "debian-keyring.tar.gz"
+        REMOTE_FILE_NAME = "CONTENTS"
         host = self.host
         # Implicitly create child host object.
@@ -874 +874 @@
     def test_subsequent_reading(self):
         # Open a file for reading.
-        with self.host.open("debian-keyring.tar.gz", "rb") as file1:
+        with self.host.open("CONTENTS", "rb") as file1:
             pass
         # Make sure that there are no problems if the connection is reused.
-        with self.host.open("debian-keyring.tar.gz", "rb") as file2:
+        with self.host.open("CONTENTS", "rb") as file2:
             pass
         assert file1._session is file2._session