Opened 6 months ago

Closed 6 months ago

Last modified 6 months ago

#145 closed defect (fixed)

ftputil 5.0.0: kwargs not being passed through to ftplib.FTP

Reported by: ftputiluser Owned by: schwa
Priority: major Milestone: 5.0.1
Component: Library Version: 5.0.0
Keywords: Cc:

Description

We specify timeout as a kwarg to the ftputil.FTPHost constructor for passthrough to ftplib.FTP, which worked through ftputil 4.0.0. However, in 5.0.0 the constructor of ftputil.session.Session does not support the other kwargs passed to ftplib.FTP-- it only supports host, user and password. As a result any code relying on the acct, timeout, or source_address kwargs breaks. I believe the solution is just to add **kwargs to the Session.__init__ method and pass them through the super.__init__ method.

Stack trace:

In [11]: host = ftputil.FTPHost(creds['host'], creds['user'], creds['pwd'], timeout=10)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-9308d6c2abeb> in <module>
----> 1 host = ftputil.FTPHost(creds['host'], creds['user'], creds['pwd'], timeout=10)

C:\Miniconda3\envs\test_feb_2021\lib\site-packages\ftputil\host.py in __init__(self, *args, **kwargs)
     82         # The time shift setting shouldn't be reset though. Make a session
     83         # according to these arguments.
---> 84         self._session = self._make_session()
     85         # Simulate `os.path`.
     86         self.path = ftputil.path._Path(self)

C:\Miniconda3\envs\test_feb_2021\lib\site-packages\ftputil\host.py in _make_session(self)
    149                     f"session instance {session!r} must have an `encoding` attribute"
    150                 )
--> 151             self._encoding = session.encoding
    152         return session
    153

C:\Miniconda3\envs\test_feb_2021\lib\site-packages\ftputil\host.py in _make_session(self)
    144         factory = kwargs.pop("session_factory", default_session_factory)
    145         with ftputil.error.ftplib_error_to_ftp_os_error:
--> 146             session = factory(*args, **kwargs)
    147             if not hasattr(session, "encoding"):
    148                 raise ftputil.error.NoEncodingError(

TypeError: __init__() got an unexpected keyword argument 'timeout'

Attachments (1)

host.py (42.0 KB) - added by schwa 6 months ago.
host.py with fix for ticket #145

Download all attachments as: .zip

Change History (11)

comment:1 Changed 6 months ago by ftputiluser

Version: 5.0.0

comment:2 Changed 6 months ago by schwa

Status: newassigned

Thanks for your report!

It seems you're right. ftputil 4.0.0 passed the *args and **kwargs from FTPHost.__init__ to the session factory. ftputil 5.0.0 replaces the default factory ftplib.FTP with a factory created by ftputil.session.session_factory, which doesn't understand "arbitrary" positional and keyword arguments.

I need to think a bit more about how best to resolve this. For example, what should happen if you pass an encoding to session_factory and later pass encoding also to FTPHost.__init__? For the other arguments and how they're used in the class returned by session_factory there could also be tricky interactions. I guess that's a reason why the class returned by session_factory doesn't accept arbitrary positional or keyword arguments, i. e. to avoid handling such edge cases.

comment:3 Changed 6 months ago by schwa

As a workaround for the time being, you can use FTPHost(..., session_factory=ftplib.FTP) (if you don't mind the default path encoding UTF-8).

comment:4 in reply to:  3 Changed 6 months ago by schwa

Replying to schwa:

As a workaround for the time being, you can use FTPHost(..., session_factory=ftplib.FTP) (if you don't mind the default path encoding UTF-8).

To be more precise, the default path encoding for ftplib.FTP is latin-1 under Python ≤3.8 and UTF-8 under Python ≥3.9.

comment:5 Changed 6 months ago by schwa

Milestone: 5.0.1

comment:6 Changed 6 months ago by schwa

Would you please replace ftputil/host.py with the attached file and give feedback if it works for you? Ideally, you would run it under Python 3.8 (or earlier) and Python 3.9. (The fix for Python 3.8 and earlier is relatively simple; the fix for Python 3.9 is more complicated.)

Last edited 6 months ago by schwa (previous) (diff)

Changed 6 months ago by schwa

Attachment: host.py added

host.py with fix for ticket #145

comment:7 Changed 6 months ago by ftputiluser

The patch seems to be working. Python 3.7 is our bread-and-butter installation, I also created a conda env for Python 3.9, and both work with the patch (and fail without), though our FTP targets all use exclusively ASCII so I can't verify the Latin-1 vs UTF-8 aspect. I verified that with the patch, timeout on the ftplib.FTP object is as set if specified, and ftplib._GLOBAL_DEFAULT_TIMEOUT if not, and connection is successful. Under Python 3.7 I was able to do an end-to-end successfully transfering data with our standard process with the patch.

comment:8 Changed 6 months ago by schwa

Many thanks for the thorough test! Nice. :-)

I'll make a new release during the next few days. If it's very important for you, I can try to make the release today.

comment:9 Changed 6 months ago by schwa

Resolution: fixed
Status: assignedclosed

Fixed as of [b5f89833bc4d].

comment:10 in reply to:  8 Changed 6 months ago by schwa

Replying to schwa:

I'll make a new release during the next few days. If it's very important for you, I can try to make the release today.

Too late, already released. ;-)

Note: See TracTickets for help on using tickets.