Opened 5 years ago

Closed 5 years ago

#75 closed enhancement (wontfix)

Add support for Pickle

Reported by: ftputiluser Owned by: schwa
Priority: minor Milestone: 3.1
Component: Library Version: 3.0
Keywords: pickle, unpickle Cc:

Description (last modified by schwa)

Attached to ticket find slight code revision to support pickling of the FTPHost object via __getstate__, __setstate__, and de-nesting of lrucache objects.

Nesting in python isn't particularly useful in this case, and the __getstate__/__setstate__ combo is required to prevent pickle from attempting to serialize the socket object in ftplib, which is an attribute of the ftputil.FTPHost object.

(Note from the ftputil maintainer: The socket object is no direct attribute of the FTPHost instance but is held by the _session attribute, usually an instance of ftplib.FTP.)

Change History (7)

comment:1 Changed 5 years ago by schwa

Keywords: pickle unpickle added; Pickle removed
Owner: changed from Crypt0s to schwa
Status: newassigned

Hi Crypt0s,

Thanks for the report. I can't find the attached patch though.

lrucache uses a nested class because it wasn't written by me and the original author used the nested class. I tried to keep the code close to its original state to make it easier to give my changes back to the original project.

I have my doubts if it makes sense to pickle and unpickle FTPHost instances. Yes, you could do something that's kind of sensible: When unpickling, re-login and change back to the last current directory.

I see a few problems though:

  • To re-connect from the pickle file, you'd need the username and password, and I would understand if someone was mad at me if I didn't just use their password in memory but also wrote it to a file. ;-)
  • The pickle file might be unpickled in a completely different environment. It might not be possible to connect to the host from there.
  • The state of the FTP server might have changed between pickling and unpickling. For example, your last current directory might no longer exist. Of course, this can also happen during a session, but the more time passes between two logins (before pickling and when unpickling), the more likely a change on the remote side is.
  • It gets much more complicated if there are open remote files when the FTPHost instance should be pickled.

Because of all these problems, I'd rather officially not support pickling of FTPHost and FTPFile instances. By the way, Python's regular files can't be pickled either:

>>> f = open("/etc/passwd")
>>> import pickle
>>> pickle.dumps(f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib64/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib64/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/usr/lib64/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle file objects

comment:2 Changed 5 years ago by schwa

Description: modified (diff)

comment:3 Changed 5 years ago by schwa

Description: modified (diff)

comment:4 Changed 5 years ago by schwa

Description: modified (diff)

comment:5 Changed 5 years ago by schwa

Resolution: fixed
Status: assignedclosed

Changeset [600fb1c435a9] now deliberately refuses to pickle FTPHost and FTPFile objects.

comment:6 Changed 5 years ago by schwa

Resolution: fixed
Status: closedreopened

comment:7 Changed 5 years ago by schwa

Resolution: wontfix
Status: reopenedclosed

Resolution "wontfix" makes more sense than "fixed" because I won't add pickling as requested in the ticket.

"Fixed" was supposed to mean that I made a code change to make it explicit that pickling isn't supported.

Note: See TracTickets for help on using tickets.