~sschwarzer/ftputil#75: 
Add support for Pickle

Attached to ticket find slight code revision to support pickling of the FTPHost object via __getstate__, __setstate__, and de-nesting of lrucache objects.

Nesting in python isn't particularly useful in this case, and the __getstate__/__setstate__ combo is required to prevent pickle from attempting to serialize the socket object in ftplib, which is an attribute of the ftputil.FTPHost object.

(Note from the ftputil maintainer: The socket object is no direct attribute of the FTPHost instance but is held by the _session attribute, usually an instance of ftplib.FTP.)

Status
RESOLVED WONT_FIX
Submitter
ftputiluser (unverified)
Assigned to
No-one
Submitted
9 years ago
Updated
9 years ago
Labels
enhancement library

schwa (unverified) 9 years ago · edit

Hi Crypt0s,

Thanks for the report. I can't find the attached patch though.

lrucache uses a nested class because it wasn't written by me and the original author used the nested class. I tried to keep the code close to its original state to make it easier to give my changes back to the original project.

I have my doubts if it makes sense to pickle and unpickle FTPHost instances. Yes, you could do something that's kind of sensible: When unpickling, re-login and change back to the last current directory.

I see a few problems though:

  • To re-connect from the pickle file, you'd need the username and password, and I would understand if someone was mad at me if I didn't just use their password in memory but also wrote it to a file. ;-)
  • The pickle file might be unpickled in a completely different environment. It might not be possible to connect to the host from there.
  • The state of the FTP server might have changed between pickling and unpickling. For example, your last current directory might no longer exist. Of course, this can also happen during a session, but the more time passes between two logins (before pickling and when unpickling), the more likely a change on the remote side is.
  • It gets much more complicated if there are open remote files when the FTPHost instance should be pickled.

Because of all these problems, I'd rather officially not support pickling of FTPHost and FTPFile instances. By the way, Python's regular files can't be pickled either:

>>> f = open("/etc/passwd")
>>> import pickle
>>> pickle.dumps(f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib64/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib64/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/usr/lib64/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle file objects

schwa (unverified) 9 years ago · edit

Changeset 51e52f45d5284bd8532eb3745bfaa6df6afc9f73 now deliberately refuses to pickle FTPHost and FTPFile objects.

schwa (unverified) 9 years ago · edit

Resolution "wontfix" makes more sense than "fixed" because I won't add pickling as requested in the ticket.

"Fixed" was supposed to mean that I made a code change to make it explicit that pickling isn't supported.

Register here or Log in to comment, or comment via email.