FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection

[ftputiluser]

Since FTPHost defines a __del__ method and the ftp_path._Path instances have a reference to the FTPHost, Python's circular reference garbage collection can't cleanup. See ​http://docs.python.org/reference/datamodel.html#object.__del__ for details.

This could represent a significant memory leak for anyone using ftputil in a long running app.

[schwa]

Fixed description to work with Trac wiki formatting.

[schwa]

Thank you for the bug report!

Replying to ftputiluser:

Since FTPHost defines a __del__ method and the ftp_path._Path instances have a reference to the FTPHost, Python's circular reference garbage collection can't cleanup. See ​http://docs.python.org/reference/datamodel.html#object.__del__ for details.

As far as I understand the documentation, uncollectable cycles occur only if several objects in the cycle have a __del__ method and thus the garbage collector can't decide which one to delete first. As _Path doesn't have a __del__ method, there are cycles, but they can be collected.

This could represent a significant memory leak for anyone using ftputil in a long running app.

I guess even most long-running apps generate one or a few FTPHost instances and use them over and over. Of course, if you instantiated many FTPHost objects, non-collectable cycles would lead to a memory leak.

As it happens, when I wrote a unit test for this bug, I found there is an uncollectable cycle regarding files and/or sockets (even though I had considered mutual references between FTPHost and FTPFile objects). I'll look into this.

[schwa]

I managed to fix the problem in changeset [dd9d9c95c7b6].

It seems there actually was a problem with the references held by _Path and _Stat but I was unable to find out why the garbage collection didn't work. In the end, I removed the __del__ method from FTPHost because I think that's a cleaner solution than trying to break various cycles explicitly.