Opened 8 years ago

Closed 8 years ago

#56 closed defect (fixed)

FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection

Reported by: ftputiluser Owned by: schwa
Priority: major Milestone: 2.6
Component: Library Version: 2.5
Keywords: Cc:

Description (last modified by schwa)

Since FTPHost defines a __del__ method and the ftp_path._Path instances have a reference to the FTPHost, Python's circular reference garbage collection can't cleanup. See http://docs.python.org/reference/datamodel.html#object.__del__ for details.

This could represent a significant memory leak for anyone using ftputil in a long running app.

Change History (5)

comment:1 Changed 8 years ago by schwa

  • Milestone set to 2.6
  • Status changed from new to assigned

comment:2 Changed 8 years ago by schwa

  • Description modified (diff)
  • Summary changed from FTPHost.__del__ and ftp_path._Path._host wreak havoc with Python's circular garbage collection to FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection

Fixed description to work with Trac wiki formatting.

comment:3 in reply to: ↑ description Changed 8 years ago by schwa

Thank you for the bug report!

Replying to ftputiluser:

Since FTPHost defines a __del__ method and the ftp_path._Path instances have a reference to the FTPHost, Python's circular reference garbage collection can't cleanup. See http://docs.python.org/reference/datamodel.html#object.__del__ for details.

As far as I understand the documentation, uncollectable cycles occur only if several objects in the cycle have a __del__ method and thus the garbage collector can't decide which one to delete first. As _Path doesn't have a __del__ method, there are cycles, but they can be collected.

This could represent a significant memory leak for anyone using ftputil in a long running app.

I guess even most long-running apps generate one or a few FTPHost instances and use them over and over. Of course, if you instantiated many FTPHost objects, non-collectable cycles would lead to a memory leak.

As it happens, when I wrote a unit test for this bug, I found there is an uncollectable cycle regarding files and/or sockets (even though I had considered mutual references between FTPHost and FTPFile objects). I'll look into this.

comment:4 Changed 8 years ago by schwa

I managed to fix the problem in changeset [dd9d9c95c7b6].

It seems there actually was a problem with the references held by _Path and _Stat but I was unable to find out why the garbage collection didn't work. In the end, I removed the __del__ method from FTPHost because I think that's a cleaner solution than trying to break various cycles explicitly.

comment:5 Changed 8 years ago by schwa

  • Resolution set to fixed
  • Status changed from assigned to closed
Note: See TracTickets for help on using tickets.