(For some background on stat caching read the documentation.)

The default stat cache size is 1000 entries. Though this is enough for most purposes, people from time to time run into the problem of directories with more items than the default cache size. In this case, old entries (further towards the top of the directory) are flushed out to make place for newer stat items. Then, by the time the directory is iterated over, the older entries are gone, so the directory listing is fetched again for each stat call (isdir, getmtime etc.).

I've been thinking about just increasing the default stat cache size, but of course this costs memory which possibly isn't necessary. Maybe an auto-resize feature is a better approach.