[Qt-interest] issues with QNetworkDiskCache and large number of cached files

[Crni Gorac]

We are using QNetworkDiskCache in an application with HTML/JavaScript
user interface, displayed in QWebView based widgets.  The purpose of
using QNetworkDiskCache is, of course, to speed-up application user
interface; the application is of type so that large number of small
images (like avatars etc.) appear to be used, so the cache directory
on disk get quickly filled with thousands of small images.  We've
noticed that this is affecting CPU usage, specifically on Windows -
whenever some operation with the cache involved, disk activity seem to
be shortly blocking main thread.  I remember facing alike problem,
albeit with FAT file-system, on another project previously, and the
approach that worked there was to split the given directory into
number of subdirectories, and then to split these into another level
of subdirectories, so that at the end only dozen or so files end up in
each directory.  So I've looked into maybe sub-classing
QNetworkDiskCache in order to implement such a scheme, but it appears
this is not workable; I've also looked into just copying QNetworkCache
implementation files into our project and then making needed changes,
but this is not so easy either (pimpl idiom used, then
QNetworkDiskCache private implementation class deriving from
QAbstractNetworkCache private implementation class).  So I'm wondering
- has anyone faced alike problem, or approached re-implementing
QNetworkDiskCache?  Any suggestions here?  Or maybe it would be better
to solve the problem some other way (I was thinking about somehow
separating disk cache operations in dedicated thread, in order to keep
main thread responsive, but it's not clear to me how that would be
possible)?

Thanks.