[Development] qsizetype and classes working with QStrings or QList

[Lars Knoll]

On 27 Aug 2020, at 08:32, Lars Knoll <lars.knoll at qt.io<mailto:lars.knoll at qt.io>> wrote:

On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development <development at qt-project.org<mailto:development at qt-project.org>> wrote:

Hi,

Il 27/08/20 02:46, Thiago Macieira ha scritto:
A QListView of 2 billion lines with where each line is a QString one to 7
characters in length would be 2G * (24 + 32) = 96 GB of memory use.
QListWidget's overhead is much worse.

This isn't accurate; QListView (with the default delegate) doesn't cache data, and only fetches and shows what's visible in its viewport. So the actual consumption is pretty much constant no matter how big is the underlying model.

Correct. But the argument for QTextDocument still holds. If you have such huge files, I believe you want to load them using some custom editing component that mmaps the file and only loads the relevant parts on the fly. QTextDocument allows for rich text editing and requires to hold the content in memory in a QString because of that.

QTextDocument should be able to handle very large regular documents this without larger issues. Just as a comparison: All Harry Potter books together are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 times larger document) into a QTextDocument should work without issues.

But a 5G set of machine generated data? I would believe you want a special class handling that and loading data on the fly, so you have some memory left for the rest of your system.

Having said that, I don’t mind changing our API to use qsizetype for QTextDocument, I just don’t believe you will find the class to be very snappy and usable with such large documents. The piece table itself should be able to handle it if you have enough RAM, but the default layout engine probably can’t (you could try using your own custom layout though).

I’ll give it a try and let’s see. There are very few to no virtual functions that would be affected, so changing the classes to use qsizetype might be worthwhile.

Cheers,
Lars

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20200827/d95711fa/attachment-0001.html>