[Development] qsizetype and classes working with QStrings or QList
lars.knoll at qt.io
Thu Aug 27 11:33:41 CEST 2020
On 27 Aug 2020, at 10:45, Lars Knoll <lars.knoll at qt.io<mailto:lars.knoll at qt.io>> wrote:
On 27 Aug 2020, at 08:32, Lars Knoll <lars.knoll at qt.io<mailto:lars.knoll at qt.io>> wrote:
On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development <development at qt-project.org<mailto:development at qt-project.org>> wrote:
Il 27/08/20 02:46, Thiago Macieira ha scritto:
A QListView of 2 billion lines with where each line is a QString one to 7
characters in length would be 2G * (24 + 32) = 96 GB of memory use.
QListWidget's overhead is much worse.
This isn't accurate; QListView (with the default delegate) doesn't cache data, and only fetches and shows what's visible in its viewport. So the actual consumption is pretty much constant no matter how big is the underlying model.
Correct. But the argument for QTextDocument still holds. If you have such huge files, I believe you want to load them using some custom editing component that mmaps the file and only loads the relevant parts on the fly. QTextDocument allows for rich text editing and requires to hold the content in memory in a QString because of that.
QTextDocument should be able to handle very large regular documents this without larger issues. Just as a comparison: All Harry Potter books together are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 times larger document) into a QTextDocument should work without issues.
But a 5G set of machine generated data? I would believe you want a special class handling that and loading data on the fly, so you have some memory left for the rest of your system.
Having said that, I don’t mind changing our API to use qsizetype for QTextDocument, I just don’t believe you will find the class to be very snappy and usable with such large documents. The piece table itself should be able to handle it if you have enough RAM, but the default layout engine probably can’t (you could try using your own custom layout though).
I’ll give it a try and let’s see. There are very few to no virtual functions that would be affected, so changing the classes to use qsizetype might be worthwhile.
Did a quick trial, and I don’t think it’s worth it. It’s quite a lot of work and will break user code in some places for very little gain. Little gain not because there aren’t use cases that need to handle very large documents, but because QTextEdit and QTextDocument are probably not the right classes to do so given that they load everything into memory at once.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Development