[Development] qsizetype and classes working with QStrings or QList

[Thiago Macieira]

On Wednesday, 26 August 2020 17:23:16 PDT Lisandro Damián Nicanor Pérez Meyer 
wrote:
> > Disagree here.  There is good reason many in my industry (Electronic
> > design automation) use the Scintilla editor widget inside Qt apps.
> > Specifically to handle extremely large, in both line count and column
> > count, files.
> > 
> > Many of the text files we work with, are generated library definitions for
> > silicon definitions.  A 5+ gb file is not uncommon.  Or files with a truth
> > table definition with 5000-1000 columns.  I kid you not.
> > 
> > Its one thing to not handle it, and say "too big to open" its another to
> > attempt to and crash because the internal integers cant handle it.
> > 
> > Scintilla handles these massive files relatively easily.
> 
> Yes, this files terms to be huge and are a valid case after all.

Not disagreeing that huge files are valid.

The question is whether the API needs to directly map to such huge files. When 
your data set becomes very big, the layer above those display classes should 
consider tiling or windowing the data.

Let's take a simplest case: a 2 billion character QPlainTextEdit. The text 
alone is 4 GB in memory. QPlainTextEdit (through QTextDocument) will also 
parse all the lines in the text and create blocks for them, whether they're 
visible or not. So not only will this require a lot of memory, it will also be 
very slow.

A QListView of 2 billion lines with where each line is a QString one to 7 
characters in length would be 2G * (24 + 32) = 96 GB of memory use. 
QListWidget's overhead is much worse.

So, I don't dispute the need to *index* very large data sets. I do dispute the 
need for a single Qt graphical class to keep the entire data set parsed, in 
memory.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel DPG Cloud Engineering