[Interest] Small survey on necessary Qt Container size

[Roland Hughes]

On 9/28/22 05:00, Ulf Hermann wrote:
> I think Andre's point is that you should not use Qt containers for such
> large amounts of data, but rather some other data structure better
> suited for your case (most trivially, std::vector instead of QList).

My point was you don't get a choice given the single-thread-i-ness of Qt 
and trying to stuff everything in the main event loop.

https://doc.qt.io/qt-5/qtablewidget.html

int *currentRow <https://doc.qt.io/qt-5/qtablewidget.html#currentRow>*() 
const

https://doc.qt.io/qt-5/qsqltablemodel.html

virtual int *rowCount 
<https://doc.qt.io/qt-5/qsqltablemodel.html#rowCount>*(const QModelIndex 
&/parent/ = QModelIndex()) const override

When your database has segments and each segment can have a ROWID 0 - 
ULLONG_MAX, honestly, even ULONG_MAX gets you in trouble when you have 
multiple segments in your return query.

Somewhere in all of that code is a "container." Not every database 
engine will allow one to move bidirectionally through a cursor. If you 
want to support a user being able to drag the scrollbar all the way to 
the bottom (or just clicking there) then going back to the first row, 
the data is either loaded into a container or the cursor must have 
bidirectional capabilities.

I could be wrong, but I believe part of Quentin's point of stuffing 
everything into RAM is the fact sliding cursor windows aren't 
implemented and many database engines don't support bidirectional 
movement. Leaving data in a cursor allows the OS and database engine to 
determine what should be "in RAM" without having to do all of the I/O.

If one is stuck with low end databases and tools, then one has little 
choice, they have to do the I/O and stuff all that data into RAM. This 
is especially true if you are making a decision spread across multiple 
databases from multiple vendors. The capabilities of Qt don't quite 
measure up to the 1980s product Cognos PowerHouse

https://www.teamblue.unicomsi.com/products/powerhouse-4gl/

which would let you pull a cursor from say, Oracle, and drive it against 
an indexed file or one of the other supported database engines. I think 
you could have up to seven different database engines in a single 
transaction, but that may be an outdated number. It never loaded that 
stuff into RAM more than a few records at a time. I know this because 
the databases I was using were many times the size of physical + virtual 
memory. It all "just worked."

People don't want data dictionaries anymore, but they all want the data 
capacity and throughput.

Yes, developers can customize their own QSQLTableModel to periscope over 
the cursor, and force bidirectional in, but they shouldn't have to in 
2022. The underlying code needs to also realize that ROWID is no longer 
a table unique number. It is __segment__ unique. I forget the official 
name for the groups of segments, but there is one.

-- 
Roland Hughes, President
Logikal Solutions
(630)-205-1593

http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog