[Qt-interest] QAbstractListModel Scrolling Performance Issue

[Stephan Rose]

First off, some background information.

My original model, also based on a QAbstractListModel, used a SQL Query to obtain data from a MySQL Database. In the data(..) function I'd seek to the appropriate record in the resultset and then return the requested column data.

This worked great but initial scrolling performance was poor. If for instance I'd have a resultset with 40,000 records it would take a couple seconds to scroll all the way from the top to bottom by dragging on the scrollbar. I could definitely feel a major lag. However, having fully scrolled from top to bottom at least one time, from that point on scrolling is perfectly smooth.

So, this week I implemented a new data server which runs on it's own thread in the background and created a new model to work with that server. This new model submits a query to the data server which is then added to a task queue and eventually processed. 

Along with submitting the query, the model also submits a slot that is connected to a signal of the data server's task. When the query is then finally executed, it batches a bunch of QSqlRecords (currently 1,000 at a time) and then emits a data received event with a QList containing those records. This also helped speed up query execution as I can now set forwardOnly to true.

The slot implemented in the model then takes these records, and together with beginInsertRows and endInsertRows appends these to it's own internal QList of records.

Works wonderfully, especially with larger queries that used to momentarily freeze the UI. However, the scrolling issue previously described still exists.

Using the same query that returned the 40,000 records as described in the initial example, I still get the problem that if I want to scroll all the way to the bottom there is a visible lag. I've verified that the data server has finished executing the query and all queries have been submitted to the model. So there is absolutely no more lag that can be caused by client / server communication. Again as in the initial example, once I've scrolled all the way from top to bottom, scrolling is smooth.

This is particularly an issue on slower systems where the symptoms are worse, but even on my quad core system this happens.

I don't think fetchMore and canFetchMore are going to be of any help since by the time I do any scrolling all data has already been fetched.