[Qt-interest] QTreeView and custom QAbstractItemModel performance problem
Felix Brack
fb at ltec.ch
Fri Jul 9 11:41:08 CEST 2010
On 09.07.2010 11:11, Jeroen De Wachter wrote:
> On Fri, 2010-07-09 at 10:44 +0200, Felix Brack wrote:
>> On 08.07.2010 16:35, Andreas Pakulat wrote:
>>> On 08.07.10 16:09:52, Felix Brack wrote:
>>>> On 08.07.2010 12:50, Andreas Pakulat wrote:
>>>>> On 08.07.10 11:51:38, Felix Brack wrote:
>>>>>> On 08.07.2010 10:58, Andreas Pakulat wrote:
>>>>>>> On 08.07.10 10:14:11, Felix Brack wrote:
>>>>>>>> By subclassing QAbstractItemModel I have create a custom model that
>>>>>>>> stores data from a communication trace row by row. Every row has
>>>>>>>> multiple columns such as a time stamp, the data traced from the
>>>>>>>> communication, etc. The number of rows in the data model is limited,
>>>>>>>> let's say to 100 rows. When adding row 101 it is appended and the first
>>>>>>>> row (i.e. the one with the oldest data) is removed.
>>>>>>>>
>>>>>>>> To show the information to the user I use a QTreeView which shows each
>>>>>>>> row of the data in the model line by line, i.e. the model is not
>>>>>>>> hierarchic, all rows have the same parent (root) and consist of multiple
>>>>>>>> columns.
>>>>>>>>
>>>>>>>> Things are working so far but performance is very poor and the CPU load
>>>>>>>> is incredible (even with just 100 rows). The reason for this is that
>>>>>>>> whenever I append a row to the model the view calls the models 'data'
>>>>>>>> member for all rows and all columns.
>>>>>>>>
>>>>>>>> It seems that I'm missing something fundamental here and that appending
>>>>>>>> a row of data to my model looks for the view like the entire data has
>>>>>>>> changed. I will have to change something so only the data from the newly
>>>>>>>> append row is queried by the view.
>>>>>>>>
>>>>>>>> I just don't know how to do that... Can anybody push me into the right
>>>>>>>> direction?
>>>>>>>
>>>>>>> Can you post your model implementation? It sure sounds like you're doing
>>>>>>> the appending wrong, but without having a look at the code its not possible
>>>>>>> to say what exactly is wrong.
>>>>>>>
>>>>>>> A blind guess could be that you're reset'ting you're model every time you
>>>>>>> add a row instead of using begin/endInsertRows correctly. See the example
>>>>>>> models in Qt.
>>>>>>>
>>>>>>> Also I'd suggest to use a QTableView instead of QTreeView as that one is
>>>>>>> significantly faster on non-tree structures (you can tweak it to have no
>>>>>>> lines etc.) in case you ever get more than 100 rows.
>>>>>>
>>>>>> Hello Andreas,
>>>>>>
>>>>>> Thanks for your response. The code below is my 'AppendRow' method. The
>>>>>> variable 'm_Data' and 'm_IconKey' are just arrays containing the textual
>>>>>> data as well as an icon that gets displayed in the first column.
>>>>>>
>>>>>> QModelIndex CMultiColumnListModel::AppendRow(QString IconKey,
>>>>>> QStringList Data)
>>>>>> {
>>>>>> beginInsertRows(QModelIndex(), m_Data.size(), m_Data.size()) ;
>>>>>> m_Data.push_back(Data);
>>>>>> m_IconKey.push_back(IconKey);
>>>>>> endInsertRows();
>>>>>>
>>>>>> if ((m_nMaxRows!=0)&& ((int)(m_Data.size())>m_nMaxRows)) {
>>>>>> // remove oldest element from list
>>>>>> beginRemoveRows(QModelIndex(), 0, 0);
>>>>>> m_Data.erase(m_Data.begin());
>>>>>> m_IconKey.erase(m_IconKey.begin());
>>>>>> endRemoveRows();
>>>>>> }
>>>>>>
>>>>>> return index(m_Data.size()-1, 0);
>>>>>> }
>>>>>>
>>>>>> Do you see any problem here?
>>>>>
>>>>> No, can't see anything obvious wrong. Could be the removal of the first
>>>>> element is the culprit. In general the data() function of your model should
>>>>> be as fast as humanly possible using as little computed data and as much
>>>>> pre-computed data as you can get.
>>>>>
>>>>> Andreas
>>>>>
>>>>
>>>> The removal operation in my AppendRow() function is only executed when
>>>> there are more rows then m_nMaxRows. Setting m_nMaxRows to 200 however
>>>> shows that the performance breakdown and the excessive CPU load occurs
>>>> way before a row has to be removed (starting at approximately 20 rows).
>>>>
>>>> Well,I just changed my data() function to look as follows:
>>>>
>>>> QVariant CMultiColumnListModel::data(const QModelIndex& Index, int
>>>> nRole) const
>>>> {
>>>> // sanity check
>>>> if (!Index.isValid()) {
>>>> return QVariant::Invalid;
>>>> }
>>>>
>>>> if (nRole==Qt::DisplayRole) {
>>>> // return text data
>>>> return QString("Test");
>>>> }
>>>>
>>>> // we do not support other roles for now
>>>> return QVariant::Invalid;
>>>> }
>>>>
>>>> I know it is stupid but I can't think of a faster data() function. As I
>>>> expected the problem is exactly the same: very slow with 100 rows and a
>>>> CPU load of more then 50%.
>>>
>>> Then you'll have to hit your app with a profiler, I don't see another way
>>> of finding out where all those cpu-cycles are burnt (certainly not in that
>>> data function).
>>>
>>> Andreas
>>>
>>
>> Running the application with gprof shows what I already new. Here are the relevant parts of the gprof output:
>>
>>
>> Flat profile:
>>
>> Each sample counts as 0.01 seconds.
>> % cumulative self self total
>> time seconds seconds calls ms/call ms/call name
>> 19.35 0.06 0.06 CMultiColumnListModel::data(QModelIndex const&, int) const
>> 16.13 0.11 0.05 2402663 0.00 0.00 QModelIndex::QModelIndex()
>> 6.45 0.13 0.02 4736435 0.00 0.00 QModelIndex::isValid() const
>> 6.45 0.15 0.02 1723103 0.00 0.00 QModelIndex::operator!=(QModelIndex const&) const
>> 6.45 0.17 0.02 600548 0.00 0.00 QString::QString(char const*)
>> 6.45 0.19 0.02 588369 0.00 0.00 QList<QString>::count() const
>> 6.45 0.21 0.02 576803 0.00 0.00 std::vector<QStringList, std::allocator<QStringList> >::size() const
>> 4.84 0.23 0.01 573595 0.00 0.00 CMultiColumnListModel::rowCount(QModelIndex const&) const
>>
>>
>> index % time self children called name
>> <spontaneous>
>> [1] 50.3 0.00 0.16 MainWindow::qt_metacall(QMetaObject::Call, int, void**) [1]
>> 0.00 0.16 304/304 MainWindow::ShowTrace(bool, QString, int, bool, bool, bool, QString, int, int, QString) [2]
>> 0.00 0.00 912/613837 QString::~QString() [15]
>> 0.00 0.00 1/1 MainWindow::SlaveConnect() [78]
>> 0.00 0.00 912/9586 QString::QString(QString const&) [105]
>> -----------------------------------------------
>> 0.00 0.16 304/304 MainWindow::qt_metacall(QMetaObject::Call, int, void**) [1]
>> [2] 50.3 0.00 0.16 304 MainWindow::ShowTrace(bool, QString, int, bool, bool, bool, QString, int, int, QString) [2]
>> 0.00 0.15 304/304 CMultiColumnListModel::AppendRow(QString, QStringList) [3]
>> 0.00 0.01 608/1043 QStringList::~QStringList() [20]
>> 0.00 0.00 5472/613837 QString::~QString() [15]
>> 0.00 0.00 3952/3965 QStringList::operator<<(QString const&) [48]
>> 0.00 0.00 3344/600548 QString::QString(char const*) [12]
>> 0.00 0.00 304/1724219 QModelIndex::~QModelIndex() [22]
>> 0.00 0.00 1824/1824 QLatin1Char::QLatin1Char(char) [113]
>> 0.00 0.00 1824/1824 QChar::QChar(QLatin1Char) [114]
>> 0.00 0.00 1824/1824 QString::arg(int, int, int, QChar const&) const [116]
>> 0.00 0.00 304/307 QStringList::QStringList() [162]
>> 0.00 0.00 304/304 QTime::QTime() [169]
>> 0.00 0.00 304/736 QStringList::QStringList(QStringList const&) [128]
>> 0.00 0.00 1/1 MainWindow::SlaveDisconnect() [310]
>> -----------------------------------------------
>> 0.00 0.15 304/304 MainWindow::ShowTrace(bool, QString, int, bool, bool, bool, QString, int, int, QString) [2]
>> [3] 48.2 0.00 0.15 304 CMultiColumnListModel::AppendRow(QString, QStringList) [3]
>> 0.01 0.14 304/304 CMultiColumnListModel::index(int, int, QModelIndex const&) const [4]
>> 0.00 0.00 204/204 std::vector<QStringList, std::allocator<QStringList> >::erase(__gnu_cxx::__normal_iterator<QStringList*, std::vector<QStringList, std::allocator<QStringList> > >) [28]
>> 0.00 0.00 304/304 std::vector<QStringList, std::allocator<QStringList> >::push_back(QStringList const&) [33]
>> 0.00 0.00 1216/576803 std::vector<QStringList, std::allocator<QStringList> >::size() const [14]
>> 0.00 0.00 812/2402663 QModelIndex::QModelIndex() [7]
>> 0.00 0.00 204/204 std::vector<QString, std::allocator<QString> >::erase(__gnu_cxx::__normal_iterator<QString*, std::vector<QString, std::allocator<QString> > >) [68]
>> 0.00 0.00 304/304 std::vector<QString, std::allocator<QString> >::push_back(QString const&) [69]
>> 0.00 0.00 812/1724219 QModelIndex::~QModelIndex() [22]
>> 0.00 0.00 204/212 std::vector<QStringList, std::allocator<QStringList> >::begin() [177]
>> 0.00 0.00 204/212 std::vector<QString, std::allocator<QString> >::begin() [178]
>> -----------------------------------------------
>> 0.01 0.14 304/304 CMultiColumnListModel::AppendRow(QString, QStringList) [3]
>> [4] 46.9 0.01 0.14 304 CMultiColumnListModel::index(int, int, QModelIndex const&) const [4]
>> 0.01 0.05 573595/573595 CMultiColumnListModel::rowCount(QModelIndex const&) const [6]
>> 0.01 0.04 573589/573589 CMultiColumnListModel::columnCount(QModelIndex const&) const [8]
>> 0.01 0.00 573595/2402663 QModelIndex::QModelIndex() [7]
>> 0.01 0.00 573589/1723103 QModelIndex::operator!=(QModelIndex const&) const [9]
>> 0.00 0.00 573589/1724219 QModelIndex::~QModelIndex() [22]
>> 0.00 0.00 573589/573589 QAbstractItemModel::createIndex(int, int, void*) const [99]
>> -----------------------------------------------
>> <spontaneous>
>> [5] 38.4 0.06 0.06 CMultiColumnListModel::data(QModelIndex const&, int) const [5]
>> 0.02 0.00 4736435/4736435 QModelIndex::isValid() const [11]
>> 0.02 0.00 597183/600548 QString::QString(char const*) [12]
>> 0.01 0.01 597183/613837 QString::~QString() [15]
>> -----------------------------------------------
>> 0.01 0.05 573595/573595 CMultiColumnListModel::index(int, int, QModelIndex const&) const [4]
>> [6] 19.5 0.01 0.05 573595 CMultiColumnListModel::rowCount(QModelIndex const&) const [6]
>> 0.02 0.00 575555/576803 std::vector<QStringList, std::allocator<QStringList> >::size() const [14]
>> 0.01 0.00 575859/2402663 QModelIndex::QModelIndex() [7]
>> 0.01 0.00 575859/1723103 QModelIndex::operator!=(QModelIndex const&) const [9]
>> 0.00 0.00 575859/1724219 QModelIndex::~QModelIndex() [22]
>> -----------------------------------------------
>>
>> I modified the application so it automatically stops adding rows
>> after 300 rows (304 rows in the gprof output above).
>>
>> I think the absolute time consumed by one of these functions (i.e.
>> the function's performance) is not really relevant here.
>> The real problem is that 304 calls to the function AppendRow() generate
>> more then halve a million of calls to rowCount(), columnCount(), etc. In fact
>> every call to AppendRow() generates r*c calls to these functions where r is
>> the number of rows and c is the number columns in my list.
>> Limitting the number of rows in my list to 100 while the number of columns is
>> constantly 12 will produce the following amount of calls to rowCount() etc.:
>>
>> roughly: 12*(100*(100+1))/2)+200*(100*12)= 275550
>>
>> This is not halve a million but quite close to it; the Qt framework will certainly
>> do more calls then calculated.
>>
>> As I can see what's wrong I still can't figure out how to solve the problem. The
>> huge amount of calls shown by the profiler are from the view updating always all
>> cells of the list.
>> The question is: why does the tree view update _all_ cells (potentially 1200 in my
>> sample above) every time a row is added to the list?
>>
>> Felix
>
> Is it at all possible that the issue is with the first row being removed
> instead? I could imagine QTreeView not handling the shift in indexes
> very well... (row 1 becomes row 0, etc.)
>
> If QTreeView proceeds to get the data for all the "changed" rows, that
> might explain the behaviour you're getting...
>
> Could you try running your code without removing the first row? Just to
> see what would happen?
>
> Also: have you tried using a table view instead? Since it doesn't work
> with parent/child relationships like QTreeView does, it might handle
> rows being removed more gracefully... (I'm just guessing though, at this
> point, haven't done any tests)
>
> Greets,
>
> Jeroen
The thing with the row shift was my first guess too, but profiling
without removing the first (oldest) row has shown that every thing
behaves identically - poor performance and high CPU load.
If there is really no solution to this I will give the table view a
chance. Doing this however would mean to surrender. Furthermore one
could say that Qt's TreeView is not able to manage a simple (100 to 500
rows) dynamically changing list with 12 columns; something I simply
can't believe (I have implemented this with native windows code and it
works without any problem even with thousands of rows); this is why I'm
sure that I just don't really understand the mechanics of the Qt
framework. I'd rather think this problem is of type 'how could I be so
stupid not to ...' once the solution is found.
Felix
More information about the Qt-interest-old
mailing list