[Interest] Can a view somehow notify a model of which rows it is going to show on screen?

[Mark Gaiser]

On Sun, Apr 27, 2014 at 11:08 PM, Mark Gaiser <markg85 at gmail.com> wrote:
> On Fri, Apr 25, 2014 at 2:35 PM, Jan Kundrát <jkt at flaska.net> wrote:
>> On Tuesday, 22 April 2014 23:51:07 CEST, Mark Gaiser wrote:
>>> There we have it. All visible item indexes accurately known. :)
>>
>> The model does not know if there's any view attached to it, though.
>
> Now it does so i'm breaking the MVC principle a bit for better
> performance. Oops ;)
>>
>>> Usecase:
>>> If you only look at small folders then this definitely is premature
>>> optimizations since the "regular sorting" will do just fine there.
>>> Bit if you look at insanely massive folders with 500.000 items. All
>>> stuffed in one ListView (where you only see ~50 or so). Sorting that
>>> by string in a natural way (using QCollator) is very slow. In my case
>>> ~6 seconds.
>>
>> Six seconds to sort 500k items doing a locale-aware string comparison
>> sounds like a bug. Check this:
>>
>> svist ~ # locale | grep CTYPE
>> LC_CTYPE=cs_CZ.utf8
>> svist ~ # wc -l /var/log/messages
>> 763524 /var/log/messages
>> svist ~ # time sort /var/log/messages > /dev/null
>>
>> real    0m0.244s
>> user    0m0.555s
>> sys     0m0.055s
>>
>> 763k items just got sorted in a fraction of a second, including (cached) IO
>> and printing to stdout. Sure, these items were actually presorted (my log
>> was the easiest think to work on, and it's ordered already), but if you
>> need six seconds for something so trivial, your program has a bug.
>>
>> Use a profiler. Take a look at where the time is spent. You mentioned "a
>> folder" -- are you doing stuff like directory IO in there, and including
>> that in your measurements? Again, profile, measure, do not guess. And do
>> not try to fight file IO by doing partial sorts.
>
> Your testcase is wrong :)
> I'm testing "natural sorting". Aka, the stuff that turns:
> 4.txt
> 2.txt
> 10.txt
> 0.txt
> 8.txt
>
> into
> 0.txt
> 2.txt
> 4.txt
> 8.txt
> 10.txt
>
> For that i'm using QCollator with setNumericMode(true). This exact
> option is what makes a natural sort slow. There is nothing that can be
> done about it other then reducing the number of items to sort.
> I promise you, if you run your same test with this then it will be a
> lot slower :)
>>
>>> I know that giving a ListView a task with that many items
>>> is insane and in most cases you're probably better of splitting the
>>> data in chunks.
>>
>> That said, introducing deliberate chunking is also ugly from the user's
>> POV.
>>
>> Cheers,
>> Jan
>>
>> --
>> Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
>> _______________________________________________
>> Interest mailing list
>> Interest at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/interest

I forgot to mention that the sorting tests are done while everything
is cached in memory (to make the test fair and not id dependent).
Loaded from SSD into memory. And that is ~6 seconds for 500.000 items
with QCollator and numeric mode set to true.