[Interest] building a light weight tree model
Frank Rueter | OHUfx
frank at ohufx.com
Wed May 3 08:34:35 CEST 2017
Hi Andre,
>>Do you really?
I do. The spec is to generate a report of a file system including number
of files and total file size among many other per-file stats.
I also need to identify file names that match certain filters, identify
empty directories etc.
It's for a pre-archive workflow.
The tree and list views are mostly for sanity checking the results (they
are colour coded and filtered to facilitate reading the amount of data)
and eventually for renaming paths if they grow too large for windows to
operate with during the actual archive process.
Currently this is all pre-flight checking and report building (I am
outputting an html file with the found stats).
Sorry, I should have outlined all of this in my initial email.
>>You mentioned being able to navigate into zip files?
Not navigating into zip files. I need to display certain groups of
physical files on disk as "virtual" zip files to show the user what will
be zipped in the next step.
I am now doing that with a simple proxy model.
Thanks for the explanation regarding the models. That all makes perfect
sense and in the next run I will endeavour to swap out QSIM with at a
custom one derived from QAIM. Then see what the performance is after
Cythonising and compiling.
I am happy to say though that the performance I achieved today seems to
be acceptable for now (early beta prototype).
Thanks again for your guidance, it's much appreciated!
Cheers,
frank
On 3/05/17 5:48 PM, André Somers wrote:
>
> Hi,
>
>
> Op 03/05/2017 om 00:18 schreef Frank Rueter | OHUfx:
>> Thanks for the tip.
>> I don't know C++ so that's not an option, but I can Cythonise and
>> compile later.
>>
>> Actually, I started with a command line app that just printed out a
>> report. In my analysis I need to visit every file and directory and
>> collect data such as size, file extensions, matches to a user filter etc.
> Do you really? Are you sure you need your tree fully expanded? Note
> that QFileSystemModel (QFSM) does its population in a lazy way: it
> does not iterate over parts of the file system that have not been
> requested yet from the model (and it does not monitor for changes in
> these parts either). Are you sure you need each and every file and
> dir's data right away? Can your user even view that much information?
>
>> Hence it felt natural to collect QStandardItems as I went, rather
>> than us QFileSystemModel and then cycle through all items after the fact.
> Perhaps you should explain your requirements a bit better. What are
> you going to do with the data? What do you need on top of what QFSM
> already supplies? You mentioned being able to navigate into zip files?
> Anything else?
>
>> I will have a look at doing that though, since you seem to be in
>> agreement that it will be faster.
>>
>> >>QStandardItemModel is not the API you are looking for
>> Can you clarify please? I thought it was one of the simplest models
>> and hence I gravitated towards that.
>> Should I use QAbstractItemModel instead (unless I end up using
>> QFileSystemModel)?
> QStandardItemModel is all but the simplest model. It is a very heavy
> and complicated one. Its item-based API provides many fields by
> default, and also sports virtual methods. You won't need many of
> these, yet you will pay the price for them for every cell in your
> entire model. It is far more efficient to create a compact data
> structure that contains just the data you really need in your case,
> and expose that via a QAbstractItemModel (QAIM) derived model directly.
>
> Also on a conceptual level, I think the class is wrong for most
> usecases - including yours. It models a collection of independent
> individual cells, like a spread sheet where each cell can recursively
> contain a child spread sheet again (as is the structure of QAIM). But
> your data does not look like that. You are dealing with a forest
> structure: a collection of trees of items, containing directories
> (branches) and files (leafs). Sure, these items have attributes that
> you may want to present in aditional columns in your model, but in
> essense, the data is just about files and directories (which also can
> be thought of as files, only files containing a list of other files).
> So, you want your API to match _that_ concept. You want your API to
> deal with files, not with a row of items that really may contain
> anything.
>
> InsHMO, the naming of this class was a mistake. It should never have
> been called "Standard", as that suggests it is the default choice,
> while in fact it should almost never be that. And the introdcution of
> QListWidget, QTableWidget and QTreeWidget was an even bigger mistake.
>
> André
>
>>
>> Cheers,
>> frank
>>
>> On 3/05/17 7:52 AM, André Somers wrote:
>>>
>>> Hi
>>>
>>>
>>> Op 02/05/2017 om 09:15 schreef Frank Rueter | OHUfx:
>>>> I need a few custom views for my data, including representing
>>>> collections of files with a certain extension as virtual zip files,
>>>> i.e. items that don't actually exist on disk.
>>> So, use a proxy model or a model that wraps the original one.
>>>> I assumed using QFileSystemModel would not be the most efficient
>>>> way to use i this case (based on various comments I read in the past).
>>> Eh... If you want to know if your code is efficient, you should not
>>> start from assumptions. You should start with measurements and some
>>> concrete goals.
>>>
>>> Other than that, I fully agree with Chris: if you want this to work
>>> fast, use C++ and avoid QStandardItemModel, especially for "huge"
>>> models. QStandardItemModel is not the API you are looking for (ever,
>>> IMHO, with a possible exception for small toy projects).
>>>
>>> André
>>>
>>>>
>>>>
>>>> On 2/05/17 6:25 PM, Ch'Gans wrote:
>>>>>
>>>>>
>>>>> On 2 May 2017 at 17:55, Frank Rueter | OHUfx <frank at ohufx.com
>>>>> <mailto:frank at ohufx.com>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Here is a snippet of test code to mimic a light weight tree model.
>>>>> All it is trying to do is mimic the parenting according to the
>>>>> directory structure on disk using QStandardItems in a
>>>>> QStandardModel.
>>>>> It works, but it took me way longer to figure out than I
>>>>> expected, and I have a feeling some of you may have good
>>>>> advise on how to make it more efficient.
>>>>> https://pastebin.com/vRkzDPtt
>>>>>
>>>>>
>>>>> What's your definition of "more efficient", what are the problems,
>>>>> what make you think it's not efficient?
>>>>>
>>>>> If you want efficiency, then don't use python and don't use
>>>>> QStandardModel, if you want expediency then use QFileSystemModel.
>>>>>
>>>>> My 2 cents.
>>>>> Chris
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I am not using QFileSystemModel as I wanted to keep things as
>>>>> efficient as possible and also need to add some custom items
>>>>> to the model.
>>>>> Also, for the Python programmers: in my actual code I am using
>>>>> scandir.walk() instead of os.walk() for speed.
>>>>>
>>>>> Any input would be greatly appreciated as I will need to run
>>>>> this over disks that have massive amounts of files.
>>>>>
>>>>> Cheers,
>>>>> frank
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> ohufxLogo 50x50 <http://www.ohufx.com>
>>>>> *vfx compositing <http://ohufx.com/compositing.html> |
>>>>> *workflow customisation and consulting
>>>>> <http://ohufx.com/customising.html>* *
>>>>> *<http://ohufx.com/compositing.html>*
>>>>> <http://www.nukepedia.com/nubridge>
>>>>>
>>>>>
>>>>> Your gateway to over 1,000 free tools... right inside of Nuke
>>>>> <http://www.nukepedia.com/nubridge>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Interest mailing list
>>>>> Interest at qt-project.org <mailto:Interest at qt-project.org>
>>>>> http://lists.qt-project.org/mailman/listinfo/interest
>>>>> <http://lists.qt-project.org/mailman/listinfo/interest>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Interest mailing list
>>>> Interest at qt-project.org
>>>> http://lists.qt-project.org/mailman/listinfo/interest
>>>
>>>
>>>
>>> _______________________________________________
>>> Interest mailing list
>>> Interest at qt-project.org
>>> http://lists.qt-project.org/mailman/listinfo/interest
>>
>>
>>
>> _______________________________________________
>> Interest mailing list
>> Interest at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/interest
>
>
>
> _______________________________________________
> Interest mailing list
> Interest at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/interest
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20170503/fb88bcfa/attachment.html>
More information about the Interest
mailing list