[Interest] [External]Re: How to get QtConcurrent to do what I want?
Sean.Murphy at centauricorp.com
Tue Feb 1 16:32:24 CET 2022
> Not knowing if a partial value makes any sense to your system.
> Qt::Concurrent::mappedReduced might make more sense, if its purely a
> speedup you are looking for, and not a "keep the GUI alive during it" possibly
I don't think mappedReduced would help me until after the remapping step
when I want to assemble the individual tiles into a QImage - although since the
ultimate destination for the image is inside a QGraphicsView, I'm tempted to just
leave it as individual tiles, but I'm not there yet as far as testing.
Regarding the keep GUI alive portion, my tileManager is already running in a
separate thread from the UI thread, so I do have the option of making blocking calls
in the tileManager class with the exception that I do need to fire progress signals out
of tileManager back to MainWindow to provide progress to the user
> If you need the gui, setting up a qfuturewatcher on the results of the
> mapped call, would be my approach
> QFutureWatcher< XXX > watcher;
> connect( &watcher, &finished, manager, &handleFinished);
> auto future = QtConcurrent::mapReduced(tiles, processTile, mergeFunction)
> watcher.setFuture( future );
I actually spent yesterday refactoring the tileManager class as you've just described,
as well as changing out the tile class to no longer inherit from QObject. I've got a couple
more things I want to try/clean up today but I still seem to be having trouble speeding
up the allocation & assignment of the tiles themselves.
As my code currently stands, I now have to vectors, each of which will have 60,000 items
in them once they're populated:
The mTileIndices vector is implementing Andrei's idea of quickly generating a list of unique tile
indices, which can then be fed to a QtConcurrent::map() call to create & uniquely assign the
tile items in parallel. The " mTiles " vector is obviously the tiles that will do the
As I build the vectors up, these are the timings I get:
1. resizing tile index vector to 60000 took 0.1514 ms
a. This is just calling QVector<int>::resize(60000) on the ID. This takes less than a
millisecond, so no complaints here.
2. allocated 60000 indices in 0.0207 ms
a. This is calling std::iota(mTileIndices.begin(), mTileIndices.end(), 0). Also takes less than
a millisecond, still no complaints
3. assigning 60000 tiles took 18087.1 ms
a. This timing is the result of QtConcurrent::mapped(mTiles, initTile) where the initTile
function takes in an integer from mTileIndices, and calls the tile constructor using the
combination of the tile index and tile size to do the assignment. The assigned tiles end
up in the mTiles
b. This step takes 18 seconds, which seems excessive to me and I'd love to continue to
reduce that time.
4. load finished in 37507.1 ms
a. this is calling tile::load() on each tile. Right now that is just a dummy function that calls
msleep for a random amount milliseconds to simulate doing the actual work
5. remapping 60000 tiles took 48519.3 ms
b. this is calling tile::remap() on each tile. Right now that is just a dummy function that
calls msleep for a random amount milliseconds to simulate doing the actual work
So the only step in this process that still bothers me is step 3 - creating and assigning each tile
object takes 18 seconds. I log the total time by each tile spent in steps 4 & 5 and compare how
long steps 4 & 5 actually take vs. the sum of how long each tile spent sleep and I routinely get
a speedup factor of about 7.5. QThread::idealThreadCount() reports 8 on my machine, so I think
I'm getting what I should expect from those steps on this machine.
I'm not sure what else to try at this point. One thing I was thinking about measuring is that even
though my tile class no longer inherits from QObject, it still is a class with a constructor and some
getter functions. And when I look at the combination of steps 1 & 2 where I both resize a vector
of integers AND assign each one a unique ID in less than a millisecond total, but then it takes
me 18 seconds to create and assign each tile, I keep wondering if there isn't room for
improvement there still?
And as I was typing this whole thing up, Konstantin gave me a different approach that I'll probably
pursue, but I would like to better understand how to solve the question of "if you absolutely need
to have a lot of items (whatever type an "item" needs to be), what's the right design approach to
be able to create and populate them quickly..."
More information about the Interest