[Interest] [External]Re: How to get QtConcurrent to do what I want?

[Tony Rietwyk]

That's looks OK.  Why does the tile object creation take so long?  Is 
all of the image handling in tile::process, or does tile constructor 
extract from the original?


Regards, Tony


On 31/01/2022 11:59 am, Murphy, Sean wrote:
> Thanks for the response, but I'm not following your suggestion - or at 
> least I'm not seeing how it's different than what I'm doing? Maybe a 
> little pseudocode will help. Here's what I'm currently doing:
>
> Tile class:
> private:
> QPointmPos;
> intmSize;
>
> tile::tile(QPointpos,intsize):
> mPos(pos),
> mSize(size)
> {
> // assigns this tile an mSize x mSize square
> // from the original image starting at mPos
> // pixel location in the original image
> }
>
> voidtile::process()
> {
> // does the work on the assigned subset
> }
> TileManager:
> private:
> QVector<QSharedPointer<tile>> mTiles;
>
> processTile(QSharedPointer<tile>& t)
> {
> t->process();
> }
>
> tileManager::setup(QSize tileGrid, int tileSize)
> {
> //generateeachtilewithitsassignment
> for(inti=0;i<tileGrid.height();++i)
> {
> for(intj=0;j<tileGrid.width();++j)
> {
> // create the new tile while assign its
> // region of the original image
> QSharedPointer<tile>t(newtile(
> QPoint(j*tileSize,i*tileSize),
> tileSize));
> mTiles.append(t);
> }
> }
> QtConcurrent::map(mTiles,processTile);
> }
>
> So I think I'm already doing what you're saying? Where I'm paying the 
> penalty is that the allocation of each tile is happening in one thread 
> and I'd like to see if I can thread out the object creation. But I 
> don't see how to simultaneously thread out the tile objection creation 
> AND correctly assign the tile its location since as far as I can tell, 
> when QtConcurrent executes tileManager's processTile function in 
> parallel there's nothing I can poll inside tileManager::processTile() 
> that allows me to know WHICH step I'm at.
>
> Or am I misunderstanding what you're saying?
>
> The best thing I can come up with is that maybe I could change the 
> type of my mTiles vector to be a QVector<QPoint>> but then I'd still 
> need to loop through nested for-loop to populate all the QPoint items 
> in the vector I want to pass to QtConcurrent::map(). I have tried that 
> yet to see if generating thousands of QPoint objects is faster than 
> generating the same number of tiles, but I can test that out.
>
> Sean
> ------------------------------------------------------------------------
> *From:* Interest <interest-bounces at qt-project.org> on behalf of Tony 
> Rietwyk <tony at rightsoft.com.au>
> *Sent:* Sunday, January 30, 2022 7:19 PM
> *To:* interest at qt-project.org <interest at qt-project.org>
> *Subject:* [External]Re: [Interest] How to get QtConcurrent to do what 
> I want?
> CAUTION: This email originated from outside of the organization. Do 
> not click links or open attachments unless you recognize the sender 
> and know the content is safe.
>
> Hi Sean,
>
>
> Can you use the position of the tile as a unique key?  Then the 
> manager only needs to calculate each tile's position in the original 
> image.  Each tile extracts the bits, processes and notifies the result 
> with its position.
>
>
> Regards, Tony
>
>
>
> On 31/01/2022 10:06 am, Murphy, Sean wrote:
>> I'm hitting a design issue with the way I'm using the QtConcurrent 
>> module to do some image processing, and I'm wondering if someone can 
>> give some pointers?
>>
>> At a high level, the software needs to do some processing on every 
>> pixel of an image. The processing can mostly be done in parallel, so 
>> I've created the following:
>>
>>  1. Tile class - responsible for doing the processing on a small
>>     subset of the original image
>>      1. Has a constructor that takes a Position and Size. From those
>>         parameters, the Tile knows what subset of the original image
>>         it is going to process
>>      2. Has a process() function which will do the work on those
>>         assigned pixels
>>  2. TileManager class - responsible for managing the Tile objects
>>      1. Contains a for-loop that creates each Tile object, assigns it
>>         a unique Position, and adds it to the QVector<Tile> vector
>>      2. Has a processTile(Tile& t) function which calls t.process()
>>         to tell a given Tile to begin its work
>>      3. Calls QtConcurrent::map(tiles, processTile) to process each tile
>>
>> So far this works well, but as I was timing different parts of the 
>> codebase, I discovered that a large portion of the time is spent 
>> allocating the QVector<Tile> vector (step 2a above) before I get to 
>> the concurrent processing call. The reason why is obvious to me - I 
>> need to ensure that each tile is created with a unique assignment and 
>> as far as I can see, that need to happen in a single thread? If I 
>> could instead pass off the Tile creation to the parallel processing 
>> step, I might be able to improve the overall performance, but I don't 
>> see a way around it within the QtConcurrent framework.
>>
>> How can I go about creating Tile objects in parallel AND ensure that 
>> each of them gets a unique Position assignment? I could easily move 
>> the Tile allocation into processTile(), but if I do that, I don't see 
>> a way make the unique position assignment since I don't see how a 
>> given call to processTile() would know where it is in the overall 
>> parallelization sequence to determine what Position to assign to the 
>> Tile it creates. If I were using something like CUDA, I could use 
>> things like blockIdx and threadIdx to do that, but as far as I can 
>> see, those concepts don't exist (or at least aren't exposed) in 
>> QtConcurrent.
>>
>> Any thoughts?
>>
>> _______________________________________________
>> Interest mailing list
>> Interest at qt-project.org  <mailto:Interest at qt-project.org>
>> https://lists.qt-project.org/listinfo/interest  <https://lists.qt-project.org/listinfo/interest>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20220131/5ef92fd8/attachment-0001.htm>