[Interest] [External]Re: How to get QtConcurrent to do what I want?

[Murphy, Sean]

> On Mon, Jan 31, 2022 at 7:15 PM Murphy, Sean <mailto:Sean.Murphy at centauricorp.com> wrote:
> >   1. Creating 60,000 QObjects in a single thread appears to be slow  
> [...]
>
> Ehm, maybe I'm not understanding something, but why do you need objects to begin with?

I do need to report progress back to the UI and I mistakenly thought I needed to have 
each tile inherit from QObject to provide that functionality. After reading up a bit more about 
QFuture and QFutureWatcher and then refactoring yesterday to use those classes, as of the 
moment, my tile class no longer inherits from anything, and I'm able to use signals from 
QFutureWatcher to relay progress back to the UI. 

> The actual loop is this:
>    // generate each tile with its assignment
> [snip]
>
> What's the significance of the tiles? As far as I can tell from your requirements, you don't care about
> the "true geometry" of the data.

Either I'm understanding what you mean by "true geometry", or this assumption is at least partially incorrect. 
Looking back on my list of requirements I've posted, I left off the last step: 

  At the end of all this processing, I do need to produce an onscreen image to the user. 

So any way I slice up the work that needs to be done using threads, once they are all finished, I do need to 
know what chunk of the original image each thread was working on to know where place its normalized pixels 
in what I display to the user.

> At least to me it seems you want something like (pseudo algorithm):
>
> 1) Start QThread::idealThreadCount threads (QThread::create<> / std::thread)
> 2) Each thread works on "total samples" / QThread::idealThreadCount buffers that are completely independent.
> 2.1) Each thread goes through each sample from a partially mapped (from the file) buffer, takes the min/max to get the dynamic range
> 2.2) Sync the threads to get the global min/max
> 2.3) Go through each of the buffers a second time to normalize the dynamic range (again no tiles involved, just samples)
> 3) Done.

I think this is an preferable approach to what I was attempting, and I'm glad you suggested it. This being my first attempt at this, 
I naively started from asking "what seems like it would be a reasonable tile size?", arbitrarily thought "256 pixels square" and worked 
backwards from there, which is how I got into this mindset of "I might have 60,000+ tiles to deal with". Your approach starts from 
what now appears to me a much better thought of "what's the ideal number of threads for your machine? Don't bother creating 
more threads than that because you're not going to benefit by having more" and then working forward from there.

> Note: As each thread works on its own piece of data in both cases there's no sync required at any one point - 
> you just read/write different parts of the same thing. Which is true both for when you load/write from/to a file 
> and from/to memory.

Not sure if I quite understand what you meant by this note? There is the sync you pointed out as your step 2.2, and then since I need to 
form the results into an onscreen image (most likely a QGraphicsPixmapItem, etc.) there's another sync at your step 3 before I can make
the final onscreen image. Otherwise I think I understand and prefer your concept to what I was doing.

Thanks again for your help! Now to test this out in practice... 
Sean