[Interest] QtConcurrent programming need help

[Philipp Menke]

Am 27.09.2012 20:32, schrieb Sujan Dasmahapatra:
> I am working on an existing application where  threads are submitted 
> concurrently. he is submitting 8 threads together and then he is 
> waiting for all 8 threads to be finished to submit the next 8 threads. 
> In this way after every 8 threads he is waiting and a lot of time is 
> being taken for waiting. Can you me some suggestions how can I skip 
> this waiting after submitting 8 threads to improve speed. Total number 
> of cpu is 8 so he is submitting 8 threads at  a time.
> Submitting threads like this
> int nCpus = Environment::Get_No_Of_Cpus(); // ==8
> for(z=0; z<21; z++)
> {
> for (int cpuID=0; cpuID<nCpus; cpuID++)
> {
> Th = QtConcurrent::run(this, zLayerFunction,z,...,cpuID);
> Threads.push_back(Th);
> }
> //then he after submitting 8 threads, he is waiting for all 8 threads 
> to be finished to submit the next 8 threads.
> for(int th=0; th<Threads.size(); th++ )
> {
>      Threads.at(th).waitForFinished();
>      Threads.cancel();
> }
> Threads.clear();
> z--;
> }
> How can i improve the speed so that it doesn't wait. How can I make it 
> continuous. Any suggestions would be highly appreciated. Thanks Sujan
Try a Thread-Pool-Pattern. So you got a queue where all work to be done 
is stored. As soon as one Thread is finished he checks the queue for new 
work.
If you have unindependent work (so step 2,3 and 4 require 1 to be 
already done) you can push 2,3 and 4 to the pool as last step of 1.

This approach has several positive effects:
- You need the time for starting the workers just one time.
- There is a way to deal with dependencies between steps.
- You got full usage of your hardware for your tasks (ignoring time to 
manage pool)

here is the Wikipedia-artical on that pattern: 
http://en.wikipedia.org/wiki/Thread_pool_pattern