[PySide] QtConcurrent support?

Tue Jan 14 12:28:57 CET 2014

thanks for your experienced suggestion,but what do you mean by Cython has nothing to do with parallelism ?have you visited this page ?
http://docs.cython.org/src/userguide/parallelism.html
BTW ,I have no experience with Cython :)

------------------ Original ------------------
From:  "Sean Fisk";<sean at seanfisk.com>;
Date:  Mon, Jan 13, 2014 02:36 AM
To:  "iMath"<2281570025 at qq.com>; 
Cc:  "pyside"<pyside at qt-project.org>; 
Subject:  Re: [PySide] QtConcurrent support?

On Sun, Jan 12, 2014 at 4:32 AM, iMath <2281570025 at qq.com> wrote:
 thanks for your great explanation ,it helped me a lot !!!

the GIL is not present under Ironpython nor Jython and so, running multithreaded Python code under them WILL give the expected performance benefits .

That is correct. However, if you still want to use CPython, can also avoid the GIL with multiple processes using multiprocessing or concurrent.futures.ProcessPool. I have also had a good experience with Cython, which of course has nothing to do with parallelism.

A short aside: I studied Python for high-performance computing in university and also at my internship with NCAR. My opinion: don’t use it, or move your performance-sensitive code to C/C++ extensions or Cython. There are a myriad of solutions out there that boil down to one of four things: fork(), pthreads, MPI, or CUDA. Tacking on one of [multiprocessing, multithreading, message-passing, or stream processing] to Python and expecting it to be fast is like fitting a square peg in a round hole. It’s a great academic exercise, but not very practical.

That’s just my opinion, so of course take it with a grain of salt. I love Python, hence being on this list. But I don’t love it for it’s performance :)

 http://www.neotitans.com/resources/python/python-threads-multithreading-tutorial.html

------------------ Original ------------------
 From:  "Sean Fisk";<sean at seanfisk.com>;
 Date:  Sun, Jan 12, 2014 02:30 PM
To:  "iMath"<2281570025 at qq.com>; 
Cc:  "pyside"<pyside at qt-project.org>; 
 Subject:  Re: [PySide] QtConcurrent support?

Hello iMath,

This is a complicated topic; one that I have been studying for the past six months or so. I would specifically avoid the term “real multithreading.” There are enough factors that come into play to make that term virtually unqualified.

The GIL and Parallelism

What it boils down to is this:

If Python code is running, the GIL is in effect.

If Python code is not running (e.g., C, C++), the GIL may or may not be in effect.

Instead of real multithreading, let us instead define the term true in-process parallelism as follows:

Given that a process has multiple instructions, true in-process parallelism occurs when two or more instructions are executed in the same instant.

Obviously, a machine would need to possess multiple processors for this to happen. But true in-process parallelism is specifically what the GIL prevents. Hence, no matter whether you use threading or QThread or anything else built on these, the GIL will prevent true in-process parallelism for pure Python code.

However, just because two instructions can’t happen in the same instant in Python code doesn’t make threading useless. If you want two operations to appear to be executing simultaneously, threading in Python will suffice. However, you won’t get any speed boost because your code can’t truly execute simultaneously, thanks to the GIL.

Extension Modules

Extension modules (e.g., written in C, C++) complicate the issue because they have the possibility of releasing the GIL. PySide, as you know, is an extension module. I can’t say much here, though, because AFAIK the only way to determine whether an extension module is releasing the GIL is to look in its source code. That’s somewhat out of my realm.

Just know that each time you call a PySide function which eventually leads to C++ code (or, I guess, binary which was C++ code) being executed there is a possibility of releasing the GIL and achieving true in-process parallelism.

QtConcurrent

As QtConcurrent is based on QThread, using it for a speedup [if it was in PySide] would be basically moot. That’s because all of QtConcurrent’s hypothetical functions take a callable as an argument, which, if written in Python, would invoke the GIL.

In Python 2’s case, the much better options are to use the standard multiprocessing module and concurrent.futures (which has been added to the standard library in Python 3.2). Each of them provide a thread and process pool approach. As implied earlier, only the process-based approach will allow for the possibility of speedup (when in pure Python). These integrate much nicer with Python and are probably better than what Qt could provide.

Bottom Line

What you choose depends on your use case. By the way, what are you trying to achieve?

I don’t claim to be a threading or Python or GIL expert, but this is where my research on the topic has pointed me. Please, someone correct me if you take issue.

Sincerely,

--
Sean Fisk

On Sat, Jan 11, 2014 at 11:09 PM, iMath <2281570025 at qq.com> wrote:
 In these 2 posts
http://nathanhorne.com/?p=353
http://www.riverbankcomputing.com/pipermail/pyqt/2011-January/028961.html

says 
"when the threaded code is calling long running c/c++ functions, such as a database select, it will release the GIL until the call completes.  This is all done automatically within python and PySide   This is “real” multithreading."

if he is right(QThread is "real" multithreading in Python),then why not add QtConcurrent to PySide? The documentation doesn't appear to show any of the functions available in the QtConcurrent namespace

_______________________________________________
 PySide mailing list
 PySide at qt-project.org
 http://lists.qt-project.org/mailman/listinfo/pyside
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140114/eb045bb9/attachment.html>