[PySide] QtConcurrent support?

[Sean Fisk]

Hello iMath,

This is a complicated topic; one that I have been studying for the past six
months or so. I would specifically avoid the term “real multithreading.”
There are enough factors that come into play to make that term virtually
unqualified.
 <#>The GIL and Parallelism

What it boils down to is this:

   - If Python code is running, the GIL *is in effect*.
   - If Python code is not running (e.g., C, C++), the GIL *may or may not
   be in effect*.

Instead of *real multithreading*, let us instead define the term *true
in-process parallelism* as follows:

*Given that a process has multiple instructions, true in-process
parallelism occurs when two or more instructions are executed in the same
instant.*

Obviously, a machine would need to possess multiple processors for this to
happen. But true in-process parallelism is specifically what the GIL
prevents. Hence, no matter whether you use threading or QThread or anything
else built on these, the GIL will prevent true in-process parallelism for
pure Python code.

However, just because two instructions can’t happen in the same instant in
Python code doesn’t make threading useless. If you want two operations to
*appear* to be executing simultaneously, threading in Python will suffice.
However, you won’t get any speed boost because your code can’t truly
execute simultaneously, thanks to the GIL.
 <#>Extension Modules

Extension modules (e.g., written in C, C++) complicate the issue because
they have the possibility of releasing the GIL. PySide, as you know, is an
extension module. I can’t say much here, though, because AFAIK the only way
to determine whether an extension module is releasing the GIL is to look in
its source code. That’s somewhat out of my realm.

Just know that each time you call a PySide function which eventually leads
to C++ code (or, I guess, binary which was C++ code) being executed there
is a possibility of releasing the GIL and achieving true in-process
parallelism.
 <#>QtConcurrent

As QtConcurrent is based on QThread, using it for a speedup [if it was in
PySide] would be basically moot. That’s because all of QtConcurrent’s
hypothetical functions take a callable as an argument, which, if written in
Python, would invoke the GIL.

In Python 2’s case, the much better options are to use the standard
multiprocessing
<http://docs.python.org/2/library/multiprocessing.html>module and
concurrent.futures <http://pythonhosted.org/futures/> (which has been added
to the standard library in Python 3.2). Each of them provide a thread and
process pool approach. As implied earlier, only the process-based approach
will allow for the possibility of speedup (when in pure Python). These
integrate much nicer with Python and are probably better than what Qt could
provide.
 <#>Bottom Line

What you choose depends on your use case. By the way, what *are* you trying
to achieve?

I don’t claim to be a threading or Python or GIL expert, but this is where
my research on the topic has pointed me. Please, someone correct me if you
take issue.

Sincerely,



--
Sean Fisk


On Sat, Jan 11, 2014 at 11:09 PM, iMath <2281570025 at qq.com> wrote:

> In these 2 posts
> http://nathanhorne.com/?p=353
> http://www.riverbankcomputing.com/pipermail/pyqt/2011-January/028961.html
>
> says
> "when the threaded code is calling long running c/c++ functions, such as a
> database select, it will release the GIL until the call completes.  This is
> all done automatically within python and PySide   This is “real”
> multithreading."
>
> if he is right(QThread is "real" multithreading in Python),then why not
> add QtConcurrent to PySide? The documentation doesn't appear to show any
> of the functions available in the QtConcurrent namespace
>
>
> _______________________________________________
> PySide mailing list
> PySide at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/pyside
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140112/20154264/attachment.html>