[PySide] QtConcurrent support?

[Sean Fisk]

On Sun, Jan 12, 2014 at 4:32 AM, iMath <2281570025 at qq.com> wrote:

> thanks for your great explanation ,it helped me a lot !!!
>
> the GIL is not present under Ironpython nor Jython and so, running
> multithreaded Python code under them WILL give the expected performance
> benefits .
>
That is correct. However, if you still want to use CPython, can also avoid
the GIL with multiple processes using multiprocessing or
concurrent.futures.ProcessPool. I have also had a good experience with
Cython <http://cython.org/>, which of course has nothing to do with
parallelism.

*A short aside:* I studied Python for high-performance computing in
university and also at my internship with NCAR. My opinion: don’t use it,
or move your performance-sensitive code to C/C++ extensions or Cython.
There are a myriad of solutions out there that boil down to one of four
things: fork(), pthreads, MPI, or CUDA. Tacking on one of [multiprocessing,
multithreading, message-passing, or stream processing] to Python and
expecting it to be fast is like fitting a square peg in a round hole. It’s
a great academic exercise, but not very practical.

That’s just my opinion, so of course take it with a grain of salt. I love
Python, hence being on this list. But I don’t love it for it’s performance
:)

>
>
> http://www.neotitans.com/resources/python/python-threads-multithreading-tutorial.html
>
> ------------------ Original ------------------
> *From: * "Sean Fisk";<sean at seanfisk.com>;
> *Date: * Sun, Jan 12, 2014 02:30 PM
> *To: * "iMath"<2281570025 at qq.com>;
> *Cc: * "pyside"<pyside at qt-project.org>;
> *Subject: * Re: [PySide] QtConcurrent support?
>
> Hello iMath,
>
> This is a complicated topic; one that I have been studying for the past
> six months or so. I would specifically avoid the term “real
> multithreading.” There are enough factors that come into play to make that
> term virtually unqualified.
> The GIL and Parallelism
>
> What it boils down to is this:
>
>    - If Python code is running, the GIL *is in effect*.
>    - If Python code is not running (e.g., C, C++), the GIL *may or may
>    not be in effect*.
>
> Instead of *real multithreading*, let us instead define the term *true
> in-process parallelism* as follows:
>
> *Given that a process has multiple instructions, true in-process
> parallelism occurs when two or more instructions are executed in the same
> instant.*
>
> Obviously, a machine would need to possess multiple processors for this to
> happen. But true in-process parallelism is specifically what the GIL
> prevents. Hence, no matter whether you use threading or QThread or
> anything else built on these, the GIL will prevent true in-process
> parallelism for pure Python code.
>
> However, just because two instructions can’t happen in the same instant in
> Python code doesn’t make threading useless. If you want two operations to
> *appear* to be executing simultaneously, threading in Python will
> suffice. However, you won’t get any speed boost because your code can’t
> truly execute simultaneously, thanks to the GIL.
> Extension Modules
>
> Extension modules (e.g., written in C, C++) complicate the issue because
> they have the possibility of releasing the GIL. PySide, as you know, is an
> extension module. I can’t say much here, though, because AFAIK the only way
> to determine whether an extension module is releasing the GIL is to look in
> its source code. That’s somewhat out of my realm.
>
> Just know that each time you call a PySide function which eventually leads
> to C++ code (or, I guess, binary which was C++ code) being executed there
> is a possibility of releasing the GIL and achieving true in-process
> parallelism.
> QtConcurrent
>
> As QtConcurrent is based on QThread, using it for a speedup [if it was in
> PySide] would be basically moot. That’s because all of QtConcurrent’s
> hypothetical functions take a callable as an argument, which, if written in
> Python, would invoke the GIL.
>
> In Python 2’s case, the much better options are to use the standard
> multiprocessing <http://docs.python.org/2/library/multiprocessing.html>module and
> concurrent.futures <http://pythonhosted.org/futures/> (which has been
> added to the standard library in Python 3.2). Each of them provide a thread
> and process pool approach. As implied earlier, only the process-based
> approach will allow for the possibility of speedup (when in pure Python).
> These integrate much nicer with Python and are probably better than what Qt
> could provide.
> Bottom Line
>
> What you choose depends on your use case. By the way, what *are* you
> trying to achieve?
>
> I don’t claim to be a threading or Python or GIL expert, but this is where
> my research on the topic has pointed me. Please, someone correct me if you
> take issue.
>
> Sincerely,
>
>
>
> --
> Sean Fisk
>
>
> On Sat, Jan 11, 2014 at 11:09 PM, iMath <2281570025 at qq.com> wrote:
>
>> In these 2 posts
>> http://nathanhorne.com/?p=353
>> http://www.riverbankcomputing.com/pipermail/pyqt/2011-January/028961.html
>>
>> says
>> "when the threaded code is calling long running c/c++ functions, such as
>> a database select, it will release the GIL until the call completes.  This
>> is all done automatically within python and PySide   This is “real”
>> multithreading."
>>
>> if he is right(QThread is "real" multithreading in Python),then why not
>> add QtConcurrent to PySide? The documentation doesn't appear to show any
>> of the functions available in the QtConcurrent namespace
>>
>>
>> _______________________________________________
>> PySide mailing list
>> PySide at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/pyside
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140112/f76f32ef/attachment.html>