[PySide] QtConcurrent support?

[Sean Fisk]

Oh, nice, I didn't know about that! I guess that I meant that parallelism
wasn't really its focus :)

You should be able to use whatever C library you want through Cython. That
means you can use MPI or pthreads, too. Not saying it's easy, but it should
be possible.


--
Sean Fisk


On Tue, Jan 14, 2014 at 6:28 AM, iMath <2281570025 at qq.com> wrote:

> thanks for your experienced suggestion,but what do you mean by Cython has
> nothing to do with parallelism ?have you visited this page ?
> http://docs.cython.org/src/userguide/parallelism.html
> BTW ,I have no experience with Cython :)
>
> ------------------ Original ------------------
> *From: * "Sean Fisk";<sean at seanfisk.com>;
> *Date: * Mon, Jan 13, 2014 02:36 AM
> *To: * "iMath"<2281570025 at qq.com>;
> *Cc: * "pyside"<pyside at qt-project.org>;
> *Subject: * Re: [PySide] QtConcurrent support?
>
> On Sun, Jan 12, 2014 at 4:32 AM, iMath <2281570025 at qq.com> wrote:
>
>> thanks for your great explanation ,it helped me a lot !!!
>>
>> the GIL is not present under Ironpython nor Jython and so, running
>> multithreaded Python code under them WILL give the expected performance
>> benefits .
>>
> That is correct. However, if you still want to use CPython, can also avoid
> the GIL with multiple processes using multiprocessing or
> concurrent.futures.ProcessPool. I have also had a good experience with
> Cython <http://cython.org/>, which of course has nothing to do with
> parallelism.
>
> *A short aside:* I studied Python for high-performance computing in
> university and also at my internship with NCAR. My opinion: don’t use it,
> or move your performance-sensitive code to C/C++ extensions or Cython.
> There are a myriad of solutions out there that boil down to one of four
> things: fork(), pthreads, MPI, or CUDA. Tacking on one of
> [multiprocessing, multithreading, message-passing, or stream processing] to
> Python and expecting it to be fast is like fitting a square peg in a round
> hole. It’s a great academic exercise, but not very practical.
>
> That’s just my opinion, so of course take it with a grain of salt. I love
> Python, hence being on this list. But I don’t love it for it’s performance
> :)
>
>>
>>
>> http://www.neotitans.com/resources/python/python-threads-multithreading-tutorial.html
>>
>> ------------------ Original ------------------
>> *From: * "Sean Fisk";<sean at seanfisk.com>;
>> *Date: * Sun, Jan 12, 2014 02:30 PM
>> *To: * "iMath"<2281570025 at qq.com>;
>> *Cc: * "pyside"<pyside at qt-project.org>;
>> *Subject: * Re: [PySide] QtConcurrent support?
>>
>> Hello iMath,
>>
>> This is a complicated topic; one that I have been studying for the past
>> six months or so. I would specifically avoid the term “real
>> multithreading.” There are enough factors that come into play to make that
>> term virtually unqualified.
>> The GIL and Parallelism
>>
>> What it boils down to is this:
>>
>>    - If Python code is running, the GIL *is in effect*.
>>    - If Python code is not running (e.g., C, C++), the GIL *may or may
>>    not be in effect*.
>>
>> Instead of *real multithreading*, let us instead define the term *true
>> in-process parallelism* as follows:
>>
>> *Given that a process has multiple instructions, true in-process
>> parallelism occurs when two or more instructions are executed in the same
>> instant.*
>>
>> Obviously, a machine would need to possess multiple processors for this
>> to happen. But true in-process parallelism is specifically what the GIL
>> prevents. Hence, no matter whether you use threading or QThread or
>> anything else built on these, the GIL will prevent true in-process
>> parallelism for pure Python code.
>>
>> However, just because two instructions can’t happen in the same instant
>> in Python code doesn’t make threading useless. If you want two operations
>> to *appear* to be executing simultaneously, threading in Python will
>> suffice. However, you won’t get any speed boost because your code can’t
>> truly execute simultaneously, thanks to the GIL.
>> Extension Modules
>>
>> Extension modules (e.g., written in C, C++) complicate the issue because
>> they have the possibility of releasing the GIL. PySide, as you know, is an
>> extension module. I can’t say much here, though, because AFAIK the only way
>> to determine whether an extension module is releasing the GIL is to look in
>> its source code. That’s somewhat out of my realm.
>>
>> Just know that each time you call a PySide function which eventually
>> leads to C++ code (or, I guess, binary which was C++ code) being executed
>> there is a possibility of releasing the GIL and achieving true in-process
>> parallelism.
>> QtConcurrent
>>
>> As QtConcurrent is based on QThread, using it for a speedup [if it was
>> in PySide] would be basically moot. That’s because all of QtConcurrent’s
>> hypothetical functions take a callable as an argument, which, if written in
>> Python, would invoke the GIL.
>>
>> In Python 2’s case, the much better options are to use the standard
>> multiprocessing <http://docs.python.org/2/library/multiprocessing.html>module and
>> concurrent.futures <http://pythonhosted.org/futures/> (which has been
>> added to the standard library in Python 3.2). Each of them provide a thread
>> and process pool approach. As implied earlier, only the process-based
>> approach will allow for the possibility of speedup (when in pure Python).
>> These integrate much nicer with Python and are probably better than what Qt
>> could provide.
>> Bottom Line
>>
>> What you choose depends on your use case. By the way, what *are* you
>> trying to achieve?
>>
>> I don’t claim to be a threading or Python or GIL expert, but this is
>> where my research on the topic has pointed me. Please, someone correct me
>> if you take issue.
>>
>> Sincerely,
>>
>>
>>
>> --
>> Sean Fisk
>>
>>
>> On Sat, Jan 11, 2014 at 11:09 PM, iMath <2281570025 at qq.com> wrote:
>>
>>> In these 2 posts
>>> http://nathanhorne.com/?p=353
>>> http://www.riverbankcomputing.com/pipermail/pyqt/2011-January/028961.html
>>>
>>> says
>>> "when the threaded code is calling long running c/c++ functions, such as
>>> a database select, it will release the GIL until the call completes.  This
>>> is all done automatically within python and PySide   This is “real”
>>> multithreading."
>>>
>>> if he is right(QThread is "real" multithreading in Python),then why not
>>> add QtConcurrent to PySide? The documentation doesn't appear to show
>>> any of the functions available in the QtConcurrent namespace
>>>
>>>
>>> _______________________________________________
>>> PySide mailing list
>>> PySide at qt-project.org
>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140114/50570696/attachment.html>