[PySide] Keeping GUI responsive
Cristián Maureira-Fredes
Cristian.Maureira-Fredes at qt.io
Fri Dec 6 17:43:10 CET 2019
Hello Israel,
Bad news, it seems you have found a similar situation
as described in:
https://bugreports.qt.io/browse/PYSIDE-803
This is a P1 bug that we have been trying to solve
since a couple of releases ago.
It's not a simple bug, and it might imply
to do a complete refactoring on how we interact
with the Python GIL.
At least on the report you can find a workaround
for what is described there, which is really hacky IMO,
but it works.
In simple words, the issue is that we interact
with the GIL too much, and due to this, we are holding
the GIL when we shouldn't:
IIRC We hold the GIL all the time, releasing it
only for the C++ calls, while PyQt for example
does the opposite, it holds the GIL when a Python call
are in progress, and it releases it the rest of the time.
We will do our best to get rid of this issue.
Cheers
On 12/5/19 9:19 PM, Israel Brewster wrote:
>> On Dec 5, 2019, at 10:40 AM, Jason H <jhihn at gmx.com
>> <mailto:jhihn at gmx.com>> wrote:
>>
>> That doesn't make any sense. Can you post a minimal example?
>
> Sure, see the end of this message (only 58 lines, so I figured inline
> was probably fine). It’s not *quite* as extreme an effect as what I am
> seeing in my application (probably due to over-simplification), but it
> still illustrates the problem: click the button, and try typing quickly
> in the the line edit. While the thread is running, typing speed is
> noticeably reduced over when the thread is not running. The thread
> apparently doesn’t *completely* block the GUI (I’m not sure why - maybe
> the GIL is released between loop iterations?), but it does introduce
> noticeable lags. It’s those lags I’m trying to get rid of.
>
> Simplified example:
>
> import time
> from PySide2.QtWidgets import (QApplication,
> QMainWindow,
> QPushButton,
> QLineEdit,
> QVBoxLayout,
> QWidget)
>
> from PySide2.QtCore import (QThread)
>
>
> class LongRunningThread(QThread):
> """A process that takes several seconds to complete, involving
> manipulation of data
> contained in large data structures that are members of the
> MainWindow instance."""
> def __init__(self, dataset):
> super().__init__()
> self.dataset = dataset
>
> def run(self):
> print("Thread running")
> start_time = time.time()
> for idx, val in enumerate(self.dataset):
> self.dataset[idx] = val * 50
> print(f"Thread complete after: {time.time()-start_time}")
>
>
> class MainWindow(QMainWindow):
> """Main application class, contains large data structures that can
> not/should
> not be easily copied to another process for various reasons such as
> memory usage."""
>
> def __init__(self):
> super().__init__()
> central_widget = QWidget(self)
> layout = QVBoxLayout()
> central_widget.setLayout(layout)
> self.setCentralWidget(central_widget)
>
> self.text_entry = QLineEdit(self)
> layout.addWidget(self.text_entry)
>
> self.button = QPushButton("Run Thread")
> self.button.clicked.connect(self._run_blocking_thread)
> layout.addWidget(self.button)
>
> # Generate a random dataset. Adjust size so operations take a
> while.
> self.large_dataset = list(range(20000000))
>
> def _run_blocking_thread(self):
> self.thread = LongRunningThread(self.large_dataset)
> self.thread.start()
> print("Thread started!")
>
>
> if __name__ == "__main__":
> app = QApplication()
> win = MainWindow()
> win.show()
> app.exec_()
>
> ---
> Israel Brewster
> Software Engineer
> Alaska Volcano Observatory
> Geophysical Institute - UAF
> 2156 Koyukuk Drive
> Fairbanks AK 99775-7320
> Work: 907-474-5172
> cell: 907-328-9145
>
>>
>>> Sent: Thursday, December 05, 2019 at 7:16 PM
>>> From: "Israel Brewster" <ijbrewster at alaska.edu
>>> <mailto:ijbrewster at alaska.edu>>
>>> To: "Jason H" <jhihn at gmx.com <mailto:jhihn at gmx.com>>
>>> Cc: "Kerby Geffrard" <kerby.geffrard at autodesk.com
>>> <mailto:kerby.geffrard at autodesk.com>>, "pyside at qt-project.org
>>> <mailto:pyside at qt-project.org>" <pyside at qt-project.org
>>> <mailto:pyside at qt-project.org>>
>>> Subject: Re: [PySide] Keeping GUI responsive
>>>
>>>>
>>>> On Dec 5, 2019, at 7:18 AM, Jason H <jhihn at gmx.com
>>>> <mailto:jhihn at gmx.com>> wrote:
>>>>
>>>>
>>>> I don't thinkso. The Python multiprocessing module uses multiple
>>>> processes, so you have to copy data between them. QThreads are in
>>>> the same process so no copy is needed.
>>>> GIL only matters when you're in interpreted code.
>>>
>>> Yes, but we’re talking about threads vs QThread - I already ruled out
>>> the python multiprocessing module due to the data copy issue. You are
>>> correct that the GIL only matters when you’re in interpreted code,
>>> but given the behavior I’m seeing, I have to assume my function *is*
>>> interpreted code - that is, the time intensive portions are not
>>> calling C libraries. Otherwise running the function in a separate
>>> thread would work to keep the GUI responsive.
>>>
>>> I did try using a QThread - with the same result as using a python
>>> thread. The GUI freezes while the function is running. As I stated
>>> before, I can only assume this is due to the GIL. And in case there
>>> is any question about if I used QThreads correctly or not, here is
>>> the code I used to start the thread:
>>>
>>> thread = ResultsThread(self)
>>> thread.resultsReady.connect(self._on_results_ready)
>>> thread.start()
>>>
>>> ResultsThread is, of course, a QThread subclass in which I
>>> re-implemented the run() function to do my calculations.
>>>
>>> As a further test, I started another thread that just spit out the
>>> “current” time (as per time.time()) every .2 seconds. What I’m seeing
>>> is that under normal operation, this other thread outputs as expected
>>> every (approximately) .2 seconds. However, while the QThread (or
>>> python thread) is running, this other thread “stutters”. This
>>> indicates that while the QThread operation doesn’t block other
>>> threads completely, there are a number of operations taking place in
>>> it that *DO* block other threads for a noticeable length of time,
>>> causing the next iteration of my timer loop to be delayed. Such
>>> blocking would obviously apply to the main GUI thread as well.
>>>
>>> So yes, running in a QThread blocks the main GUI thread the same as
>>> running in a python thread does.
>>> ---
>>> Israel Brewster
>>> Software Engineer
>>> Alaska Volcano Observatory
>>> Geophysical Institute - UAF
>>> 2156 Koyukuk Drive
>>> Fairbanks AK 99775-7320
>>> Work: 907-474-5172
>>> cell: 907-328-9145
>>>
>>>>
>>>>
>>>> Sent: Thursday, December 05, 2019 at 9:42 AM
>>>> From: "Israel Brewster" <ijbrewster at alaska.edu
>>>> <mailto:ijbrewster at alaska.edu>>
>>>> To: "Kerby Geffrard" <kerby.geffrard at autodesk.com
>>>> <mailto:kerby.geffrard at autodesk.com>>
>>>> Cc: "pyside at qt-project.org <mailto:pyside at qt-project.org>"
>>>> <pyside at qt-project.org <mailto:pyside at qt-project.org>>
>>>> Subject: Re: [PySide] Keeping GUI responsive
>>>>
>>>> On Dec 4, 2019, at 4:53 PM, Kerby Geffrard
>>>> <kerby.geffrard at autodesk.com
>>>> <mailto:kerby.geffrard at autodesk.com><mailto:kerby.geffrard at autodesk.com>>
>>>> wrote:
>>>>
>>>> I think you can try to use a QThread for this.
>>>>
>>>> I was under the impression that QThreads had the same limitations
>>>> that regular python threads had, but I’ll give it a shot. The need
>>>> to subclass QThread rather than simply being able to execute a
>>>> function complicates things however - I have to make sure that said
>>>> subclass has access to all the original class members it needs.
>>>> ---
>>>> Israel Brewster
>>>> Software Engineer
>>>> Alaska Volcano Observatory
>>>> Geophysical Institute - UAF
>>>> 2156 Koyukuk Drive
>>>> Fairbanks AK 99775-7320
>>>> Work: 907-474-5172
>>>> cell: 907-328-9145
>>>>
>>>>
>>>> Le 4 déc. 2019 à 18:41, Israel Brewster <ijbrewster at alaska.edu
>>>> <mailto:ijbrewster at alaska.edu><mailto:ijbrewster at alaska.edu>> a écrit :
>>>>
>>>> I know this is a FAQ, however I haven’t been able to make any of
>>>> the standard answers work for me. Here’s the situation:
>>>>
>>>> - using PySide2 5.12.2
>>>> - I have an object (QMainWindow subclass) that contains most of the
>>>> code for my application
>>>> - One of the functions that runs in response to user input takes
>>>> around 2 seconds to run. The GUI obviously freezes during this time
>>>> (BAD!)
>>>> - Said function needs to access and modify several large variables
>>>> (pandas data frames) from the main object
>>>>
>>>> So here’s the problem: If I run this function as a separate (python)
>>>> thread, that doesn’t help - the GUI is still frozen. I’m thinking
>>>> this is due to the GIL, but I could be wrong about that. Running
>>>> under the multiprocessing module, however, doesn’t appear to be an
>>>> option due to the number and size of the data structures that the
>>>> function needs to modify, and if I try just to see what happens, the
>>>> process actually crashes.
>>>>
>>>> So what are my options here? How can I keep the GUI responsive while
>>>> this function runs, without being able to spin it off as a separate
>>>> process? Or is the only option going to be to completely rip apart
>>>> the function and try to re-build it in such a way that it can,
>>>> somehow, still access the memory from the main thread, while doing
>>>> the processing in a separate function?
>>>> ---
>>>> Israel Brewster
>>>> Software Engineer
>>>> Alaska Volcano Observatory
>>>> Geophysical Institute - UAF
>>>> 2156 Koyukuk Drive
>>>> Fairbanks AK 99775-7320
>>>> Work: 907-474-5172
>>>> cell: 907-328-9145
>>>> _______________________________________________
>>>> PySide mailing list
>>>> PySide at qt-project.org
>>>> <mailto:PySide at qt-project.org><mailto:PySide at qt-project.org>
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.qt-2Dproject.org_listinfo_pyside&d=DwIGaQ&c=76Q6Tcqc-t2x0ciWn7KFdCiqt6IQ7a_IF9uzNzd_2pA&r=Sh-EhW_r32FEgrsAfK3Hc0du2ebOZ7PrbchABO_xtMo&m=r4Ron7tNrQN9BNhzXVlsKvYoyWrnjjOzLwALVV_CgnA&s=T2UjSQWTnYFAfFbfdTlMgLZnfC1dlXhNy1SmPijkWbs&e=<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.qt-2Dproject.org_listinfo_pyside&d=DwIGaQ&c=76Q6Tcqc-t2x0ciWn7KFdCiqt6IQ7a_IF9uzNzd_2pA&r=Sh-EhW_r32FEgrsAfK3Hc0du2ebOZ7PrbchABO_xtMo&m=r4Ron7tNrQN9BNhzXVlsKvYoyWrnjjOzLwALVV_CgnA&s=T2UjSQWTnYFAfFbfdTlMgLZnfC1dlXhNy1SmPijkWbs&e=>
>>>> _______________________________________________ PySide mailing
>>>> listPySide at qt-project.org
>>>> <mailto:PySide at qt-project.org>https://lists.qt-project.org/listinfo/pyside<https://lists.qt-project.org/listinfo/pyside>
>
--
Dr. Cristian Maureira-Fredes
R&D Manager
The Qt Company GmbH
Erich-Thilo-Str. 10
D-12489 Berlin
Geschäftsführer: Mika Pälsi,
Juha Varelius, Mika Harjuaho
Sitz der Gesellschaft: Berlin,
Registergericht: Amtsgericht
Charlottenburg, HRB 144331 B
--
More information about the PySide
mailing list