[PySide] Keeping GUI responsive

Jason H jhihn at gmx.com
Fri Dec 6 17:33:00 CET 2019


What those links say and what is actually happening here are two different things. 

It looks like the only way he can do this is to write the thread code in C++, and export that through Shiboken to Python. So the lines:
    for idx, val in enumerate(self.dataset):
        self.dataset[idx] = val * 50

Have to:
1) Operate on Qt data (QMap?)
2) Have the QRunnable implemented in C++
3) Export the class to Python,
4) Have Python start the thread

I find myself asking why QThread exits at all n PySide or at least not have documentation saying that using the class(es) as one would in C++ will not have the same behavior as in C++.


> Sent: Friday, December 06, 2019 at 11:04 AM
> From: "Tommaso Massimi" <tmassimi at gmail.com>
> To: "Jason H" <jhihn at gmx.com>
> Cc: "pyside at qt-project.org" <pyside at qt-project.org>
> Subject: Re: [PySide] Keeping GUI responsive
>
> Hi,
> 
> at this link you can find (a link to) a deep discussion about QT
> Threads vs Python threads
> 
> https://stackoverflow.com/questions/1595649/threading-in-a-pyqt-application-use-qt-threads-or-python-threads
> 
> 
> at this link you can found golden rules about threads vs multiprocess in python
> 
> https://medium.com/towards-artificial-intelligence/the-why-when-and-how-of-using-python-multi-threading-and-multi-processing-afd1b8a8ecca
> 
> Hope this helps you
> 
> On Fri, Dec 6, 2019 at 4:56 PM Jason H <jhihn at gmx.com> wrote:
> >
> > I tried this, myself and it's not much better. It stops only runs threads = #cores and queues the rest which is slightly better but not much. As a result each thread completes faster, but the overall time is the same.
> >
> > Help?
> >
> >
> >
> > Sent: Friday, December 06, 2019 at 10:40 AM
> > From: "Jason H" <jhihn at gmx.com>
> > To: "Jason H" <jhihn at gmx.com>
> > Cc: "Israel Brewster" <ijbrewster at alaska.edu>, "pyside at qt-project.org" <pyside at qt-project.org>
> > Subject: Re: [PySide] Keeping GUI responsive
> >
> > I found this: https://www.learnpyqt.com/courses/concurrent-execution/multithreading-pyqt-applications-qthreadpool/
> > Which advises use of QRunnable. In my C++ code, I've pretty much stopped using QThread and am using QRunnables anyway.
> >
> > Give that a try.
> >
> > Sent: Friday, December 06, 2019 at 10:33 AM
> > From: "Jason H" <jhihn at gmx.com>
> > To: "Israel Brewster" <ijbrewster at alaska.edu>
> > Cc: "pyside at qt-project.org" <pyside at qt-project.org>
> > Subject: Re: [PySide] Keeping GUI responsive
> > I can confirm your experience on Mojave.
> >
> > If you increase your dataset 100-fold, then I get a beach ball. Based on what I am seeing here, QThread doesn't actually quite work. To test this, I changed it to:
> >    self.threads = [ LongRunningThread(self.large_dataset, x) for x in range(24)]
> >
> > and
> >    [thread.start() for thread in self.threads]
> >
> > Which should saturate my 16 core CPU, at least briefly. At no time did it go over 15%. OSX's activity Monitor showed Python maxed out at 100.25%
> > The threads increase from 4 at idle to 28. But they were not sistricnuted across cores as one would expect.
> > it took ~69s per thread at 32 threads
> > ~34 for 16
> > 16 for 8
> > 8 for 4
> > 4 for 2
> >
> > Which means that at no time did any of my other cores get used. It is like PySide is only running on the main thread, or only running one thread itself.
> >
> >
> > Sent: Thursday, December 05, 2019 at 3:19 PM
> > From: "Israel Brewster" <ijbrewster at alaska.edu>
> > To: "Jason H" <jhihn at gmx.com>
> > Cc: "Kerby Geffrard" <kerby.geffrard at autodesk.com>, "pyside at qt-project.org" <pyside at qt-project.org>
> > Subject: Re: [PySide] Keeping GUI responsive
> >
> > On Dec 5, 2019, at 10:40 AM, Jason H <jhihn at gmx.com> wrote:
> >
> > That doesn't make any sense. Can you post a minimal example?
> >
> >
> > Sure, see the end of this message (only 58 lines, so I figured inline was probably fine). It’s not *quite* as extreme an effect as what I am seeing in my application (probably due to over-simplification), but it still illustrates the problem: click the button, and try typing quickly in the the line edit. While the thread is running, typing speed is noticeably reduced over when the thread is not running. The thread apparently doesn’t *completely* block the GUI (I’m not sure why - maybe the GIL is released between loop iterations?), but it does introduce noticeable lags. It’s those lags I’m trying to get rid of.
> >
> > Simplified example:
> >
> > import time
> > from PySide2.QtWidgets import (QApplication,
> >                                QMainWindow,
> >                                QPushButton,
> >                                QLineEdit,
> >                                QVBoxLayout,
> >                                QWidget)
> >
> > from PySide2.QtCore import (QThread)
> >
> >
> > class LongRunningThread(QThread):
> >     """A process that takes several seconds to complete, involving manipulation of data
> >     contained in large data structures that are members of the MainWindow instance."""
> >     def __init__(self, dataset):
> >         super().__init__()
> >         self.dataset = dataset
> >
> >     def run(self):
> >         print("Thread running")
> >         start_time = time.time()
> >         for idx, val in enumerate(self.dataset):
> >             self.dataset[idx] = val * 50
> >         print(f"Thread complete after: {time.time()-start_time}")
> >
> >
> > class MainWindow(QMainWindow):
> >     """Main application class, contains large data structures that can not/should
> >     not be easily copied to another process for various reasons such as memory usage."""
> >
> >     def __init__(self):
> >         super().__init__()
> >         central_widget = QWidget(self)
> >         layout = QVBoxLayout()
> >         central_widget.setLayout(layout)
> >         self.setCentralWidget(central_widget)
> >
> >         self.text_entry = QLineEdit(self)
> >         layout.addWidget(self.text_entry)
> >
> >         self.button = QPushButton("Run Thread")
> >         self.button.clicked.connect(self._run_blocking_thread)
> >         layout.addWidget(self.button)
> >
> >         # Generate a random dataset. Adjust size so operations take a while.
> >         self.large_dataset = list(range(20000000))
> >
> >     def _run_blocking_thread(self):
> >         self.thread = LongRunningThread(self.large_dataset)
> >         self.thread.start()
> >         print("Thread started!")
> >
> >
> > if __name__ == "__main__":
> >     app = QApplication()
> >     win = MainWindow()
> >     win.show()
> >     app.exec_()
> >
> >
> >
> > ---
> > Israel Brewster
> > Software Engineer
> > Alaska Volcano Observatory
> > Geophysical Institute - UAF
> > 2156 Koyukuk Drive
> > Fairbanks AK 99775-7320
> > Work: 907-474-5172
> > cell:  907-328-9145
> >
> >
> >
> >
> > Sent: Thursday, December 05, 2019 at 7:16 PM
> > From: "Israel Brewster" <ijbrewster at alaska.edu>
> > To: "Jason H" <jhihn at gmx.com>
> > Cc: "Kerby Geffrard" <kerby.geffrard at autodesk.com>, "pyside at qt-project.org" <pyside at qt-project.org>
> > Subject: Re: [PySide] Keeping GUI responsive
> >
> >
> >
> > On Dec 5, 2019, at 7:18 AM, Jason H <jhihn at gmx.com> wrote:
> >
> >
> > I don't thinkso. The Python multiprocessing module uses multiple processes, so you have to copy data between them. QThreads are in the same process so no copy is needed.
> > GIL only matters when you're in interpreted code.
> >
> >
> > Yes, but we’re talking about threads vs QThread - I already ruled out the python multiprocessing module due to the data copy issue. You are correct that the GIL only matters when you’re in interpreted code, but given the behavior I’m seeing, I have to assume my function *is* interpreted code - that is, the time intensive portions are not calling C libraries. Otherwise running the function in a separate thread would work to keep the GUI responsive.
> >
> > I did try using a QThread - with the same result as using a python thread. The GUI freezes while the function is running. As I stated before, I can only assume this is due to the GIL. And in case there is any question about if I used QThreads correctly or not, here is the code I used to start the thread:
> >
> > thread = ResultsThread(self)
> > thread.resultsReady.connect(self._on_results_ready)
> > thread.start()
> >
> > ResultsThread is, of course, a QThread subclass in which I re-implemented the run() function to do my calculations.
> >
> > As a further test, I started another thread that just spit out the “current” time (as per time.time()) every .2 seconds. What I’m seeing is that under normal operation, this other thread outputs as expected every (approximately) .2 seconds. However, while the QThread (or python thread) is running, this other thread “stutters”. This indicates that while the QThread operation doesn’t block other threads completely, there are a number of operations taking place in it that *DO* block other threads for a noticeable length of time, causing the next iteration of my timer loop to be delayed. Such blocking would obviously apply to the main GUI thread as well.
> >
> > So yes, running in a QThread blocks the main GUI thread the same as running in a python thread does.
> > ---
> > Israel Brewster
> > Software Engineer
> > Alaska Volcano Observatory
> > Geophysical Institute - UAF
> > 2156 Koyukuk Drive
> > Fairbanks AK 99775-7320
> > Work: 907-474-5172
> > cell:  907-328-9145
> >
> >
> >
> >
> > Sent: Thursday, December 05, 2019 at 9:42 AM
> > From: "Israel Brewster" <ijbrewster at alaska.edu>
> > To: "Kerby Geffrard" <kerby.geffrard at autodesk.com>
> > Cc: "pyside at qt-project.org" <pyside at qt-project.org>
> > Subject: Re: [PySide] Keeping GUI responsive
> >
> > On Dec 4, 2019, at 4:53 PM, Kerby Geffrard <kerby.geffrard at autodesk.com <mailto:kerby.geffrard at autodesk.com>> wrote:
> >
> > I think you can try to use a QThread for this.
> >
> > I was under the impression that QThreads had the same limitations that regular python threads had, but I’ll give it a shot. The need to subclass QThread rather than simply being able to execute a function complicates things however - I have to make sure that said subclass has access to all the original class members it needs.
> > ---
> > Israel Brewster
> > Software Engineer
> > Alaska Volcano Observatory
> > Geophysical Institute - UAF
> > 2156 Koyukuk Drive
> > Fairbanks AK 99775-7320
> > Work: 907-474-5172
> > cell:  907-328-9145
> >
> >
> > Le 4 déc. 2019 à 18:41, Israel Brewster <ijbrewster at alaska.edu <mailto:ijbrewster at alaska.edu>> a écrit :
> >
> >  I know this is a FAQ, however I haven’t been able to make any of the standard answers work for me. Here’s the situation:
> >
> > - using PySide2 5.12.2
> > - I have an object (QMainWindow subclass) that contains most of the code for my application
> > - One of the functions that runs in response to user input takes around 2 seconds to run. The GUI obviously freezes during this time (BAD!)
> > - Said function needs to access and modify several large variables (pandas data frames) from the main object
> >
> > So here’s the problem: If I run this function as a separate (python) thread, that doesn’t help - the GUI is still frozen. I’m thinking this is due to the GIL, but I could be wrong about that. Running under the multiprocessing module, however, doesn’t appear to be an option due to the number and size of the data structures that the function needs to modify, and if I try just to see what happens, the process actually crashes.
> >
> > So what are my options here? How can I keep the GUI responsive while this function runs, without being able to spin it off as a separate process? Or is the only option going to be to completely rip apart the function and try to re-build it in such a way that it can, somehow, still access the memory from the main thread, while doing the processing in a separate function?
> > ---
> > Israel Brewster
> > Software Engineer
> > Alaska Volcano Observatory
> > Geophysical Institute - UAF
> > 2156 Koyukuk Drive
> > Fairbanks AK 99775-7320
> > Work: 907-474-5172
> > cell:  907-328-9145
> > _______________________________________________
> > PySide mailing list
> > PySide at qt-project.org <mailto:PySide at qt-project.org>
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.qt-2Dproject.org_listinfo_pyside&d=DwIGaQ&c=76Q6Tcqc-t2x0ciWn7KFdCiqt6IQ7a_IF9uzNzd_2pA&r=Sh-EhW_r32FEgrsAfK3Hc0du2ebOZ7PrbchABO_xtMo&m=r4Ron7tNrQN9BNhzXVlsKvYoyWrnjjOzLwALVV_CgnA&s=T2UjSQWTnYFAfFbfdTlMgLZnfC1dlXhNy1SmPijkWbs&e=<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.qt-2Dproject.org_listinfo_pyside&d=DwIGaQ&c=76Q6Tcqc-t2x0ciWn7KFdCiqt6IQ7a_IF9uzNzd_2pA&r=Sh-EhW_r32FEgrsAfK3Hc0du2ebOZ7PrbchABO_xtMo&m=r4Ron7tNrQN9BNhzXVlsKvYoyWrnjjOzLwALVV_CgnA&s=T2UjSQWTnYFAfFbfdTlMgLZnfC1dlXhNy1SmPijkWbs&e=>
> > _______________________________________________ PySide mailing list PySide at qt-project.org https://lists.qt-project.org/listinfo/pyside <https://lists.qt-project.org/listinfo/pyside>
> >
> > _______________________________________________ PySide mailing list PySide at qt-project.org https://lists.qt-project.org/listinfo/pyside
> > _______________________________________________
> > PySide mailing list
> > PySide at qt-project.org
> > https://lists.qt-project.org/listinfo/pyside
>


More information about the PySide mailing list