[PySide] using QProcess to run python function

Sean Fisk sean at seanfisk.com
Sat Jan 25 08:03:52 CET 2014


Hi Frank,

I updated your example to hopefully work as you would like. I added a
progress bar update as well. I’m going to make a last-ditch effort to
convince you to stop doing things manually with QThread, and move to using
QThreadPool<http://pyside.github.io/docs/pyside/PySide/QtCore/QThreadPool.html>.
Everything about the QThreadPool API is much nicer, and it’s worked much
better for me in real projects. Also, the Python ‘s
multiprocessing<http://docs.python.org/2/library/multiprocessing.html>and
concurrent.futures <http://pythonhosted.org/futures/> modules can work well
if you’re careful about your callbacks.

Also, there is a large discussion about not overriding the run() method of
QThread. I don’t think overriding it is so bad if you don’t need the thread
to be running an event loop of its own. I still prefer the QThreadPool API,
though. Someone please correct me if this is totally wrong and there is
never a reason to override it.

Here is the code:

#!/usr/bin/env python
import sysimport time
from PySide import QtGuifrom PySide import QtCore

TOTAL_WIDGETS = 10
class Worker(QtCore.QObject):
    processed = QtCore.Signal(int)
    finished = QtCore.Signal()

    # Overriding this is not necessary if you're not doing anything in it.

    # def __init__(self):
    #     super(Worker, self).__init__()

    def helloWorld(self):
        for i in xrange(TOTAL_WIDGETS):
            # We sleep first to simulate an operation taking place.
            time.sleep(0.5)
            print 'hello %s' % i
            self.processed.emit(i + 1)
        # Must manually emit the finished signal.
        self.finished.emit()
class MainUI(QtGui.QWidget):
    def __init__(self, parent=None):
        super(MainUI, self).__init__(parent)
        self.extraThread = QtCore.QThread()

        # IMPORTANT: Don't quit the app until the thread has
completed. Prevents errors like:
        #
        #     QThread: Destroyed while thread is still running
        #
        QtGui.QApplication.instance().aboutToQuit.connect(self.quit)

        self.worker = Worker()
        self.worker.moveToThread(self.extraThread)
        self.setupUI()
        self.connectSignalsAndSlots()

    def setupUI(self):
        # CREAT MAIN LAYOUT AND WIDGETS
        mainLayout = QtGui.QVBoxLayout()
        btnLayout = QtGui.QHBoxLayout()
        mainLayout.addLayout(btnLayout)
        self.setLayout(mainLayout)

        self.progressBar = QtGui.QProgressBar(self)
        self.progressBar.setRange(0, TOTAL_WIDGETS)
        self.progressBar.setVisible(False)
        self.btnWork = QtGui.QPushButton('Do Work')
        self.btnCancel = QtGui.QPushButton('Cancel')
        self.btnCancel.setDisabled(True)

        self.guiResponseProgressbar = QtGui.QProgressBar(self)
        self.guiResponseProgressbar.setRange(0, 0)

        self.outputWindow = QtGui.QTextEdit()

        mainLayout.addWidget(self.progressBar)
        mainLayout.addWidget(self.outputWindow)
        mainLayout.addWidget(self.guiResponseProgressbar)

        btnLayout.addWidget(self.btnWork)
        btnLayout.addWidget(self.btnCancel)

    def connectSignalsAndSlots(self):
        print 'connecting signals'
        self.btnWork.clicked.connect(self.startWorker)

        # Pleas see <http://nooooooooooooooo.com/>. Bad bad bad bad bad.
        # self.btnCancel.clicked.connect(self.extraThread.terminate)

        # THREAD STARTED
        # Not necessary; just do this in startWorker.
        # self.extraThread.started.connect(lambda:
self.btnWork.setDisabled(True))
        # self.extraThread.started.connect(lambda:
self.btnCancel.setEnabled(True))
        # self.extraThread.started.connect(self.progressBar.show)
        self.extraThread.started.connect(self.worker.helloWorld)

        # THREAD FINISHED
        # self.extraThread.finished.connect(lambda:
self.btnCancel.setDisabled(True))
        # self.extraThread.finished.connect(self.extraThread.deleteLater)
        # self.extraThread.finished.connect(self.worker.deleteLater)
        self.extraThread.finished.connect(self.finished)

        # Connect worker signals.
        self.worker.processed.connect(self.progressBar.setValue)
        self.worker.finished.connect(self.finished)

        # SHOW PROGRESS BAR WHEN PUBLISHING STARTS
        # self.extraThread.started.connect(self.progressBar.show)
        # CONNECT PROCESS TO PROGRESS BAR AND OUTPUT WINDOW
        # NOT DONE YET

    def startWorker(self):
        # GO
        self.btnWork.setDisabled(True)
        self.btnCancel.setEnabled(True)
        self.progressBar.show()
        print 'starting worker thread'
        self.extraThread.start()

        # THIS IS BLOCKING THE GUI THREAD! Try putting this back in and seeing
        # what happens to the gui_response_progressbar.

        # for i in xrange(10):
        #     print 'from main thread:', i
        #     time.sleep(.3)

    def finished(self):
        print 'finished'
        self.btnCancel.setDisabled(True)
        # self.extraThread.deleteLater()
        # self.worker.deleteLater()

    def quit(self):
        # Quit the thread's event loop. Note that this *doesn't* stop tasks
        # running in the thread, it just stops the thread from dispatching
        # events.
        self.extraThread.quit()
        # Wait for the thread to complete. If the thread's task is not done,
        # this will block.
        self.extraThread.wait()
if __name__ == '__main__':
    args = sys.argv
    app = QtGui.QApplication(args)
    p = MainUI()
    p.show()
    # Annoyance on Mac OS X.
    p.raise_()
    sys.exit(app.exec_())

Cheers,


--
Sean Fisk


On Sat, Jan 25, 2014 at 12:30 AM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:

>  quick update:
> I just learnt about connection types but those didn't help either in this
> case:
>
> http://qt-project.org/doc/qt-5/threads-qobject.html#signals-and-slots-across-threads
>
>
>
> On 25/01/14 18:15, Frank Rueter | OHUfx wrote:
>
> And of course I ran into trouble :-D
> Here is my test code trying to ustilise QThread:
> http://pastebin.com/Q26Q9M1M
>
> The basic structure seems to work (the extra thread and the main thread
> are running at the same time), but my UI does not update according to the
> signal connections, e.g.:
> When the thread starts the progress bar is supposed to be shown, the "Do
> Work" button should be disabled, the "Cancel" button disabled.
> etc.
>
> I tried calling self.update() in the MainUI when the thread starts  but to
> no avail.
> I'm sure I'm missing something obvious as usual.
>
> I tried to adhere to what I learnt on this list about threading and avoid
> sub-classing QThread.
>
> Cheers,
> frank
>
> On 24/01/14 17:41, Frank Rueter | OHUfx wrote:
>
> Great, thanks. I shall adjust my code.
> I am interested in seeing how you would go about it, but let me have a go
> from scratch, I will be able to understand things much better afterwards :)
>
> Cheers,
> frank
>
>
> On 24/01/14 17:38, Sean Fisk wrote:
>
>  On Thu, Jan 23, 2014 at 10:44 PM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:
>
>>  Much appreciated but as I mentioned, I think I jumped the gun with my
>> post and should be using QThread to hook the python code to my QProgressBar
>> and debug output window (QTextEdit).
>>
>  That would be a correct way to use QProgressBar. Also, for subprocess
> output, I would consider using QPlainTextEdit instead of QTextEdit as is
> is more performant with high amounts of text.
>
> If you are interested, I just wrote an asynchronous module for PySide for
> the project on which I am currently working. It is based on Python’s
> futures <http://pythonhosted.org/futures/> module and works great with
> Qt’s signals/slots. Although the project is primarily closed-source, I
> would be happy to share the async implementation with you. I don’t know if
> would fit your needs, but for us it’s much easier than manually spawning
> QThreads and even easier than QThreadPool.
>
> Good luck!
>
>>  I will investigate that now
>>
>>
>>
>> On 24/01/14 16:08, Sean Fisk wrote:
>>
>>  On Thu, Jan 23, 2014 at 9:42 PM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:
>>
>>>  Thanks Sean and Ryan,
>>>
>>> I'm still not quite clear on how this ties into QProcess.start()
>>>
>>
>>  It doesn’t tie in with QProcess at all. We’re advising to avoid using
>> that :)
>>
>>>  I do have a if __name__ ... block in the script in question.
>>> An example would certainly be awesome, but if it's less hassle,
>>> explaining how your and Ryan's advise helps use QProcess on a python module
>>> might already suffice. Maybe a simlpe example says it all though?!
>>>
>>
>>  I will whip up a simple example for you, but it might take a few hours
>> (lots of stuff to do right now).
>>
>>>  I'm not using python 3 btw
>>>
>>> Thanks guys for your help!!
>>>
>>> frank
>>>
>>>
>>> On 24/01/14 15:33, Sean Fisk wrote:
>>>
>>>  Hi Frank,
>>>
>>> You should definitely avoid calling Python as a subprocess if you can.
>>> As far as Ryan’s example, I agree with the if __name__... but I think
>>> that using the imp module is a bit overkill. I would recommend using
>>> Setuptool’s entry_points keyword<http://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation>.
>>> Or distutils’ scripts keyword<http://docs.python.org/2/distutils/setupscript.html#installing-scripts>,
>>> if you must.
>>>
>>> An example of a well-known Python package which does this is Pygments<https://bitbucket.org/birkenfeld/pygments-main>,
>>> which has a large “library” component but also comes with the pygmentizecommand-line script. The Pygments codebase is pretty large, so if you would
>>> like me to whip up a simpler example I’d be glad to do so.
>>>
>>> Cheers,
>>>   --
>>> Sean Fisk
>>>
>>>
>>> On Thu, Jan 23, 2014 at 9:17 PM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:
>>>
>>>>  Sorry if I'm being thick, but I'm not quite understanding how this
>>>> helps to connect a python function to qprocess?! All your code does is
>>>> execute the script, right?!
>>>> I can already call myscript.main() straight up, but maybe I'm missing
>>>> the point as I'm unfamiliar with the imp module.
>>>>
>>>> Let me elaborate a little bit more:
>>>> myscript.main() calls a bunch of other python scripts that (directly or
>>>> through other scripts again) execute external programs to do some
>>>> conversion work. Those external programs spit out their progress to stdout
>>>> which I can see fine when I run myscript.main() manually in a python
>>>> terminal.
>>>>
>>>> Now I need run myscript.main() via QProcess and grab stdout to do be
>>>> able to show a progress bar as well as show stdout and stderr in a debug
>>>> window inside my QT code.
>>>>
>>>>
>>>> Cheers,
>>>> frank
>>>>
>>>>
>>>>
>>>>
>>>> On 24/01/14 14:58, Ryan Gonzalez wrote:
>>>>
>>>> If you put an "if __name__ == '__main__'" and a main functions, you
>>>> could always import the script from the GUI frontend. Example:
>>>>
>>>>  myscript.py
>>>>
>>>>  def main(argv):
>>>>     do_cool_stuff()
>>>>     return 0
>>>>
>>>>  if __name__ == '__main__':
>>>>     sys.exit(main(sys.argv))
>>>>
>>>>  mygui.py(Python 2):
>>>>
>>>>  import imp
>>>>
>>>>  ...
>>>>
>>>>  main = imp.load_module('myscript', *imp.find_module('myscript'))
>>>>
>>>>  main.main(my_argv)
>>>>
>>>>  mygui.py(Python 3):
>>>>
>>>>  import importlib.machinery
>>>>
>>>>  main = importlib.machinery.SourceFileLoader('myscript',
>>>> 'myscript.py').load_module('myscript')
>>>>
>>>>  main.main(my_argv)
>>>>
>>>>
>>>> On Thu, Jan 23, 2014 at 7:48 PM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I got a little code design question:
>>>>>
>>>>> I have a python script that does a lot of file
>>>>> processing/converting/uploading etc and I'd like to write a decent
>>>>> interface for it now.
>>>>> The main goal is to be able to show the user detailed info about the
>>>>> current step and progress as well as clean up properly in case the
>>>>> whole
>>>>> thing is cancelled.
>>>>>
>>>>> My existing python code needs to stay independent of QT so any
>>>>> application that supports python can use it.
>>>>> I am wondering now how to best connect the python script and the PySide
>>>>> code. Should I just run the script as an argument to the python
>>>>> interpreter like I would with any other program? E.g.:
>>>>>
>>>>> process = QtCore.QProcess(self)
>>>>> process.start(<path_to_python>, <path_to_python_script>)
>>>>>
>>>>> As simple as this seems, it feels odd to use python to call itself as
>>>>> an
>>>>> external program.
>>>>>
>>>>>
>>>>> I'm happy to go that way but am curious how others are doing this?!
>>>>>
>>>>> Cheers,
>>>>> frank
>>>>>
>>>>> _______________________________________________
>>>>> PySide mailing list
>>>>> PySide at qt-project.org
>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>
>>>>
>>>>
>>>>
>>>>  --
>>>> Ryan
>>>> If anybody ever asks me why I prefer C++ to C, my answer will be
>>>> simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>>> that was nul-terminated."
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> PySide mailing list
>>>> PySide at qt-project.org
>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>
>>>>
>>>
>>>
>>
>>
>
>
>
> _______________________________________________
> PySide mailing listPySide at qt-project.orghttp://lists.qt-project.org/mailman/listinfo/pyside
>
>
>
>
> _______________________________________________
> PySide mailing listPySide at qt-project.orghttp://lists.qt-project.org/mailman/listinfo/pyside
>
>
>
> _______________________________________________
> PySide mailing list
> PySide at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/pyside
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140125/92161043/attachment.html>


More information about the PySide mailing list