[PySide] using QProcess to run python function
Sean Fisk
sean at seanfisk.com
Fri Mar 14 05:15:46 CET 2014
Hi Frank,
Glad you got that figured out. One more important thing:
To achieve a "clean exit", you should wait for the completion of all tasks
associated with the QThreadPool before exiting. Use something like:
thread_pool = QtCore.QThreadPool.globalInstance()
thread_pool.start(hello)
thread_pool.waitForDone()
It is a common idiom to have the application wait for this before it exits:
If you are using a QApplication or QCoreApplication (i.e., a Qt event loop):
from PySide import QtCore
QtCore.QCoreApplication.instance().aboutToQuit.connect(thread_pool.waitForDone)
Or with Python's exit handlers:
import atexit
atexit.register(thread_pool.waitForDone)
In general, I would prefer Qt's method if you are running an event loop.
Hope this helps!
--
Sean Fisk
On Fri, Mar 14, 2014 at 12:00 AM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:
> False alarm:
> Looks like the culprit wasn't the code but the debugger I was using spat
> it's dummy more or less silently.
> When I run the same code in a different interpreter it works as expected.
>
>
>
>
> On 14/03/14 14:30, Frank Rueter | OHUfx wrote:
>
> after a much longer absence from this than anticipated, I'm finally trying
> to get back into this.
> I had a look at your example Sean and it makes sense so far.
> Now I'm trying to follow your advise and use QThreadPool, but am not
> finding any good examples on it's usage.
> Unfortunately, the first example mentioned in the docs<http://srinikom.github.io/pyside-docs/PySide/QtCore/QThreadPool.html>already throws an error for me:
>
> from PySide import QtCore
> class HelloWorldTask(QtCore.QRunnable):
> def run(self):
> print "Hello world from thread", QtCore.QThread.currentThread()
>
> hello = HelloWorldTask()
> # QThreadPool takes ownership and deletes 'hello' automatically
> QtCore.QThreadPool.globalInstance().start(hello)
>
> result:
> Hello world from thread
> Traceback (most recent call last):
> File "/ohufx/pipeline/tools/python/sandbox/QThreadPoolTest.py", line 6,
> in run
> print "Hello world from thread", QtCore.QThread.currentThread()
> AttributeError: 'NoneType' object has no attribute 'QThread'
>
>
>
> What am I missing here?
>
> Cheers,
> frank
>
>
>
>
>
> On 28/01/14 08:00, Sean Fisk wrote:
>
> On Sat, Jan 25, 2014 at 2:38 AM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:
>
>> fantastic, thanks Sean!!
>> I will examine this to make sure I understand everything, then give
>> QThreadPool a whirl. Am more than happy to learn from more experienced
>> people and adjust my approaches accordingly, so thanks a lot for your time
>> with this!
>>
>
> No problem! Let me know if you have any questions about the code.
>
>>
>> Cheers,
>> frank
>>
>>
>>
>> On 25/01/14 20:03, Sean Fisk wrote:
>>
>> Hi Frank,
>>
>> I updated your example to hopefully work as you would like. I added a
>> progress bar update as well. I'm going to make a last-ditch effort to
>> convince you to stop doing things manually with QThread, and move to
>> using QThreadPool<http://pyside.github.io/docs/pyside/PySide/QtCore/QThreadPool.html>.
>> Everything about the QThreadPool API is much nicer, and it's worked much
>> better for me in real projects. Also, the Python 's multiprocessing<http://docs.python.org/2/library/multiprocessing.html>and
>> concurrent.futures <http://pythonhosted.org/futures/> modules can work
>> well if you're careful about your callbacks.
>>
>> Also, there is a large discussion about not overriding the run() method
>> of QThread. I don't think overriding it is so bad if you don't need the
>> thread to be running an event loop of its own. I still prefer the
>> QThreadPool API, though. Someone please correct me if this is totally
>> wrong and there is never a reason to override it.
>>
>> Here is the code:
>>
>> #!/usr/bin/env python
>> import sysimport time
>> from PySide import QtGuifrom PySide import QtCore
>>
>> TOTAL_WIDGETS = 10
>> class Worker(QtCore.QObject):
>> processed = QtCore.Signal(int)
>> finished = QtCore.Signal()
>>
>> # Overriding this is not necessary if you're not doing anything in it.
>>
>> # def __init__(self):
>> # super(Worker, self).__init__()
>>
>> def helloWorld(self):
>> for i in xrange(TOTAL_WIDGETS):
>> # We sleep first to simulate an operation taking place.
>> time.sleep(0.5)
>> print 'hello %s' % i
>> self.processed.emit(i + 1)
>> # Must manually emit the finished signal.
>> self.finished.emit()
>> class MainUI(QtGui.QWidget):
>> def __init__(self, parent=None):
>> super(MainUI, self).__init__(parent)
>> self.extraThread = QtCore.QThread()
>>
>> # IMPORTANT: Don't quit the app until the thread has completed. Prevents errors like:
>> #
>> # QThread: Destroyed while thread is still running
>> #
>> QtGui.QApplication.instance().aboutToQuit.connect(self.quit)
>>
>> self.worker = Worker()
>> self.worker.moveToThread(self.extraThread)
>> self.setupUI()
>> self.connectSignalsAndSlots()
>>
>> def setupUI(self):
>> # CREAT MAIN LAYOUT AND WIDGETS
>> mainLayout = QtGui.QVBoxLayout()
>> btnLayout = QtGui.QHBoxLayout()
>> mainLayout.addLayout(btnLayout)
>> self.setLayout(mainLayout)
>>
>> self.progressBar = QtGui.QProgressBar(self)
>> self.progressBar.setRange(0, TOTAL_WIDGETS)
>> self.progressBar.setVisible(False)
>> self.btnWork = QtGui.QPushButton('Do Work')
>> self.btnCancel = QtGui.QPushButton('Cancel')
>> self.btnCancel.setDisabled(True)
>>
>> self.guiResponseProgressbar = QtGui.QProgressBar(self)
>> self.guiResponseProgressbar.setRange(0, 0)
>>
>> self.outputWindow = QtGui.QTextEdit()
>>
>> mainLayout.addWidget(self.progressBar)
>> mainLayout.addWidget(self.outputWindow)
>> mainLayout.addWidget(self.guiResponseProgressbar)
>>
>> btnLayout.addWidget(self.btnWork)
>> btnLayout.addWidget(self.btnCancel)
>>
>> def connectSignalsAndSlots(self):
>> print 'connecting signals'
>> self.btnWork.clicked.connect(self.startWorker)
>>
>> # Pleas see <http://nooooooooooooooo.com/>. Bad bad bad bad bad.
>> # self.btnCancel.clicked.connect(self.extraThread.terminate)
>>
>> # THREAD STARTED
>> # Not necessary; just do this in startWorker.
>> # self.extraThread.started.connect(lambda: self.btnWork.setDisabled(True))
>> # self.extraThread.started.connect(lambda: self.btnCancel.setEnabled(True))
>> # self.extraThread.started.connect(self.progressBar.show)
>> self.extraThread.started.connect(self.worker.helloWorld)
>>
>> # THREAD FINISHED
>> # self.extraThread.finished.connect(lambda: self.btnCancel.setDisabled(True))
>> # self.extraThread.finished.connect(self.extraThread.deleteLater)
>> # self.extraThread.finished.connect(self.worker.deleteLater)
>> self.extraThread.finished.connect(self.finished)
>>
>> # Connect worker signals.
>> self.worker.processed.connect(self.progressBar.setValue)
>> self.worker.finished.connect(self.finished)
>>
>> # SHOW PROGRESS BAR WHEN PUBLISHING STARTS
>> # self.extraThread.started.connect(self.progressBar.show)
>> # CONNECT PROCESS TO PROGRESS BAR AND OUTPUT WINDOW
>> # NOT DONE YET
>>
>> def startWorker(self):
>> # GO
>> self.btnWork.setDisabled(True)
>> self.btnCancel.setEnabled(True)
>> self.progressBar.show()
>> print 'starting worker thread'
>> self.extraThread.start()
>>
>> # THIS IS BLOCKING THE GUI THREAD! Try putting this back in and seeing
>> # what happens to the gui_response_progressbar.
>>
>> # for i in xrange(10):
>> # print 'from main thread:', i
>> # time.sleep(.3)
>>
>> def finished(self):
>> print 'finished'
>> self.btnCancel.setDisabled(True)
>> # self.extraThread.deleteLater()
>> # self.worker.deleteLater()
>>
>> def quit(self):
>> # Quit the thread's event loop. Note that this *doesn't* stop tasks
>> # running in the thread, it just stops the thread from dispatching
>> # events.
>> self.extraThread.quit()
>> # Wait for the thread to complete. If the thread's task is not done,
>> # this will block.
>> self.extraThread.wait()
>> if __name__ == '__main__':
>> args = sys.argv
>> app = QtGui.QApplication(args)
>> p = MainUI()
>> p.show()
>> # Annoyance on Mac OS X.
>> p.raise_()
>> sys.exit(app.exec_())
>>
>> Cheers,
>>
>>
>> --
>> Sean Fisk
>>
>>
>> On Sat, Jan 25, 2014 at 12:30 AM, Frank Rueter | OHUfx <frank at ohufx.com>wrote:
>>
>>> quick update:
>>> I just learnt about connection types but those didn't help either in
>>> this case:
>>>
>>> http://qt-project.org/doc/qt-5/threads-qobject.html#signals-and-slots-across-threads
>>>
>>>
>>>
>>> On 25/01/14 18:15, Frank Rueter | OHUfx wrote:
>>>
>>> And of course I ran into trouble :-D
>>> Here is my test code trying to ustilise QThread:
>>> http://pastebin.com/Q26Q9M1M
>>>
>>> The basic structure seems to work (the extra thread and the main thread
>>> are running at the same time), but my UI does not update according to the
>>> signal connections, e.g.:
>>> When the thread starts the progress bar is supposed to be shown, the "Do
>>> Work" button should be disabled, the "Cancel" button disabled.
>>> etc.
>>>
>>> I tried calling self.update() in the MainUI when the thread starts but
>>> to no avail.
>>> I'm sure I'm missing something obvious as usual.
>>>
>>> I tried to adhere to what I learnt on this list about threading and
>>> avoid sub-classing QThread.
>>>
>>> Cheers,
>>> frank
>>>
>>> On 24/01/14 17:41, Frank Rueter | OHUfx wrote:
>>>
>>> Great, thanks. I shall adjust my code.
>>> I am interested in seeing how you would go about it, but let me have a
>>> go from scratch, I will be able to understand things much better afterwards
>>> :)
>>>
>>> Cheers,
>>> frank
>>>
>>>
>>> On 24/01/14 17:38, Sean Fisk wrote:
>>>
>>> On Thu, Jan 23, 2014 at 10:44 PM, Frank Rueter | OHUfx <frank at ohufx.com
>>> > wrote:
>>>
>>>> Much appreciated but as I mentioned, I think I jumped the gun with my
>>>> post and should be using QThread to hook the python code to my QProgressBar
>>>> and debug output window (QTextEdit).
>>>>
>>> That would be a correct way to use QProgressBar. Also, for subprocess
>>> output, I would consider using QPlainTextEdit instead of QTextEdit as
>>> is is more performant with high amounts of text.
>>>
>>> If you are interested, I just wrote an asynchronous module for PySide
>>> for the project on which I am currently working. It is based on Python's
>>> futures <http://pythonhosted.org/futures/> module and works great with
>>> Qt's signals/slots. Although the project is primarily closed-source, I
>>> would be happy to share the async implementation with you. I don't know if
>>> would fit your needs, but for us it's much easier than manually spawning
>>> QThreads and even easier than QThreadPool.
>>>
>>> Good luck!
>>>
>>>> I will investigate that now
>>>>
>>>>
>>>>
>>>> On 24/01/14 16:08, Sean Fisk wrote:
>>>>
>>>> On Thu, Jan 23, 2014 at 9:42 PM, Frank Rueter | OHUfx <frank at ohufx.com
>>>> > wrote:
>>>>
>>>>> Thanks Sean and Ryan,
>>>>>
>>>>> I'm still not quite clear on how this ties into QProcess.start()
>>>>>
>>>>
>>>> It doesn't tie in with QProcess at all. We're advising to avoid using
>>>> that :)
>>>>
>>>>> I do have a if __name__ ... block in the script in question.
>>>>> An example would certainly be awesome, but if it's less hassle,
>>>>> explaining how your and Ryan's advise helps use QProcess on a python module
>>>>> might already suffice. Maybe a simlpe example says it all though?!
>>>>>
>>>>
>>>> I will whip up a simple example for you, but it might take a few
>>>> hours (lots of stuff to do right now).
>>>>
>>>>> I'm not using python 3 btw
>>>>>
>>>>> Thanks guys for your help!!
>>>>>
>>>>> frank
>>>>>
>>>>>
>>>>> On 24/01/14 15:33, Sean Fisk wrote:
>>>>>
>>>>> Hi Frank,
>>>>>
>>>>> You should definitely avoid calling Python as a subprocess if you can.
>>>>> As far as Ryan's example, I agree with the if __name__... but I think
>>>>> that using the imp module is a bit overkill. I would recommend using
>>>>> Setuptool's entry_points keyword<http://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation>.
>>>>> Or distutils' scripts keyword<http://docs.python.org/2/distutils/setupscript.html#installing-scripts>,
>>>>> if you must.
>>>>>
>>>>> An example of a well-known Python package which does this is Pygments<https://bitbucket.org/birkenfeld/pygments-main>,
>>>>> which has a large "library" component but also comes with the
>>>>> pygmentize command-line script. The Pygments codebase is pretty
>>>>> large, so if you would like me to whip up a simpler example I'd be glad to
>>>>> do so.
>>>>>
>>>>> Cheers,
>>>>> --
>>>>> Sean Fisk
>>>>>
>>>>>
>>>>> On Thu, Jan 23, 2014 at 9:17 PM, Frank Rueter | OHUfx <frank at ohufx.com
>>>>> > wrote:
>>>>>
>>>>>> Sorry if I'm being thick, but I'm not quite understanding how this
>>>>>> helps to connect a python function to qprocess?! All your code does is
>>>>>> execute the script, right?!
>>>>>> I can already call myscript.main() straight up, but maybe I'm missing
>>>>>> the point as I'm unfamiliar with the imp module.
>>>>>>
>>>>>> Let me elaborate a little bit more:
>>>>>> myscript.main() calls a bunch of other python scripts that (directly
>>>>>> or through other scripts again) execute external programs to do some
>>>>>> conversion work. Those external programs spit out their progress to stdout
>>>>>> which I can see fine when I run myscript.main() manually in a python
>>>>>> terminal.
>>>>>>
>>>>>> Now I need run myscript.main() via QProcess and grab stdout to do be
>>>>>> able to show a progress bar as well as show stdout and stderr in a debug
>>>>>> window inside my QT code.
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> frank
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 24/01/14 14:58, Ryan Gonzalez wrote:
>>>>>>
>>>>>> If you put an "if __name__ == '__main__'" and a main functions, you
>>>>>> could always import the script from the GUI frontend. Example:
>>>>>>
>>>>>> myscript.py
>>>>>>
>>>>>> def main(argv):
>>>>>> do_cool_stuff()
>>>>>> return 0
>>>>>>
>>>>>> if __name__ == '__main__':
>>>>>> sys.exit(main(sys.argv))
>>>>>>
>>>>>> mygui.py(Python 2):
>>>>>>
>>>>>> import imp
>>>>>>
>>>>>> ...
>>>>>>
>>>>>> main = imp.load_module('myscript', *imp.find_module('myscript'))
>>>>>>
>>>>>> main.main(my_argv)
>>>>>>
>>>>>> mygui.py(Python 3):
>>>>>>
>>>>>> import importlib.machinery
>>>>>>
>>>>>> main = importlib.machinery.SourceFileLoader('myscript',
>>>>>> 'myscript.py').load_module('myscript')
>>>>>>
>>>>>> main.main(my_argv)
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 23, 2014 at 7:48 PM, Frank Rueter | OHUfx <
>>>>>> frank at ohufx.com> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I got a little code design question:
>>>>>>>
>>>>>>> I have a python script that does a lot of file
>>>>>>> processing/converting/uploading etc and I'd like to write a decent
>>>>>>> interface for it now.
>>>>>>> The main goal is to be able to show the user detailed info about the
>>>>>>> current step and progress as well as clean up properly in case the
>>>>>>> whole
>>>>>>> thing is cancelled.
>>>>>>>
>>>>>>> My existing python code needs to stay independent of QT so any
>>>>>>> application that supports python can use it.
>>>>>>> I am wondering now how to best connect the python script and the
>>>>>>> PySide
>>>>>>> code. Should I just run the script as an argument to the python
>>>>>>> interpreter like I would with any other program? E.g.:
>>>>>>>
>>>>>>> process = QtCore.QProcess(self)
>>>>>>> process.start(<path_to_python>, <path_to_python_script>)
>>>>>>>
>>>>>>> As simple as this seems, it feels odd to use python to call itself
>>>>>>> as an
>>>>>>> external program.
>>>>>>>
>>>>>>>
>>>>>>> I'm happy to go that way but am curious how others are doing this?!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> frank
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> PySide mailing list
>>>>>>> PySide at qt-project.org
>>>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ryan
>>>>>> If anybody ever asks me why I prefer C++ to C, my answer will be
>>>>>> simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>>>>> that was nul-terminated."
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> PySide mailing list
>>>>>> PySide at qt-project.org
>>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> PySide mailing listPySide at qt-project.orghttp://lists.qt-project.org/mailman/listinfo/pyside
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> PySide mailing listPySide at qt-project.orghttp://lists.qt-project.org/mailman/listinfo/pyside
>>>
>>>
>>>
>>> _______________________________________________
>>> PySide mailing list
>>> PySide at qt-project.org
>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>
>>>
>>
>>
>
> --
> [image: ohufxLogo 50x50] <http://www.ohufx.com> *vfx compositing
> <http://ohufx.com/index.php/vfx-compositing> | workflow customisation and
> consulting <http://ohufx.com/index.php/vfx-customising> *
>
>
> _______________________________________________
> PySide mailing listPySide at qt-project.orghttp://lists.qt-project.org/mailman/listinfo/pyside
>
>
> --
> [image: ohufxLogo 50x50] <http://www.ohufx.com> *vfx compositing
> <http://ohufx.com/index.php/vfx-compositing> | workflow customisation and
> consulting <http://ohufx.com/index.php/vfx-customising> *
>
> _______________________________________________
> PySide mailing list
> PySide at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/pyside
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/6b971c2d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 2666 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/6b971c2d/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ohufxLogo_50x50.png
Type: image/png
Size: 2666 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/6b971c2d/attachment-0001.png>
More information about the PySide
mailing list