[PySide] using QProcess to run python function
Frank Rueter | OHUfx
frank at ohufx.com
Fri Mar 14 05:00:23 CET 2014
False alarm:
Looks like the culprit wasn't the code but the debugger I was using spat
it's dummy more or less silently.
When I run the same code in a different interpreter it works as expected.
On 14/03/14 14:30, Frank Rueter | OHUfx wrote:
> after a much longer absence from this than anticipated, I'm finally
> trying to get back into this.
> I had a look at your example Sean and it makes sense so far.
> Now I'm trying to follow your advise and use QThreadPool, but am not
> finding any good examples on it's usage.
> Unfortunately, the first example mentioned in the docs
> <http://srinikom.github.io/pyside-docs/PySide/QtCore/QThreadPool.html>
> already throws an error for me:
>
> from PySide import QtCore
> class HelloWorldTask(QtCore.QRunnable):
> def run(self):
> print "Hello world from thread", QtCore.QThread.currentThread()
>
> hello = HelloWorldTask()
> # QThreadPool takes ownership and deletes 'hello' automatically
> QtCore.QThreadPool.globalInstance().start(hello)
>
> result:
> Hello world from thread
> Traceback (most recent call last):
> File "/ohufx/pipeline/tools/python/sandbox/QThreadPoolTest.py", line
> 6, in run
> print "Hello world from thread", QtCore.QThread.currentThread()
> AttributeError: 'NoneType' object has no attribute 'QThread'
>
>
>
> What am I missing here?
>
> Cheers,
> frank
>
>
>
>
>
> On 28/01/14 08:00, Sean Fisk wrote:
>> On Sat, Jan 25, 2014 at 2:38 AM, Frank Rueter | OHUfx
>> <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>
>> fantastic, thanks Sean!!
>> I will examine this to make sure I understand everything, then
>> give QThreadPool a whirl. Am more than happy to learn from more
>> experienced people and adjust my approaches accordingly, so
>> thanks a lot for your time with this!
>>
>>
>> No problem! Let me know if you have any questions about the code.
>>
>>
>> Cheers,
>> frank
>>
>>
>>
>> On 25/01/14 20:03, Sean Fisk wrote:
>>>
>>> Hi Frank,
>>>
>>> I updated your example to hopefully work as you would like. I
>>> added a progress bar update as well. I'm going to make a
>>> last-ditch effort to convince you to stop doing things manually
>>> with |QThread|, and move to using |QThreadPool|
>>> <http://pyside.github.io/docs/pyside/PySide/QtCore/QThreadPool.html>.
>>> Everything about the |QThreadPool| API is much nicer, and it's
>>> worked much better for me in real projects. Also, the Python 's
>>> |multiprocessing|
>>> <http://docs.python.org/2/library/multiprocessing.html> and
>>> |concurrent.futures| <http://pythonhosted.org/futures/> modules
>>> can work well if you're careful about your callbacks.
>>>
>>> Also, there is a large discussion about not overriding the
>>> |run()| method of |QThread|. I don't think overriding it is so
>>> bad if you don't need the thread to be running an event loop of
>>> its own. I still prefer the |QThreadPool| API, though. Someone
>>> please correct me if this is totally wrong and there is never a
>>> reason to override it.
>>>
>>> Here is the code:
>>>
>>> |#!/usr/bin/env python
>>>
>>> import sys
>>> import time
>>>
>>> from PySideimport QtGui
>>> from PySideimport QtCore
>>>
>>> TOTAL_WIDGETS =10
>>>
>>> class Worker(QtCore.QObject):
>>> processed = QtCore.Signal(int)
>>> finished = QtCore.Signal()
>>>
>>> # Overriding this is not necessary if you're not doing anything in it.
>>>
>>> # def __init__(self):
>>> # super(Worker, self).__init__()
>>>
>>> def helloWorld(self):
>>> for iin xrange(TOTAL_WIDGETS):
>>> # We sleep first to simulate an operation taking place.
>>> time.sleep(0.5)
>>> print 'hello %s' % i
>>> self.processed.emit(i +1)
>>> # Must manually emit the finished signal.
>>> self.finished.emit()
>>>
>>> class MainUI(QtGui.QWidget):
>>> def __init__(self, parent=None):
>>> super(MainUI, self).__init__(parent)
>>> self.extraThread = QtCore.QThread()
>>>
>>> # IMPORTANT: Don't quit the app until the thread has completed. Prevents errors like:
>>> #
>>> # QThread: Destroyed while thread is still running
>>> #
>>> QtGui.QApplication.instance().aboutToQuit.connect(self.quit)
>>>
>>> self.worker = Worker()
>>> self.worker.moveToThread(self.extraThread)
>>> self.setupUI()
>>> self.connectSignalsAndSlots()
>>>
>>> def setupUI(self):
>>> # CREAT MAIN LAYOUT AND WIDGETS
>>> mainLayout = QtGui.QVBoxLayout()
>>> btnLayout = QtGui.QHBoxLayout()
>>> mainLayout.addLayout(btnLayout)
>>> self.setLayout(mainLayout)
>>>
>>> self.progressBar = QtGui.QProgressBar(self)
>>> self.progressBar.setRange(0, TOTAL_WIDGETS)
>>> self.progressBar.setVisible(False)
>>> self.btnWork = QtGui.QPushButton('Do Work')
>>> self.btnCancel = QtGui.QPushButton('Cancel')
>>> self.btnCancel.setDisabled(True)
>>>
>>> self.guiResponseProgressbar = QtGui.QProgressBar(self)
>>> self.guiResponseProgressbar.setRange(0,0)
>>>
>>> self.outputWindow = QtGui.QTextEdit()
>>>
>>> mainLayout.addWidget(self.progressBar)
>>> mainLayout.addWidget(self.outputWindow)
>>> mainLayout.addWidget(self.guiResponseProgressbar)
>>>
>>> btnLayout.addWidget(self.btnWork)
>>> btnLayout.addWidget(self.btnCancel)
>>>
>>> def connectSignalsAndSlots(self):
>>> print 'connecting signals'
>>> self.btnWork.clicked.connect(self.startWorker)
>>>
>>> # Pleas see <http://nooooooooooooooo.com/>. Bad bad bad bad bad.
>>> # self.btnCancel.clicked.connect(self.extraThread.terminate)
>>>
>>> # THREAD STARTED
>>> # Not necessary; just do this in startWorker.
>>> # self.extraThread.started.connect(lambda: self.btnWork.setDisabled(True))
>>> # self.extraThread.started.connect(lambda: self.btnCancel.setEnabled(True))
>>> # self.extraThread.started.connect(self.progressBar.show)
>>> self.extraThread.started.connect(self.worker.helloWorld)
>>>
>>> # THREAD FINISHED
>>> # self.extraThread.finished.connect(lambda: self.btnCancel.setDisabled(True))
>>> # self.extraThread.finished.connect(self.extraThread.deleteLater)
>>> # self.extraThread.finished.connect(self.worker.deleteLater)
>>> self.extraThread.finished.connect(self.finished)
>>>
>>> # Connect worker signals.
>>> self.worker.processed.connect(self.progressBar.setValue)
>>> self.worker.finished.connect(self.finished)
>>>
>>> # SHOW PROGRESS BAR WHEN PUBLISHING STARTS
>>> # self.extraThread.started.connect(self.progressBar.show)
>>> # CONNECT PROCESS TO PROGRESS BAR AND OUTPUT WINDOW
>>> # NOT DONE YET
>>>
>>> def startWorker(self):
>>> # GO
>>> self.btnWork.setDisabled(True)
>>> self.btnCancel.setEnabled(True)
>>> self.progressBar.show()
>>> print 'starting worker thread'
>>> self.extraThread.start()
>>>
>>> # THIS IS BLOCKING THE GUI THREAD! Try putting this back in and seeing
>>> # what happens to the gui_response_progressbar.
>>>
>>> # for i in xrange(10):
>>> # print 'from main thread:', i
>>> # time.sleep(.3)
>>>
>>> def finished(self):
>>> print 'finished'
>>> self.btnCancel.setDisabled(True)
>>> # self.extraThread.deleteLater()
>>> # self.worker.deleteLater()
>>>
>>> def quit(self):
>>> # Quit the thread's event loop. Note that this *doesn't* stop tasks
>>> # running in the thread, it just stops the thread from dispatching
>>> # events.
>>> self.extraThread.quit()
>>> # Wait for the thread to complete. If the thread's task is not done,
>>> # this will block.
>>> self.extraThread.wait()
>>>
>>> if __name__ =='__main__':
>>> args = sys.argv
>>> app = QtGui.QApplication(args)
>>> p = MainUI()
>>> p.show()
>>> # Annoyance on Mac OS X.
>>> p.raise_()
>>> sys.exit(app.exec_())|
>>>
>>> Cheers,
>>>
>>>
>>>
>>> --
>>> Sean Fisk
>>>
>>>
>>> On Sat, Jan 25, 2014 at 12:30 AM, Frank Rueter | OHUfx
>>> <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>
>>> quick update:
>>> I just learnt about connection types but those didn't help
>>> either in this case:
>>> http://qt-project.org/doc/qt-5/threads-qobject.html#signals-and-slots-across-threads
>>>
>>>
>>>
>>>
>>> On 25/01/14 18:15, Frank Rueter | OHUfx wrote:
>>>> And of course I ran into trouble :-D
>>>> Here is my test code trying to ustilise QThread:
>>>> http://pastebin.com/Q26Q9M1M
>>>>
>>>> The basic structure seems to work (the extra thread and the
>>>> main thread are running at the same time), but my UI does
>>>> not update according to the signal connections, e.g.:
>>>> When the thread starts the progress bar is supposed to be
>>>> shown, the "Do Work" button should be disabled, the
>>>> "Cancel" button disabled.
>>>> etc.
>>>>
>>>> I tried calling self.update() in the MainUI when the thread
>>>> starts but to no avail.
>>>> I'm sure I'm missing something obvious as usual.
>>>>
>>>> I tried to adhere to what I learnt on this list about
>>>> threading and avoid sub-classing QThread.
>>>>
>>>> Cheers,
>>>> frank
>>>>
>>>> On 24/01/14 17:41, Frank Rueter | OHUfx wrote:
>>>>> Great, thanks. I shall adjust my code.
>>>>> I am interested in seeing how you would go about it, but
>>>>> let me have a go from scratch, I will be able to
>>>>> understand things much better afterwards :)
>>>>>
>>>>> Cheers,
>>>>> frank
>>>>>
>>>>>
>>>>> On 24/01/14 17:38, Sean Fisk wrote:
>>>>>> On Thu, Jan 23, 2014 at 10:44 PM, Frank Rueter | OHUfx
>>>>>> <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>>>>
>>>>>> Much appreciated but as I mentioned, I think I jumped
>>>>>> the gun with my post and should be using QThread to
>>>>>> hook the python code to my QProgressBar and debug
>>>>>> output window (QTextEdit).
>>>>>>
>>>>>> That would be a correct way to use |QProgressBar|. Also,
>>>>>> for subprocess output, I would consider using
>>>>>> |QPlainTextEdit| instead of |QTextEdit| as is is more
>>>>>> performant with high amounts of text.
>>>>>>
>>>>>> If you are interested, I just wrote an asynchronous
>>>>>> module for PySide for the project on which I am currently
>>>>>> working. It is based on Python's futures
>>>>>> <http://pythonhosted.org/futures/> module and works great
>>>>>> with Qt's signals/slots. Although the project is
>>>>>> primarily closed-source, I would be happy to share the
>>>>>> async implementation with you. I don't know if would fit
>>>>>> your needs, but for us it's much easier than manually
>>>>>> spawning |QThread|s and even easier than |QThreadPool|.
>>>>>>
>>>>>> Good luck!
>>>>>>
>>>>>> I will investigate that now
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 24/01/14 16:08, Sean Fisk wrote:
>>>>>>> On Thu, Jan 23, 2014 at 9:42 PM, Frank Rueter |
>>>>>>> OHUfx <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>>>>>
>>>>>>> Thanks Sean and Ryan,
>>>>>>>
>>>>>>> I'm still not quite clear on how this ties into
>>>>>>> QProcess.start()
>>>>>>>
>>>>>>>
>>>>>>> It doesn't tie in with |QProcess| at all. We're
>>>>>>> advising to avoid using that :)
>>>>>>>
>>>>>>> I do have a if __name__ ... block in the script
>>>>>>> in question.
>>>>>>> An example would certainly be awesome, but if
>>>>>>> it's less hassle, explaining how your and Ryan's
>>>>>>> advise helps use QProcess on a python module
>>>>>>> might already suffice. Maybe a simlpe example
>>>>>>> says it all though?!
>>>>>>>
>>>>>>>
>>>>>>> I will whip up a simple example for you, but it
>>>>>>> might take a few hours (lots of stuff to do right now).
>>>>>>>
>>>>>>> I'm not using python 3 btw
>>>>>>>
>>>>>>> Thanks guys for your help!!
>>>>>>>
>>>>>>> frank
>>>>>>>
>>>>>>>
>>>>>>> On 24/01/14 15:33, Sean Fisk wrote:
>>>>>>>>
>>>>>>>> Hi Frank,
>>>>>>>>
>>>>>>>> You should definitely avoid calling Python as a
>>>>>>>> subprocess if you can. As far as Ryan's
>>>>>>>> example, I agree with the |if __name__...| but
>>>>>>>> I think that using the |imp| module is a bit
>>>>>>>> overkill. I would recommend using Setuptool's
>>>>>>>> |entry_points| keyword
>>>>>>>> <http://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation>.
>>>>>>>> Or distutils' |scripts| keyword
>>>>>>>> <http://docs.python.org/2/distutils/setupscript.html#installing-scripts>,
>>>>>>>> if you must.
>>>>>>>>
>>>>>>>> An example of a well-known Python package which
>>>>>>>> does this is Pygments
>>>>>>>> <https://bitbucket.org/birkenfeld/pygments-main>,
>>>>>>>> which has a large "library" component but also
>>>>>>>> comes with the |pygmentize| command-line
>>>>>>>> script. The Pygments codebase is pretty large,
>>>>>>>> so if you would like me to whip up a simpler
>>>>>>>> example I'd be glad to do so.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sean Fisk
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 23, 2014 at 9:17 PM, Frank Rueter |
>>>>>>>> OHUfx <frank at ohufx.com
>>>>>>>> <mailto:frank at ohufx.com>> wrote:
>>>>>>>>
>>>>>>>> Sorry if I'm being thick, but I'm not quite
>>>>>>>> understanding how this helps to connect a
>>>>>>>> python function to qprocess?! All your code
>>>>>>>> does is execute the script, right?!
>>>>>>>> I can already call myscript.main() straight
>>>>>>>> up, but maybe I'm missing the point as I'm
>>>>>>>> unfamiliar with the imp module.
>>>>>>>>
>>>>>>>> Let me elaborate a little bit more:
>>>>>>>> myscript.main() calls a bunch of other
>>>>>>>> python scripts that (directly or through
>>>>>>>> other scripts again) execute external
>>>>>>>> programs to do some conversion work. Those
>>>>>>>> external programs spit out their progress
>>>>>>>> to stdout which I can see fine when I run
>>>>>>>> myscript.main() manually in a python terminal.
>>>>>>>>
>>>>>>>> Now I need run myscript.main() via QProcess
>>>>>>>> and grab stdout to do be able to show a
>>>>>>>> progress bar as well as show stdout and
>>>>>>>> stderr in a debug window inside my QT code.
>>>>>>>>
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> frank
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 24/01/14 14:58, Ryan Gonzalez wrote:
>>>>>>>>> If you put an "if __name__ == '__main__'"
>>>>>>>>> and a main functions, you could always
>>>>>>>>> import the script from the GUI frontend.
>>>>>>>>> Example:
>>>>>>>>>
>>>>>>>>> myscript.py
>>>>>>>>>
>>>>>>>>> def main(argv):
>>>>>>>>> do_cool_stuff()
>>>>>>>>> return 0
>>>>>>>>>
>>>>>>>>> if __name__ == '__main__':
>>>>>>>>> sys.exit(main(sys.argv))
>>>>>>>>>
>>>>>>>>> mygui.py(Python 2):
>>>>>>>>>
>>>>>>>>> import imp
>>>>>>>>>
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> main = imp.load_module('myscript',
>>>>>>>>> *imp.find_module('myscript'))
>>>>>>>>>
>>>>>>>>> main.main(my_argv)
>>>>>>>>>
>>>>>>>>> mygui.py(Python 3):
>>>>>>>>>
>>>>>>>>> import importlib.machinery
>>>>>>>>>
>>>>>>>>> main =
>>>>>>>>> importlib.machinery.SourceFileLoader('myscript',
>>>>>>>>> 'myscript.py').load_module('myscript')
>>>>>>>>>
>>>>>>>>> main.main(my_argv)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 23, 2014 at 7:48 PM, Frank
>>>>>>>>> Rueter | OHUfx <frank at ohufx.com
>>>>>>>>> <mailto:frank at ohufx.com>> wrote:
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I got a little code design question:
>>>>>>>>>
>>>>>>>>> I have a python script that does a lot
>>>>>>>>> of file
>>>>>>>>> processing/converting/uploading etc
>>>>>>>>> and I'd like to write a decent
>>>>>>>>> interface for it now.
>>>>>>>>> The main goal is to be able to show
>>>>>>>>> the user detailed info about the
>>>>>>>>> current step and progress as well as
>>>>>>>>> clean up properly in case the whole
>>>>>>>>> thing is cancelled.
>>>>>>>>>
>>>>>>>>> My existing python code needs to stay
>>>>>>>>> independent of QT so any
>>>>>>>>> application that supports python can
>>>>>>>>> use it.
>>>>>>>>> I am wondering now how to best connect
>>>>>>>>> the python script and the PySide
>>>>>>>>> code. Should I just run the script as
>>>>>>>>> an argument to the python
>>>>>>>>> interpreter like I would with any
>>>>>>>>> other program? E.g.:
>>>>>>>>>
>>>>>>>>> process = QtCore.QProcess(self)
>>>>>>>>> process.start(<path_to_python>,
>>>>>>>>> <path_to_python_script>)
>>>>>>>>>
>>>>>>>>> As simple as this seems, it feels odd
>>>>>>>>> to use python to call itself as an
>>>>>>>>> external program.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm happy to go that way but am
>>>>>>>>> curious how others are doing this?!
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> frank
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> PySide mailing list
>>>>>>>>> PySide at qt-project.org
>>>>>>>>> <mailto:PySide at qt-project.org>
>>>>>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ryan
>>>>>>>>> If anybody ever asks me why I prefer C++
>>>>>>>>> to C, my answer will be simple: "It's
>>>>>>>>> becauseslejfp23(@#Q*(E*EIdc-SEGFAULT.
>>>>>>>>> Wait, I don't think that was nul-terminated."
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> PySide mailing list
>>>>>>>> PySide at qt-project.org
>>>>>>>> <mailto:PySide at qt-project.org>
>>>>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> PySide mailing list
>>>>> PySide at qt-project.org <mailto:PySide at qt-project.org>
>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> PySide mailing list
>>>> PySide at qt-project.org <mailto:PySide at qt-project.org>
>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>
>>>
>>> _______________________________________________
>>> PySide mailing list
>>> PySide at qt-project.org <mailto:PySide at qt-project.org>
>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>
>>>
>>
>>
>
> --
> ohufxLogo 50x50 <http://www.ohufx.com> *vfx compositing
> <http://ohufx.com/index.php/vfx-compositing> | *workflow customisation
> and consulting <http://ohufx.com/index.php/vfx-customising>* *
>
>
>
> _______________________________________________
> PySide mailing list
> PySide at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/pyside
--
ohufxLogo 50x50 <http://www.ohufx.com> *vfx compositing
<http://ohufx.com/index.php/vfx-compositing> | *workflow customisation
and consulting <http://ohufx.com/index.php/vfx-customising>* *
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/b45daf5b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 2666 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/b45daf5b/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ohufxLogo_50x50.png
Type: image/png
Size: 2666 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/b45daf5b/attachment-0001.png>
More information about the PySide
mailing list