[PySide] using QProcess to run python function
Frank Rueter | OHUfx
frank at ohufx.com
Fri Mar 14 02:30:16 CET 2014
after a much longer absence from this than anticipated, I'm finally
trying to get back into this.
I had a look at your example Sean and it makes sense so far.
Now I'm trying to follow your advise and use QThreadPool, but am not
finding any good examples on it's usage.
Unfortunately, the first example mentioned in the docs
<http://srinikom.github.io/pyside-docs/PySide/QtCore/QThreadPool.html>
already throws an error for me:
from PySide import QtCore
class HelloWorldTask(QtCore.QRunnable):
def run(self):
print "Hello world from thread", QtCore.QThread.currentThread()
hello = HelloWorldTask()
# QThreadPool takes ownership and deletes 'hello' automatically
QtCore.QThreadPool.globalInstance().start(hello)
result:
Hello world from thread
Traceback (most recent call last):
File "/ohufx/pipeline/tools/python/sandbox/QThreadPoolTest.py", line
6, in run
print "Hello world from thread", QtCore.QThread.currentThread()
AttributeError: 'NoneType' object has no attribute 'QThread'
What am I missing here?
Cheers,
frank
On 28/01/14 08:00, Sean Fisk wrote:
> On Sat, Jan 25, 2014 at 2:38 AM, Frank Rueter | OHUfx <frank at ohufx.com
> <mailto:frank at ohufx.com>> wrote:
>
> fantastic, thanks Sean!!
> I will examine this to make sure I understand everything, then
> give QThreadPool a whirl. Am more than happy to learn from more
> experienced people and adjust my approaches accordingly, so thanks
> a lot for your time with this!
>
>
> No problem! Let me know if you have any questions about the code.
>
>
> Cheers,
> frank
>
>
>
> On 25/01/14 20:03, Sean Fisk wrote:
>>
>> Hi Frank,
>>
>> I updated your example to hopefully work as you would like. I
>> added a progress bar update as well. I’m going to make a
>> last-ditch effort to convince you to stop doing things manually
>> with |QThread|, and move to using |QThreadPool|
>> <http://pyside.github.io/docs/pyside/PySide/QtCore/QThreadPool.html>.
>> Everything about the |QThreadPool| API is much nicer, and it’s
>> worked much better for me in real projects. Also, the Python ‘s
>> |multiprocessing|
>> <http://docs.python.org/2/library/multiprocessing.html> and
>> |concurrent.futures| <http://pythonhosted.org/futures/> modules
>> can work well if you’re careful about your callbacks.
>>
>> Also, there is a large discussion about not overriding the
>> |run()| method of |QThread|. I don’t think overriding it is so
>> bad if you don’t need the thread to be running an event loop of
>> its own. I still prefer the |QThreadPool| API, though. Someone
>> please correct me if this is totally wrong and there is never a
>> reason to override it.
>>
>> Here is the code:
>>
>> |#!/usr/bin/env python
>>
>> import sys
>> import time
>>
>> from PySideimport QtGui
>> from PySideimport QtCore
>>
>> TOTAL_WIDGETS =10
>>
>> class Worker(QtCore.QObject):
>> processed = QtCore.Signal(int)
>> finished = QtCore.Signal()
>>
>> # Overriding this is not necessary if you're not doing anything in it.
>>
>> # def __init__(self):
>> # super(Worker, self).__init__()
>>
>> def helloWorld(self):
>> for iin xrange(TOTAL_WIDGETS):
>> # We sleep first to simulate an operation taking place.
>> time.sleep(0.5)
>> print 'hello %s' % i
>> self.processed.emit(i +1)
>> # Must manually emit the finished signal.
>> self.finished.emit()
>>
>> class MainUI(QtGui.QWidget):
>> def __init__(self, parent=None):
>> super(MainUI, self).__init__(parent)
>> self.extraThread = QtCore.QThread()
>>
>> # IMPORTANT: Don't quit the app until the thread has completed. Prevents errors like:
>> #
>> # QThread: Destroyed while thread is still running
>> #
>> QtGui.QApplication.instance().aboutToQuit.connect(self.quit)
>>
>> self.worker = Worker()
>> self.worker.moveToThread(self.extraThread)
>> self.setupUI()
>> self.connectSignalsAndSlots()
>>
>> def setupUI(self):
>> # CREAT MAIN LAYOUT AND WIDGETS
>> mainLayout = QtGui.QVBoxLayout()
>> btnLayout = QtGui.QHBoxLayout()
>> mainLayout.addLayout(btnLayout)
>> self.setLayout(mainLayout)
>>
>> self.progressBar = QtGui.QProgressBar(self)
>> self.progressBar.setRange(0, TOTAL_WIDGETS)
>> self.progressBar.setVisible(False)
>> self.btnWork = QtGui.QPushButton('Do Work')
>> self.btnCancel = QtGui.QPushButton('Cancel')
>> self.btnCancel.setDisabled(True)
>>
>> self.guiResponseProgressbar = QtGui.QProgressBar(self)
>> self.guiResponseProgressbar.setRange(0,0)
>>
>> self.outputWindow = QtGui.QTextEdit()
>>
>> mainLayout.addWidget(self.progressBar)
>> mainLayout.addWidget(self.outputWindow)
>> mainLayout.addWidget(self.guiResponseProgressbar)
>>
>> btnLayout.addWidget(self.btnWork)
>> btnLayout.addWidget(self.btnCancel)
>>
>> def connectSignalsAndSlots(self):
>> print 'connecting signals'
>> self.btnWork.clicked.connect(self.startWorker)
>>
>> # Pleas see <http://nooooooooooooooo.com/>. Bad bad bad bad bad.
>> # self.btnCancel.clicked.connect(self.extraThread.terminate)
>>
>> # THREAD STARTED
>> # Not necessary; just do this in startWorker.
>> # self.extraThread.started.connect(lambda: self.btnWork.setDisabled(True))
>> # self.extraThread.started.connect(lambda: self.btnCancel.setEnabled(True))
>> # self.extraThread.started.connect(self.progressBar.show)
>> self.extraThread.started.connect(self.worker.helloWorld)
>>
>> # THREAD FINISHED
>> # self.extraThread.finished.connect(lambda: self.btnCancel.setDisabled(True))
>> # self.extraThread.finished.connect(self.extraThread.deleteLater)
>> # self.extraThread.finished.connect(self.worker.deleteLater)
>> self.extraThread.finished.connect(self.finished)
>>
>> # Connect worker signals.
>> self.worker.processed.connect(self.progressBar.setValue)
>> self.worker.finished.connect(self.finished)
>>
>> # SHOW PROGRESS BAR WHEN PUBLISHING STARTS
>> # self.extraThread.started.connect(self.progressBar.show)
>> # CONNECT PROCESS TO PROGRESS BAR AND OUTPUT WINDOW
>> # NOT DONE YET
>>
>> def startWorker(self):
>> # GO
>> self.btnWork.setDisabled(True)
>> self.btnCancel.setEnabled(True)
>> self.progressBar.show()
>> print 'starting worker thread'
>> self.extraThread.start()
>>
>> # THIS IS BLOCKING THE GUI THREAD! Try putting this back in and seeing
>> # what happens to the gui_response_progressbar.
>>
>> # for i in xrange(10):
>> # print 'from main thread:', i
>> # time.sleep(.3)
>>
>> def finished(self):
>> print 'finished'
>> self.btnCancel.setDisabled(True)
>> # self.extraThread.deleteLater()
>> # self.worker.deleteLater()
>>
>> def quit(self):
>> # Quit the thread's event loop. Note that this *doesn't* stop tasks
>> # running in the thread, it just stops the thread from dispatching
>> # events.
>> self.extraThread.quit()
>> # Wait for the thread to complete. If the thread's task is not done,
>> # this will block.
>> self.extraThread.wait()
>>
>> if __name__ =='__main__':
>> args = sys.argv
>> app = QtGui.QApplication(args)
>> p = MainUI()
>> p.show()
>> # Annoyance on Mac OS X.
>> p.raise_()
>> sys.exit(app.exec_())|
>>
>> Cheers,
>>
>>
>>
>> --
>> Sean Fisk
>>
>>
>> On Sat, Jan 25, 2014 at 12:30 AM, Frank Rueter | OHUfx
>> <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>
>> quick update:
>> I just learnt about connection types but those didn't help
>> either in this case:
>> http://qt-project.org/doc/qt-5/threads-qobject.html#signals-and-slots-across-threads
>>
>>
>>
>>
>> On 25/01/14 18:15, Frank Rueter | OHUfx wrote:
>>> And of course I ran into trouble :-D
>>> Here is my test code trying to ustilise QThread:
>>> http://pastebin.com/Q26Q9M1M
>>>
>>> The basic structure seems to work (the extra thread and the
>>> main thread are running at the same time), but my UI does
>>> not update according to the signal connections, e.g.:
>>> When the thread starts the progress bar is supposed to be
>>> shown, the "Do Work" button should be disabled, the "Cancel"
>>> button disabled.
>>> etc.
>>>
>>> I tried calling self.update() in the MainUI when the thread
>>> starts but to no avail.
>>> I'm sure I'm missing something obvious as usual.
>>>
>>> I tried to adhere to what I learnt on this list about
>>> threading and avoid sub-classing QThread.
>>>
>>> Cheers,
>>> frank
>>>
>>> On 24/01/14 17:41, Frank Rueter | OHUfx wrote:
>>>> Great, thanks. I shall adjust my code.
>>>> I am interested in seeing how you would go about it, but
>>>> let me have a go from scratch, I will be able to understand
>>>> things much better afterwards :)
>>>>
>>>> Cheers,
>>>> frank
>>>>
>>>>
>>>> On 24/01/14 17:38, Sean Fisk wrote:
>>>>> On Thu, Jan 23, 2014 at 10:44 PM, Frank Rueter | OHUfx
>>>>> <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>>>
>>>>> Much appreciated but as I mentioned, I think I jumped
>>>>> the gun with my post and should be using QThread to
>>>>> hook the python code to my QProgressBar and debug
>>>>> output window (QTextEdit).
>>>>>
>>>>> That would be a correct way to use |QProgressBar|. Also,
>>>>> for subprocess output, I would consider using
>>>>> |QPlainTextEdit| instead of |QTextEdit| as is is more
>>>>> performant with high amounts of text.
>>>>>
>>>>> If you are interested, I just wrote an asynchronous module
>>>>> for PySide for the project on which I am currently
>>>>> working. It is based on Python’s futures
>>>>> <http://pythonhosted.org/futures/> module and works great
>>>>> with Qt’s signals/slots. Although the project is primarily
>>>>> closed-source, I would be happy to share the async
>>>>> implementation with you. I don’t know if would fit your
>>>>> needs, but for us it’s much easier than manually spawning
>>>>> |QThread|s and even easier than |QThreadPool|.
>>>>>
>>>>> Good luck!
>>>>>
>>>>> I will investigate that now
>>>>>
>>>>>
>>>>>
>>>>> On 24/01/14 16:08, Sean Fisk wrote:
>>>>>> On Thu, Jan 23, 2014 at 9:42 PM, Frank Rueter | OHUfx
>>>>>> <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>>>>
>>>>>> Thanks Sean and Ryan,
>>>>>>
>>>>>> I'm still not quite clear on how this ties into
>>>>>> QProcess.start()
>>>>>>
>>>>>>
>>>>>> It doesn’t tie in with |QProcess| at all. We’re
>>>>>> advising to avoid using that :)
>>>>>>
>>>>>> I do have a if __name__ ... block in the script
>>>>>> in question.
>>>>>> An example would certainly be awesome, but if
>>>>>> it's less hassle, explaining how your and Ryan's
>>>>>> advise helps use QProcess on a python module
>>>>>> might already suffice. Maybe a simlpe example
>>>>>> says it all though?!
>>>>>>
>>>>>>
>>>>>> I will whip up a simple example for you, but it might
>>>>>> take a few hours (lots of stuff to do right now).
>>>>>>
>>>>>> I'm not using python 3 btw
>>>>>>
>>>>>> Thanks guys for your help!!
>>>>>>
>>>>>> frank
>>>>>>
>>>>>>
>>>>>> On 24/01/14 15:33, Sean Fisk wrote:
>>>>>>>
>>>>>>> Hi Frank,
>>>>>>>
>>>>>>> You should definitely avoid calling Python as a
>>>>>>> subprocess if you can. As far as Ryan’s example,
>>>>>>> I agree with the |if __name__...| but I think
>>>>>>> that using the |imp| module is a bit overkill. I
>>>>>>> would recommend using Setuptool’s |entry_points|
>>>>>>> keyword
>>>>>>> <http://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation>.
>>>>>>> Or distutils’ |scripts| keyword
>>>>>>> <http://docs.python.org/2/distutils/setupscript.html#installing-scripts>,
>>>>>>> if you must.
>>>>>>>
>>>>>>> An example of a well-known Python package which
>>>>>>> does this is Pygments
>>>>>>> <https://bitbucket.org/birkenfeld/pygments-main>, which
>>>>>>> has a large “library” component but also comes
>>>>>>> with the |pygmentize| command-line script. The
>>>>>>> Pygments codebase is pretty large, so if you
>>>>>>> would like me to whip up a simpler example I’d
>>>>>>> be glad to do so.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> --
>>>>>>> Sean Fisk
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 23, 2014 at 9:17 PM, Frank Rueter |
>>>>>>> OHUfx <frank at ohufx.com <mailto:frank at ohufx.com>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Sorry if I'm being thick, but I'm not quite
>>>>>>> understanding how this helps to connect a
>>>>>>> python function to qprocess?! All your code
>>>>>>> does is execute the script, right?!
>>>>>>> I can already call myscript.main() straight
>>>>>>> up, but maybe I'm missing the point as I'm
>>>>>>> unfamiliar with the imp module.
>>>>>>>
>>>>>>> Let me elaborate a little bit more:
>>>>>>> myscript.main() calls a bunch of other
>>>>>>> python scripts that (directly or through
>>>>>>> other scripts again) execute external
>>>>>>> programs to do some conversion work. Those
>>>>>>> external programs spit out their progress to
>>>>>>> stdout which I can see fine when I run
>>>>>>> myscript.main() manually in a python terminal.
>>>>>>>
>>>>>>> Now I need run myscript.main() via QProcess
>>>>>>> and grab stdout to do be able to show a
>>>>>>> progress bar as well as show stdout and
>>>>>>> stderr in a debug window inside my QT code.
>>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>> frank
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 24/01/14 14:58, Ryan Gonzalez wrote:
>>>>>>>> If you put an "if __name__ == '__main__'"
>>>>>>>> and a main functions, you could always
>>>>>>>> import the script from the GUI frontend.
>>>>>>>> Example:
>>>>>>>>
>>>>>>>> myscript.py
>>>>>>>>
>>>>>>>> def main(argv):
>>>>>>>> do_cool_stuff()
>>>>>>>> return 0
>>>>>>>>
>>>>>>>> if __name__ == '__main__':
>>>>>>>> sys.exit(main(sys.argv))
>>>>>>>>
>>>>>>>> mygui.py(Python 2):
>>>>>>>>
>>>>>>>> import imp
>>>>>>>>
>>>>>>>> ...
>>>>>>>>
>>>>>>>> main = imp.load_module('myscript',
>>>>>>>> *imp.find_module('myscript'))
>>>>>>>>
>>>>>>>> main.main(my_argv)
>>>>>>>>
>>>>>>>> mygui.py(Python 3):
>>>>>>>>
>>>>>>>> import importlib.machinery
>>>>>>>>
>>>>>>>> main =
>>>>>>>> importlib.machinery.SourceFileLoader('myscript',
>>>>>>>> 'myscript.py').load_module('myscript')
>>>>>>>>
>>>>>>>> main.main(my_argv)
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 23, 2014 at 7:48 PM, Frank
>>>>>>>> Rueter | OHUfx <frank at ohufx.com
>>>>>>>> <mailto:frank at ohufx.com>> wrote:
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I got a little code design question:
>>>>>>>>
>>>>>>>> I have a python script that does a lot
>>>>>>>> of file
>>>>>>>> processing/converting/uploading etc and
>>>>>>>> I'd like to write a decent
>>>>>>>> interface for it now.
>>>>>>>> The main goal is to be able to show the
>>>>>>>> user detailed info about the
>>>>>>>> current step and progress as well as
>>>>>>>> clean up properly in case the whole
>>>>>>>> thing is cancelled.
>>>>>>>>
>>>>>>>> My existing python code needs to stay
>>>>>>>> independent of QT so any
>>>>>>>> application that supports python can
>>>>>>>> use it.
>>>>>>>> I am wondering now how to best connect
>>>>>>>> the python script and the PySide
>>>>>>>> code. Should I just run the script as
>>>>>>>> an argument to the python
>>>>>>>> interpreter like I would with any other
>>>>>>>> program? E.g.:
>>>>>>>>
>>>>>>>> process = QtCore.QProcess(self)
>>>>>>>> process.start(<path_to_python>,
>>>>>>>> <path_to_python_script>)
>>>>>>>>
>>>>>>>> As simple as this seems, it feels odd
>>>>>>>> to use python to call itself as an
>>>>>>>> external program.
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm happy to go that way but am curious
>>>>>>>> how others are doing this?!
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> frank
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> PySide mailing list
>>>>>>>> PySide at qt-project.org
>>>>>>>> <mailto:PySide at qt-project.org>
>>>>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ryan
>>>>>>>> If anybody ever asks me why I prefer C++ to
>>>>>>>> C, my answer will be simple: "It's
>>>>>>>> becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait,
>>>>>>>> I don't think that was nul-terminated."
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> PySide mailing list
>>>>>>> PySide at qt-project.org
>>>>>>> <mailto:PySide at qt-project.org>
>>>>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> PySide mailing list
>>>> PySide at qt-project.org <mailto:PySide at qt-project.org>
>>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>>
>>>
>>>
>>> _______________________________________________
>>> PySide mailing list
>>> PySide at qt-project.org <mailto:PySide at qt-project.org>
>>> http://lists.qt-project.org/mailman/listinfo/pyside
>>
>>
>> _______________________________________________
>> PySide mailing list
>> PySide at qt-project.org <mailto:PySide at qt-project.org>
>> http://lists.qt-project.org/mailman/listinfo/pyside
>>
>>
>
>
--
ohufxLogo 50x50 <http://www.ohufx.com> *vfx compositing
<http://ohufx.com/index.php/vfx-compositing> | *workflow customisation
and consulting <http://ohufx.com/index.php/vfx-customising>* *
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/bef2d136/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ohufxLogo_50x50.png
Type: image/png
Size: 2666 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140314/bef2d136/attachment.png>
More information about the PySide
mailing list