[PySide] using QProcess to run python function

Frank Rueter | OHUfx frank at ohufx.com
Sat Jan 25 08:38:53 CET 2014


fantastic, thanks Sean!!
I will examine this to make sure I understand everything, then give 
QThreadPool a whirl. Am more than happy to learn from more experienced 
people and adjust my approaches accordingly, so thanks a lot for your 
time with this!

Cheers,
frank


On 25/01/14 20:03, Sean Fisk wrote:
>
> Hi Frank,
>
> I updated your example to hopefully work as you would like. I added a 
> progress bar update as well. I’m going to make a last-ditch effort to 
> convince you to stop doing things manually with |QThread|, and move to 
> using |QThreadPool| 
> <http://pyside.github.io/docs/pyside/PySide/QtCore/QThreadPool.html>. 
> Everything about the |QThreadPool| API is much nicer, and it’s worked 
> much better for me in real projects. Also, the Python ‘s 
> |multiprocessing| 
> <http://docs.python.org/2/library/multiprocessing.html> and 
> |concurrent.futures| <http://pythonhosted.org/futures/> modules can 
> work well if you’re careful about your callbacks.
>
> Also, there is a large discussion about not overriding the |run()| 
> method of |QThread|. I don’t think overriding it is so bad if you 
> don’t need the thread to be running an event loop of its own. I still 
> prefer the |QThreadPool| API, though. Someone please correct me if 
> this is totally wrong and there is never a reason to override it.
>
> Here is the code:
>
> |#!/usr/bin/env python
>
> import  sys
> import  time
>
> from  PySideimport  QtGui
> from  PySideimport  QtCore
>
> TOTAL_WIDGETS =10
>
> class  Worker(QtCore.QObject):
>      processed = QtCore.Signal(int)
>      finished = QtCore.Signal()
>
>      # Overriding this is not necessary if you're not doing anything in it.
>
>      # def __init__(self):
>      #     super(Worker, self).__init__()
>
>      def  helloWorld(self):
>          for  iin  xrange(TOTAL_WIDGETS):
>              # We sleep first to simulate an operation taking place.
>              time.sleep(0.5)
>              print  'hello %s'  % i
>              self.processed.emit(i +1)
>          # Must manually emit the finished signal.
>          self.finished.emit()
>
> class  MainUI(QtGui.QWidget):
>      def  __init__(self, parent=None):
>          super(MainUI, self).__init__(parent)
>          self.extraThread = QtCore.QThread()
>
>          # IMPORTANT: Don't quit the app until the thread has completed. Prevents errors like:
>          #
>          #     QThread: Destroyed while thread is still running
>          #
>          QtGui.QApplication.instance().aboutToQuit.connect(self.quit)
>
>          self.worker = Worker()
>          self.worker.moveToThread(self.extraThread)
>          self.setupUI()
>          self.connectSignalsAndSlots()
>
>      def  setupUI(self):
>          # CREAT MAIN LAYOUT AND WIDGETS
>          mainLayout = QtGui.QVBoxLayout()
>          btnLayout = QtGui.QHBoxLayout()
>          mainLayout.addLayout(btnLayout)
>          self.setLayout(mainLayout)
>
>          self.progressBar = QtGui.QProgressBar(self)
>          self.progressBar.setRange(0, TOTAL_WIDGETS)
>          self.progressBar.setVisible(False)
>          self.btnWork = QtGui.QPushButton('Do Work')
>          self.btnCancel = QtGui.QPushButton('Cancel')
>          self.btnCancel.setDisabled(True)
>
>          self.guiResponseProgressbar = QtGui.QProgressBar(self)
>          self.guiResponseProgressbar.setRange(0,0)
>
>          self.outputWindow = QtGui.QTextEdit()
>
>          mainLayout.addWidget(self.progressBar)
>          mainLayout.addWidget(self.outputWindow)
>          mainLayout.addWidget(self.guiResponseProgressbar)
>
>          btnLayout.addWidget(self.btnWork)
>          btnLayout.addWidget(self.btnCancel)
>
>      def  connectSignalsAndSlots(self):
>          print  'connecting signals'
>          self.btnWork.clicked.connect(self.startWorker)
>
>          # Pleas see <http://nooooooooooooooo.com/>. Bad bad bad bad bad.
>          # self.btnCancel.clicked.connect(self.extraThread.terminate)
>
>          # THREAD STARTED
>          # Not necessary; just do this in startWorker.
>          # self.extraThread.started.connect(lambda: self.btnWork.setDisabled(True))
>          # self.extraThread.started.connect(lambda: self.btnCancel.setEnabled(True))
>          # self.extraThread.started.connect(self.progressBar.show)
>          self.extraThread.started.connect(self.worker.helloWorld)
>
>          # THREAD FINISHED
>          # self.extraThread.finished.connect(lambda: self.btnCancel.setDisabled(True))
>          # self.extraThread.finished.connect(self.extraThread.deleteLater)
>          # self.extraThread.finished.connect(self.worker.deleteLater)
>          self.extraThread.finished.connect(self.finished)
>
>          # Connect worker signals.
>          self.worker.processed.connect(self.progressBar.setValue)
>          self.worker.finished.connect(self.finished)
>
>          # SHOW PROGRESS BAR WHEN PUBLISHING STARTS
>          # self.extraThread.started.connect(self.progressBar.show)
>          # CONNECT PROCESS TO PROGRESS BAR AND OUTPUT WINDOW
>          # NOT DONE YET
>
>      def  startWorker(self):
>          # GO
>          self.btnWork.setDisabled(True)
>          self.btnCancel.setEnabled(True)
>          self.progressBar.show()
>          print  'starting worker thread'
>          self.extraThread.start()
>
>          # THIS IS BLOCKING THE GUI THREAD! Try putting this back in and seeing
>          # what happens to the gui_response_progressbar.
>
>          # for i in xrange(10):
>          #     print 'from main thread:', i
>          #     time.sleep(.3)
>
>      def  finished(self):
>          print  'finished'
>          self.btnCancel.setDisabled(True)
>          # self.extraThread.deleteLater()
>          # self.worker.deleteLater()
>
>      def  quit(self):
>          # Quit the thread's event loop. Note that this *doesn't* stop tasks
>          # running in the thread, it just stops the thread from dispatching
>          # events.
>          self.extraThread.quit()
>          # Wait for the thread to complete. If the thread's task is not done,
>          # this will block.
>          self.extraThread.wait()
>
> if  __name__ =='__main__':
>      args = sys.argv
>      app = QtGui.QApplication(args)
>      p = MainUI()
>      p.show()
>      # Annoyance on Mac OS X.
>      p.raise_()
>      sys.exit(app.exec_())|
>
> Cheers,
>
>
>
> --
> Sean Fisk
>
>
> On Sat, Jan 25, 2014 at 12:30 AM, Frank Rueter | OHUfx 
> <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>
>     quick update:
>     I just learnt about connection types but those didn't help either
>     in this case:
>     http://qt-project.org/doc/qt-5/threads-qobject.html#signals-and-slots-across-threads
>
>
>
>
>     On 25/01/14 18:15, Frank Rueter | OHUfx wrote:
>>     And of course I ran into trouble :-D
>>     Here is my test code trying to ustilise QThread:
>>     http://pastebin.com/Q26Q9M1M
>>
>>     The basic structure seems to work (the extra thread and the main
>>     thread are running at the same time), but my UI does not update
>>     according to the signal connections, e.g.:
>>     When the thread starts the progress bar is supposed to be shown,
>>     the "Do Work" button should be disabled, the "Cancel" button
>>     disabled.
>>     etc.
>>
>>     I tried calling self.update() in the MainUI when the thread
>>     starts  but to no avail.
>>     I'm sure I'm missing something obvious as usual.
>>
>>     I tried to adhere to what I learnt on this list about threading
>>     and avoid sub-classing QThread.
>>
>>     Cheers,
>>     frank
>>
>>     On 24/01/14 17:41, Frank Rueter | OHUfx wrote:
>>>     Great, thanks. I shall adjust my code.
>>>     I am interested in seeing how you would go about it, but let me
>>>     have a go from scratch, I will be able to understand things much
>>>     better afterwards :)
>>>
>>>     Cheers,
>>>     frank
>>>
>>>
>>>     On 24/01/14 17:38, Sean Fisk wrote:
>>>>     On Thu, Jan 23, 2014 at 10:44 PM, Frank Rueter | OHUfx
>>>>     <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>>
>>>>         Much appreciated but as I mentioned, I think I jumped the
>>>>         gun with my post and should be using QThread to hook the
>>>>         python code to my QProgressBar and debug output window
>>>>         (QTextEdit).
>>>>
>>>>     That would be a correct way to use |QProgressBar|. Also, for
>>>>     subprocess output, I would consider using |QPlainTextEdit|
>>>>     instead of |QTextEdit| as is is more performant with high
>>>>     amounts of text.
>>>>
>>>>     If you are interested, I just wrote an asynchronous module for
>>>>     PySide for the project on which I am currently working. It is
>>>>     based on Python’s futures <http://pythonhosted.org/futures/>
>>>>     module and works great with Qt’s signals/slots. Although the
>>>>     project is primarily closed-source, I would be happy to share
>>>>     the async implementation with you. I don’t know if would fit
>>>>     your needs, but for us it’s much easier than manually spawning
>>>>     |QThread|s and even easier than |QThreadPool|.
>>>>
>>>>     Good luck!
>>>>
>>>>         I will investigate that now
>>>>
>>>>
>>>>
>>>>         On 24/01/14 16:08, Sean Fisk wrote:
>>>>>         On Thu, Jan 23, 2014 at 9:42 PM, Frank Rueter | OHUfx
>>>>>         <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>>>
>>>>>             Thanks Sean and Ryan,
>>>>>
>>>>>             I'm still not quite clear on how this ties into
>>>>>             QProcess.start()
>>>>>
>>>>>
>>>>>         It doesn’t tie in with |QProcess| at all. We’re advising
>>>>>         to avoid using that :)
>>>>>
>>>>>             I do have a if __name__ ... block in the script in
>>>>>             question.
>>>>>             An example would certainly be awesome, but if it's
>>>>>             less hassle, explaining how your and Ryan's advise
>>>>>             helps use QProcess on a python module might already
>>>>>             suffice. Maybe a simlpe example says it all though?!
>>>>>
>>>>>
>>>>>         I will whip up a simple example for you, but it might take
>>>>>         a few hours (lots of stuff to do right now).
>>>>>
>>>>>             I'm not using python 3 btw
>>>>>
>>>>>             Thanks guys for your help!!
>>>>>
>>>>>             frank
>>>>>
>>>>>
>>>>>             On 24/01/14 15:33, Sean Fisk wrote:
>>>>>>
>>>>>>             Hi Frank,
>>>>>>
>>>>>>             You should definitely avoid calling Python as a
>>>>>>             subprocess if you can. As far as Ryan’s example, I
>>>>>>             agree with the |if __name__...| but I think that
>>>>>>             using the |imp| module is a bit overkill. I would
>>>>>>             recommend using Setuptool’s |entry_points| keyword
>>>>>>             <http://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation>.
>>>>>>             Or distutils’ |scripts| keyword
>>>>>>             <http://docs.python.org/2/distutils/setupscript.html#installing-scripts>,
>>>>>>             if you must.
>>>>>>
>>>>>>             An example of a well-known Python package which does
>>>>>>             this is Pygments
>>>>>>             <https://bitbucket.org/birkenfeld/pygments-main>,
>>>>>>             which has a large “library” component but also comes
>>>>>>             with the |pygmentize| command-line script. The
>>>>>>             Pygments codebase is pretty large, so if you would
>>>>>>             like me to whip up a simpler example I’d be glad to
>>>>>>             do so.
>>>>>>
>>>>>>             Cheers,
>>>>>>
>>>>>>             --
>>>>>>             Sean Fisk
>>>>>>
>>>>>>
>>>>>>             On Thu, Jan 23, 2014 at 9:17 PM, Frank Rueter | OHUfx
>>>>>>             <frank at ohufx.com <mailto:frank at ohufx.com>> wrote:
>>>>>>
>>>>>>                 Sorry if I'm being thick, but I'm not quite
>>>>>>                 understanding how this helps to connect a python
>>>>>>                 function to qprocess?! All your code does is
>>>>>>                 execute the script, right?!
>>>>>>                 I can already call myscript.main() straight up,
>>>>>>                 but maybe I'm missing the point as I'm unfamiliar
>>>>>>                 with the imp module.
>>>>>>
>>>>>>                 Let me elaborate a little bit more:
>>>>>>                 myscript.main() calls a bunch of other python
>>>>>>                 scripts that (directly or through other scripts
>>>>>>                 again) execute external programs to do some
>>>>>>                 conversion work. Those external programs spit out
>>>>>>                 their progress to stdout which I can see fine
>>>>>>                 when I run myscript.main() manually in a python
>>>>>>                 terminal.
>>>>>>
>>>>>>                 Now I need run myscript.main() via QProcess and
>>>>>>                 grab stdout to do be able to show a progress bar
>>>>>>                 as well as show stdout and stderr in a debug
>>>>>>                 window inside my QT code.
>>>>>>
>>>>>>
>>>>>>                 Cheers,
>>>>>>                 frank
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>                 On 24/01/14 14:58, Ryan Gonzalez wrote:
>>>>>>>                 If you put an "if __name__ == '__main__'" and a
>>>>>>>                 main functions, you could always import the
>>>>>>>                 script from the GUI frontend. Example:
>>>>>>>
>>>>>>>                 myscript.py
>>>>>>>
>>>>>>>                 def main(argv):
>>>>>>>                 do_cool_stuff()
>>>>>>>                     return 0
>>>>>>>
>>>>>>>                 if __name__ == '__main__':
>>>>>>>                 sys.exit(main(sys.argv))
>>>>>>>
>>>>>>>                 mygui.py(Python 2):
>>>>>>>
>>>>>>>                 import imp
>>>>>>>
>>>>>>>                 ...
>>>>>>>
>>>>>>>                 main = imp.load_module('myscript',
>>>>>>>                 *imp.find_module('myscript'))
>>>>>>>
>>>>>>>                 main.main(my_argv)
>>>>>>>
>>>>>>>                 mygui.py(Python 3):
>>>>>>>
>>>>>>>                 import importlib.machinery
>>>>>>>
>>>>>>>                 main =
>>>>>>>                 importlib.machinery.SourceFileLoader('myscript',
>>>>>>>                 'myscript.py').load_module('myscript')
>>>>>>>
>>>>>>>                 main.main(my_argv)
>>>>>>>
>>>>>>>
>>>>>>>                 On Thu, Jan 23, 2014 at 7:48 PM, Frank Rueter |
>>>>>>>                 OHUfx <frank at ohufx.com <mailto:frank at ohufx.com>>
>>>>>>>                 wrote:
>>>>>>>
>>>>>>>                     Hi all,
>>>>>>>
>>>>>>>                     I got a little code design question:
>>>>>>>
>>>>>>>                     I have a python script that does a lot of file
>>>>>>>                     processing/converting/uploading etc and I'd
>>>>>>>                     like to write a decent
>>>>>>>                     interface for it now.
>>>>>>>                     The main goal is to be able to show the user
>>>>>>>                     detailed info about the
>>>>>>>                     current step and progress as well as clean
>>>>>>>                     up properly in case the whole
>>>>>>>                     thing is cancelled.
>>>>>>>
>>>>>>>                     My existing python code needs to stay
>>>>>>>                     independent of QT so any
>>>>>>>                     application that supports python can use it.
>>>>>>>                     I am wondering now how to best connect the
>>>>>>>                     python script and the PySide
>>>>>>>                     code. Should I just run the script as an
>>>>>>>                     argument to the python
>>>>>>>                     interpreter like I would with any other
>>>>>>>                     program? E.g.:
>>>>>>>
>>>>>>>                     process = QtCore.QProcess(self)
>>>>>>>                     process.start(<path_to_python>,
>>>>>>>                     <path_to_python_script>)
>>>>>>>
>>>>>>>                     As simple as this seems, it feels odd to use
>>>>>>>                     python to call itself as an
>>>>>>>                     external program.
>>>>>>>
>>>>>>>
>>>>>>>                     I'm happy to go that way but am curious how
>>>>>>>                     others are doing this?!
>>>>>>>
>>>>>>>                     Cheers,
>>>>>>>                     frank
>>>>>>>
>>>>>>>                     _______________________________________________
>>>>>>>                     PySide mailing list
>>>>>>>                     PySide at qt-project.org
>>>>>>>                     <mailto:PySide at qt-project.org>
>>>>>>>                     http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                 -- 
>>>>>>>                 Ryan
>>>>>>>                 If anybody ever asks me why I prefer C++ to C,
>>>>>>>                 my answer will be simple: "It's
>>>>>>>                 becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
>>>>>>>                 don't think that was nul-terminated."
>>>>>>>
>>>>>>
>>>>>>
>>>>>>                 _______________________________________________
>>>>>>                 PySide mailing list
>>>>>>                 PySide at qt-project.org <mailto:PySide at qt-project.org>
>>>>>>                 http://lists.qt-project.org/mailman/listinfo/pyside
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>     _______________________________________________
>>>     PySide mailing list
>>>     PySide at qt-project.org  <mailto:PySide at qt-project.org>
>>>     http://lists.qt-project.org/mailman/listinfo/pyside
>>
>>
>>
>>     _______________________________________________
>>     PySide mailing list
>>     PySide at qt-project.org  <mailto:PySide at qt-project.org>
>>     http://lists.qt-project.org/mailman/listinfo/pyside
>
>
>     _______________________________________________
>     PySide mailing list
>     PySide at qt-project.org <mailto:PySide at qt-project.org>
>     http://lists.qt-project.org/mailman/listinfo/pyside
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/pyside/attachments/20140125/da4c9e85/attachment.html>


More information about the PySide mailing list