wxPython, capturing an output from subprocess in real-time

Question

I'm working on application in wxPython which is a GUI for a command line utility. In the GUI there is a text control which should display the output from the application. I'm launching the shell command using subprocess, but I don't get any output from it until it has completed.

I have tried several solutions but none of them seems to work. Below is the code I'm using at the moment (updated):

       def onOk(self,event):        
        self.getControl('infotxt').Clear()
        try:
            thread = threading.Thread(target=self.run)
            thread.setDaemon(True)
            thread.start()

        except Exception:
            print 'Error starting thread'    

    def run(self):    
        args = dict()
        # creating a command to execute...

        cmd = ["aplcorr", "-vvfile", args['vvfile'], "-navfile", args['navfile'], "-lev1file", args['lev1file'], "-dem", args['dem'], "-igmfile", args['outfile']]

        proc = subprocess.Popen(' '.join(cmd), shell=True, stdout=subprocess.PIPE, stderr.subprocess.PIPE)         

        print
        while True:
            line = proc.stdout.readline()
            wx.Yield()
            if line.strip() == "":
                pass
            else:                
                print line.strip()                
            if not line: break
        proc.wait()

class RedirectInfoText:
    """ Class to redirect stdout text """
    def __init__(self,wxTextCtrl):
        self.out=wxTextCtrl

    def write(self,string):
        self.out.WriteText(string)

class RedirectErrorText:
    """ Class to redirect stderr text """    
    def __init__(self,wxTextCtrl):
        self.out.SetDefailtStyle(wx.TextAttr())
        self.out=wxTextCtrl

    def write(self,string):
        self.out.SetDefaultStyle(wx.TextAttr(wx.RED))
        self.out.WriteText(string)

In particular I'm going to need the output in real-time to create a progress-bar.

Edit: I changed my code, based on Mike Driscoll's suggestion. It seems to work sometimes, but most of the time I'm getting one of the following errors:

(python:7698): Gtk-CRITICAL **: gtk_text_layout_real_invalidate: assertion `layout->wrap_loop_count == 0' failed

or

(python:7893): Gtk-WARNING **: Invalid text buffer iterator: either the iterator is uninitialized, or the characters/pixbufs/widgets in the buffer have been modified since the iterator was created. You must use marks, character numbers, or line numbers to preserve a position across buffer modifications. You can apply tags and insert marks without invalidating your iterators, but any mutation that affects 'indexable' buffer contents (contents that can be referred to by character offset) will invalidate all outstanding iterators Segmentation fault (core dumped)

Any clues?

Answer 1

The problem is because you are trying to wx.Yield and to update the output widgets from the context of the thread running the process, instead of doing the update from the GUI thread.

1. Since you are running the process from a thread there should be no need to call wx.Yield, because you are not blocking the GUI thread, and so any pending UI events should be processed normally anyway.

2. Take a look at the wx.PyOnDemandOutputWindow class for an example of how to handle prints or other output that originate from a non-GUI thread.

Answer 2

This can be a little tricky, but I figured out one way to do it which I wrote about here: http://www.blog.pythonlibrary.org/2010/06/05/python-running-ping-traceroute-and-more/

After you have set up the redirection of the text, you just need to do something like this:

def pingIP(self, ip):
    proc = subprocess.Popen("ping %s" % ip, shell=True,
                            stdout=subprocess.PIPE)
    print
    while True:
        line = proc.stdout.readline()
        wx.Yield()
        if line.strip() == "":
            pass
        else:
            print line.strip()
        if not line: break
    proc.wait()

The article shows how to redirect the text too. Hopefully that will help!