Why does the Popen() function behave differently in Python and PyGTK?

Question

I wrote these two lines in a Python program and they worked fine:

subprocess.Popen("./prog infile outfile >/dev/null", cwd="/path/to/prog", stdout=subprocess.PIPE, shell=True)

output = subprocess.Popen("prog1 infile1 2>/dev/null", stdout=subprocess.PIPE, shell=True).stdout.read()

However these two lines of code do not work correctly in my PyGTK application. I invoke these lines from a handler called on the "row-activated" signal of a TreeView widget. Neither does prog write the outfile , nor do I get the output of prog1 in the output variable.

What could be the problem?

---

@Sven Marnach, thanks for your inputs. I replicated the situation in a smaller PyGTK program, where Popen works just as it should.

There is no reason why Popen should have a different behaviour in a PyGTK app.

Which means I am doing something else that is creating the problem, which I will write down once fixed.

#!/usr/bin/env python
import pygtk,sys,gtk,os,subprocess
class C:
   def main(self, argv=None):
      gtk.main()

   def __init__(self):

      # Main window
      self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
      self.window.set_border_width(2)
      self.window.set_position(gtk.WIN_POS_CENTER)
      self.window.connect("destroy", self._destroy_window)

      # TextView
      self.v = gtk.TextView()
      self.v.set_name("v")
      self.vsw = gtk.ScrolledWindow()
      self.vsw.set_policy(gtk.POLICY_AUTOMATIC, gtk.POLICY_AUTOMATIC)
      self.vsw.add(self.v)

      # TextView
      self.v1 = gtk.TextView()
      self.v1.set_name("v1")
      self.v1sw = gtk.ScrolledWindow()
      self.v1sw.set_policy(gtk.POLICY_AUTOMATIC, gtk.POLICY_AUTOMATIC)
      self.v1sw.add(self.v1)

      # TreeView
      self.model = gtk.ListStore(str, str)
      self.tv = gtk.TreeView(self.model)
      self.tv.connect("row-activated", self._f, self.v)
      self.tv.connect("row-activated", self._f, self.v1)
      self.c = gtk.CellRendererText()
      self.c1 = gtk.CellRendererText()
      self.col = gtk.TreeViewColumn("C", self.c, text=0)
      self.col1 = gtk.TreeViewColumn("C1", self.c1, text=1)
      self.tv.append_column(self.col)
      self.tv.append_column(self.col1)
      self.tvsw = gtk.ScrolledWindow()
      self.tvsw.set_policy(gtk.POLICY_AUTOMATIC, gtk.POLICY_AUTOMATIC)
      self.tvsw.add(self.tv)

      self.fill_model(self.model)

      # Layout
      self.rbox = gtk.VBox(False, 0)
      self.rbox.pack_start(self.vsw, False, False, 0)
      self.rbox.pack_start(self.v1sw, False, False, 0)
      self.box = gtk.HBox(False, 0)
      self.box.pack_start(self.tvsw, False, False, 0)
      self.box.pack_start(self.rbox, False, False, 0)

      self.window.add(self.box)
      self.window.show_all()

   def fill_model(self, model):
      self.dbg("fill_model()")
      model.clear()
      fd = open("file", "r"); rows = fd.readlines(); fd.close()
      for l in rows:
          a = l.split()
          model.append([l[0], l[1]])
      return

   def _f(self, tview, path, column, textview):
       self.dbg("_f()")
       tsel = tview.get_selection()
       model, iter = tsel.get_selected()
       buf = textview.get_buffer()
       buf.set_text("")
       if(textview.get_name() == "v"):
           self.dbg("_f():v")
           buf.set_text("hello")
       elif(textview.get_name() == "v1"):
           self.dbg("_f():v1")
           t = self.g()
           buf.set_text(t)
       return

   def g(self):
       self.dbg("g()")
       t = subprocess.Popen("anubadok file1 2>/dev/null", 
                            stdout=subprocess.PIPE, shell=True).stdout.read()
       self.dbg("g(): t = " + t)
       return t

   def _destroy_window(self, widget, data = None):
      self.dbg("_destroy_window()")
      gtk.main_quit()
      return

   def dbg(self, msg):
       sys.stderr.write("dbg: %s\n" % msg)

if __name__ == "__main__":
   ui = C()
   ui.main()

Where,

file:

a cat
b bat
c mat

file1:

hello world

And the program anubadok is here .

Answer 1

First , I don't see that you wait for the child process to complete.

What is likely to happen here is that your pygtk starts a child process and exits right away, the python garbage collector destroys the Popen object.

When you run same code in python shell or some other long-running function, Popen gets more time and hopefully child process completes.

To quickly test if this is in fact the problem, try adding time.sleep(1) after Popen call.

What you should be doing is calling .communicate() on Popen object to make sure child process does its thing and terminates.

Second , I have come across cases when pygtk program temporarily changes working directory to something else and then changes it back.

To quickly test this case, try to supply full path both to /path/to/prog as well as infile, outfile, redirect, etc.

Answer 2

The

OSError: [Errno 2] No such file or directory

is caused by your Popen call. Subprocess wants a list:

subprocess.Popen(["./prog","infile","outfile",">/dev/null"], cwd="/path/to/prog", stdout=subprocess.PIPE, shell=True)