从子进程读取的实时stdout仅在从PyCharm运行而不在Terminal运行时有效

Question

What the application does: runs a subprocess and displays the stdout in real-time to a Tkinter textbox widget.

This works perfectly when I run the application from PyCharm.

When I run the application from terminal ./application.py it doesn't display in real-time, but instead will display it all after the process has finished.

---

Some details:

I have a subprocess running (the subprocess prints out "SPAM" every 1 second for 10 seconds):

process = subprocess.Popen(<some file path>, stdout=subprocess.PIPE, universal_newlines=True)

I am printing the stdout to a Tkinter textbox widget:

for stdout_line in iter(process.stdout.readline, ""):
     self.app.pm_textbox.see(tk.END)
     self.app.pm_textbox.insert(tk.INSERT, stdout_line)

---

So my question is what could possibly cause running from terminal and PyCharm to display stdout data differently?

Answer 1

I know this is an old thread, but still -

I had the same issue as you did. I asked a question that seemed to help me get in the right direction as to working with real time output. My issue was I needed to differentiate output that resulted of a CR and simulate that behavior in my program. But then I encountered your problem as well - I worked for 3 days trying to find a solution as to why this happens.

After reading this and having no luck with what they did there, I found this thread without answers but one comment that after a little modification, finally helped me.

What finally worked for me, is this:

#                        This seems to handle    This seems to handle
#                        the terminal output     the PyCharm output
proc = subprocess.Popen(["stdbuf", "-oL"] + cmd, stdout=subprocess.PIPE)

out = io.open(proc.stdout.fileno(), mode='r', encoding="utf-8", newline='')
should_cr = False
for line in out:
    if should_cr is True:
        sys.stdout.write("\r")
        sys.stdout.flush()

    if line.endswith(os.linesep):
        should_cr = False
        sys.stdout.write(line)
        sys.stdout.flush()

    elif line.endswith("\r"):
        should_cr = True
        line = line.rstrip()
        sys.stdout.write(line)
        sys.stdout.flush()

If you wouldn't regard CR output then it is more simple of course:

#                        This seems to handle    This seems to handle
#                        the terminal output     the PyCharm output
proc = subprocess.Popen(["stdbuf", "-oL"] + cmd, stdout=subprocess.PIPE)
for line in iter(proc.stdout.readline, ''):
    print(line)

Notice this works for me in Python 2.7