How to get output in the subprocess in real time?

Question

I'm trying to get tail -f /var/log/syslog to play the result in variable data0 but without success.

from subprocess import Popen,PIPE
 
def exit_data():
    with Popen(['tail -f', '/var/log/syslog'],stdout=PIPE,stderr=PIPE) as b:
        out,err = b.communicate()
    data0 = out.decode('utf-8')
    return data0

Answer 1

From the documentation, calling the communicate() method will block until the child process exits. Since you're calling tail -f , this will not return until the tail process exits, which only happens on EOF, errors, etc. So you don't see anything.

It looks like you want to continuously print the output of the tail subprocess in Python. To do this, you'd need to start the process, and continually (in a loop) read from its stdout and print the result. Do not call communicate() , and instead just read from the stdout attribute, which is a standard file-like object.

For example, this script would be reader.py :

import subprocess as sp

# A dummy file to tail
filename = "/tmp/logfile"

proc = sp.Popen(
    ["tail", "-f", filename],
    stdout=sp.PIPE,
    stderr=sp.PIPE,
    text=True,  # I'm using text files, may not apply to your case
)
try:
    while True:
        print(proc.stdout.readline().rstrip("\n"))
except KeyboardInterrupt:
    print("Received interrupt, exiting")
    proc.terminate()
    proc.wait()
    print("Reaped child")

You can test this works by running the following snippet in another Python script, call it writer.py :

import time
N_LINES = 100

filename = "/tmp/logfile"
with open(filename, "wt") as f:
    for _ in range(N_LINES):
        time.sleep(1)
        f.write("a new line of data\n")
        f.flush()

Run them with:

$ python3 writer.py &
$ python3 reader.py
a new line of data
a new line of data
a new line of data
a new line of data
a new line of data
^CReceived interrupt, exiting
Reaped child