examine output of shell command using subprocess in Python

Question

I'm running a shell command in a Jupyter Notebook using subprocess or os.system() . The actual output is a dump of thousands of lines of code which takes at least a minute to stdout in terminal. In my notebook, I just want to know if the output is more than a couple of lines because if it was an error, the output would only be 1 or 2 lines. What's the best way to check if I'm receiving 20+ lines and then stop the process and move on to the next?

Answer 1

you could read line by line using subprocess.Popen and count the lines (redirecting & merging output and error streams, maybe merging is not needed, depends on the process)

• If the number of lines exceeds 20, kill the process and break the loop.
• If the loop ends before the number of lines reaches 20, print/handle an error

code:

p = subprocess.Popen(cmd,
                     stdout=subprocess.PIPE,
                     stderr=subprocess.STDOUT)


for lineno,line in enumerate(iter(p.stdout.readline, b'')):
    if lineno == 20:
        print("process okay")
        p.kill()
        break
else:
    # too short, break wasn't reached
    print("process failed return code: {}".format(p.wait()))

note that p.poll() is not None can help to figure out if the process has ended prematurely too