Unable to read file with python

Question

I'm trying to read the content of a file with python 3.8.5 but the output is empty, I don't understand what I'm doing wrong.

Here is the code:

import subprocess
import os

filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
subprocess.Popen(ls_command, stdout=file, shell=True)
file.close()

# So far, all is ok. The file "ls.out" is correctly created and filled with the output of "ls -la" command"

file = open(filename, "r")
for line in file:
    print(line)
file.close()

The output of this script is empty, it doesn't print anything. I'm not able to see the content of ls.out .

What is not correct here?

Answer 1

Popen creates a new process and launches it but returns immediately. So the end result is that you've fork ed your code and have both processes running at once. Your python code in executing faster than the start and finish of ls . Thus, you need to wait for the process to finish by adding a call to wait() :

import subprocess
import os

filename = "ls.out"
ls_command = "ls -la"
file = open(filename, "w")
proc = subprocess.Popen(ls_command, stdout=file, shell=True)
proc.wait()
file.close()


file = open(filename, "r")
for line in file:
    print(line)
file.close()

Answer 2

Popen merely starts the subprocess. Chances are the file is not yet populated when you open it.

If you want to wait for the Popen object to finish, you have to call its wait method, etc; but a much better and simpler solution is to use subprocess.check_call() or one of the other higher-level wrappers.

If the command prints to standard output, why don't you read it drectly?

import subprocess
import shlex

result = subprocess.run(
    shlex.split(ls_command), # avoid shell=True
    check=True, text=True, capture_output=True)
line = result.stdout