cant access content of a file while using subprocess python

Question

Script A.py (python 3.0) does some work and creates a output.csv file. Script B.py (python 2.5) takes output.csv as input and does some work.

Example A.py:

Outfile = open (path_to_csv, "w+")
Outfile.write(string_to_be_written)
Outfile.close
subprocess.call(["path/to/B.py", path_to_csv, args+])

Example B.py:

csv_file = open(path_to_csv, "rb")
csv_reader = csv.reader(csv_file)
for row in csv_reader:
    print row

Now when I run B.py directly, it works perfectly and prints each row of the csv. But doesn't read anything when called from A.py with subprocess.call. It doesnt throw any errors while reading either.

I have tried using os.system and subprocess.Popen but they encounter the same problem. What am I missing here?

Answer 1

Outfile.write = open (path_to_csv, "w+") is probably a typo, next line could work as is. You probably mean Outfile = open (path_to_csv, "w+")

Then the problem is that you don't actually close your file:

Outfile.close

accomplishes nothing. You need:

Outfile = open (path_to_csv, "w+")
Outfile.write(string_to_be_written)
Outfile.close()

else the file stays open/unflushed. On Windows, you cannot open it again (locked), on Linux it may contain unflushed data.

The most annoying thing as that when a finishes, the file is properly closed and you see the proper csv output, so it's misleading as why it doesn't work.

Better: use a with block to close file on exiting the block

with open (path_to_csv, "w+") as Outfile:
   Outfile.write(string_to_be_written)
# now call the subprocess outside the with block: the file is closed
subprocess.call(["path/to/B.py", path_to_csv, args+])

Even better: rewrite both scripts so output of the first one is consumed by input of the second one (avoids writing a temporary file just to pass data)