Unable to write to file in“w” mode (IOError: [Errno 2] No such file or directory: 'items.csv')

Question

I have a python script which calls a bunch of other python scripts via subprocess . One of these scripts has the following line:

items = open("items.csv","w")

I also tried:

path = os.getcwd()
items = open("%s/items.csv","w") %path

But this gives me the following error:

IOError: [Errno 2] No such file or directory: 'items.csv'

If I am using the file in "w" mode, it should get created if it doesn't exist. Then why am I getting this error.

Thanks

EDIT: Also tried using items = open("%s/items.csv" % path,"w") . But getting the same error again

EDIT2: The calling script:

import subprocess
import sys
import shlex
import fileinput
import os

cmd = "python2 convert.py ebay.csv ebay.xml"
arg = shlex.split(cmd)
p = subprocess.Popen(arg)
p.wait()

cmd1 = "python2 convert.py all_products.csv all_products.xml"
arg1 = shlex.split(cmd1)
p1 = subprocess.Popen(arg1)
p1.wait()

ebay = open("ebay.xml")
voylla = open("all_products.xml")

for line in fileinput.input("ebay.xml", inplace=True):
    print(line.replace("&", "and"))

for line in fileinput.input("all_products.xml", inplace=True):
    print(line.replace("&", "and"))

path = os.getcwd()
print path
cmd2 = "python2 compare.py ebay.xml all_products.xml"
arg2 = shlex.split(cmd2)
print cmd2
p2 = subprocess.Popen(arg2)
p2.wait()

cmd4 = "python2 convert.py items.csv items.xml"
arg4 = shlex.split(cmd4)
p4 = subprocess.Popen(arg4)
p4.wait()

#cmd4 = "python2 ReviseItem.py"
#arg4 = shlex.split(cmd4)
#p4 = subprocess.Popen(arg4)
#p4.wait()

compare.py:

from xml.dom.minidom import parse, parseString
import xml.etree.ElementTree as ET
import sys
import os

#sys.stdout = open('file', 'w')
ebay = sys.argv[1]
voylla = sys.argv[2]

tree = ET.parse(ebay)
root = tree.getroot()

tree1 = ET.parse(voylla)
root1 = tree1.getroot()

path = os.getcwd()

items = open("%s/items.csv" % path,"w")

Answer 1

The main issue here, since you are using subprocess in Python 2.7 to direct other python processes, is that the file will get locked for whatever process first attempts to create and write to the file. From the documentation:

Popen.wait()
Wait for child process to terminate. Set and return returncode attribute.

Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE and the child process 
generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to 
accept more data. Use communicate() to avoid that.

Another issue to be aware of: you are using the "w" permission with open , which means all previous output will be overwritten. You would do better to open the file in a master process, create and close it, then use the "a" permission with open to append to the file for each process sequentially.

Also note that a data stream from a process is not actually written to the file until the file is closed.

Answer 2

You tried to open the file '%s/items.csv' , then apply the % path to the *return value of open() .

Move the string formatting to the string:

items = open("%s/items.csv" % path, "w")

However, you should use the os.path library to handle path manipulation:

items = open(os.path.join(path, 'items.csv'), 'w')

or, in this case, use os.path.abspath instead, as it'll use os.getcwd() for you for relative paths:

items = open(os.path.abspath('items.csv'), 'w')