Python: Looping through multiple csv files and making multiple new csv files

Question

I am starting out in Python, and I am looking at csv files.

Basically my situation is this:

I have coordinates X, Y, Z in a csv.

X Y Z
1 1 1
2 2 2
3 3 3

and I want to go through and add a user defined offset value to all Z values and make a new file with the edited z-values.

here is my code so far which I think is right:

# list of lists we store all data in
allCoords = []
# get offset from user
offset = int(input("Enter an offset value: "))
# read all values into memory
with open('in.csv', 'r') as inFile: # input csv file
    reader = csv.reader(inFile, delimiter=',') 
    for row in reader:
        # do not add the first row to the list
        if row[0] != "X": 
            # create a new coord list
            coord = []
            # get a row and put it into new list
            coord.append(int(row[0]))
            coord.append(int(row[1]))
            coord.append(int(row[2]) + offset)
            # add list to list of lists
            allCoords.append(coord)

# write all values into new csv file
with open(in".out.csv", "w", newline="") as f:
    writer = csv.writer(f)
    firstRow = ['X', 'Y', 'Z']
    allCoords.insert(0, firstRow)
    writer.writerows(allCoords)

But now come's the hard part. How would I go about going through a bunch of csv files (in the same location), and producing a new file for each of the csv's.

I am hoping to have something like: "filename.csv" turns into "filename_offset.csv" using the original file name as a starter for the new filename, appending ".offset" to the end.

I think I need to use "os." functions, but I am not sure how to, so any explanation would be much appreciated along with the code! :)

Sorry if I didn't make much sense, let me know if I need to explain more clearly. :)

Thanks a bunch! :)

Answer 1

shutil.copy2(src, dst)¶
Similar to shutil.copy(), but metadata is copied as well

shutil

The glob module finds all the pathnames matching a specified pattern
according to the rules used by the Unix shell. No tilde expansion is
done, but *, ?, and character ranges expressed with [] will be correctly matched 

glob

import glob
from shutil import copy2
import shutil
files = glob.glob('cvs_DIR/*csv')

for file in files:

    try:
        # need to have full path of cvs_DIR
        oldName = os.path.join(cvs_DIR, file)
        newName = os.path.join(cvs_DIR, file[:4] + '_offset.csv')
        copy2(oldName,newName)

    except shutil.Error as e:
        print('Error: {}'.format(e))

Answer 2

BTW, you can write ...

for row in reader:
    if row[0] == "X":
        break
for row in reader:
    coord = []
    ...

... instead of ...

for row in reader:
    if row[0] != "X": 
        coord = []
        ...

This stops checking for 'X' es after the first line. It works because you dont work with a real list here but with a self consuming iterator , which you can stop and restart.

See also: Detecting if an iterator will be consumed .