for loop using to_csv, is only looping once

Question

I have a script that uses a for loop to read and clean a csv file then the result will save it as a new csv file. The loop for reading and cleaning works fine for all my csv files, until it reaches the 'to_csv' function. It appears it only saves the first csv file but not all.

Here is my script

 files_directory = 'C:/Users/Downloads/data/raw_data' raw_files = os.listdir(files_directory) csv_files = [] def clean_df(csv_files): for files in raw_files: csv_files.append('{}/{}'.format(files_directory,files)) for file in csv_files: df = pd.read_csv(file, parse_dates=True) ### Clean leap years and create just one colum with all data df = df.dropna(axis=0) #remove row with feb 29 df1 = df.drop(df.columns[[0,1]], axis = 1) #remove month and day column data = pd.Series(df1.values.ravel('A')) ##Create years dataframe year=list(df1) a = [np.repeat(yr, 366) for yr in year] df3= pd.DataFrame(a) years = pd.Series(df3.values.ravel('C')) ### Create dataframe with D/Y Dataframe months = df.drop(df.columns[[2,3,4,5,6,7,8,9,10,11,12,13,14]], axis = 1) months = pd.concat([months]*13, ignore_index=True) ### Create dataframe with M/D/Y timestep = pd.concat(([months, years]), axis=1, join='inner') timestep.columns = ['Month', 'Day', 'Year'] nat = pd.concat([timestep, data], axis=1, join='inner') print(nat) ## Save it to csv only_file_name = csv_files[0].split("/")[-1][0:-4] nat.to_csv('{}/{}_new.csv'.format(files_directory, only_file_name), index=False, mode='w') #if mode is a then it will copy paste below return csv_files clean_df(csv_files) 

Answer 1

Here:

only_file_name = csv_files[0].split("/")[-1][0:-4]

You are always using a modified version of the first file name at each iteration of the loop. So every time, you write over the same file. It seems like instead you should use:

only_file_name = file.split("/")[-1][0:-4]

(I would also avoid using file as a variable name, since that is a builtin in Python 2.)