Python，pandas循环仅在最后一项有效

Question

edit : my apology : I find the issue, I am using Rstudio to run the code, which mess up something, I just try it from the console and it is working fine

I know, I am doing something silly but I can't figure out what I am doing wrong, I built this script that read a zip file, do some transformation and write a final csv, but for some reason, only the last file got written the script is fully reproducible, the source files are located in the link below if you want to try and debug it.

files = os.listdir(os.curdir)
files = [i for i in files if i.endswith('.zip')]
print(files)
for x in files:
     path_file = os.path.join(curDir ,x)
     print(path_file)
     source = pd.read_csv(path_file,
     skiprows=1,
     usecols=["DISPATCH","1" ,"SETTLEMENTDATE", "RUNNO","INTERVENTION","CASESUBTYPE","SOLUTIONSTATUS","NONPHYSICALLOSSES"],
     dtype=str)

     source.rename(columns={'1': 'version'}, inplace=True)
     source.query('version=="2"')

      ################ Extract UNIT, SETTLEMENTDATE,DUID,INITIALMW AND EXPORT TO CSV
     df_unit=source
     df_unit=df_unit.query('DISPATCH=="DUNIT" or DISPATCH=="TUNIT"')
     #Make first row a header
     df_unit.columns = df_unit.iloc[0]
     df_unit = df_unit[1:]
     #create a conditional column
     df_unit.loc[df_unit['DUNIT'] == 'TUNIT', 'INITIALMW1'] = df_unit['INTERVENTION']
     df_unit.loc[df_unit['DUNIT'] == 'DUNIT', 'INITIALMW1'] = df_unit['INITIALMW']
     df_unit.drop(columns=['RUNNO','2','INTERVENTION','INITIALMW','DISPATCHMODE'],inplace=True)
     df_unit.rename(columns={'INITIALMW1': 'INITIALMW','DUNIT': 'UNIT'}, inplace=True)
     df_unit=df_unit.query('SETTLEMENTDATE!="SETTLEMENTDATE" and INITIALMW !="0"')
     df_unit["INITIALMW"] = pd.to_numeric(df_unit["INITIALMW"])
     df_unit['SETTLEMENTDATE']=pd.to_datetime(df_unit['SETTLEMENTDATE'])
     df_unit.head()
     df_unit.to_csv(x.rsplit('.', 1)[0] + '.csv',float_format="%.4f",
     index=False,date_format='%Y-%m-%dT%H:%M:%S.%fZ',compression='gzip')
     print(path_file) 

EDIT: I added list files:

['PUBLIC_DAILY_201906040000_20190605040502.zip', 'PUBLIC_DAILY_201906050000_20190606040501.zip', 'PUBLIC_DAILY_201907140000_20190715040502.zip']

The files are downloaded from here .

Answer 1

The below code is working for me. Probably you only have one downloaded zip file in the current directory, or actually you are invoking jupyter notebook or python script from the incorrect directory . You can print os.getcwd() . Otherwise, there is nothing wrong with the code. All the zip files has to be in the same directory from where you this code is getting run either through python script or jupyter notebook.

files = os.listdir(os.getcwd())
files = [i for i in files if i.endswith('.zip')]
print(files)
for x in files:
    path_file = os.path.join(os.getcwd() ,x)
    print(path_file)
    ...
    ... 
    df_unit.to_csv(x.rsplit('.', 1)[0] + '.csv',float_format="%.4f",
    index=False,date_format='%Y-%m-%dT%H:%M:%S.%fZ',compression='gzip')
    print(path_file)