将列添加到几个文件，合并到一个文件中并对其进行排序

Question

I am new to the Python world, and I am fighting with basic operations. I have a huge amount of files named inlet_10_00_00.csv , inlet_10_00_01.csv , inlet_10_00_02.csv , etc. corresponding to different time inlet_Hour_minute_sec. The content looks like:

x, y, z, temperature, pressure, u, v, w, k , omega

0, 0, 0, 295, 100001, ...

0, 1, 1, 296, 100002, ...

...

I would like to do the next 2 operations:

1/ Add a t column to each file with value 0 for inlet_10_00_00 , value 1 for inlet_10_00_01 , etc. Adding a column t for a single file was quite easy with cols tools. But how can I add it to all files?

data=pd.read_csv('C:/myPath/inlet_10_00_00.csv',sep=',',skiprows=7)
data["t (s)"]="0"
cols=list(data.columns.values)
cols=  cols[0:3]+ cols[-1:] + cols[3:10]
cols
newdata=data[cols]
newdata.to_csv('out.csv', index=False)

2/ I want to combine all these files in one file and sort it by the t variable. I should end up with such format: x, y, z, t, temperature,pressure, u, v, w, k , omega

0, 0, 0, 0, 295,100001, ....   this part comes from inlet_10_00_00.csv

0, 1, 1, 0, 296,100002, ...

0, 0, 0, 1, 292,100008, ...    this part comes from inlet_10_00_01.csv

0, 1, 1, 1, 294,100012, ...

Any ideas, how to proceed?

Answer 1

The following steps should help you achieve the expected result.

Explanation:

1. Step 1: Here we'll list the file names in the current directory. Then, we'll sort the file names to ensure files are looped in desired sequence.

2. Step 2: Here we'll iterate through the list we created in step 1, read a file, add a column and save the data frame with the same name.

3. Step 3: Finally, using pd.concat we join all the files in a data frame and sort the data frame by t .

# Step 1
path = 'C:/myPath/'
files = sorted(os.listdir(path)) # sorting will ensure _001, 002, 003 will bein sequence


## Step 2 
for ix, file in enumerate(files,1):
    df = pd.read_csv(file, skiprows=7) ## assuming you want to skip first 7 rows in every file
    df['t'] = ix
    df.to_csv(os.path.join(path, file), index=False)

## Step 3
master_df = pd.concat([pd.read_csv(file) for file in files])
master_df = master_df.sort_values('t').reset_index()