从 csv 文件的集合中提取一列并用所述数据构建一个新表

Question

I'm a newbie when it comes to Python with a bit more experience in MATLAB. I'm currently trying to write a script that basically loops through a folder to pick up all the.csv files, extract column 14 from csv file 1 and adding it to column 1 of the new table, extract column 14 from csv file 2 and adding it to column 2 of the new table, to build up a table of column 14 from all csvfiles in the folder. I'd ideally like to have the headers of the new table to show the respective filename that said column 14 has been extracted from.

I've considered that Python is base0 so I've double checked that it reads the desired column, but as my code stands, i can only get it to print all the files' 14th columns in the one array and I'm not sure how to split it up to put it into a table. Perhaps via dataframe, although I'm not entirely sure how they work. Any help would be greatly appreciated!

Code attached below:

import os
import sys
import csv
pathName = "D:/GLaDOS-CAMPUS/data/TestData-AB/"
numFiles = []
fileNames = os.listdir(pathName)
for fileNames in fileNames:
    if fileNames.endswith(".csv"):
        numFiles.append(fileNames)
        print(numFiles)
for i in numFiles:
    file = open(os.path.join(pathName, i), "rU")
    reader = csv.reader(file, delimiter=',')
    for column in reader:
         print(column[13])

Answer 1

Finding files.

I'm not sure if your way of finding files is right or not. Since I do not have a folder with csv files. But I can say it is way better to use glob for getting list of files:

from glob import glob
files = glob("/Path/To/Files/*.csv")

This will return all csv files.

Reading CSV files

Now we need to find a way to read all files and get 13th column. I don't know if it is an overkill but I prefer to use pandas and numpy to get 13th column.

To read a column of a csv file using pandas one can use:

pd.read_csv(file, usecols=[COL])

Now we can loop over files and get 13th columns:

columns = [pd.read_csv(file, usecols=[2]).values[:, 0] for file in files]

Notice we converted all values to numpy arrays.

Merging all columns

In columns we have our each column as an element of a list. So it is technical rows. Not columns. Now we should get the transpose of the array so it will become columns:

pd.DataFrame(np.transpose(columns))

The code

The whole code would look like:

from glob import glob
import pandas as pd
import numpy as np

files = glob("/Path/To/Files/*.csv")
columns = [pd.read_csv(file, usecols=[2]).values[:, 0] for file in files]
print(pd.DataFrame(np.transpose(columns)))