读取CSV文件导致列表索引超出范围

Question

I have a csv file with numerous columns and some columns have more rows than others. I am trying to read certain columns and extract those columns to a different csv file. However, I keep getting list index out of range because the columns I'm reading are shorter than the other columns in the csv file.

import csv
import os
import os.path

'Open csv directory to extract data from'
path = 'S://Raw Data/VehicleData/'               

'Get file names and print number of files in the directory'
files = [f for f in os.listdir(path)]
print(files)
print(len(files))

'Start loop to scan through files in directory'
for j in range(len(files)):

'determine name of file and the output name of the file'
name = files[j]
print(name)

'path to file to open'
fpath = os.path.join(path, files[j])

    if files[j].endswith(".csv"):
        nameout = 'VehicleN3STPData' + '.csv'
        with open (fpath, 'r', newline='') as csvfile:
            testcsv = csv.reader(csvfile,delimiter=',')
            with open (nameout,'a+',newline='') as outfile:
                writer = csv.writer(outfile)
                for row in testcsv:
                    my_row = []
                    my_row.append(row[59])
                    my_row.append(row[111])
                    my_row.append(row[120])
                    print(my_row)
                    writer.writerow(my_row)
                outfile.close()    
        csvfile.close()

Answer 1

You need a conditional statement to check for length or you need a try & except clause depending on your intended behavior.

Here's an example of using a conditional statement that preserves the column values as blanks if they don't exist.

Instead of:

my_row.append(row[59]) 
my_row.append(row[111]) 
my_row.append(row[120])

Do:

for col_num in [59, 111, 120]:
    my_row.append(row[col_num] if len(row) > col_num else '')

Answer 2

Thanks for your help, here is the final code that worked, removing the header with islice, using your conditional statement, and then removing the blank lines with if any(my_row) thanks!!

    if files[j].endswith(".csv"):
    nameout = 'VehicleN3STPData' + '.csv'
    with open (fpath, 'r', newline='') as csvfile:
        testcsv = csv.reader(csvfile,delimiter=',')
        with open (nameout,'a+',newline='') as outfile:
            writer = csv.writer(outfile)
            for row in islice(testcsv,3,None):
                    my_row = []
                    for col_num in [59,111,120]:
                        my_row.append(row[col_num] if len(row) > col_num else '')
                    if any(my_row):
                        writer.writerow(my_row)
            outfile.close()    
    csvfile.close()enter code here