How do use python to iterate through a directory and delete specific columns from all csvs?

Question

I have a directory with several csvs.

files = glob('C:/Users/jj/Desktop/Bulk_Wav/*.csv')

Each csv has the same below columns. Reprex below-

yes no maybe ofcourse
1   2  3     4

I want my script to iterate through all csvs in the folder and delete the columns maybe and ofcourse.

Answer 1

Do you mean by:

files = glob('C:/Users/jj/Desktop/Bulk_Wav/*.csv')
for filename in files:
    df = pd.read_csv(filename)
    df = df.drop(['maybe ', 'ofcourse'], axis=1)
    df.to_csv(filename)

This code will remove the maybe and ofcourse columns and save it back to the csv.

Answer 2

If glob provides you with file paths, you can do the following with pandas :

import pandas as pd

files = glob('C:/Users/jj/Desktop/Bulk_Wav/*.csv')
drop = ['maybe ', 'ofcourse']

for file in files:
    df = pd.read_csv(file)
    for col in drop:
        if col in df:
            df = df.drop(col, axis=1)
    df.to_csv(file)

Alternatively if you want a cleaner way to not get KeyError s from drop you can do this:

import pandas as pd

files = glob('C:/Users/jj/Desktop/Bulk_Wav/*.csv')
drop = ['maybe ', 'ofcourse']

for file in files:
    df = pd.read_csv(file)
    df = df.drop([c for c in drop if c in df], axis=1)
    df.to_csv(file)

Answer 3

You can use panda to read csv file to a dataframe then use drop() to drop specific columns. something like below:

df = pd.read_csv(csv_filename)
df.drop(['maybe', 'ofcourse'], axis=1)

Answer 4

import pandas as pd
from glob import glob

files = glob(r'C:/Users/jj/Desktop/Bulk_Wav/*.csv')
for filename in files:
    df = pd.read_csv(filename, sep='\t')
    df.drop(['maybe', 'ofcourse'], axis=1, inplace=True)
    df.to_csv(filename, sep='\t', index=False)

If the files look exactly like what you have there, then maybe something like this