Open a csv file, delete columns containing a word in their name, save in a new csv Pandas, Python

Question

I have a csv file. In this file there are a lot of columns named as:

value_number, species_name, color_name, unnamed:1 etc...

How can I implement a python script using pandas that open a csv file, search in its columns that have the word unnamed and then delete them and save the cleaned csv as a new csv with an updated name.

Here is the script:

import os, glob
import pandas as pd



path= "/home/CaptainSnake/Desktop/Test_class/class_data"

all_files = glob.glob(os.path.join(path, "Classification_testData_*.csv"))
df_from_each_file = (pd.read_csv(f, sep=',') for f in all_files)
df_merged   = pd.concat(df_from_each_file, ignore_index=False, sort=False)
df_merged.columns

df_merged.columns.str.match('Unnamed')
df_merged.loc[:, ~df_merged.columns.str.match('Unnamed')]
df_merged.to_csv( "Classification_testData_cleaned.csv")

This script in particular, does the merging of all csv with a particular name...then It should clear the new csv from unnamed: etc..

Answer 1

This will remove all the columns that starts with 'Unnamed':

filtered_cols = [i for i in df.columns if not i.startswith("Unnamed")]
df[filtered_cols].to_csv('filename.csv')