Python : How do you filter out columns from a dataset based on substring match in Column names

Question

df_train = pd.read_csv('../xyz.csv')
headers = df_train.columns

I want to filter out those columns in headers which have _pct in their substring.

Answer 1

Use df.filter

df = pd.DataFrame({'a':[1,2,3], 'b_pct':[1,2,3],'c_pct':[1,2,3],'d':[1]*3})

print(df.filter(items=[i for i in df.columns if '_pct' not in i]))

## or as jezrael suggested
# print(df[[i for i in df.columns if '_pct' not in i]])

Output:

   a  d                                                                                                                                                           
0  1  1                                                                                                                                                           
1  2  1                                                                                                                                                           
2  3  1 

Answer 2

Use:

#data from AkshayNevrekar answer
df = df.loc[:, ~df.columns.str.contains('_pct')]
print (df)

Filter solution is not trivial:

df = df.filter(regex=r'^(?!.*_pct).*$')

---

   a  d
0  1  1
1  2  1
2  3  1

---

Thank you, @IanS for another solutions:

df[df.columns.difference(df.filter(like='_pct').columns).tolist()]

df.drop(df.filter(like='_pct').columns, axis=1)

Answer 3

由于df.columns返回列名列表，您可以使用列表理解并使用简单条件构建新列表：

new_headers = [x for x in headers if '_pct' not in x]