删除 pandas dataframe 中的每 n 列

Question

i have a pandas dataframe where the columns are named like:

0,1,2,3,4,.....,n

i would like to drop every 3rd column so that i get a new dataframe where i would have the columns like:

0,1,3,4,6,7,9,.....,n

I have tried like this:

shape = df.shape[1]
for i in range(2,shape,3):
    df = df.drop(df.columns[i], axis=1) 

but i get an error saying index is out of bound and i assume this happens because the shape of the dataframe changes when i am dropping the columns. if i just don't store the output of the "for" loop, then the code works but i don't get my new dataframe.

How do i solve this? Thanks

Answer 1

Here is solution with inverted logic - select all columns with removed each 3rd column.

You can filter values by compare added 1 to helper array, with 3 modulo compare for not equal 0 and pass to DataFrame.loc :

df = pd.DataFrame({
        'A':list('abcdef'),
         'B':[4,5,4,5,5,4],
         'C':[7,8,9,4,2,3],
         'D':[1,3,5,7,1,0],
         'E':[5,3,6,9,2,4],
         'F':list('aaabbb')
})

df = df.loc[:, (np.arange(len(df.columns)) + 1) % 3 != 0]
print (df)
   A  B  D  E
0  a  4  1  5
1  b  5  3  3
2  c  4  5  6
3  d  5  7  9
4  e  5  1  2
5  f  4  0  4

Answer 2

The issue with code is , each time you drop a column in your loop, you end up with a different set of columns because you overwrite the df back after each iteration. When you try to drop the next 3rd column of THAT new set of columns, you not only drop the wrong one, you end up running out of columns eventually. That's why you get the error you are getting.

iter1 -> 0,1,3,4,5,6,7,8,9,10 ... n #first you drop 2 which is 3rd col
iter2 -> 0,1,3,4,5,7,8,9,10 ... n   #next you drop 6 which is 6th col (should be 5)
iter3 -> 0,1,3,4,5,7,8,9, ... n     #next you drop 10 which is 9th col (should be 8)

What you want to do is calculate the indexes beforehand and then remove them in one go.

---

You can simply just get the indexes of columns you want to remove with range and then drop those.

drop_idx = list(range(2,df.shape[1],3)) #Indexes to drop
df2 = df.drop(drop_idx, axis=1)         #Drop them at once over axis=1


print('old columns->', list(df.columns))
print('idx to drop->', drop_idx)
print('new columns->',list(df2.columns))

old columns-> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
idx to drop-> [2, 5, 8]
new columns-> [0, 1, 3, 4, 6, 7, 9]

---

Note: This works only because your columns names are same as indexes. If however, your column names are not like that, you will have to do an extra step of fetching the column names based on the index you want to drop.

drop_idx = list(range(2,df.shape[1],3))
drop_cols = [j for i,j in enumerate(df.columns) if i in drop_idx] #<--
df2 = df.drop(drop_idx, axis=1)

Answer 3

You can use list comprehension to filter columns:

df = df[[k for k in df.columns if (k + 1) % 3 != 0]]

If the names are different (eg strings) and you want to discard every 3rd column regardless of its name, then:

df = df[[k for i, k in enumerate(df.columns, 1) if i % 3 != 0]]