[英]Drop columns in Dataframe if more than 90% of the values in the column are 0's
First of all, next time please give an example dataset, not an image or copy of one.首先,下次请给出示例数据集,而不是图像或副本。 It's best to give a minimal example that reproduces your problem (it's also a good way to investigate your problem).最好给出一个重现问题的最小示例(这也是调查问题的好方法)。 This df, for example, will do the trick:例如,这个 df 可以解决问题:
df = pd.DataFrame.from_dict({
'a':[1,0,0,0,0,0,0,0,0,0,0],
'b':[1,1,1,0,1,0,0,0,0,0,0]})
Now, the previous answers help, but if you can avoid a loop, it's preferable.现在,先前的答案有所帮助,但如果您可以避免循环,则最好。 You can write something simpler and more concise that will do the trick:你可以写一些更简单、更简洁的东西来解决这个问题:
df.drop(columns=df.columns[df.eq(0).mean()>0.9])
Let's go through it step by step:让我们一步一步来:
The df.eq(0)
returns True
\\ False
in each cell. df.eq(0)
在每个单元格中返回True
\\ False
。
The .mean()
method treats True as 1 and False as 0, so comparing that mean to 0.9 is what you want. .mean()
方法将 True 视为 1,将 False 视为 0,因此将该均值与 0.9 进行比较就是您想要的。
Calling df.columns[...]
at these places will return only those where the >0.9
holds, and drop
just drops them.在这些地方调用df.columns[...]
将只返回那些>0.9
地方,而drop
只是丢弃它们。
The following should do the trick for you:以下应该为您解决问题:
row_count = df.shape[0]
columns_to_drop = []
for column, count in df.apply(lambda column: (column == 0).sum()).iteritems():
if count / row_count >= 0.9:
columns_to_drop.append(column)
df = df.drop(columns_to_drop, axis=1, inplace=True)
bad_col = []
for i, x in enumerate(df.columns):
if sorted(list(df[x].value_counts(normalize = True).values))[-1] >= 0.9 :
bad_col.append(x)
Explanation inline the code .解释内联代码。
#Suppose df is your DataFrame then execute the following code.
df_float=df.loc[:, df.dtypes == np.float64] #checks if the column contains numbers
for i in df_float.columns:
if ((len(df_float[i].loc[df_float[i]==0])/len(df_float))>0.9): #checking if 90% data is zero
df_float.drop(i,axis=1,inplace=True) #delete the column
#Your results are stored in df_float
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.