删除值显示在熊猫所有列中的行

Question

Here is a very simple dataframe:

df = pd.DataFrame({'col1' :[1,2,3], 
                   'col2' :[1,3,3] })

I'm trying to remove rows where there are duplicate values (eg, row 3)

This doesn't work,

df = df[(df.col1 != 3 & df.col2 != 3)]

and the documentation is pretty clear about why, which makes sense.

But I still don't know how to delete that row.

Does anyone have any ideas? Thanks. Monica.

Answer 1

If I understand your question correctly, I think you were close.

Starting from your data:

In [20]: df
Out[20]: 
   col1  col2
0     1     1
1     2     3
2     3     3

And doing this:

In [21]: df = df[df['col1'] != df['col2']]

Returns:

In [22]: df
Out[22]: 
   col1  col2
1     2     3

Answer 2

What about:

In [43]: df = pd.DataFrame({'col1' :[1,2,3], 
                   'col2' :[1,3,3] })

In [44]: df[df.max(axis=1) != df.min(axis=1)]
Out[44]: 
   col1  col2
1     2     3

[1 rows x 2 columns]

We want to remove rows whose values show up in all columns, or in other words the values are equal => their minimums and maximums are equal. This is method works on a DataFrame with any number of columns. If we apply the above, we remove rows 0 and 2.

Answer 3

Any row with all the same values with have zero as the standard deviation. One way to filter them is as follows:

import pandas as pd
import numpy as np

df = pd.DataFrame({'col1' :[1, 2, 3, np.nan], 
                   'col2' :[1, 3, 3, np.nan]}

>>> df.loc[df.std(axis=1, skipna=False) > 0]
   col1  col2
    1     2