[英]How to drop Pandas DataFrame rows with condition to keep specific column value
I know similar questions have been asked before, but they didn't quite seem to help with my issue so I decided to ask a new question.我知道以前有人问过类似的问题,但它们似乎对我的问题没有帮助,所以我决定提出一个新问题。
What I have are three separate DataFrames - let's call them a
, b
, and c
- that are merged into one large dataframe.我拥有的是三个独立的 DataFrames - 我们称它们为
a
、 b
和c
- 它们合并为一个大型数据帧。 In each of these three DataFrames, there may be duplicate pairs of column values that I want to drop, but the condition is that if the pair belongs to DataFrame c
, then I want to keep that pair.在这三个 DataFrame 中的每一个中,可能都有重复的列值对要删除,但条件是如果该对属于 DataFrame
c
,那么我想保留那对。 For example:例如:
>>> a.head()
unit value target
0 3 23 'a'
1 2 24 'd'
2 8 56 'e'
3 9 89 'p'
4 0 32 'q'
>>> b.head()
unit value target
0 3 34 'a'
1 2 36 'd'
2 8 23 'a'
3 9 89 'p'
4 0 48 'm'
>>> c.head()
unit value target
0 3 34 'a'
1 5 23 'a'
2 2 48 'm'
3 9 56 'e'
4 0 98 'z'
The particular columns that I'm looking to find duplicates in is ( value
, target
).我要在其中查找重复项的特定列是 (
value
, target
)。 As you can tell, there are a total of four different duplicate scenarios: ( a
, b
), ( b
, c
), ( a
, c
), ( a
, b
, c
).如您所知,共有四种不同的重复场景:(
a
, b
), ( b
, c
), ( a
, c
), ( a
, b
, c
)。 In the above example, the ( value
, target
) pairs that would occur for each scenario would be: ( 89
, 'p'
), ( 34
, 'a'
), ( 56
, 'e'
), and ( 23
, 'a'
), respectively.在上面的例子中,每个场景会出现的 (
value
, target
) 对是: ( 89
, 'p'
), ( 34
, 'a'
), ( 56
, 'e'
), and ( 23
, 'a'
),分别。
If the duplicate occurs in ( a
, b
) it's not a huge problem because I can just simply choose from one of them, but if it occurs in any of the other three scenarios, I want to choose the pair from c
and discard the duplicates from a
and/or b
.如果重复出现在 (
a
, b
) 中,这不是一个大问题,因为我可以简单地从其中一个中进行选择,但是如果它出现在其他三个场景中的任何一个中,我想从c
选择一对并丢弃重复项来自a
和/或b
。
The original idea that I had was to use the following code:我最初的想法是使用以下代码:
>>> df = pd.concat([a, b, c], axis=0)
>>> df.drop_duplicates(subset=['value', 'target'], keep='last', inplace=True)
Since we're adding c
to the end of the concatenated DataFrame df
, we're guaranteed to retain that value should it occur as a duplicate.由于我们将
c
添加到连接的 DataFrame df
的末尾,因此如果它作为重复出现,我们保证保留该值。 However, I was wondering if anyone knew of a way where if ( a
, b
) were to occur, we would select one by random and if c
is included then we always choose c
.但是,我想知道是否有人知道如果 (
a
, b
) 发生的方式,我们会随机选择一个,如果包含c
则我们总是选择c
。
Thanks in advance.提前致谢。
we can use sample
before we combine with c
我们可以在与
c
结合之前使用sample
a_b=pd.concat([a,b]).sample(n=len(a)+len(b))
new=pd.concat([a_b,c]).drop_duplicates(['value', 'target'], keep='last')
new
Out[11]:
unit value target
1 2 24 'd'
4 0 32 'q'
3 9 89 'p'
1 2 36 'd'
0 3 34 'a'
1 5 23 'a'
2 2 48 'm'
3 9 56 'e'
4 0 98 'z'
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.