在熊猫数据框中按组分组数据

Question

I need to subset a dataframe using groups and three conditional rules. If within a group all values in the Value column are none, I need to retain the first row for that group. If within a group all values in the Value column are not none, I need to retain all the values. If within a group some of the values in the Value column are none and others not none, I need to drop all rows where there is a none. Columns Region and ID together define a unique group within the dataframe.

My first approach was to separate the dataframe into two chunks. The first chunk is rows where for a group there are all nulls. The second chunk is everything else. For the chunk of data where rows for a group contained all nulls, I would create a rownumber using a cumulative count of rows by group and query rows where the cumulative count = 1. For the second chunk, I would drop all rows where Value is null. Then I would append the dataframes.

Sample source dataframe

dfInput = pd.DataFrame({
'Region':     [1, 1, 2, 2, 2, 2, 2],
'ID':     ['A', 'A', 'B', 'B', 'B', 'A', 'A'],
'Value':[0, 1, 1, None, 2, None, None],
})

Desired output dataframe:

dfOutput = pd.DataFrame({
'Region':     [1, 1, 2, 2, 2],
'ID':     ['A', 'A', 'B', 'B', 'A'],
'Value':[0, 1, 1, 2, None],
})

Answer 1

Just follow your logic and using groupby

dfInput.groupby(['Region','ID']).Value.apply(lambda x : x.head(1) if x.isnull().all() else x.dropna()).\
        reset_index(level=[0,1]).sort_index()
Out[86]: 
   Region ID  Value
0       1  A    0.0
1       1  A    1.0
2       2  B    1.0
4       2  B    2.0
5       2  A    NaN