使用python pandas将csv文件中的多行合并为一行

Question

I have a csv file with multiple rows as stated below

Id  Name  Marks1 Marks2 Marks3 Marks4 Marks5
1   ABC   10     NAN    NAN    NAN    NAN
2   BCD   15     NAN    NAN    NAN    NAN
3   CDE   17     NAN    NAN    NAN    NAN
1   ABC   NAN    18     NAN    17     NAN
2   BCD   NAN    10     NAN    15     NAN
1   ABC   NAN    NAN    16     NAN    NAN
3   CDE   NAN    NAN    19     NAN    NAN

I want to merge the rows having the same id and name into a single row using pandas in python. The output should be :

Id  Name  Marks1 Marks2 Marks3 Marks4 Marks5
1   ABC   10     18     16     17     NAN
2   BCD   15     10     NAN    15     NAN
3   CDE   17     NAN    19     NAN    NAN

Answer 1

IIUC, DataFrame.groupby + as_index=False with GroupBy.first to eliminate NaN .

#df = df.replace('NAN',np.nan) #If necessary
df.groupby(['Id','Name'],as_index=False).first()

if you think it could have a pair Id Name with non-null values ​​in some column you could use GroupBy.apply with Series.ffill and Series.bfill + DataFrame.drop_duplicates to keep all the information.

df.groupby(['Id','Name']).apply(lambda x: x.ffill().bfill()).drop_duplicates()

Output

   Id Name Marks1 Marks2 Marks3 Marks4  Marks5
0   1  ABC     10     18     16     17     NaN
1   2  BCD     15     10    NaN     15     NaN
2   3  CDE     17    NaN     19    NaN     NaN

Answer 2

Hacky answer:

pd.groupby(“Name”).mean().reset_index()

This will only work if for each column there is only one valid value for each Name.