如何将同一类别的多行合并为大熊猫？

Question

I'm trying to get from table 1 to table 2 from the image but I can't seem to get it right. I tried pivot table to change col A - D from rows to cols. Then I try groupby but it doesn't give me one row but messes up my dataframe instead.

Answer 1

You can fill the null values with the value in the column and drop duplicates:

with :

df = pd.DataFrame([["A", pd.np.nan, pd.np.nan, "Y", "Z"],
              [pd.np.nan, "B", pd.np.nan, "Y", "Z"],
              [pd.np.nan,pd.np.nan, "C", "Y", "Z"]], columns=list("ABCDE"))
df
     A    B    C  D  E
0    A  NaN  NaN  Y  Z
1  NaN    B  NaN  Y  Z
2  NaN  NaN    C  Y  Z

df.ffill().bfill().drop_duplicates()
   A  B  C  D  E
0  A  B  C  Y  Z

df.ffill().bfill() gives:

   A  B  C  D  E
0  A  B  C  Y  Z
1  A  B  C  Y  Z
2  A  B  C  Y  Z

As per your comment, you could define a function that fill the missing value of the first row by the unique value that lies somewhere else in the same column.

def fillna_uniq(df, col):
    if isinstance(col, list):
        for c in col:
            df.loc[df.index[0], c] = df[c].dropna().iloc[0]
    else:
        df.loc[df.index[0], col] = df[col].dropna().iloc[0]
    return df.iloc[[0]]

You could then do:

fillna_uniq(df.copy(), ["B", "C", "D"])
       A  B   C     D       E     F
0  Hello  I  am  lost  Pandas  Data

It is a bit faster I think. You can modify your df inplace by passing directly the dataframe, not a copy.

HTH

Answer 2

One way you can do this is using apply and dropna :

Assuming those blanks in your table above are really nulls:

df = pd.DataFrame({'A':['Hello',np.nan,np.nan,np.nan],'B':[np.nan,'I',np.nan,np.nan],
                   'C':[np.nan,np.nan,'am',np.nan],
                  'D':[np.nan,np.nan,np.nan,'lost'],
                  'E':['Pandas']*4,
                  'F':['Data']*4})

print(df)
       A    B    C     D       E     F
0  Hello  NaN  NaN   NaN  Pandas  Data
1    NaN    I  NaN   NaN  Pandas  Data
2    NaN  NaN   am   NaN  Pandas  Data
3    NaN  NaN  NaN  lost  Pandas  Data

Using apply , you can apply the lambda function to each column of the dataframe, first dropping null values then find the max:

df.apply(lambda x: x.dropna().max()).to_frame().T

       A  B   C     D       E     F
0  Hello  I  am  lost  Pandas  Data

Or if your blanks are really empty strings, then you can do this:

df1 = df.replace(np.nan,'')
df1
       A  B   C     D       E     F
0  Hello               Pandas  Data
1         I            Pandas  Data
2            am        Pandas  Data
3                lost  Pandas  Data

df1.apply(lambda x: x[x!=''].max()).to_frame().T

       A  B   C     D       E     F
0  Hello  I  am  lost  Pandas  Data