合并行并用 pandas 替换 NaN 值

Question

I am trying to merge rows with each other to get one row containing all the values that are present. Currently the df look like this: dataframe

What i want is something like:

| index | scan .. | snel. | kool .. | note ..  |
| ----- | ------- | ----- | ------- | -------  |
| 0     | 7,8     | 4,0   | 20.0    | Fiasp, ..|

I can get that output in the code example below but it just seems really messy.

I tried to use groupby, agg, sum, max, and all those do is that it removes columns and looks like this: df2.groupby('Tijdstempel apparaat').max().reset_index()

I tried filling the row with the values of the previous rows, and then drop the rows that dont contain every value. But this seems like a long work around and really messy.

df2 = df2.loc[df['Tijdstempel apparaat'] == '20-01-2023 13:24']
df2 = df2.reset_index()
del df2['index']
df2['Snelwerkende insuline (eenheden)'].fillna(method='pad', inplace=True)
df2['Koolhydraten (gram)'].fillna(method='pad', inplace=True)
df2['Notities'].fillna(method='pad', inplace=True)
df2['Scan Glucose mmol/l'].fillna(method='pad', inplace=True)
print(df2)
# df2.loc[df2[0,'Snelwerkende insuline (eenheden)']] = df2.loc[df2[1, 'Snelwerkende insuline (eenheden)']]
df2.drop([0, 1, 2])

Output:

When i have to do this for the entire data.csv (whenever a time stamp like "20-01-2023 13:24" is found multiple times) i am worried it wil be really slow and time consuming.

Answer 1

sample data as your data

df = pd.DataFrame(data={
    "times":["date1","date1","date1","date1","date1"],
    "type":[1,2,3,4,5],
    "key1":[1,None,None,None,None],
    "key2":[None,"2",None,None,None],
    "key3":[None,None,3,None,None],
    "key4":[None,None,None,"val",None],
    "key5":[None,None,None,None,5],
})

solution

melt = df.melt(id_vars="times",
        value_vars=df.columns[1:],)

melt = melt.dropna()

pivot = melt.pivot_table(values="value", index="times", columns="variable", aggfunc=lambda x: x)

change type column location

index = list(pivot.columns).index("type")
pivot = pd.concat([pivot.iloc[:,index:], pivot.iloc[:,:index]], axis=1)