如何附加到 Pandas DataFrame 中的各个列

Question

So I want to add/append data to a specific pandas dataFrame column but without it causing NaN values in the remaining columns

Ie

DataFrame = pd.DataFrame(columns=["column1", "column2", "column3"])
for i in range():
    DataFrame = DataFrame.append({"column1":int(i)}, ignore_index=True)
    DataFrame = DataFrame.append({"column2":float(i*2)}, ignore_index=True)
    DataFrame = DataFrame.append({"column3":int(i*5)}, ignore_index=True)
print(DataFrame)

This will return:

   column1  column2  column3
0      0.0      NaN      NaN
1      NaN      0.0      NaN
2      NaN      NaN      0.0
3      1.0      NaN      NaN
4      NaN      2.0      NaN
5      NaN      NaN      5.0
6      2.0      NaN      NaN
7      NaN      4.0      NaN
8      NaN      NaN     10.0

What we want returned:

   column1  column2  column3
0      0.0      0.0      0.0
1      1.0      2.0      5.0
2      2.0      4.0     10.0

I know I can in this case use one .append for all the different columns. But I have some cases where the data to be appended will vary based on multiple conditions. Hence I'd like to know if it's possible to append to single columns in a dataframe without producing NaN values in the remaining columns. So that I can avoid writing hundreds of if else statements.

Or if someone has any good idea regarding how to 'collapse' the NaN values (removing the NaN values without removing the entire row so that if there is a NaN value at index 0 in column 3 and there is a integer 5 at index 1 in the same column the integer 5 gets moved up to index 0)

Happy to hear any ideas.

Answer 1

IIUC for your current example you can try this:

DataFrame[['column2','column3']]=DataFrame[['column2','column3']].bfill()

Output:

 column1  column2   column3
0   0.0     0.0     0.0
1   NaN     0.0     0.0
2   NaN     2.0     0.0
3   1.0     2.0     5.0
4   NaN     2.0     5.0
5   NaN     4.0     5.0
6   2.0     4.0     10.0
7   NaN     4.0     10.0
8   NaN     6.0     10.0
9   3.0     6.0     15.0
10  NaN     6.0     15.0
11  NaN     8.0     15.0
12  4.0     8.0     20.0
13  NaN     8.0     20.0
14  NaN     NaN     20.0

then remove the NaN :

DataFrame.dropna(inplace=True)

Outpt:

 column1  column2   column3
0   0.0     0.0     0.0
3   1.0     2.0     5.0
6   2.0     4.0     10.0
9   3.0     6.0     15.0
12  4.0     8.0     20.0