Merging and subtracting DataFrame columns in pandas?

Question

I have a pandas DataFrame, something like:

col1  col2 col3 col5
NaN    1    2    8
2     NaN   4    8
4     NaN   4    8

I want to do two things:

1) Merge columns 1 and 2:

newcol1 col3 col5
1       2    8
2       4    8
4       4    8

I have tried using .concat, but that just concatenates the rows. Doesn't seem like I can use standard + operators with NaN values.

2) Subtract column 5 from new column 1 and column 3, so I end up with:

newcol1    col3
-7         -6
-6         -4
-4         -4

Tried doing it this way:

dataframe[['newcol1', 'col2']] - dataframe['col5']

and

dataframe[['newcol1', 'col2']].subtract(dataframe['col5'])

but neither works.

Answer 1

To get the new column, you could use fillna (or combine_first ):

df['newcol1'] = df.col1.fillna(df.col2)

Then for the subtraction, use sub and specify axis=0 since we want to consider the row indices when matching up labels (not the column indices as is the default):

>>> df[['newcol1', 'col3']].sub(df['col5'], axis=0)
   newcol1  col3
0       -7    -6
1       -6    -4
2       -4    -4

Answer 2

Here's one approach.

You could create newcol1 by sum(axis=1)

In [256]: df['newcol1'] = df[['col1', 'col2']].sum(axis=1)

In [257]: df
Out[257]:
   col1  col2  col3  col5  newcol1
0   NaN     1     2     8        1
1     2   NaN     4     8        2
2     4   NaN     4     8        4

Then use df.sub() on axis=0

In [258]: df[['newcol1', 'col3']].sub(df['col5'], axis=0)
Out[258]:
   newcol1  col3
0       -7    -6
1       -6    -4
2       -4    -4

Answer 3

In [58]:

df['newcol'] = df[['col1','col2']].sum(axis=1) - df['col5']
df['col3'] = df['col3'] - df['col5']
df
Out[58]:
   col1  col2  col3  col5  newcol
0   NaN     1    -6     8      -7
1     2   NaN    -4     8      -6
2     4   NaN    -4     8      -4

You can then drop col1 and col2:

In [59]:

df = df.drop(['col1','col2'],axis=1)
df
Out[59]:
   col3  col5  newcol
0    -6     8      -7
1    -4     8      -6
2    -4     8      -4