Merging two columns in a single pandas data frame

Question

I have the following data frame:

df = pd.DataFrame({'id': [0.1, 0.2, 0.3, 0.4],'A': [1,2, np.NaN, np.NaN], 'A1': [np.NaN, np.NaN, 3,4]})

I'm looking to merge A1 into A (drop A1) that should result into:

df = pd.DataFrame({'id': [0.1, 0.2, 0.3, 0.4, 0.1, 0.2, 0.3, 0.4],'A': [1,2, np.NaN, np.NaN, np.NaN, np.NaN, 3,4]})

Appreciate any help.

Answer 1

You can concat the two columns:

pd.concat([df[['id', 'A']], df[['id', 'A1']].rename(columns={'A1': 'A'})],
          ignore_index=True)

Or melt and rename / drop :

df.melt('id').rename({'value': 'A'}).drop(columns='variable')

output:

    id    A
0  0.1  1.0
1  0.2  2.0
2  0.3  NaN
3  0.4  NaN
4  0.1  NaN
5  0.2  NaN
6  0.3  3.0
7  0.4  4.0

Answer 2

one option is with pivot_longer from pyjanitor ; the columns have a pattern, they both start with A , we can use that to our advantage in the reshaping:

# pip install pyjanitor
import janitor
import pandas as pd
df.pivot_longer('id', names_to = 'A', names_pattern=['A'])

    id    A
0  0.1  1.0
1  0.2  2.0
2  0.3  NaN
3  0.4  NaN
4  0.1  NaN
5  0.2  NaN
6  0.3  3.0
7  0.4  4.0