How can I extract a column from dataframe and attach it to rows while keeping other columns intact

Question

How can I extract a column from pandas dataframe attach it to rows while keeping the other columns same.

This is my example dataset.

import pandas as pd
import numpy as np
df = pd.DataFrame({'ID':  np.arange(0,5),
                  'sample_1' : [5,6,7,8,9],
                  'sample_2' : [10,11,12,13,14],
                  'group_id' : ["A","B","C","D","E"]})

The output I'm looking for is:

df2 = pd.DataFrame({'ID':  [0, 1, 2, 3, 4, 0, 1, 2, 3, 4],
                  'sample_1' : [5,6,7,8,9,10,11,12,13,14],
                  'group_id' : ["A","B","C","D","E","A","B","C","D","E"]})

I have tried to slice the dataframe and concat using pd.concat but it was giving NaN values.

My original dataset is large.

Answer 1

You could do this using stack : Set the index to the columns you don't want to modify, call stack, sort by the "sample" column, then reset your index:

df.set_index(['ID','group_id']).stack().sort_values(0).reset_index([0,1]).reset_index(drop=True)

   ID group_id   0
0   0        A   5
1   1        B   6
2   2        C   7
3   3        D   8
4   4        E   9
5   0        A  10
6   1        B  11
7   2        C  12
8   3        D  13
9   4        E  14

Answer 2

Using pd.wide_to_long :

res = pd.wide_to_long(df, stubnames='sample_', i='ID', j='group_id')
res.index = res.index.droplevel(1)
res = res.rename(columns={'sample_': 'sample_1'}).reset_index()

print(res)

   ID group_id  sample_1
0   0        A         5
1   1        B         6
2   2        C         7
3   3        D         8
4   4        E         9
5   0        A        10
6   1        B        11
7   2        C        12
8   3        D        13
9   4        E        14

Answer 3

The function you are looking for is called melt

For example:

df2 = pd.melt(df, id_vars=['ID', 'group_id'], value_vars=['sample_1', 'sample_2'], value_name='sample_1')

df2 = df2.drop('variable', axis=1)