Concatenating Time Column Headers with Corresponding dates in rows in a DataFrame Pandas

Question

SO I basically have this data set where I have the time of the day in intervals of 15 minutes(12:15, 12:30, 12:45 etc.) as my column headers. Each row has a date from 2010 to 2020 and what I want to do is basically match the time(column headers) with the rows.

print (df)
        Date     0:15     0:30     0:45     1:00     1:15     1:30     1:45  
0  01May2010  2.98298  2.30478  2.57654  2.44110  2.25174  2.20100  2.15370   
1  02May2010  2.31606  2.20325  2.12952  2.09236  2.04150  2.08978  1.01500   
2  03May2010  2.07710  2.13000  2.07249  2.05315  2.08925  1.94481  1.85551   

The following is how I want the rows to look like

01-May-2010 0:15
01-May-2010 0:30
01-May-2010 0:45
... till 
01-May-2010 11:45
01-May-2010 12:00
02-May-2010 12:15
etc etc

So essentially I just want 2 columns instead of 100 columns. One with the value and the other being date+time.

How can I do that? I know I need to use pandas but I'm really confused as to what to do here.

Answer 1

Use DataFrame.melt with to_datetime with joined columns with DataFrame.pop for use and remove column variable :

df = df.melt('Date', value_name='val')
df['Date'] = pd.to_datetime(df['Date'] + ' ' + df.pop('variable'), format='%d%b%Y %H:%M')

df = df.sort_values('Date', ignore_index=True)
print (df.head(10))
                 Date      val
0 2010-05-01 00:15:00  2.98298
1 2010-05-01 00:30:00  2.30478
2 2010-05-01 00:45:00  2.57654
3 2010-05-01 01:00:00  2.44110
4 2010-05-01 01:15:00  2.25174
5 2010-05-01 01:30:00  2.20100
6 2010-05-01 01:45:00  2.15370
7 2010-05-02 00:15:00  2.31606
8 2010-05-02 00:30:00  2.20325
9 2010-05-02 00:45:00  2.12952

Solution with no convert to datetimes with DataFrame.set_index and DataFrame.stack :

df = df.set_index('Date').stack()
df.index = df.index.map(' '.join)
df = df.rename_axis('date').reset_index(name='val')

print (df.head(10))
             date      val
0  01May2010 0:15  2.98298
1  01May2010 0:30  2.30478
2  01May2010 0:45  2.57654
3  01May2010 1:00  2.44110
4  01May2010 1:15  2.25174
5  01May2010 1:30  2.20100
6  01May2010 1:45  2.15370
7  02May2010 0:15  2.31606
8  02May2010 0:30  2.20325
9  02May2010 0:45  2.12952