pivot 列名称和索引 pandas df 到列本身的有效方法，对应的值作为第三列？

Question

I have a very large dataset in python that came from a net CDF file. The column names are latitudes, and the indexes are longitudes. For each index/column in the database there is az value I am interested in. I want a new dataframe with the columns ['Latitude','Longitude','Z'] . I was able to come up with one solutions using itertools, but I my dataframe dimensions are (7200,14400) giving me 103,680,000 values to iterate over. Is there a more efficient way to do this. I provide here a sample input and output to make testing easy. Is there a pivot function in pandas or another efficient way to solve this problem?

#import libraries
import numpy as np
import pandas as pd
import itertools

#Create Sample Data
columns=['a','b','c']
rows=['1','2','3']
d_base=np.array([0.1,0.2,0.3])
data=np.tile(d_base,(3,1))

#create df
df=pd.DataFrame(data,columns=columns,index=rows)

df


Out[]
     a    b    c
1  0.1  0.2  0.3
2  0.1  0.2  0.3
3  0.1  0.2  0.3

This is the solution that works but is slow.

#iterate all combinations of columns and rows
col_index_pairs=list(itertools.product(columns, rows))

desired_output=pd.DataFrame()

#lookup the value of each possible pair in the original dataframe and put it into a new one.
for item in col_index_pairs:
    desired_output[item]=[item[0],item[1],df.loc[item[1],item[0]]]
    
desired_output=desired_output.T
desired_output.columns=['Latitude','Longitude','Z']
desired_output

Out[]: 
       Latitude Longitude    Z
       a         1           0.1
       a         2           0.1
       a         3           0.1
       b         1           0.2
       b         2           0.2
       b         3           0.2
       c         1           0.3
       c         2           0.3
       c         3           0.3
    

Answer 1

You may check with melt

s = df.reset_index().melt('index')
Out[18]: 
   index variable  value
0      1        a    0.1
1      2        a    0.1
2      3        a    0.1
3      1        b    0.2
4      2        b    0.2
5      3        b    0.2
6      1        c    0.3
7      2        c    0.3
8      3        c    0.3