如何在 Pandas 中将多列转换为单独的行/值？

Question

I'm sure this question has been answered, but unfortunately I don't know what to call this operation, and so my search is failing me. It is almost like a reverse pivot table.

Lets say I have the following payroll data:

data = [
    {'employee': 1, 'date': '2020-01-04', 'reg': 8, 'ot': 0, 'dt': 0},
    {'employee': 1, 'date': '2020-01-05', 'reg': 4, 'ot': 4, 'dt': 0},
    {'employee': 1, 'date': '2020-01-06', 'reg': 0, 'ot': 0, 'dt': 4},
    {'employee': 2, 'date': '2020-01-04', 'reg': 6, 'ot': 2, 'dt': 0},
    {'employee': 2, 'date': '2020-01-05', 'reg': 3, 'ot': 5, 'dt': 0},
    {'employee': 2, 'date': '2020-01-06', 'reg': 0, 'ot': 4, 'dt': 0},
]

data_df = pd.DataFrame(data)

What I need to do is break each rate ('reg', 'ot', and 'dt') for each employee/date, out into its own row that has a column for the rate label, and a column for the number of hours, keeping the other non-rate-based columns. Additionally, I dont want a row for any rates where the value is zero. For the data above, I am looking to get:

result = [
    {'employee': 1, 'date': '2020-01-04', 'rate': 'reg', 'hours': 8},
    {'employee': 1, 'date': '2020-01-05', 'rate': 'reg', 'hours': 4},
    {'employee': 1, 'date': '2020-01-05', 'rate': 'ot', 'hours': 4},
    {'employee': 1, 'date': '2020-01-06', 'rate': 'dt', 'hours': 4},
    {'employee': 2, 'date': '2020-01-04', 'rate': 'reg', 'hours': 6},
    {'employee': 2, 'date': '2020-01-04', 'rate': 'ot', 'hours': 2},
    {'employee': 2, 'date': '2020-01-05', 'rate': 'reg', 'hours': 3},
    {'employee': 2, 'date': '2020-01-05', 'rate': 'ot', 'hours': 5},
    {'employee': 2, 'date': '2020-01-06', 'rate': 'ot', 'hours': 4},
]

result_df = pd.DataFrame(result)

Any thoughts on how to accomplish this would be greatly appreciated!

Answer 1

Try using melt :

(data_df.melt(['employee','date'], 
             var_name='rate', 
             value_name='hours')
        .query('hours != 0'))

Output:

    employee        date rate  hours
0          1  2020-01-04  reg      8
1          1  2020-01-05  reg      4
3          2  2020-01-04  reg      6
4          2  2020-01-05  reg      3
7          1  2020-01-05   ot      4
9          2  2020-01-04   ot      2
10         2  2020-01-05   ot      5
11         2  2020-01-06   ot      4
14         1  2020-01-06   dt      4

Answer 2

This should do the trick:

data_df=data_df.set_index(["employee", "date"]).stack().reset_index().rename(columns={"level_2": "rate", 0: "hours"})

Output:

    employee        date rate  hours
0          1  2020-01-04  reg      8
1          1  2020-01-04   ot      0
2          1  2020-01-04   dt      0
3          1  2020-01-05  reg      4
4          1  2020-01-05   ot      4
5          1  2020-01-05   dt      0
6          1  2020-01-06  reg      0
7          1  2020-01-06   ot      0
8          1  2020-01-06   dt      4
9          2  2020-01-04  reg      6
10         2  2020-01-04   ot      2
11         2  2020-01-04   dt      0
12         2  2020-01-05  reg      3
13         2  2020-01-05   ot      5
14         2  2020-01-05   dt      0
15         2  2020-01-06  reg      0
16         2  2020-01-06   ot      4
17         2  2020-01-06   dt      0