Sort pandas pivot table by sum of rows and columns

Question

I have (for example) this DataFrame:

 COLUMN1 COLUMN2  VALUE
0    0102    1020      1
1    0102    1220      8
2    0102    1210      2
3    0103    1020      1
4    0103    1210      3
5    0103    1222      8
6    0104    1020      3
7    0104    1120      2

(In reailty, it's ~9000 rows long.)

From this, I create the pivot table where indexes are COLUMN1, columns are COLUMN2, and the values are from VALUES, filled by 0 where NaN.

COLUMN2  1020  1120  1210  1220  1222
COLUMN1                              
0102        1     0     2     8     0
0103        1     0     3     0     8
0104        3     2     0     0     0

I have to sort this pivot by the grand total of rows, then by the grand total of columns. That would look like this:

COLUMN2  1220  1222  1020  1210  1120| (GT)
COLUMN1                              |     HIGHEST
0103        0     8     1     3     0| (12) |
0102        8     0     1     2     0| (11) |
0104        0     0     3     0     2| (5)  V
--------------------------------------
(GT:        8     8     5     5     2)
 HIGHTEST---------------------------->  LOWEST

Is there a way to do this? I have tried creating the pivot by importing the indexes and columns as lists, sorted in the order I would like them to appear, but pandas seems to automatically sort them AZ when creating the table.

Code for the example:

import pandas as pd

exampledata=[['0102','1020',1],['0102','1220',8],['0102','1210',2],
             ['0103','1020',1],['0103','1210',3], ['0103','1222',8],
             ['0104','1020',3],['0104','1120',2]]

df = pd.DataFrame(exampledata,columns=['COLUMN1','COLUMN2','VALUE'])
print(df)
pivot = pd.pivot_table(df,
                       index='COLUMN1',
                       columns='COLUMN2',
                       values='VALUE',
                       aggfunc='sum',
                       fill_value=0)
print(pivot)

Answer 1

pivot_table has an option margin which is convenient for this case:

(df.pivot_table(index='COLUMN1', columns='COLUMN2', values='VALUE',
               aggfunc='sum', fill_value=0, margins=True)   # pivot with margins 
   .sort_values('All', ascending=False)  # sort by row sum
   .drop('All', axis=1)                  # drop column `All`
   .sort_values('All', ascending=False, axis=1) # sort by column sum
   .drop('All')    # drop row `All`
)

Output:

COLUMN2  1220  1222  1020  1210  1120
COLUMN1                              
103         0     8     1     3     0
102         8     0     1     2     0
104         0     0     3     0     2

Answer 2

I will try something like this

pivot['sum_cols'] = pivot.sum(axis=1)
pivot = pivot.sort_values('sum_cols' , ascending=False)

Answer 3

The index of your pivot table (values from COLUMN1 and COLUMN2 ) are of type String , and sorting of String is done from A to Z. Perhaps you should input indexes of Integer type, and then the sorting will be done numerically. Considering the pivot_table documentation Integer type is allowed for columns and index .

df = df.astype('int')

Now, your pivot_table function outputs a DataFrame , which you can sort by index or by columns in the same manner you do with any DataFrame .

According to sort_index documentation : For sorting the index you should do:

pivot = pivot.sort_index(ascending=0)

For sorting the columns you should do:

pivot = pivot.sort_index(axis=1, ascending=0)