操作 pandas DataFrame 中多行列的排序/排序

Question

This is a side-problem caused by an answer form another question .

I do combine two crosstab() results with counted and normalized values. The problem is that the resulting column names are not in the right order. "Right" means that the margins_name (in my example it is "gesamt" ) should always appear at the last row/column and not like this:

sex    female        gesamt         male
            n      %      n       %    n      %
age

What I need is

sex    female          male       gesamt
            n      %      n       %    n      %
age

This is the minimal working example

#!/usr/bin/env python3
import pandas as pd
import pydataset

# sample data
df = pydataset.data('agefat')
df = df.loc[df.age < 35]

# Label of the margin column/row
mn = 'gesamt'

# count absolute
taba = pd.crosstab(df.age, df.sex, margins=True, margins_name=mn)
# percentage / normalized
tabb = pd.crosstab(df.age, df.sex, margins=True, margins_name=mn,
                   normalize=True).round(4)*100

# combine (based on: https://stackoverflow.com/a/68362010/4865723)
tab = pd.concat([taba, tabb], axis=1, keys=['n', '%']).swaplevel(axis=1)

# sort the columns        
tab = tab.sort_index(axis=1, ascending=[True, False])

print(tab)

Also I have a possible solution which works but I am not sure if this is a good panda's way . I do manipulate the sorting-algorithm this way that the margins_name always get the highest possible chr() value to make it appear always at the end of a lexicographical ordering.

# workaround
tab = tab.sort_index(axis=1, ascending=[False, False],
                     key=lambda x: x.where(x.isin([mn]), chr(0x10ffff)))

print(tab)  # looks like I expect

The result output

sex    female        male        gesamt
            n      %    n      %      n       %
age
23          1  16.67    1  16.67      2   33.33
24          0   0.00    1  16.67      1   16.67
27          0   0.00    2  33.33      2   33.33
31          1  16.67    0   0.00      1   16.67
gesamt      2  33.33    4  66.67      6  100.00

Answer 1

Use ordered CategoricalIndex for custom ordering of first level of MultiIndex :

i = tab.columns.levels[0]
out = sorted(i.difference([mn]))
out.append(mn)

new = pd.CategoricalIndex(i, ordered=True, categories=out)
tab.columns = tab.columns.set_levels(new,level=0)

tab = tab.sort_index(axis=1, ascending=[True, False])

print(tab)
sex    female        male        gesamt        
            n      %    n      %      n       %
age                                            
2000        2  33.33    0   0.00      2   33.33
2001        1  16.67    1  16.67      2   33.33
2002        1  16.67    1  16.67      2   33.33
gesamt      4  66.67    2  33.33      6  100.00

Answer 2

I would just select the total columns using a list comprehension and piece together the columns selection as desired:

cols_tot = [c for c in tab.columns if c[0] == mn]
print(tab[[c for c in tab.columns if not c in cols_tot] + cols_tot])

sex    female        male        gesamt        
            n      %    n      %      n       %
age                                            
23          1  16.67    1  16.67      2   33.33
24          0   0.00    1  16.67      1   16.67
27          0   0.00    2  33.33      2   33.33
31          1  16.67    0   0.00      1   16.67
gesamt      2  33.33    4  66.67      6  100.00

Answer 3

Please let me highlight a detail in addition to @jezrael's original answer .

We still know from @jezrael 's answer that .sort_index() does take the ordering of categories into account. This has consequences when you crosstab() on a column that still is an ordered categorical and you adding a margin= (eg a total column) to the crosstab.

Going back to the MWE of my question. Lets assume that age is not a number but a ordered cateogry .

['younger then 20' < '20 till 60' < 'older then 60']

The column will lose its categorical order and .sort_index() will sort it only by its lexicographical order when you do it like this (as in the original MWE):

# Label of the margin column/row
mn = 'gesamt'
# count absolute
taba = pd.crosstab(df.age, df.sex, margins=True, margins_name=mn)

What you have to do is to add the margin= column as one of the categories before calling .crosstab() .

df.age = df.age.cat.add_categories([mn])  # mn=='gesamt'