如何通过 pandas MultiIndex 高效过滤和求和

Question

I have a DataFrame with a MultiIndex where I would like to, as efficiently as possible:

1. Filter by one index ( flag & flag_filter != 0 )
2. Group and sum by the other two ( df.groupby(['time', 'sensor']).sum(['col1','col2','col3']) )

So as a setup:

import pandas as pd
import numpy as np

index = pd.MultiIndex.from_product(
    [
        range(0, 0xff),
        range(0, 5000),
        range(1, 3),
    ], names = ["flags", "time", "sensor"]
)

data = pd.DataFrame({
    "col1": np.random.uniform(size=len(index), low=0.0, high=0.5),
    "col2": np.random.uniform(size=len(index), low=0.0, high=0.5),
    "col3": np.random.uniform(size=len(index), low=0.0, high=0.5),
}, index = index)

I'm hoping to get, from this, a DataFrame with the same columns, but an index of just time, sensor . The idea is we threw out rows that didn't match the filter, and summed the rows that did, while still maintaining the time, sensor grouping.

Answer 1

Combine .loc with droplevel :

# Let's say we want to filter for even flags
flag_filter = data.index.get_level_values("flags") % 2 == 0

# Select matching rows and drop the first level 
data.loc[flag_filter, :].droplevel(0)