Filtering Pandas Dataframe according to indices

Question

Consider following dataframe:

m.transmission

                                         eff    inv-cost    fix-cost  var-cost
Site In Site Out Transmission Commodity

Mid     North    hvac         Elec       0.90   1650000     16500         0
Mid     South    hvac         Elec       0.90   1650000     16500         0
North   Mid      hvac         Elec       0.90   1650000     16500         0
North   South    hvac         Elec       0.85   3000000     30000         0
South   Mid      hvac         Elec       0.90   1650000     16500         0
South   North    hvac         Elec       0.85   3000000     30000         0

I would like to filter values according to, if Site In == 'Mid' or Site Out == 'Mid'

How would I do it? Before saying anything, this is not the desired outcome:

m.transmission.loc[['Mid']]

Site In Site Out Transmission Commodity 
Mid     North    hvac         Elec
Mid     South    hvac         Elec

because it just filters if Site In == 'Mid'

Desired output would be (ofc with the columns; such as eff, inv-cost, fix-cost, var-cost):

Site In Site Out Transmission Commodity 
Mid     North    hvac         Elec
Mid     South    hvac         Elec
North   Mid      hvac         Elec
South   Mid      hvac         Elec

EXTRA

(Pdb) m.transmission.columns
Index(['eff', 'inv-cost', 'fix-cost', 'var-cost', 'inst-cap', 'cap-lo',
       'cap-up', 'wacc', 'depreciation'],
      dtype='object')
(Pdb) m.transmission.index
MultiIndex(levels=[['Mid', 'North', 'South'], ['Mid', 'North', 'South'], ['hvac'], ['Elec']],
           labels=[[0, 0, 1, 1, 2, 2], [1, 2, 0, 2, 0, 1], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]],
           names=['Site In', 'Site Out', 'Transmission', 'Commodity'])

Answer 1

If 'columns' are in index.

In [217]: df.loc[(df.index.get_level_values('Site In') == 'Mid') | 
                 (df.index.get_level_values('Site Out') == 'Mid')]
Out[217]:
                                         v
Site In Site Out Transmission Commodity
Mid     North    hvac         Elec       1
        South    hvac         Elec       1
North   Mid      hvac         Elec       1
South   Mid      hvac         Elec       1

---

If it's a flat dataframe, use

In [168]: df.loc[(df['Site In'] == 'Mid') | (df['Site Out'] == 'Mid')]
Out[168]:
  Site In Site Out Transmission Commodity
0     Mid    North         hvac      Elec
1     Mid    South         hvac      Elec
2   North      Mid         hvac      Elec
4   South      Mid         hvac      Elec

Or

In [169]: df.loc[df['Site In'].eq('Mid') | df['Site Out'].eq('Mid')]
Out[169]:
  Site In Site Out Transmission Commodity
0     Mid    North         hvac      Elec
1     Mid    South         hvac      Elec
2   North      Mid         hvac      Elec
4   South      Mid         hvac      Elec

Answer 2

UPDATE:

Demo:

In [94]: df
Out[94]:
                                         val
Site_In Site_Out Transmission Commodity
Mid     North    hvac         Elec         1
        South    hvac         Elec         2
North   Mid      hvac         Elec         3
        South    hvac         Elec         4
South   Mid      hvac         Elec         5
        North    hvac         Elec         6

In [95]: df.query("Site_In == 'Mid' or Site_Out == 'Mid'")
Out[95]:
                                         val
Site_In Site_Out Transmission Commodity
Mid     North    hvac         Elec         1
        South    hvac         Elec         2
North   Mid      hvac         Elec         3
South   Mid      hvac         Elec         5

NOTE: this approach works only for index/column names that don't contain spaces

Answer 3

You can try this.. It's very easy to achieve.

In [12]: df[(df["Site In"]=="Mid")|(df["Site Out"]=="Mid")]
Out[12]:
  Site In Site Out Transmission Commodity
0     Mid    North         hvac      Elec
1     Mid    South         hvac      Elec
2   North      Mid         hvac      Elec
4   South      Mid         hvac      Elec