Find values that exceed the minimum or maximum

Question

I am attempting. My dataframe looks similar to this:

Name    DateTime    Na  Na Err  Mg  Mg Err  Al  Al Err  Si  Si Err
STD1    2/11/2020   0.3 0.11    1.6 0.08    0.6 0.12    21.5    0.14
STD2    2/11/2020   0.2 0.10    1.6 0.08    0.2 0.12    21.6    0.14
STD3    2/11/2020   0.2 0.10    1.6 0.08    0.5 0.12    21.7    0.14
STD4    2/11/2020   0.1 0.10    1.3 0.08    0.5 0.12    21.4    0.14

Here is what I have:

elements=['Na','Mg', 'Al', 'Si',...]
quant=df[elements].quantile([lower, upper]) #obtain upper/lower limits
outsideBounds=(quant.loc[lower_bound, elements] < df[elements].to_numpy()) \
& (df[elements].to_numpy()<quant.loc[lower_bound, elements])

However, this gives me a "ValueError: Lengths must match to compare". Any help would be appreciated

Answer 1

Here's a solution (I chose 0.3 and 0.7 for lower and upper bounds, respectively, but that can be changed of course):

lower = 0.3
upper = 0.7
elements=['Na','Mg', 'Al', 'Si']
df[elements]
bounds = df[elements].quantile([lower, upper]) #obtain upper/lower limits
out_of_bounds = df[elements].lt(bounds.loc[lower, :]) | df[elements].gt(bounds.loc[upper, :])
df[elements][out_of_bounds]

The resulting bounds are:

       Na    Mg    Al     Si
0.3  0.19  1.57  0.47  21.49
0.7  0.21  1.60  0.51  21.61

The result of df[elements][out_of_bounds] is:

    Na   Mg   Al    Si
0  0.3  NaN  0.6   NaN
1  NaN  NaN  0.2   NaN
2  NaN  NaN  NaN  21.7
3  0.1  1.3  NaN  21.4