Python数据框：在一行上使用Groupby计算置信度或预测间隔

Question

I have a table like below:

Type    Actual  Predicted
A       4       3
A       10      18
A       13      11
B       3       10
B       4       2
B       8       33
C       20      17
C       40      33
C       87      80
C       32      30

I wanted to calculate the R^2 and RMSE for each Type. The code to do that is below:

import numpy as np
import pandas as pd
from sklearn.metrics import r2_score, mean_squared_error

def r2_rmse( g ):
    r2 = r2_score( g['Actual'], g['Predicted'] )
    rmse = np.sqrt( mean_squared_error( g['Actual'], g['Predicted'] ) )
    return pd.Series( dict(  r2 = r2, rmse = rmse ) )

your_df.groupby( 'Type' ).apply( r2_rmse ).reset_index()

Sample Output Table (values are hypothetical):

Type    R^2     RMSE    
A       0.66    4   
B       1.00    6   
C       0.03    1

The above code worked and gave me the output I wanted. But now I want to add confidence / prediction intervals into the table at the Type level. I have literally scoured the internet on how to do this with no luck.

Conceptual Question: If I want the range of values in which the Actual value is captured with 95% confidence, do I run the confidence interval on the Actual column or the Predicted column?

Below is the sample table I want:

Type    Conf_Int_90%  Conf_Int_80%
    A    (21, 100)       (5, 55)
    B    (10, 46)        (3, 14)
    C    (1, 19)         (12, 19)

I have a sense that the confidence interval code is something like this:

st.t.interval(0.95, len(a)-1, loc=np.mean(a), scale=st.sem(a)) BUT ... 

What specific code do I incorporate into my existing code (shown above) so I get the table output I want?

Answer 1

try following, and from my understanding, the confidence interval should be operated in the predicted columns. Hope it helps you :)

import numpy as np
import pandas as pd
import scipy.stats as st
from sklearn.metrics import r2_score, mean_squared_error

def r2_rmse_interval(g):
    r2 = r2_score( g['Actual'], g['Predicted'] )
    rmse = np.sqrt( mean_squared_error( g['Actual'], g['Predicted'] ))
    st_interval = st.t.interval(0.95, len(g) -1, loc=np.mean(g.Predicted), scale=st.sem(g.Predicted))
    return pd.Series( dict(  r2 = r2, rmse = rmse, st_interval = st_interval) )


df = pd.DataFrame({'Type': ['A', 'A', 'A', 'B', 'B', 'B', 'C', 'C', 'C', 'C'],
               'Actual': [4, 10, 13, 3, 4, 8, 20, 40, 87, 32],
               'Predicted': [3, 18, 11, 10, 2, 33, 17, 33, 80, 30]}, 
                columns=['Type', 'Actual', 'Predicted'])

df.groupby( 'Type' ).apply( r2_rmse_interval ).reset_index()

Answer 2

Using the standard formula for 95% CI :

sample mean +/- 1.96 * std.err

You can do everything in one go with apply :

def stats(g):
    r2 = r2_score(g.Actual, g.Predicted)
    rmse = np.sqrt(mean_squared_error(g.Actual, g.Predicted))
    ci95_hi = g.Predicted.mean() + g.Predicted.sem() * 1.96
    ci95_lo = g.Predicted.mean() - g.Predicted.sem() * 1.96
    return r2, rmse,(ci95_lo, ci95_hi)

df.groupby("Type").apply(stats)