Calculate T statistics for beta in linear regression model

Question

i have the following equation for calculating the t statistics of a simple linear regression model.

t= beta1/SE(beta1)

SE(beta1)=sqrt((RSS/var(x1))*(1/n-2))

If i want to do this for an simple example wit R, i am not able to get the same results as the linear model in R.

x <- c(1,2,4,8,16)
y <- c(1,2,3,4,5)

mod <- lm(y~x)
summary(mod)

Call:
lm(formula = y ~ x)

Residuals:
       1        2        3        4        5 
-0.74194  0.01613  0.53226  0.56452 -0.37097 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)  
(Intercept)  1.50000    0.44400   3.378   0.0431 *
x            0.24194    0.05376   4.500   0.0205 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.6558 on 3 degrees of freedom
Multiple R-squared:  0.871, Adjusted R-squared:  0.828 
F-statistic: 20.25 on 1 and 3 DF,  p-value: 0.02049

If i do this by hand i get a other value.

var(x)
37.2
sum(resid(mod)^2)
1.290323

beta1=0.24194

SE(beta1)=sqrt((1.290323/37.2)*(1/3)) SE(beta1)=0.1075269

So t= 0.24194/0.1075269=2.250042

So why is my calculation exact the half of the value from R? Has it something to do with one/two tailed tests? The value for t(0.05/2) is 3.18

Regards, Jan

Answer 1

The different result was caused by a missing term in your formula for se(beta) . It should be:

se(beta) = sqrt((1 / (n - 2)) * rss / (var(x) * (n - 1)))

The formula is usually written out as:

se(beta) = sqrt((1 / (n - 2)) * rss / sum((x - mean(x)) ^ 2))

rather than in terms of var(x) .

For the sake of completeness, here's also the computational check:

reprex::reprex_info()
#> Created by the reprex package v0.1.1.9000 on 2017-10-30

x <- c(1, 2, 4, 8, 16)
y <- c(1, 2, 3, 4, 5)
n <- length(x)

mod <- lm(y ~ x)
summary(mod)
#> 
#> Call:
#> lm(formula = y ~ x)
#> 
#> Residuals:
#>        1        2        3        4        5 
#> -0.74194  0.01613  0.53226  0.56452 -0.37097 
#> 
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)  
#> (Intercept)  1.50000    0.44400   3.378   0.0431 *
#> x            0.24194    0.05376   4.500   0.0205 *
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> Residual standard error: 0.6558 on 3 degrees of freedom
#> Multiple R-squared:  0.871,  Adjusted R-squared:  0.828 
#> F-statistic: 20.25 on 1 and 3 DF,  p-value: 0.02049

mod_se_b <- summary(mod)$coefficients[2, 2]

rss <- sum(resid(mod) ^ 2)
se_b <- sqrt((1 / (n - 2)) * rss / (var(x) * (n - 1)))

all.equal(se_b, mod_se_b)
#> [1] TRUE