Simlulating a t-test in R

Question

I am looking for a way to simulate the power of a simple t-test in different sample sizes. My idea is to generate 400 random normal distribution samples, each with mean 5 and variance 1, and perform a t-test on each one concerning the hypothesis that the true mean is 4, ie the t-test would take the form:

t=(mean(x)-4)*sqrt(n)/sd(x) # for each sample x which consists of n observations.

For comparison I would like, the first 100 samples to consist of 10 observations, the next 100 ones of 100, the next 100 of 1000 and finally the last 100 of 5000, which I think is the upper limit. A t-test will have to be performed on each and every sample.

Lastly, I would like to see on what percentage of each sample group- let's call them, n10,n100,n1000,n5000, depending on how many observations they comprise- my (false) hypothesis is rejected.

Could you please help me write the corresponding R-code? I know the small commands but have trouble putting it all together. This is a nice exercise and hopefully I shall then be able to modify it a bit and use it for different purposes as well.

Thank you in advance.

Answer 1

Here's a one liner for 400 t.tests of n=10:

R>simulations <- replicate(400, t.test(rnorm(10, mean=5, sd=1), mu=4),
                           simplify=FALSE);

Then you can analyze it:

R>table(sapply(simulations, "[[", "p.value") < .05)
FALSE  TRUE 
   75   325 

Answer 2

I'm still learning R, too, so handle with care:

n <- 5
N <- 100
samplesizes <- as.integer(10^(1:n))
set.seed(1) 

# generate samples
samples <- replicate(N, mapply(rnorm, samplesizes, mean=4, sd=sqrt(1)))

# perform t-tests
t.tests <- lapply(samples, function(x) t.test(x, mu=5, alternative="two.sided"))

# get p-values
t.test.pvalues <- sapply(t.tests, function(x) x$p.value)

rejected <- t.test.pvalues > .05
sampleIndices <- rep(1:n, N)
res <- aggregate(rejected, list(sample=sampleIndices), FUN=function(x) sum(x)/length(x) )
names(res)[2] <- "percRejected"
print(res, row.names=F)
# sample percRejected
# 1         0.16
# 2         0.00
# 3         0.00
# 4         0.00
# 5         0.00