How to calculate the Fisher information matrix in Gaussian Mixture model with R

Question

For the Gaussian Mixture model with k components,

Fisher information matrix is given by,

How do I calculate the Fisher information matrix? Is there an R function available for this calculation?

Answer 1

Take a look at mle.tools package. For example, in normal distributions,

library(mle.tools)
## Normal distribution
pdf <- quote(1 / (sqrt(2 * pi) * sigma) * exp(-0.5 / sigma ^ 2 * (x - mu) ^ 2))
lpdf <- quote(-log(sigma) - 0.5 / sigma ^ 2 * (x - mu) ^ 2)
x <- rnorm(n = 100, mean = 0.0, sd = 1.0)
expected.varcov(density = pdf, logdensity = lpdf, n = length(x), parms = c("mu", "sigma"),
                mle = c(mean(x), sd(x)), lower = '-Inf', upper = 'Inf')

$mle
         mu       sigma 
-0.01889498  0.96256386 

$varcov
                 mu         sigma
mu     9.265292e-03 -6.989460e-13
sigma -6.989460e-13  4.632646e-03




##Normal distribution
lpdf <- quote(-log(sigma) - 0.5 / sigma ^ 2 * (x - mu) ^ 2)
x <- rnorm(n = 100, mean = 0.0, sd = 1.0)
observed.varcov(logdensity = lpdf, X = x, parms = c("mu", "sigma"),
                mle = c(mean(x), sd(x)))

$mle
        mu      sigma 
0.02476464 1.12332365 

$varcov
                mu        sigma
mu    1.261856e-02 2.617048e-19
sigma 2.617048e-19 6.405360e-03

For more details, seemle.tools.pdf