Programming Newton Raphson in R for Maximum Likelihood estimation

Question

I need to programm the Newton-Raphson method in R to estimate the parameter of a Poisson distribution. I am just getting started with programmation and with R. When i run my program with simulated data, R return some errors.

Error in if (abs(x1 - x0) < stoptol) break : 
missing value where TRUE/FALSE needed
In addition: Warning messages:
1: In log(mu) : NaNs produced
2: In log(mu) : NaNs produced
3: In log(mu) : NaNs produced

This is what i have so far:

#
#  NEWTON-RAPHSON METHOD 
#

#generate the data 
lambda=3.2
y=rpois(500,lambda)

#declare the log likehood function
poisson.lik<-function(mu,ydata=y){
   n<-length(ydata)
   logl<-sum(ydata)*log(mu)-n*mu-sum(lfactorial(ydata))
   return(-logl)
}

## Newton-Raphson 

NR<-function(initval, f, stoptol=1e-05, imax=25){
   i=0
   h=1e-05
   x0=initval-0.1
   x1=initval

   while(i<=imax){
     df.dx=(f(x0+h)-f(x0))/h
     x1=(x0-(f(x0)/df.dx))
     i=i+1
     if(abs(x1-x0)<stoptol) break
     x0=x1
    }
    list(nstep=i, initial=initval, final=x1, fctval=f(x1))
}

NR(initval=3,poisson.lik)

To my understanding, one problem comes from the value that the parameter mu takes in the iterations of NR function and the computation of the log. Maybe i should force mu taking only a range of values... The other error is about the condition "if" (the stop criterion) but i really dont know what the problem is.

Answer 1

Your NaN issue was coming from your poisson.lik function. You need to have log(abs(mu)) in the case where mu is negative.

#
#  NEWTON-RAPHSON METHOD 
#

#generate the data 
lambda <- 3.2
ydata <- rpois(500, lambda)

#declare the log likehood function
poisson.lik <- function(mu = ""){
    n <- length(ydata)
    loglik <- -n * mu - sum(log(factorial(ydata))) + log(abs(mu)) * sum(ydata) 
    return(-loglik)
}

#creating the Newton Raphson loop
NR <- function ( mu = "", initval = "", f = "", stoptol = 1e-05, imax = "") {
    i = 0
    h = .1
    mu0 = initval - 0.1
    mu1 = initval
    df.dx <- double(1) #predeclare your variable types     
    while ( abs(f(mu) - f(mu1)) > stoptol && i <= imax ) {
        mu0 <- mu1
        df.dx <- (f(mu0 + h) - f(mu0)) / h
        mu1 <- mu0 + (mu - mu1) / abs(df.dx)
        i <- i + 1        
    }
    return(list("nstep" = i, "Final" = mu1, "fctval" = f(mu1)))
} 

mu <- 3.2
initval <- 1
f <- poisson.lik
stoptol <- 0.0001
imax <- 10^4
NR(mu, initval, f, stoptol, imax)

The main issue I saw was that you need to know where you want the function to go to. It is usually used when you know some aspect about the data. For instance if you knew what you wanted the loglikelihood was, the skewness or some type of statistic so you can solve for the parameter that yields that same statistic. The code above solve the problem. Hope this helps.