Fit distribution to empirical data

Question

I am trying to fit a beta distribution to a histogram created from empirical data.

The problem I encounter is that the fitted distribution is much higher than the bars in the original histogram.

The original data is outside the range of [0,1] which is the range in which the beta distribution can be evaluated so I rescale the original data so that it lies in the interval [0,1].

Here's my code:

 load("https://www.dropbox.com/s/c3psxx8jjbc20mo/data.Rdata?dl=0")

  #create histogram with values normalized between 0 and 1
  h <- hist((data-min(data)) / (max(data)-min(data)),lty="blank",col="grey")
  #normalize the density so the y-axis goes from 0 to 1
  h$density <- h$counts/max(h$counts)
  #plot the results
  plot(h,freq=FALSE,cex.main=1,cex.axis=1,yaxt='n',ylim=c(0,1.5),col='grey',lty='blank',xaxt='n')
  axis(2,at=seq(0,1,0.5),labels=seq(0,1,0.5))
  axis(1,at=seq(0,1,0.5),labels=seq(0,1,0.5))

  #fit beta distribution
  a <- (data-min(data)) / (max(data)-min(data))
  a[a==1] <- 0.9999
  a[a==0] <- 0.0001
  fit.beta <- suppressWarnings(fitdistr(a, "beta", start = list( shape1=0.1, shape2=0.1 ) ))

  #overlay curve from beta distribution
  alpha <- fit.beta$estimate[1]
  beta <- fit.beta$estimate[2]
  b <- rbeta(length(data),alpha,beta)
  lines(density(b))

What am I missing?

Answer 1

First, you will want to use hist(..., freq=TRUE) for the histogram. Then, to correctly set the y-axis range, you can compute the maximum value of your beta distribution ( see eg here ). Finally, it woul be much better to use dbeta than generate a random sample followed by estimating the density:

maxibeta <- dbeta((alpha-1)/(alpha+beta-2), alpha, beta)
hist( (data-min(data)) / (max(data)-min(data)), 
      prob=TRUE, col="grey", border="white", ylim=c(0, maxibeta), 
      main="Histogram + fitted distribution")
plot(function(x) dbeta(x,alpha,beta), add=TRUE, col=2, lwd=2)

---

EDIT: a more general solution, but which makes me a little sad because it does not use the nice properties of the beta distribution:

fbeta <- function(x)  dbeta(x,alpha,beta)
maxibeta <- optimize(fbeta, interval = c(0,1), maximum = TRUE)$objective

histo <- hist((data-min(data)) / (max(data)-min(data)), plot = FALSE)

plot(histo, freq=FALSE, col="grey", border="white", 
     ylim=c(0, max(maxibeta, max(histo$density))), 
     main="Histogram + fitted distribution")
plot(fbeta, add=TRUE, col=2, lwd=2)