How can optimization be used as a solver?

Question

In a question on Cross Validated ( How to simulate censored data ), I saw that the optim function was used as a kind of solver instead of as an optimizer. Here is an example:

 optim(1, fn=function(scl){(pweibull(.88, shape=.5, scale=scl, lower.tail=F)-.15)^2}) # $par # [1] 0.2445312 # ... pweibull(.88, shape=.5, scale=0.2445312, lower.tail=F) # [1] 0.1500135 

I have found a tutorial on optim here , but I am still not able to figure out how to use optim to work as a solver. I have several questions:

1. What is first parameter (ie, the value 1 being passed in)?

2. What is the function that is passed in?

3. I can understand that it is taking the Weibull probability distribution and subtracting 0.15, but why are we squaring the result?

Answer 1

I believe you are referring to my answer. Let's walk through a few points:

• The OP (of that question) wanted to generate (pseudo-)random data from a Weibull distribution with specified shape and scale parameters, and where the censoring would be applied for all data past a certain censoring time, and end up with a prespecified censoring rate. The problem is that once you have specified any three of those, the fourth is necessarily fixed. You cannot specify all four simultaneously unless you are very lucky and the values you specify happen to fit together perfectly. As it happened, the OP was not so lucky with the four preferred values—it was impossible to have all four as they were inconsistent. At that point, you can decide to specify any three and solve for the last. The code I presented were examples of how to do that.

• As noted in the documentation for ?optim , the first argument is par "[i]nitial values for the parameters to be optimized over".

  Very loosely, the way the optimization routine works is that it calculates an output value given a function and an input value. Then it 'looks around' to see if moving to a different input value would lead to a better output value. If that appears to be the case, it moves in that direction and starts the process again. (It stops when it does not appear that moving in either direction will yield a better output value.)

  The point is that is has to start somewhere, and the user is obliged to specify that value. In each case, I started with the OP's preferred value (although really I could have started most anywhere).

• The function that I passed in is ?pweibull . It is the cumulative distribution function (CDF) of the Weibull distribution . It takes a quantile ( X value) as its input and returns the proportion of the distribution that has been passed through up to that point. Because the OP wanted to censor the most extreme 15% of that distribution, I specified that pweibull return the proportion that had not yet been passed through instead (that is the lower.tail=F part). I then subtracted .15 from the result.

• Thus, the ideal output (from my point of view) would be 0 . However, it is possible to get values below zero by finding a scale parameter that makes the output of pweibull < .15. Since optim (or really most any optimizer) finds the input value that minimizes the output value, that is what it would have done. To keep that from happening, I squared the difference. That means that when the optimizer went 'too far' and found a scale parameter that yielded an output of .05 from pweibull , and the difference was -.10 (ie, < 0 ), the squaring makes the ultimate output +.01 (ie, > 0 , or worse). This would push the optimizer back towards the scale parameter that makes pweibull output ( .15 -.15)^2 = 0 .

• In general, the distinction you are making between an "optimizer" and a "solver" is opaque to me. They seem like two different views of the same elephant .

• Another possible confusion here involves optimization vs. regression. Optimization is simply about finding an input value[s] that minimizes (maximizes) the output of a function. In regression , we conceptualize data as draws from a data generating process that is a stochastic function. Given a set of realized values and a functional form, we use optimization techniques to estimate the parameters of the function, thus extracting the data generating process from noisy instances. Part of regression analyses partakes of optimization then, but other aspects of regression are less concerned with optimization and optimization itself is much larger than regression. For example, the functions optimized in my answer to the other question are deterministic, and there were no "data" being analyzed.