Why is AD Model builder better than using spreadsheet solvers or other statistical packages?
The major problem in nonlinear statistical modeling is fitting the model to data. This involves nonlinear optimization. For the kinds of problems generally encountered in statistical modeling the best nonlinear optimization routines employ the derivatives of the function being maximized. Spreadsheet solvers and other statistical modeling packages use finite difference approximations for the derivatives of the function to be maximized in their solvers. This approach has two major limitations. The inaccuracy of the derivative approximations leads to instability in the solver. The result is that the solver becomes unreliable for ill-conditioned problems (naturally most real problems of interest are ill-conditioned). Also with finite difference approximations it takes at least n function evaluations to obtain the finite difference approximation for a function with n independent variables. As a result it is generally impossible to fit models with more than 20 or so parameters reliably with
