Checking Objective Function Reproducibility

Why Repeat an Objective-Function Evaluation?

An optimizer decides where to move by comparing objective-function values. That comparison is only useful if the values are stable enough to distinguish meaningful changes in the parameters from variation in the calculation itself. In a nonlinear mixed-effects model, evaluating the likelihood can involve integrating over random effects. Depending on the implementation, the calculation may use deterministic approximations, simulation, or numerical routines whose results are sensitive to tolerances and computation order. A run can therefore appear to improve, stall, or converge partly because the objective itself varies when nothing in the model or data has changed.

A reproducibility check isolates this issue. Hold the model, data, parameter values, and evaluation settings fixed; evaluate the objective repeatedly; then quantify how much the reported value varies. This is not a test of whether the model fits well, nor does a stable value prove that the optimizer found the best estimate. It asks a narrower but important question: does the calculation return essentially the same objective for the same input?

Many NLME programs report an objective proportional to the negative log likelihood, often \(F=-2\ell\), where \(\ell\) is the log likelihood. Under that convention, a smaller objective corresponds to a larger likelihood. Software may report a different scaling or add a constant, so use the same definition and settings throughout a check. A constant added to all evaluations does not affect their differences, but mixing objective conventions makes the comparison meaningless.

Conditions for a Valid Check

The purpose is to vary only the evaluation itself. Keep the parameter vector exactly fixed, including any transformations used internally, and use the same data records, model code, likelihood definition, and numerical settings. Do not allow an optimizer step, parameter update, data filtering operation, or change in integration settings between repetitions. Record the random-number seed and whether the program reuses a random stream. If the data or settings change, a difference in objective values cannot be attributed specifically to repeatability.

There are two useful but distinct tests. Repeating with the same seed and same execution conditions checks whether the computation is deterministic given its random stream and whether program state is controlled. Repeating with different seeds estimates variation attributable to stochastic integration or simulation. If the evaluator is intended to be deterministic, same-condition repeats should agree within an appropriate numerical tolerance. If it is intentionally stochastic, different-seed repeats need not agree exactly; the key question is whether their spread is small enough for the intended optimization decisions.

The data should also be the same in the sense relevant to the likelihood: identical observations, covariates, event records, and missing-data handling. In a parallel calculation, thread count and reduction order can affect rounding. These are part of the computation conditions and should be recorded rather than silently changed. Before interpreting a discrepancy, verify that the repeated calls really used the same input and did not inherit a changed state from an earlier call.

How to Summarize Repeated Values

For \(R\) repeated evaluations, calculate their mean, sample standard deviation, and range. The mean describes the average reported objective in these repetitions; the standard deviation and range describe observed variation. They are diagnostics, not a universal pass/fail test. The acceptable difference depends on the objective scale, the optimizer's stopping rules, and the size of objective changes that matter in the fit.

A short set of repetitions can reveal substantial instability, but a small observed range does not guarantee that all possible evaluations will agree. For stochastic methods, more repetitions provide a more informative estimate of variability. For deterministic methods, repeat the exact call first; if values differ, then change one computational condition at a time to help locate the source.

Why Evaluation Noise Can Mislead Optimization

An optimizer compares objective values at different parameter vectors. Write a stochastic evaluation as \(\hat F(\theta)=F(\theta)+\epsilon\), where \(F(\theta)\) is the target objective and \(\epsilon\) is evaluation error. For a proposed move from \(\theta_0\) to \(\theta_1\), the observed difference is \(\hat F(\theta_1)-\hat F(\theta_0)=F(\theta_1)-F(\theta_0)+\epsilon_1-\epsilon_0\). Thus, the comparison includes the difference in evaluation errors as well as the actual change in the objective.

A simple justification makes the practical issue clear. If the errors at the two points are independent, have mean zero, and each have variance \(\tau^2\), then their difference has variance \(2\tau^2\), because \(\operatorname{Var}(\epsilon_1-\epsilon_0)=\operatorname{Var}(\epsilon_1)+\operatorname{Var}(\epsilon_0)=2\tau^2\). Its standard deviation is therefore \(\sqrt{2}\tau\). When the true objective change is small relative to this scale, the observed comparison can have the wrong sign. If the errors are approximately normal, the probability of a wrong sign for a true change of magnitude \(|\Delta|\) is \(\Phi(-|\Delta|/(\sqrt{2}\tau))\), where \(\Phi\) is the standard normal cumulative distribution function. The normal approximation is not required for the diagnostic; it illustrates why small optimizer steps are vulnerable to noisy evaluations.

Using common random numbers at nearby parameter values can sometimes make the errors positively correlated and reduce the variance of their difference. That may help a comparison, but it does not make either objective value deterministic. A reproducibility report should say whether the evaluations used common or independent random streams rather than treating the resulting variability as interchangeable.

Worked Diagnostics

Failure Modes, Alternatives, and Limits

If identical-condition repeats differ more than expected, do not immediately label the optimizer as failed. First confirm that the inputs are truly fixed. Then investigate potential sources in a controlled order. Randomized integration can cause variation across seeds; insufficient integration effort can leave variation large enough to affect comparisons. Numerical solvers and quadrature can be sensitive to tolerances. Parallel calculations can change the order in which floating-point values are combined. A stateful implementation, unintended cache reuse, or an accidental change to records or settings can also produce a discrepancy. Change one feature at a time so the result is interpretable.

The main alternative to repeated evaluation is to inspect a single objective value and rely on the software's convergence output. That is faster, but it cannot reveal whether the evaluator itself changes at a fixed point. A complementary approach is to repeat evaluations with a more accurate or deterministic integration method, where available. Agreement between methods is useful evidence about numerical stability, but it is not guaranteed: different approximations can share bias, and changing methods can change the numerical objective. These checks address computation, not model adequacy.

There is no universal cutoff for an acceptable objective-function range. A variation of 0.1 may be irrelevant if meaningful comparisons differ by many units, or consequential if the optimizer is distinguishing changes much smaller than 0.1. Similarly, the sample standard deviation across a few stochastic evaluations is uncertain. Report the actual repeated values and the conditions used, not only a declaration that reproducibility “passed.”

Finally, fixed-point reproducibility is not a substitute for examining parameter estimates, gradients, or other convergence evidence. A perfectly repeatable objective can be evaluated at a poor parameter vector, and stable repeated evaluations do not prove global optimality. Conversely, noisy objective evaluations can complicate optimization even when the underlying statistical model is appropriate. The check tells you whether the computational objective is stable enough to support the decisions being made; it does not settle those other questions.

Common Mistake