Why a Small Objective-Function Change Needs a Numerical Check
In a covariate test, the change in the objective function may be compared with a statistical cutoff. That comparison is meaningful only if the fitted objective values are sufficiently accurate. A difference close to a cutoff can change sides because of rounding or small optimization errors; a larger discrepancy can arise when one fit stops at a local, rather than the best available, optimum. The resulting test can then reflect the numerical behavior of the fitting procedure as much as the evidence for the covariate.
This tutorial focuses on diagnosing that problem and checking the fits before accepting a test. It does not replace the likelihood-ratio test assumptions: the models must be appropriately nested and fitted to the same observations under the same likelihood and objective-function conventions. Numerical stability cannot correct a mismatch in data, likelihood, or model specification.
The likelihood-ratio statistic is this difference when the models and fitting conventions make the likelihood-ratio comparison appropriate. Its statistical reference distribution and cutoff are separate from the numerical question: how accurately have the two objective values been obtained? A test statistic should not be treated as reliable merely because the software printed a value or reported convergence.
How Rounding Affects a Difference
Suppose the reported objective values are rounded versions of the values that would be obtained at the numerical optima. If values are rounded to a resolution \(q\), each reported value can differ from its unrounded value by at most \(q/2\), assuming ordinary rounding to nearest. Because a difference combines two values, the worst-case error in that difference is the sum of the two possible rounding errors.
This bound concerns rounding alone. It does not bound error from incomplete optimization, local minima, or other computation. If a test cutoff is \(c\), a rounding-only interval \([\widetilde{\Delta OFV}-q,\widetilde{\Delta OFV}+q]\) entirely above \(c\) or entirely below \(c\) indicates that rounding at that precision would not reverse the comparison. If the interval includes values on both sides of \(c\), the displayed precision is not enough to settle the comparison.
Worked Example: A Difference Stable to Two-Decimal Rounding
Suppose a reduced model has a reported \(OFV\) of 804.23 and its expanded, nested model has a reported \(OFV\) of 800.23. Both values are rounded to two decimal places, so \(q=0.01\). For illustration, compare the result with a prespecified cutoff of 3.84.
Calculate the displayed difference.
Bound rounding error. Each objective value has a maximum rounding error of 0.005. The difference can therefore have a maximum rounding error of \(0.005+0.005=0.01\). Its rounding-only range is approximately \(4.00-0.01=3.99\) to \(4.00+0.01=4.01\).
The entire range is above 3.84, so rounding to two decimal places alone would not reverse the comparison. This does not establish that the optimizations are accurate; it only rules out this particular source of uncertainty at the stated precision.
Worked Example: A Difference Too Close to the Cutoff for Rounded Values
A reduced model has a reported \(OFV\) of 510.24, and a nested expanded model has a reported \(OFV\) of 506.40. The values are rounded to two decimal places. Use 3.84 as an illustrative cutoff.
Calculate the displayed difference.
The rounding-only error bound is \(q=0.01\), so the difference consistent with these displayed values is approximately \(3.84-0.01=3.83\) to \(3.84+0.01=3.85\). This range includes values below and above 3.84. The rounded values therefore do not establish on which side of the cutoff the unrounded difference lies.
The appropriate next step is to obtain and compare higher-precision objective values, then check the fits themselves. Reporting more digits can resolve display rounding, but it cannot resolve an optimization error.
Why Convergence and Local Minima Require Separate Checks
An optimizer typically stops when one or more numerical criteria are met, such as a small change in the objective, small parameter updates, or a sufficiently small gradient. These criteria are evidence about the algorithm's stopping behavior; they are not a universal guarantee that the true optimum has been found. In a nonlinear mixed-effects model, different starting values can lead to different regions of the parameter space, and a fit may stop at a local optimum or a poorly determined point.
A nested expanded model should be able to reproduce the reduced model by setting its added covariate effect to the null value. At the global maximum likelihoods, this nesting implies that the expanded model cannot have a worse likelihood than the reduced model. Thus, a negative \(\Delta OFV\) is a warning to investigate. It is not proof of a particular failure: differences in data, likelihood settings, parameter constraints, or objective-function definitions can also make the comparison invalid.
A small positive difference can be unreliable, too. If the reduced and expanded models are optimized to different degrees of accuracy, the apparent improvement may be driven by unequal fitting error. Even if both runs report convergence, inspect the estimates, gradients or other available convergence diagnostics, boundary behavior, and sensitivity to starting values and tighter tolerances.
Worked Example: A Covariate Result Changes After Refitting
Consider a hypothetical nested-model comparison. An initial run reports \(OFV_{\mathrm{reduced}}=650.00\) and \(OFV_{\mathrm{expanded}}=646.70\). A 3.84 cutoff is used for illustration.
Initial comparison.
The initial difference is below 3.84. However, diagnostics show that estimates vary with starting values and the expanded-model fit has not been stable. It would be premature to interpret 3.30 as a dependable test statistic.
The analyst refits both models with tighter convergence settings and several dispersed, plausible starting values, retaining the best valid objective value found for each model. The reduced model reaches 649.82, while the expanded model reaches 645.96.
Recomputed comparison.
The refitted difference is above 3.84. The initial and refitted differences lead to different cutoff comparisons, so the initial result is numerically unstable. The refit is better supported by the checks described, but multiple starts do not mathematically prove that a global optimum has been found. The conclusion should be based on a documented, sufficiently stable fitting procedure, not on choosing whichever run gives a preferred test result.
A Practical Refitting and Review Sequence
Before accepting a covariate test whose objective-function difference may be affected by numerical noise, check comparability and optimization in a systematic order. The checks should apply to both models, not only the one with the covariate.
Confirm that the models use the same observations, likelihood definition, estimation settings, and objective-function scale. Confirm that the expanded model is genuinely nested in the reduced model and that the added effect is set to its null value in the reduced model.
Record objective values with enough digits to assess rounding relative to the cutoff. If the rounding-only range can cross the cutoff, do not make the cutoff decision from the displayed values.
Use tighter tolerances and multiple reasonable starting values. Compare objective values and parameter estimates across runs. A substantially lower objective from a repeat fit indicates that the earlier fit was not sufficiently stable.
Review convergence status, gradients or equivalent optimizer diagnostics, implausible estimates, parameter boundaries, and warnings. A convergence label by itself is not enough when results change materially across fits.
Use the best supported, comparable fits to calculate \(OFV_{\mathrm{reduced}}-OFV_{\mathrm{expanded}}\). Report the fitting checks and whether plausible numerical variation could change the comparison with the prespecified cutoff.
There is no universal numerical tolerance that makes every objective-function difference trustworthy. The needed precision depends on the scale of the objective, the closeness of the difference to the decision cutoff, and the behavior of the model and optimizer. Repeat-fit variability is a practical diagnostic, not a formal error bound on the unknown global optimum.
Distinguishing Numerical Instability from Statistical Evidence
The likelihood-ratio, Wald, and score tests address covariate hypotheses in different ways. A likelihood-ratio test depends directly on the difference between fitted objective values, so numerical error in either fit directly affects its statistic. A Wald test uses an estimated effect and its uncertainty; a score test uses information evaluated under the null model. These alternatives can be useful comparisons, but neither automatically repairs a poorly optimized model. Their assumptions and numerical calculations also require scrutiny.
The numerical review should therefore precede interpretation of the statistical test. A difference that is stable across refits and precise enough relative to the cutoff supports a more dependable test calculation; it does not by itself establish that the covariate effect is clinically important, correctly specified, or free from confounding. Conversely, a difference that changes across reasonable fitting runs is not reliable evidence for or against the covariate until the source of instability is understood.
Common Mistakes and Limitations
- Treating printed digits as exact. If both objective values are rounded, their difference inherits uncertainty from both values. Calculate the rounding bound rather than assuming the subtraction is exact.
- Assuming convergence means global optimization. A convergence message describes a stopping condition. Compare starts, estimates, and diagnostics before concluding that the objective is stable.
- Refitting only the expanded model. The difference depends on both objective values. A reduced-model fit can also be trapped or insufficiently optimized, so check both.
- Using a negative difference without investigation. For comparable nested models at their global maximum likelihoods, the expanded model should fit at least as well. Check optimization and comparability rather than interpreting a negative value as ordinary evidence.
- Changing settings until a desired result appears. Refitting is a diagnostic, not an invitation to select a favorable statistic. Prespecify or document a consistent fitting strategy and report material sensitivity.
These checks have limits. Multiple starting values cannot prove that every region of a complex parameter space has been explored, and tighter tolerances may increase computation without resolving weak identifiability or a misspecified model. If objective values and estimates remain unstable, report that the test is numerically unreliable and investigate the model or estimation problem before drawing a covariate conclusion.
Check Your Understanding
Use the stated assumptions in each question. Solutions follow the questions.
- Two objective values are rounded to the nearest 0.1. What is the maximum rounding error in their difference?
- A reported difference is 4.0, each objective value was rounded to two decimal places, and the cutoff is 3.84. Can rounding alone reverse the comparison?
- A reduced model has an \(OFV\) of 420.10 and a nested expanded model has an \(OFV\) of 420.14. What should be checked before interpreting the negative difference?
- Both models report convergence, but their objective values and parameter estimates change materially across starting values. Is the test ready to interpret? Give the next checks.
Solutions. (1) Here \(q=0.1\). Each objective value has rounding error at most \(0.05\), so the difference has maximum rounding error \(0.05+0.05=0.10\). (2) With two-decimal rounding, \(q=0.01\), so the rounding-only range is approximately \(4.00-0.01=3.99\) to \(4.00+0.01=4.01\). This range is entirely above 3.84; rounding alone would not reverse the comparison. (3) The displayed difference is \(420.10-420.14=-0.04\). For comparable nested models at their global maximum likelihoods, the difference should be nonnegative. Check that data, likelihood settings, objective scale, nesting, and constraints are the same, then inspect convergence and refit both models with tighter tolerances and multiple starting values. (4) No. Convergence messages do not establish stable optima. Review diagnostics and estimates, refit both models with tighter tolerances and varied plausible starts, and recalculate the difference only after the objective values are sufficiently stable. If the instability remains, report the test as numerically unreliable rather than choosing a favorable run.